Actually Useful AI

Instructions and methodology are open source: https://github.com/open-wisdom/views

I'm tired of hearing about vibecoding on Lobsters, so I've written up three of my side tasks for coding agents. Talk is cheap; show us the code.

cross-posted from: https://lemmy.world/post/41992574

Hello!

As a handsome local AI enjoyer™ you’ve probably noticed one of the big flaws with LLMs:

It lies. Confidently. ALL THE TIME.

(Technically, it “bullshits” - https://link.springer.com/article/10.1007/s10676-024-09775-5

I’m autistic and extremely allergic to vibes-based tooling, so … I built a thing. Maybe it’s useful to you too.

The thing: llama-conductor

llama-conductor is a router that sits between your frontend (OWUI / SillyTavern / LibreChat / etc) and your backend (llama.cpp + llama-swap, or any OpenAI-compatible endpoint). Local-first (because fuck big AI), but it should talk to anything OpenAI-compatible if you point it there (note: experimental so YMMV).

Not a model, not a UI, not magic voodoo.

A glass-box that makes the stack behave like a deterministic system, instead of a drunk telling a story about the fish that got away.

TL;DR: “In God we trust. All others must bring data.”

Three examples:

1) KB mechanics that don’t suck (1990s engineering: markdown, JSON, checksums)

You keep “knowledge” as dumb folders on disk. Drop docs (.txt, .md, .pdf) in them. Then:

• >>attach <kb> — attaches a KB folder
• >>summ new — generates SUMM_*.md files with SHA-256 provenance baked in
• `>> moves the original to a sub-folder

Now, when you ask something like:

“yo, what did the Commodore C64 retail for in 1982?”

…it answers from the attached KBs only. If the fact isn’t there, it tells you - explicitly - instead of winging it. Eg:

The provided facts state the Commodore 64 launched at $595 and was reduced to $250, but do not specify a 1982 retail price. The Amiga’s pricing and timeline are also not detailed in the given facts.

Missing information includes the exact 1982 retail price for Commodore’s product line and which specific model(s) were sold then. The answer assumes the C64 is the intended product but cannot confirm this from the facts.

Confidence: medium | Source: Mixed

No vibes. No “well probably…”. Just: here’s what’s in your docs, here’s what’s missing, don't GIGO yourself into stupid.

And when you’re happy with your summaries, you can:

• >>move to vault — promote those SUMMs into Qdrant for the heavy mode.

2) Mentats: proof-or-refusal mode (Vault-only)

Mentats is the “deep think” pipeline against your curated sources. It’s enforced isolation:

• no chat history
• no filesystem KBs
• no Vodka
• Vault-only grounding (Qdrant)

It runs triple-pass (thinker → critic → thinker). It’s slow on purpose. You can audit it. And if the Vault has nothing relevant? It refuses and tells you to go pound sand:

FINAL_ANSWER:
The provided facts do not contain information about the Acorn computer or its 1995 sale price.

Sources: Vault
FACTS_USED: NONE
[ZARDOZ HATH SPOKEN]

Also yes, it writes a mentats_debug.log, because of course it does. Go look at it any time you want.

The flow is basically: Attach KBs → SUMM → Move to Vault → Mentats. No mystery meat. No “trust me bro, embeddings.”

3) Vodka: deterministic memory on a potato budget

Local LLMs have two classic problems: goldfish memory + context bloat that murders your VRAM.

Vodka fixes both without extra model compute. (Yes, I used the power of JSON files to hack the planet instead of buying more VRAM from NVIDIA).

• !! stores facts verbatim (JSON on disk)
• ?? recalls them verbatim (TTL + touch limits so memory doesn’t become landfill)
• CTC (Cut The Crap) hard-caps context (last N messages + char cap) so you don’t get VRAM spikes after 400 messages

So instead of:

“Remember my server is 203.0.113.42” → “Got it!” → [100 msgs later] → “127.0.0.1 🥰”

you get:

!! my server is 203.0.113.42 ?? server ip → 203.0.113.42 (with TTL/touch metadata)

And because context stays bounded: stable KV cache, stable speed, your potato PC stops crying.

There’s more (a lot more) in the README, but I’ve already over-autism’ed this post.

TL;DR:

If you want your local LLM to shut up when it doesn’t know and show receipts when it does, come poke it:

• Primary (Codeberg): https://codeberg.org/BobbyLLM/llama-conductor
• Mirror (GitHub): https://github.com/BobbyLLM/llama-conductor

PS: Sorry about the AI slop image. I can't draw for shit.

PPS: A human with ASD wrote this using Notepad++. If it the formatting is weird, now you know why.

cross-posted from: https://lemmy.bestiver.se/post/844165

Comments

I'm looking for the least biased for a large training set on water parks and water slides where there is a lot of content models struggle to deal with.

No, not in some creepy way. I actually want to eventually try training a LoRA on O'Neill cylinders but I don't have the experience, and have a limited dataset. Water slides are a similar set of issues with model internal thinking and comprehension. Presently, all images of water slides start off as stairs, models do not understand gravity as a propulsion method and water as a lubricant on a slide. The water falling is literally a waterfall and falling/slipping/sliding are associated.

I've made several trained attempts at LoRAs already with limited success and around 200 images but there are too many mixed concepts and difficulties.

Now I'm trying to make a much larger training set. I have a well tuned Qwen 2.5 VL 7B doing captions, but want to compare the results with tags training in various models. I need something that is not going to freak out because a normal-clothed not-at-all-sexualized kid is in some images or especially ignore someone in the middle of the air on a launch slide or making a thrill-ride facial expression.

My dataset is a bunch of YT type videos and I am just using FFMPG to do periodic screenshots. The videos are not at all NSFW and most are years old, so they pass moderation. I only focused on getting as many images as possible that include sections of people sliding with the camera on their face. My goal is to capture the true cultural diversity and unique cultural norms of a water park similar to how models know beaches have different cultural norms with internal thinking and alignment.

So, do you all know what is the most recent and reliable tags generating model where alignment does not get in the way?

cross-posted from: https://programming.dev/post/30549287

An amusing chat I had with ChatGPT. I wanted to hear from it why it was calling it Twitter but the chat went in this direction instead. (It did eventually.)

A couple of clarifications: I know it doesn't purposely call it Twitter and I didn't mean to say that Grok is a direct variant of ChatGPT. Sorry! Also, I'm aware it could probably start praising Elon at any moment. lol

“This is a completely different approach to what people have done before. The writing’s on the wall that this is going to transform things, it’s going to be the new way of doing forecasting,” Turner said. He said the model would eventually be able to produce accurate eight-day forecasts, compared with five-day forecast at present, as well as hyper-localised predictions.

Dr Scott Hosking, the director of science and innovation for environment and sustainability at the Alan Turing Institute, said the breakthrough could “democratise forecasting” by making powerful technologies available to developing nations around the world, as well as assisting policymakers, emergency planners and industries that rely on accurate weather forecasts.

cross-posted from: https://lemmygrad.ml/post/7137748

And yet, China is using AI.

...I... don't know what to think about that.

...I really don't.

Because it seems that AI is just a scam.

It may "exist" but what it can do is a scam.

Maybe China thinks we have to use it just to "keep up" with the Western powers, but I dunno.

Anyway, interesting discussion with Adam Conover and Ed Zitron. It's long, but you can listen to it while doing other things. And the comments are interesting too, but then again, there are also trolls in the comments as well (AI supporters here and there).

Frankly, though? I oppose AI. I'm anti-AI. I'm anti-AI in China and anti-AI in America and anti-AI in the whole damn planet.

This is an app for iOS and Android by ElevenLabs, where you can import a web article or ebook, and listen to it with some of its text-to-speech voices.

I've tried it out for a few hours so far and so far it's pretty OK. However, I did notice a couple of problems that could be ironed out:

• When using a PDF, ElevenReader will read out all of the text in the document, including page numbers and repeated headings. I couldn't find a way to disable this, so I just resorted to using an ePub version of my book instead.
• The Lily voice will sometimes drop the first one or two syllables of a sentence, especially after it has paused for a few seconds. I tested a few other voices, and they didn't seem to have this problem.

Anyway it's nice that an app like this is being offered for free and without ads. Apparently a paid tier will be introduced later, but I hope this isn't at the expense of any of the free features. (It would be even better for an open source alternative of comparable quality to be developed in case, but I don't know any.)

(Feel free to crosspost this to other relevant communities! :)

If one wants to try a full size DeepSeek R1 model that is hosted within the EU, what provider do you recommended?

It's a big plus if privacy is good. And I'm not interested in big tech solutions.

An extremely informative article that goes into great lengths to explain why Nvidia of overvalued currently. Long but worth a read.

So hopefully you don't end up needing to fork out cash too in order to test out its capabilities.

I'll be using my own frontend here, so tool support is limited, but if you provide a full json I can send the request via curl instead (see documentation here.

Format:

• Model: (4o unless otherwise specified)
• System prompt: (My default assistant prompt unless otherwise specified)
• User message

Requests will be passed through the moderations api first (I'd rather not get banned), but in this case I'll post back the moderations api result.

I'll keep replying until the time is up or the credits are out.

Go ham!

Ten years ago, Dzmitry Bahdanau from Yoshua Bengio's group recognized a flaw in RNNs and the information bottleneck of a fixed length hidden state. They put out a paper introducing attention to rectify this issue. Not long after that, a group of researchers at Google found that you can just get rid of the RNN altogether and you still get great results with improved training performance, giving us the transformer architecture in their Attention Is All You Need paper. But transformers are expensive at inference time and scale poorly with increasing context length, unlike RNNs. Clearly, the solution is to just use RNNs. Two days ago, we got Were RNNs All We Needed?

Hello there, fellow lemmings!

As the title suggests, I've made an app that changes your wallpaper regularly based on the parameters you set. It uses AI Horde (which uses Stable Diffusion) to generate the images and you can set the interval at which your wallpaper changes (from 1 time per day to 48 times per day).

Other features:

• completely free (in fact, will be open source next week)
• easy to use, but offers also advanced options
• there's a help section with all the parameters explained
• works on both a phone and tablet

There's also an optional Premium subscription which puts you higher in the priority queue, but most of the time it's not needed - unless you're generating during the most busy time, the wait is not really that long even without Premium. Note that if you're a user of the AI Horde, you can put your own api key there and get the same benefits as having a Premium.

The open source version (released next week) will also allow (toggleable on or off) NSFW content, which is not possible in the Google Play version. It also doesn't contain the Premium subscription.

Also please check https://dontkillmyapp.com/ and see steps for your phone vendor, this app needs to work in the background and most vendors kill background apps. It will ask for an exception for battery saver on its own when you set the schedule for the first time, but for some vendors that's not enough.

Do let me know what do you think about the app! And if you want me to tag you when I make a release post for the open source version, let me know. Also huge thanks to @db0@lemmy.dbzer0.com for creating and maintaining the AI Horde, which makes this (and other cool apps) possible!