lexiw

The harness is as important as the model

It really depends on the camera you have and how you use it. If it’s high enough megapixels (20+) and you bought it for high fps burst shot (needed to shoot moving subjects or kids), you might need a v90 cards which will be 4 times that price. Otherwise the Canvas is ok, but these days you can find them in the same price range of Prograde which would be my suggestion. I have and use both.

Exactly where people who need to read this message are. What’s your point?

None of what this does is new, the issue is that’s being commodified: in a short amount of time these techniques will be available to non experts in the field.

Those values are most of what I was looking for. An LLM is just predicting the next token (for simplicity, a word). It does so by generating every possible word with a probability associated with it, and then picking a random word from this list, influenced by its probability. So for the sentence “a cat sat” it might generate “on: 0.6”, “down: 0.2”, and so on. 0.6 just means 60%, and all the values add up to 1 (100%). Now, the number of tokens generated can be as big as the context, so you might want to pick randomly from the top 10, you control this with the parameter top_k, or you might want to discard all the words below 20%, you control this with min_p. And finally, in cases where you have a token with a big probability followed by tokens with very low probability, you might want to squash these probabilities to be closer together, by decreasing the higher tokens and increasing the lower tokens. You control this with the temperature parameter where 0.1 is very little squashing, and 1 a lot of it. In layman terms this is the amount of creativity of your model. 0 is none, 1 is a lot, 2 is mentally insane.

Now, without knowing your hardware or why you need docker, it is hard to suggest a tool to run LLMs. I am not familiar with what you are using, but it seems to not be maintained, and likely lacks the features needed for a modern LLM to work properly. For consumer grade hardware and personal use, the best tool these days is llamacpp, usually through a newbie friendly wrapper like LMStudio which support other backends as well and provide so much more than just a UI to download and run models. My advice is to download it and start there (it will download the right backend for you, so no need to install anything else manually).

I agree, it is a very expensive hobby, and it gets decent in the range 30-80b. However, the model you are using should not perform that bad, it seems that you might be hitting a config issue. Would you mind sharing the cli command you use to run it?

You are playing with ancient stuff that wasn’t even good at release. Try these:

A 4b model performing like a 30b model: https://huggingface.co/Nanbeige/Nanbeige4.1-3B

Google open source version of Gemini: https://huggingface.co/google/gemma-3-4b-it

This is with the extra brightness and contrast added by the iOS screenshot functionality during screenshot. I would need a second phone to truly show you the horror of liquid

I don’t dislike aero or glass, I just dislike iOS implementation of it. With the wrong background sometimes you can’t read buttons for example, and a myriad of little things that are insignificant on their own but a pain in numbers.

Yeah, that’s the same feeling I have. In the past it was “I don’t like it but can get used to it”, now it’s so bad it’s borderline unusable.

Everything is better than Liquid Glass