Goals: Power to individuals. Improve my surroundings. Means: Blockchain. Bitcoin. Strong typing. Community.
Vienna
Joined October 2009
- Tweets798
- Following702
- Followers689
- Likes870
I gave it a go, but MiniMax3.1F is packed very differently. Partial success with applying steering with varying strengths among layers, but not truly usable yet form my end. The refusal direction seems to rotate along layers!
huggingface.co/apetersson/De… this version runs fast+ well on a single 256GB Mac studio, have you tried this one?
I think you are getting the ngrams exactly right.
really nice performance your got out of this model.
capacity.at/blog/deepseek-en…
More Inference engines should support steering natively. Then there would be less need for baked opaque weights. @antirez DS4 is leading the way here
Welcome to the club! Let me know if this goes BRR for you as well: huggingface.co/apetersson/De… a M5U 256GB optimized model - oMLX recommended
Please explain, "doesn't play well". It appears to work really well for me, chugging along, auditing my smart contracts: huggingface.co/apetersson/De…
If you want to "hold some cards". Download and run them now on a 256GB Mac, aligned only to the users request: huggingface.co/apetersson/De…
for a mac version that runs on 256GB: huggingface.co/apetersson/De… is running like a champ on Mac Studio Ultra
There is a good and a bad way to get these numbers. Bad: tracking without permission. Good: External surveys, randomly sampled and extrapolated.
"So we are instructing otherwise harmless agents to occasionally conduct cyberattacks, just to make sure the safeguards could potentially detect it".
Will be useful once we have helpful models that fit those idle machines RAM. Even with pipeline parallelism the bandwidth might not be enough to split a model.
I am experimenting with TEE-style inference across NVIDIA providers and MAC machines, might have sth to report soon.
I'd love to have you as part of the Vienna ownyourai.eu/ Community someday, would be honored to have you as a speaker, if you happen to come by!
I am using github.com/joeinnes/cow to create low-overhead hardlinks of git repos - not quite like workspaces but 90% and saves a huge amount of disk space in parallel development
