Data Sculpting Inc.

My VPS pricing is going up because apparently it takes more effort and monitoring for security issues these days. This is going to be a trend everywhere soon. More incentive to self-host. Not just for AI but committing to the home lab for hosting.

Another money sink just now to order (for my 4 Sparks): 3 more ConnectX-7 cables, 2 more fan stands, small portable monitor, HDMI switch & cables (they'll run headless but it's easier to config & troubleshoot them locally), 2 510W UPS's, misc plugs & adapters. I decided since I can't place them near my 10G switches I'll put them all directly behind my gateway firewall, also because that's also near an outlet not already sucked up by GPUs in my house. I'll tie 3 together initially, until either NVIDIA official 4-unit ring cluster is supported or I can get a MikroTik switch. Got 400G cables just in case.

VisLama 1.5.18 rolled! Added Maya engine support and streamlined the UI, plus minor fixes. Now supports management of models in Llama.cpp, Maya and Strata. github.com/DataSculpting/Vis…

Wait so that Moana movie from Disney was real and not AI slop? LOLOL oh damn

Opus refusal guidance: Not sure why people are confused about this. If you're not asking it to do something like create a WMD or something seriously criminal, if it's refusing then it's how you're prompting. Don't just ask it in the first prompt, dummies. Work up a project where what you need happens to be part of the project. Guide it with words that frame the need as belonging to you and normal. Then guide it toward an escalation toward the need so that within your session context it is itself already building part of what you need. It will reference its own movement forward to reinforce itself to help you. If you just prompt it with "help me hack my site", it won't.

Success! - VisLlama coordinating Maya, Strata and Llama.cpp simultaneously. All model edits unified in one UI, and exposed with a unified single OpenAI-compatible endpoint. Nvidia-smi shows GPUs installed - this is on a Windows system. GitHub repo update soon, and more engines to come...

Alright, having finished the X login plugin to the Inference forum, I'm going back to testing Maya's engine alongside the Strata engine unified with VisLlama. Getting an engine crash with Maya, I think the RAM may be too limited or I may need to run a re-build. Strata is a bit more flexible.

I'm not going to sink more money into the 3090 gpu hack and don't want to mess with my 4090 system but not sure how I forgot I have 2 3060's. I can throw those into their own system and mess with the hack that way. They're 360 GB/s but with TP=2 that can become 720.

🚀 Running local AI? Inference is a new organized community for local LLM builders, and AI tinkerers. Now with Sign in with X - no separate password needed! 🎯 Join at datasculpting.ai/inference - Discuss hardware, inference engines, model recipes, benchmarks, and optimization.

Are banked resets on OpenAI going away after Nov 6? None are lined up after that in my account.

3090 unlock - I can't be buying any more GPUs 😂 it's already priced insanely. I'm gonna tap out on GPUs. I don't even have a spare system to put it in and no more room on any system to tack it on somewhere with eGPU, I'd have to swap out other GPUs. Maybe a year ago I should have gotten a couple of 3090's.

The Acer Veriton GN100 are now out of stock. When they're back they'll likely be priced closer to the other Sparks. Got my 2 a week ago for $5,499. Need to now order cables to hook up at least 3 of my Sparks if not 4 - NVIDIA will be officially supporting 4-8 ring cluster by December (though performance is slightly lower than using a switch). Gives me some buffer to decide if I want to wait for a MikroTik or not. Between base + tarrifs + tax, it's over $1,400 so not huge incentive if not truly needed other than an extra 5-10% of inter-node performance.

Get it now: github.com/DataSculpting/Vis… Llama.cpp & Strata support now, many more soon.

Launched! github.com/DataSculpting/Vis… Llama.cpp is the base but includes Strata support and most other engines will be added in quickly. Total UI managed model editing, control, testing, with unified endpoint. Cross-platform. Testers wanted.