Re: Re: Using Ai For Coding
By: phigan to Lonewolf on Fri Feb 27 2026 08:02 am
I coded my Quantasia AI door to use
tokenized cache so it always remembe
the conversation. But that doesn't m
Have you looked at all into connecting
to or interfacing with a self-hosted
LLM? And, have you read whether any of
them are better than others?
I kinda have an itch to install a self
hosted model for coding.
Yes, on my Quantasia door besides connection to cloud hosted LLMs, it connects to locally hosted ones too. I currently have both Ollama and LM Studio setup for local hosting on a Dell Precision 5280 Intel i9 workstation that has two Nvidia RTX 3060 GPUs for a total of 24 GB vram. I'm liking LM Studio the most so far, as it seems faster than Ollama. But I haven't had a chance to really tweak Ollama to get the most speed out of it either. Plus, I need to really dig into some of the quantized models. I'm currently running a Cogito 30b LLM that is pretty quick and gives a good tokens per second performance and comes back with pretty good results most of the time. I think AI and LLMs are only going to improve from here on out, so its really exciting to play with this stuff. I recommend diving in man, you'll enjoy it.
LW
---
þ Synchronet þ Fireside BBS - AI-WX - firesidebbs.com:23231