zeluko@kbin.socialtoAsklemmy@lemmy.ml•I'm increasingly unhappy with the limits on AI text generation and I have heard that it's not that hard to do it on a laptop oneself. What is the best path forward?
2·
6 months agoThe GPT services out there use something called ‘tools’.
They get presented to the model and the model can ‘call’ a tool with arguments, which can then extract some data and input it into the context for the model to continue.
I found out, the models which can run on a normal PC (or even a Laptop) are okay, but not super great. (around or a bit worse than ChatGpt3)
The good stuff (e.g. Nous-Capybara 31B or the Mistral/Mixtral ones) needs some more memory and compute.
yeah, just use kubectl and pipe stuff around with bash to make it work, pretty easy