Work in progress. The Nexus Version is a challenge and there is no financing in sight for any hardware which means the Cortex version remains a vision.
At this point HAProxy load balancing to use three nodes for RAGing this time the whole wiki dump works fine while still being able to use Firefox or play music through Navidrome on the headnode.
For everyone normal and leaving me out, but being into that Cyberpunk estatic of terminal computing... this is not the Beowulf Cluster part of the LAN I work on. You work on a Computer and use a LAN to connect to the internet and by now hopefully created a shared folder getting rid of the USB sticks in your company. This is, horribly simplified, an AI Beowulf Cluster.
So, the Ollama LLM models the open source guys use being tiny versions of Gemini and DeepSeek, need one computer with a lot of RAM. If it does not fit into the RAM, it won't work. Ollama does not and can not connect to other Ollamas on other computers connecting several LLM models.
I did that.
What you see there is creating a database for an AI it uses as its datapoints from a large file. The wiki dump being a copy of all English wiki pages with history has about 100GB. This time I did not RAG only the main pages, but decided to go through all of its content. That will take about 60 days, even so I use three computers.
The problem is that it only works using the same AI model (aka LLM model) which is fortunately a very small and dedicated AI made for RAGing. This time I use a proper database called chromadb, that comes with a database server and have HAProxy installed which acts as a load balancing software managing which computer gets which pages to chunk into the wiki database. So, my script pulls wiki dump data off the .xml file, gives it to HAProxy and HAProxy gives it to three LLMs that happen to be on different computers each. They give it back to HAProxy and HAProxy gives it to the chromadb server that writes it into one db.
It is not flexible yet... I cannot use a larger AI to use the database or several dbs. Having still some days I also have still some time to get that happen by using Docker for all LLMs and working on a load balancing rule set, I guess.
Others would by another computer to add it to the nodes, but ... ... #gfyALL #neversurrender #deathbeforedishonour #provos
#cyberpunkcoltoure #deggers
