This is how its done. The Intelligence Factory will have plenty of machines using GPUs.
What we have to understand here is, that this will practically be a really large LAN set up and each GPU has to load one Ollama model. It won't be One SuperLarge Ollama model.
Then another software, called load balancing, sends each new session to a free GPU.
That means, even so each of their GPUs most likely will have rather ridiculous amounts of GB RAM, it is still plenty of small ones. 32GB ones cost 300 Euros easily used on eBay and I guess they use 512GB models I did not even check if for sale or in need of being ... acquired differently.
Beowulf Clusters are solving a CPU problem, but the GPUs are way more depending on hardware pipelines limitations. So, in a Beowulf Cluster software that is really deep in the Operating System uses the Ethernet connection to use other CPUs around for number crunching under certain conditions. Not all number crushing tasks can be used. (1+1)x(1+1)x(1+1)=x is a simple example. Based on the rules the (1+1) need to be first solved. Here that means 3 computers can do that. (1+1)=a; (a+1)=b; (b+1)=c can't use that design.
GPUs sit in PCI slots on IBM compatible computers. The AS400 was scaled down to Desktop size by someone in IBM in the 1980s and this ended Commodore and Atari which had significantly different designs.
The only thing this Guy needs beside unused Gas from Oil Riggs, is someone that builds him basically a motherboard with thousands of GPU slots to enable GPU RAM and processing power sharing.
This means you get rid of the motherboard and wire them all to a CPU which is no Central Processing Unit, but a Controlling Processes Unit.
Anyway.
#cyberpunkcoltoure
