Distributed AI Breakthrough: New Software Puts Powerful AI in Reach of Everyone
Table of Contents
A new software platform, Anyway Systems, is challenging the dominance of Big Tech in the artificial intelligence landscape by enabling organizations too deploy and run large AI models locally, without relying on cloud infrastructure. This innovation promises to enhance data privacy, sovereignty, and sustainability while dramatically reducing costs.
the rapid growth of AI over the past three years has led to increased reliance on cloud-based services for processing sensitive data – from patient records to confidential work documents. Each query to an AI model currently travels to remote data centers for processing, a process known as inference, before returning an answer.This reliance concentrates immense computing power, and control over AI development and deployment, in the hands of a few major technology companies.
Though,this paradigm is shifting. Researchers at the Voron, Distributed Computing Laboratory (DCL) within the School of Computer and Communication Sciences have released software that allows users to download open-source AI models and utilize them on-premise.
On-Premise AI: A New Approach
Anyway Systems coordinates and combines existing machines on a local network into a functional cluster. Utilizing robust self-stabilization techniques, the software optimizes hardware usage, defying the conventional wisdom that large data centers are essential for AI deployment. Installation takes approximately 30 minutes,and crucially,no data leaves the network,guaranteeing both privacy and control.
The cost savings are considerable. A leading-edge AI model like GPT-120B, the latest offering from OpenAI, can be deployed using just four machines equipped with standard GPUs (approximately 2,300 CHF each) – a fraction of the 100,000 CHF typically required for specialized rack enclosures.
“For years people have believed that it’s not possible to have large language models and AI tools without huge resources, and that data privacy, sovereignty and sustainability were just victims of this, but this is not the case and smarter, frugal approaches are possible,” stated a leading researcher involved in the project.
Privacy, Sovereignty, and Sustainability Concerns
The current cloud-centric model raises critical questions about data security and privacy.Sending data to external providers introduces risks regarding its potential use in further AI training. Furthermore, dependence on global cloud providers impacts the trajectory of future AI development at EPFL to consume fewer resources with the new deployment of LLM models such as Open.
While currently not designed for single-user desktops or laptops,developers anticipate rapid optimization. “Your phone contains crazy amounts of information that would have been unimaginable a few years ago and now you do everything on it. History tells us this is the way things go. What we’re saying is that we will be able to do everything locally in terms of AI.We could download our open-source AI of choice, contextualize it to our needs, and we, not big-tech, could be the master of all the pieces,” concluded a senior researcher.
Q&A
What is the difference between Anyway Systems and Google AI edge?
Google AI Edge is designed for mobile phones and limited to smaller, Google-developed models constrained by device capacity. It lacks the distributed computing capabilities of Anyway Systems, which can handle large, powerful AI models shared by multiple users in a scalable and fault-tolerant manner.
What is the difference between Anyway Systems and other solutions that allow people to run local LLMs such as Llama or msty.ai?
Most existing solutions focus on deploying models on a single machine, creating a single point of failure. Deploying powerful models with these solutions often requires expensive, data center-grade hardware. Anyway Systems transparently, robustly, and automatically combines multiple commodity machines to deploy large models, even accommodating machine failures with minimal impact on response time.
AI models are constantly being improved and fed, how are these improvements reflected locally?
Because the Anyway System supports any open-source AI model, local deployment allows for safe and secure integration of sensitive data, returning control to the user.
