Meet the world’s smallest AI supercomputer — it packs ‘doctorate-level intelligence’, its makers say, and can fit into your pocket
An American startup has developed what it claims to be the world’s smallest artificial intelligence (AI) supercomputer. Featuring high-performance hardware and plenty of RAM, company representatives say it can run “Ph.D. Intelligence” AI models, although it’s compact enough to fit in your pocket. This means they are able to solve problems independently, reason abstractly, and plan strategically.
The “AI Pocket Lab,” as its creators at Tiiny AI have christened the device, is capable of locally running a complex language model (LLM) of 120 billion parameters, without any reliance on Internet connectivity. You would normally need data center-style infrastructure to run these systems, which opens up the possibility of local expert-level coding, document evaluation and refinement, or multi-step reasoning capabilities.
It’s built around a 12-core ARM processor, the type commonly found in smartphones, laptops, and tablets. Despite its small frame – the device measures just 5.59 × 3.15 × 1.00 inches (14.2 × 8 × 2.53 cm) – it packs 80GB of LPDDR5X RAM. Most laptops today come with between 8GB and 32GB of RAM, for comparison.
A whopping 48GB of RAM from the Pocket Lab is also reserved exclusively for the neural processing unit (NPU), a chip optimized for AI-related calculations. Intel and AMD have been manufacturing processors for several years that include dedicated NPUs to handle AI workloads and to meet Microsoft’s 40 trillion operations per second (TOPS) threshold for running AI features on Windows 11.
The Pocket Lab is considered a supercomputer (rather than a standard mini-PC or workstation) because of its computing power, capable of running workloads – particularly local inference on more than 100 billion parameter language models – that normally require data center-class multi-GPU systems. Current models that the device can run include the GPT-OSS 120B, large Phi models, and high-parameter Llama family models.
This is part of a recent push toward edge computing for AI, with the goal of reducing some of the energy constraints and environmental impact of distributed AI processing.
Pocket power
Although it is far from rivaling that of the world most powerful supercomputersthe AI Pocket Lab is capable of providing 190 TOPS of computing power between its NPU and its CPU. It represents a new step towards miniaturization in the wake of Nvidia’s recent announcement. Digits mini PC project. While it doesn’t have the same power as the Nvidia project, it’s only a fraction of its size.
To pack so much power into such a modest chassis, the Tiiny AI team relied on a number of technologies and optimizations. Chief among them was something the company calls TurboSparse – an innovation that allows massive LLMs to run faster on more limited hardware by ensuring that a system only calls on the parts of a model that it needs at any given time. While traditional models use each parameter for each processing/output word, a TurboSparse model only uses specific parameters per step.
Another important feature is PowerInfer, which allows heterogeneous scheduling of the device’s CPU, GPU, and NPU. This means that each processor only receives the workload it is most capable of handling, making the entire system more efficient and reducing power consumption. PowerInfer also includes intelligent power management, which decides when full power is needed and when less power can be used, in part by eliminating unnecessary calculations.
The implications of a miniature AI supercomputer go beyond reducing our reliance on environmentally harmful data centers. This is a boon for privacy, with users able to deploy the power of a sophisticated LLM without being connected to the Internet and without their data being processed in the cloud by third parties, while allowing access to AI in field work situations such as remote research stations or on ships or aircraft beyond connectivity range.
