1 Foundation
Why Local AI? The Business Case for Ownership
In the early 2020s, artificial intelligence was a service you rented — by the hour, by the token, by the API call. By 2026, the paradigm has shifted. The hardware required to run GPT-4 class
intelligence now fits on your desk and costs less than a used car.
Continued reliance on cloud-only AI presents a strategic trilemma:
- Escalating costs. Per-token API fees scale linearly with usage. A legal firm processing 1,000 contracts per day can face €30,500+ in annual API costs.
- Data exposure. Every query sent to a cloud API is data that leaves your network and is exposed to data security and privacy risks.
- Zero or costly customization. Cloud models are generic. They cannot easily or cost efficiently be fine-tuned on custom data, internal business processes, or business intelligence.
Local AI hardware resolves all three. It transforms variable API fees into a fixed capital asset, ensures data never leaves the LAN, and enables deep customization through fine-tuning on business data.
2 Reducing Costs
Quantization: Run Bigger AI Models on Cheaper Hardware
Quantization is a concept that fundamentally changes the economics of local AI.
In simple terms, quantization compresses an AI model's memory footprint. A standard model stores every parameter as a 16-bit floating-point number (FP16). Quantization reduces this to 8-bit (Int8), 4-bit (Int4), or even lower — dramatically shrinking the amount of memory required to run the model.
Quantization results in a slight reduction in output quality — often imperceptible for business tasks like summarization, drafting, and analysis — in exchange for a massive reduction in hardware cost.
A 70B model at full precision requires ~140 GB of memory — a €5,100+ server investment. The same model quantized to Int4 requires only ~40 GB, and can run on a €2,600 used workstation with two GPUs.
3 Mini-PCs
AI Mini-PCs €1,300 – €8,500
The most disruptive development of 2026 is high-capacity AI computing in the mini-PC form factor. Devices no larger than a hardcover book now run AI models that required server rooms two years ago.
The NVIDIA GB10 Ecosystem (DGX Spark)
Performance Leader
The NVIDIA DGX Spark has defined this category. In 2026, the GB10 Superchip — combining an ARM Grace CPU with a Blackwell GPU — has spawned an entire ecosystem. ASUS, GIGABYTE, Dell, Lenovo, HP, MSI, and Supermicro all produce GB10-based systems, each with different form factors, cooling solutions, and bundled software.
By connecting two GB10 units via the dedicated high-speed network port, the system pools resources into a 256 GB memory space. This unlocks the ability to run very large models — 400B+ parameters quantized — entirely on your desk for approximately €5,100 – €6,000 total hardware investment.
AMD Ryzen AI Max (Strix Halo) Mini-PCs
Lowest Cost
AMD's Ryzen AI Max+ Strix Halo
architecture has spawned an entirely new category of budget AI mini-PCs. A wave of manufacturers — GMKtec, Beelink, Corsair, NIMO, Bosgame, FAVM — now ship 128 GB unified-memory systems for under €1,700.
Apple Mac Studio (M4 Ultra)
Capacity Leader
The Mac Studio occupies a unique position in the local AI landscape. Apple's Unified Memory Architecture (UMA) provides up to 256 GB of memory accessible to both CPU and GPU in a single, compact desktop unit — no clustering required.
This makes it the only affordable
single device capable of loading the largest open-source models. A 400-billion parameter model quantized to Int4 fits entirely in memory on the 256 GB configuration.
Apple Mac Studio (M5 Ultra)
Upcoming Contender
Apple's next-generation M5 Ultra, expected in late 2026, is rumored to address the M4's primary weakness: AI model training performance. Built on TSMC's 2nm process, it is expected to offer configurations up to 512 GB of unified memory with bandwidth exceeding 1.2 TB/s.
The 512 GB M5 Ultra would be the first consumer device capable of running unquantized (full precision) frontier models. The high memory bandwidth of 1.2+ TB/s supports agentic AI workflows that require sustained high-throughput inference with very long context windows.
Tenstorrent
Open Source Hardware
Led by legendary chip architect Jim Keller, Tenstorrent represents a fundamentally different philosophy: open-source hardware built on RISC-V, open-source software, and modular scaling through daisy-chaining.
The Tensix
AI cores are designed to scale linearly: unlike GPUs, which struggle with communication overhead when you add more cards, Tenstorrent chips are built to be tiled efficiently.
In partnership with Razer, Tenstorrent has released a compact external AI accelerator that connects to any laptop or desktop via Thunderbolt — transforming existing hardware into an AI workstation without replacing anything.
AI NAS — Network Attached Storage
Storage + AI
The definition of NAS has shifted from passive storage to active intelligence. A new generation of network storage devices integrates AI processing directly — from lightweight NPU-based inference to full GPU-accelerated LLM deployment.
An AI-capable NAS eliminates the need for a separate AI device and allows direct processing of larger amounts of data with zero network transfer latency.
Need help choosing the right AI mini-PC for your business?
Our engineers can assess your AI hardware requirements and deploy a fully configured AI system.
Get a Free Hardware Assessment →4 Workstations
AI Workstations & Desktop PCs €2,600 – €13,000
The workstation tier utilizes discrete PCIe graphics cards and standard tower chassis. Unlike the mini-PC tier's fixed unified architectures, this tier offers modularity — you can upgrade individual components, add more GPUs, or swap cards as technology evolves.
Understanding VRAM vs. Speed
Two competing factors define the GPU choice for AI:
Consumer cards (like the RTX 5090) maximize speed but offer limited VRAM — typically 24–32 GB. Professional cards (like the RTX PRO 6000 Blackwell) maximize VRAM — up to 96 GB per card — but cost more per unit of compute.
VRAM is the binding constraint. A fast card with insufficient memory cannot load the AI model at all. A slower card with sufficient memory runs the model — just with longer response times.
Consumer GPUs
| Configuration | Total VRAM | Linking | Est. Cost |
|---|---|---|---|
| 2× RTX 3090 (Used) | 48 GB | NVLink | €2,600 |
| 2× RTX 4090 | 48 GB | PCIe Gen 5 | €3,400 |
| 2× RTX 5090 | 64 GB | PCIe Gen 5 | €6,000 |
Professional GPUs
| Configuration | Total VRAM | Linking | Est. Cost |
|---|---|---|---|
| 2× RTX A6000 Best Value | 96 GB | NVLink | €6,000 |
| 2× RTX 6000 Ada | 96 GB | PCIe Gen 5 | €11,000 |
| 1× RTX PRO 6000 Blackwell | 96 GB | NVLink | €6,800 |
| 4× RTX PRO 6000 Blackwell | 384 GB | PCIe Gen 5 | €27,000 |
Data Center GPUs
| Configuration | Total VRAM | Linking | Est. Cost |
|---|---|---|---|
| 1× L40S | 48 GB | PCIe 4.0 (passive cooling) | €6,000 |
| 1× A100 PCIe | 80 GB | PCIe 4.0 | €8,500 |
| 1× H200 NVL | 141 GB | NVLink | €25,500 |
| 4× H200 NVL | 564 GB | NVLink | €100,000 |
| 1× B200 SXM | 180 GB | NVLink 5 (1.8 TB/s) | €25,500 |
| 8× B200 SXM | 1,440 GB | NVLink 5 (1.8 TB/s) | €200,000 |
Chinese GPUs
China's domestic GPU ecosystem has matured rapidly. Several Chinese manufacturers now offer workstation-class AI GPUs with competitive specifications and significantly lower prices.
| Configuration | Total VRAM | Memory Type | Est. Cost |
|---|---|---|---|
| 1× Moore Threads MTT S4000 | 48 GB | GDDR6 | €680 |
| 4× Moore Threads MTT S4000 | 192 GB | GDDR6 | €3,000 |
| 8× Moore Threads MTT S4000 | 384 GB | GDDR6 | €5,500 |
| 1× Hygon DCU Z100 | 32 GB | HBM2 | €2,100 |
| 1× Biren BR104 | 32 GB | HBM2e | €2,600 |
| 8× Biren BR104 | 256 GB | HBM2e | €20,500 |
| 1× Huawei Ascend Atlas 300I Duo | 96 GB | HBM2e | €1,000 |
| 8× Huawei Ascend Atlas 300I Duo | 768 GB | HBM2e | €8,500 |
Upcoming
| Configuration | Total VRAM | Status | Est. Cost |
|---|---|---|---|
| RTX 5090 128 GB | 128 GB | Chinese mod. — not a standard SKU | €4,300 |
| RTX Titan AI | 64 GB | Expected 2027 | €2,600 |
Pre-Built Workstations
For SMBs that prefer a single vendor, single warranty, and certified configuration, various vendors — like Dell and HP — offer pre-configured systems. These are the safe choice
for non-technical offices — order, plug in, and start working.
NVIDIA DGX Station
Enterprise Apex
The NVIDIA DGX Station is a water-cooled, deskside supercomputer
that brings data-center performance to an office environment. The latest version utilizes the GB300 Grace Blackwell Superchip.
The Blackwell Ultra
version increases memory density and compute power, designed for organizations that need to train custom models from scratch or run massive MoE (Mixture of Experts) architectures locally.
-4438-p.jpg)
The "Value King" for SMBs. While based on the previous-generation Ampere architecture, it remains the industry standard for reliable inference and fine-tuning. Ideally suited for teams entering the AI space without the budget for Blackwell.
While expensive, the DGX Station replaces a €260,000+ server rack and its associated cooling infrastructure. It plugs into a standard wall outlet. This eliminates the server room
overhead entirely.
Need help choosing the right AI workstation for your business?
Our engineers can assess your AI hardware requirements and deploy a fully configured AI system.
Get a Free Hardware Assessment →5 Servers
AI Servers €13,000 – €170,000
When your business needs to serve 50 or more employees simultaneously, run foundation-class models at full precision, or fine-tune custom models on proprietary data — you enter the server tier.
This is the domain of dedicated AI accelerator cards with high-bandwidth memory (HBM), specialized interconnects, and rack-mountable or deskside form factors. The hardware is more expensive, but the per-user cost drops dramatically at scale.
Intel Gaudi 3
Best Value at Scale
Intel's Gaudi 3 accelerator was designed from the ground up as an AI training and inference chip — not a repurposed graphics card. Each card provides 128 GB of HBM2e memory with integrated 400 Gb Ethernet networking, eliminating the need for separate network adapters.
An 8-card Gaudi 3 server delivers 1 TB of total AI memory at much lower cost than a comparable NVIDIA H100 system. For SMBs that need server-class AI but cannot justify NVIDIA pricing, Gaudi 3 is the most compelling alternative available today.
The integrated 400 GbE networking on each Gaudi 3 card enables direct card-to-card communication without external switches — simplifying the server architecture and reducing total system cost. An 8-card server runs the largest open-source models at interactive speeds for dozens of simultaneous users.
AMD Instinct MI325X
Maximum Density
The AMD Instinct MI325X packs 256 GB of HBM3e memory per card — double Intel Gaudi 3, double NVIDIA H100. Only 4 cards are needed to reach 1 TB of total AI memory, compared to 8 cards for Intel or NVIDIA.
The MI325X is more expensive per system than Gaudi 3, but faster and denser. For workloads that demand maximum throughput — real-time inference for hundreds of users, or training custom models on large datasets — the higher investment pays for itself in reduced latency and simpler infrastructure.
Huawei Ascend
Full-Stack Alternative
Huawei has replicated the full AI infrastructure stack: custom silicon (Ascend 910B/C), proprietary interconnects (HCCS), and a complete software framework (CANN). The result is a self-contained ecosystem that operates independently of Western supply chains and at much lower cost than comparable NVIDIA H100 clusters.
Intel Xeon 6 (Granite Rapids)
Budget Server
A quiet revolution in 2026 is the rise of CPU-based AI inference. Intel Xeon 6 processors include AMX (Advanced Matrix Extensions) that enable AI workloads on standard DDR5 RAM — which is dramatically cheaper than GPU memory.
A dual-socket Xeon 6 server can hold 1 TB to 4 TB of DDR5 RAM at a fraction of the cost of GPU memory. Inference speeds are slow, but for batch processing — where speed is irrelevant but intelligence and capacity are paramount — this is transformative.
Example: An SMB uploads 100,000 scanned invoices overnight. The Xeon 6 server runs a +400B AI model to extract data perfectly. The task takes 10 hours, but the hardware cost is much lower than a GPU server.
Need help choosing the right AI server infrastructure?
Our infrastructure team designs and deploys complete AI server solutions — from Intel Gaudi to NVIDIA DGX — combined with tailor made software — to unlock the capabilities of AI for your business.
Request a Server Architecture Proposal →6 Edge AI
Edge AI & Retrofit Upgrading Existing Infrastructure
Not every SMB needs a dedicated AI server or mini-PC. Many can embed intelligence into existing infrastructure — upgrading laptops, desktops, and network devices with AI capabilities at minimal cost.
M.2 AI Accelerators: The Hailo-10
The Hailo-10 is a standard M.2 2280 module — the same slot used for SSDs — that adds dedicated AI processing to any existing PC. At ~€130 per unit and consuming only 5–8W of power, it enables fleet-wide AI upgrades without replacing hardware.
Use cases: Local meeting transcription (Whisper), real-time captioning, voice dictation, small model inference (Phi-3 Mini). These cards cannot run large LLMs, but they excel at specific, persistent AI tasks — ensuring voice data is processed locally and never sent to the cloud.
Copilot+ PCs (NPU Laptops)
Laptops with Qualcomm Snapdragon X Elite, Intel Core Ultra, or AMD Ryzen AI chips contain dedicated NPUs. These cannot run large LLMs, but they handle small, persistent AI tasks: live transcription, background blur, local Recall
features, and running lightweight models like Microsoft Phi-3.
9 AI Models
Open-Source AI Models (2026–2027)
The choice of AI model dictates the hardware requirements — but as the chapter on AI Model Quantization demonstrated, quantization allows frontier-class models to run on hardware costing a fraction of what full-precision deployment demands.
The table below provides an overview of current and upcoming open-source AI models.
| Model | Size | Architecture | Memory (FP16) | Memory (INT4) |
|---|---|---|---|---|
| Llama 4 Behemoth | 288B (active) | MoE (~2T total) | ~4 TB | ~1 TB |
| Llama 4 Maverick | 17B (active) | MoE (400B total) | ~800 GB | ~200 GB |
| Llama 4 Scout | 17B (active) | MoE (109B total) | ~220 GB | ~55 GB |
| DeepSeek V4 | ~70B (active) | MoE (671B total) | ~680 GB | ~170 GB |
| DeepSeek R1 | 37B (active) | MoE (671B total) | ~140 GB | ~35 GB |
| DeepSeek V3.2 | ~37B (active) | MoE (671B total) | ~140 GB | ~35 GB |
| Kimi K2.5 | 32B (active) | MoE (1T total) | ~2 TB | ~500 GB |
| Qwen 3.5 | 397B (active) | MoE (A17B) | ~1.5 TB | ~375 GB |
| Qwen 3-Max-Thinking | Large | Dense | ~2 TB | ~500 GB |
| Qwen 3-Coder-Next | 480B (A35B active) | MoE | ~960 GB | ~240 GB |
| Mistral Large 3 | 123B (41B active) | MoE (675B total) | ~246 GB | ~62 GB |
| Ministral 3 (3B, 8B, 14B) | 3B–14B | Dense | ~6–28 GB | ~2–7 GB |
| GLM-5 | 44B (active) | MoE (744B total) | ~1.5 TB | ~370 GB |
| GLM-4.7 (Thinking) | Large | Dense | ~1.5 TB | ~375 GB |
| MiMo-V2-Flash | 15B (active) | MoE (309B total) | ~30 GB | ~8 GB |
| MiniMax M2.5 | ~10B (active) | MoE (~230B total) | ~460 GB | ~115 GB |
| Phi-5 Reasoning | 14B | Dense | ~28 GB | ~7 GB |
| Phi-4 | 14B | Dense | ~28 GB | ~7 GB |
| Gemma 3 | 27B | Dense | ~54 GB | ~14 GB |
| Pixtral 2 Large | 90B | Dense | ~180 GB | ~45 GB |
| Stable Diffusion 4 | ~12B | DiT | ~24 GB | ~6 GB |
| FLUX.2 Pro | 15B | DiT | ~30 GB | ~8 GB |
| Open-Sora 2.0 | 30B | DiT | ~60 GB | ~15 GB |
| Whisper V4 | 1.5B | Dense | ~3 GB | ~1 GB |
| Med-Llama 4 | 70B | Dense | ~140 GB | ~35 GB |
| Legal-BERT 2026 | 35B | Dense | ~70 GB | ~18 GB |
| Finance-LLM 3 | 15B | Dense | ~30 GB | ~8 GB |
| CodeLlama 4 | 70B | Dense | ~140 GB | ~35 GB |
| Molmo 2 | 80B | Dense | ~160 GB | ~40 GB |
| Granite 4.0 | 32B (9B active) | Hybrid Mamba-Transformer | ~64 GB | ~16 GB |
| Nemotron 3 | 8B, 70B | Dense | ~16–140 GB | ~4–35 GB |
| EXAONE 4.0 | 32B | Dense | ~64 GB | ~16 GB |
| Llama 5 Frontier | ~1.2T (total) | MoE | ~2.4 TB | ~600 GB |
| Llama 5 Base | 70B–150B | Dense | ~140–300 GB | ~35–75 GB |
| DeepSeek V5 | ~600B (total) | MoE | ~1.2 TB | ~300 GB |
| Stable Diffusion 5 | TBD | DiT | — | — |
| Falcon 3 | 200B | Dense | ~400 GB | ~100 GB |
Do not buy hardware first. Identify the model class that fits your business needs, then apply quantization to determine the most affordable hardware tier.
The difference between a €2,600 and a €130,000 investment often comes down to model size requirements and the number of concurrent users.
Trends Shaping the AI Model Landscape
- Native multimodality as standard. New models are trained on text, images, audio, and video simultaneously — not as separate capabilities bolted on after training. This means a single model handles document analysis, image understanding, and voice interaction.
- Small models achieving large-model capabilities. Phi-5 (14B) and MiMo-V2-Flash demonstrate that architectural innovation can compress frontier-level reasoning into models that run on a laptop. The "bigger is better" era is ending.
- Specialization over generalization. Instead of one massive model for everything, the trend is toward ensembles of specialized models — a coding model, a reasoning model, a vision model — orchestrated by an agent framework. This reduces hardware requirements per model while improving overall quality.
- Agentic AI. Models like Kimi K2.5 and Qwen 3 are designed to autonomously decompose complex tasks, call external tools, and coordinate with other models. This
agent swarm
paradigm demands sustained throughput over long sessions — favoring high-bandwidth hardware like the GB10 and M5 Ultra. - Video and 3D generation maturing. Open-Sora 2.0 and FLUX.2 Pro signal that local video generation is becoming practical. By 2027, expect real-time video editing assistants running on workstation-class hardware.
10 Security
Architecture for Maximum Security
Acquiring powerful hardware is only step one. For SMBs handling sensitive data the architecture of the connection between your employees and the AI system is as critical as the hardware itself.
The standard security model for local AI in 2026 is the Air-Gapped API Architecture: a design pattern that physically isolates the AI server from the internet while making it accessible to authorized employees through an API interface.
This architecture creates a Digital Vault
. Even if the Broker Server were compromised, an attacker could only send text queries — they could not access the AI Server's file system, model weights, fine-tuning data, or any stored documents.
Need a secure AI deployment with tailor made AI solutions?
Our engineers design and deploy air-gapped AI architectures ensuring data never leaves the premises while providing your business with state-of-the-art AI capabilities.
Discuss Secure AI Architecture →11 Economics
The Economic Verdict: Local vs. Cloud
The transition to local AI hardware is a shift from OpEx (operational expenditure — monthly cloud API fees) to CapEx (capital expenditure — a one-time hardware investment that becomes an asset on your balance sheet).
Consider a legal firm running a 70B model to analyze contracts:
At 100 queries per day (a typical small team workload), a €3,100 DGX Spark pays for itself in under 2 months compared to cloud API costs. At higher usage levels, the break-even period shortens to weeks.
The economics become even more favorable when you factor in:
- Multiple employees sharing the same hardware (the DGX Spark serves 2–5 simultaneous users)
- No per-token pricing — complex, multi-step reasoning tasks cost nothing extra
- Fine-tuning on proprietary data — impossible with most cloud APIs, free on local hardware
- Hardware resale value — AI hardware retains significant value on the secondary market