Meta has released Llama 4, and it's a landmark moment for open-source AI. The 405-billion parameter model uses a mixture-of-experts (MoE) architecture that activates only a fraction of parameters per inference, making it significantly more efficient than its size suggests. On most benchmarks, Llama 4 matches or exceeds GPT-4 Turbo — and it's free to download and run. The implications for the AI industry, for enterprise adoption, and for the competitive dynamics between open and closed AI development are profound.

We've spent three weeks testing Llama 4 across a range of tasks — coding, reasoning, creative writing, instruction following, and multilingual performance — and compared it directly against GPT-4 Turbo, Claude 3.5 Sonnet, and Gemini 1.5 Pro. Here is our comprehensive assessment.

Architecture: Why Mixture of Experts Changes Everything

Llama 4 uses a sparse mixture-of-experts (MoE) architecture with 405 billion total parameters but only approximately 70 billion active parameters per forward pass. This means inference costs are comparable to a 70B dense model while benefiting from the knowledge capacity of a 405B model. The architecture is similar to Mistral's Mixtral and Google's Gemini 1.5, but at a significantly larger scale.

The MoE approach works by routing each input token to a subset of "expert" sub-networks, each specialised for different types of content. A question about mathematics might activate different experts than a question about creative writing. This specialisation allows the model to develop deeper expertise in specific domains without the computational cost of activating all parameters for every input.

The practical result is a model that is both more capable and more efficient than a dense model of equivalent quality. Running Llama 4 at the quality level of GPT-4 Turbo costs approximately 40% less in compute than running a comparable dense model — a significant advantage for organisations deploying AI at scale.

Benchmark Performance: Where It Stands

Llama 4 scores 87.6% on MMLU (Massive Multitask Language Understanding), compared to GPT-4 Turbo's 86.4% and Claude 3.5 Sonnet's 88.7%. On HumanEval coding, it scores 82.3% — below GPT-4 Turbo's 87.1% but significantly ahead of previous open-source models. On GPQA (graduate-level science questions), it scores 71.2%, demonstrating genuine expert-level knowledge in scientific domains.

The coding performance deserves particular attention. Previous open-source models have consistently lagged behind closed models on coding tasks, which are among the most commercially valuable AI applications. Llama 4's 82.3% HumanEval score closes that gap significantly, making it a credible choice for code generation and review applications that previously required GPT-4 or Claude.

Where Llama 4 falls short: on the most demanding reasoning tasks — multi-step mathematical proofs, complex logical deduction, and tasks requiring sustained coherent reasoning over very long contexts — it falls behind GPT-5 and Claude 4 Opus. For the vast majority of real-world applications, this gap is negligible. For the most demanding applications, it matters.

Running Llama 4: Hardware Requirements and Options

The 70B active parameter footprint means Llama 4 can run on a single high-end workstation with 4× NVIDIA RTX 5090 GPUs (96GB VRAM total). This is expensive hardware — approximately $12,000 for the GPUs alone — but it puts frontier-quality AI within reach of well-funded research teams and enterprises that need on-premises deployment for data privacy reasons.

For most users, cloud inference is more practical. Groq's LPU hardware runs Llama 4 at 800+ tokens per second — faster than any other inference provider and dramatically faster than the 50–100 tokens per second typical of GPU-based inference. Together AI, Replicate, and Fireworks AI all offer Llama 4 inference at competitive prices, typically $0.50–$1.00 per million tokens — a fraction of GPT-4 Turbo's $10 per million input tokens.

The quantised versions of Llama 4 (4-bit and 8-bit) can run on more modest hardware. The 4-bit quantised version runs on a single RTX 4090 (24GB VRAM) with modest performance degradation — approximately 3–5% on most benchmarks. This makes Llama 4 accessible to individual developers and small teams who want to experiment with frontier-quality AI without cloud costs.

Commercial Use and Licensing

Llama 4 is released under Meta's custom licence that allows commercial use for companies with fewer than 700 million monthly active users. This covers the vast majority of businesses — only a handful of companies in the world exceed this threshold. The model weights are available on Hugging Face and Meta's website, and fine-tuning is explicitly permitted.

Meta has released multiple variants: the base model (for fine-tuning), an instruction-tuned version (for general use), and a chat-optimised version (for conversational applications). The instruction-tuned version is the most useful for most applications — it follows instructions reliably and produces well-formatted outputs without additional fine-tuning.

The Open vs. Closed AI Debate

Llama 4's release has reignited the debate about the relative merits of open and closed AI development. Proponents of open-source AI argue that Llama 4 demonstrates that open models can match closed models in capability while offering significant advantages in cost, privacy, and customisability. Critics argue that releasing powerful AI models without restrictions creates risks that outweigh the benefits.

Meta's position is that open-source AI is safer than closed AI because it allows the broader research community to identify and fix safety issues, rather than concentrating that responsibility in a single company. OpenAI and Anthropic disagree, arguing that the ability to control deployment is essential for managing the risks of increasingly powerful AI systems.

The practical reality is that Llama 4 is already being used by thousands of organisations worldwide, and the genie is out of the bottle. The question is not whether powerful open-source AI will exist, but how the industry and regulators will manage its deployment.

Fine-Tuning Llama 4: The Enterprise Opportunity

The most significant commercial opportunity with Llama 4 is fine-tuning for specific domains and use cases. A fine-tuned Llama 4 model trained on a company's internal documents, code, and processes can outperform generic closed models on company-specific tasks while keeping sensitive data on-premises.

Several enterprise AI platforms — including Databricks, Scale AI, and Anyscale — have built fine-tuning pipelines specifically for Llama 4. The cost of fine-tuning a Llama 4 model for a specific domain has fallen to approximately $10,000–$50,000 for a typical enterprise dataset, making it accessible to mid-market companies that previously could not afford custom AI development.

Further Reading