On July 23, 2024, Meta released Llama 3.1 405B—the largest and most capable open-source AI model ever.

405 billion parameters. Performance matching GPT-4. Completely open weights, free for commercial use (with conditions).

Open source had finally caught up to closed models on every dimension.

The Scale

405B parameters: Massive scale 128K context: Long-form processing Multilingual: Strong non-English performance Truly open: Full weights, not just API

Plus smaller 70B and 8B versions for different needs.

The Performance

Llama 3.1 405B matched:

• GPT-4 on MMLU: General knowledge
• Claude 3 Opus on coding: Programming tasks
• Gemini 1.5 Pro on math: Quantitative reasoning

For the first time, an open model genuinely competed with the best closed models.

The Training

Meta trained on 15.6 trillion tokens using 16,000 H100 GPUs over months. The compute cost was staggering, but Meta made it free for everyone to use.

Why Meta Did This

Cloud neutrality: Meta doesn't sell cloud services, so open-sourcing doesn't hurt them Ecosystem development: A thriving Llama ecosystem benefits Meta Regulatory positioning: "Look how open we are" Recruiting: Top AI talent wants to work on open models

Strategic open source.

Where Are They Now?

Llama 3.1 405B powers countless applications and became the foundation of the open-source AI stack. Derivatives and fine-tunes number in the thousands.

July 23, 2024 was when open source definitively proved it could match closed models—ending the debate about whether openness meant compromising on capability.