NVIDIANemotronAI2 / OLMoEleutherAIBLOOMHugging FaceDBRXGranitePhiFalcon
On the wall: NVIDIA, Nemotron, AI2 / OLMo, EleutherAI, BLOOM, Hugging Face, DBRX, Granite, Phi, Falcon.
LLM History Series — Presentation 11

NVIDIA, AI2 & The Open-Model Builders

The producers outside the consumer-chat race. The chip vendor that builds models to sell hardware, the non-profits that publish everything down to the training logs, the platform that hosts it all, and the enterprise labs that ship Apache-2.0 weights. Nemotron, OLMo, the Pile, BLOOM, DBRX, Granite, Phi and Falcon — and why none of them are trying to be ChatGPT.

NVIDIANemotronAI2 / OLMo EleutherAIBLOOMHugging Face DBRXGranitePhiFalcon
Megatron-LM (2019) → The Pile (2020) → BLOOM (2022) → OLMo (Feb 2024) → Nemotron-4 340B (2024) → open + hardware-vendor models (2026)
00

What This Deck Covers

Decks 05–09 profiled the labs racing to ship the best chat assistant — OpenAI, Anthropic, Google DeepMind, Meta/Mistral/xAI, and the Chinese labs. This deck profiles a different population: producers whose reason for shipping a model is not to win the consumer-chat market. A chip vendor seeding demand for its hardware (NVIDIA). Non-profits doing reproducible open science (AI2, EleutherAI, BigScience). The platform that distributes everyone's weights (Hugging Face). Data and cloud companies shipping permissively-licensed enterprise models (Databricks, IBM, Snowflake, AI21). And the small-model and sovereign builders (Microsoft Phi, TII Falcon).

01

A Different Axis — Not Everyone Wants to Be ChatGPT

The frontier labs are easy to rank because they are all playing the same game: the best general-purpose assistant, sold by subscription and API. The producers in this deck are harder to compare because they are optimising for different things. Ranking them on chatbot-arena Elo misses the point of why they exist.

The four motivations

  • Sell the shovels. A model that makes your hardware or cloud more valuable (NVIDIA, Amazon, Snowflake).
  • Do the science. Reproducible, fully-documented models so researchers can study how LLMs actually work (AI2, EleutherAI).
  • Own the distribution. Be the place everyone downloads models from (Hugging Face).
  • Serve the enterprise. Permissively-licensed, indemnified, on-prem-deployable models for regulated buyers (IBM, Databricks).

Why they matter to the field

  • They supply the open weights, datasets and training code that the rest of the ecosystem is built on.
  • They produce most of the publicly studyable science — the frontier labs publish results, these labs publish artifacts.
  • They set the floor on price and the ceiling on lock-in: every open model is a competitive constraint on the closed ones.
  • They are where synthetic data, reward models and small-model recipes are pioneered.
A note on the word "open"

"Open" is used loosely in this field and the differences matter. Open weights (Llama, Mistral, Nemotron, DBRX, Falcon) means you can download and run the model; the data and recipe stay private. Open source / open science (OLMo, Pythia, BLOOM) means the weights plus the training data, the code, and often the intermediate checkpoints and logs — enough to reproduce the model from scratch. Slide 05 lays out the full spectrum.

02

NVIDIA — The Compute Vendor That Also Builds Models

NVIDIA sells the GPUs that almost every model in this entire series was trained on. It does not need a chatbot business. But it builds models anyway — and has done since before the LLM boom — for a precise strategic reason: every reference model, training recipe and software library it ships makes its hardware easier to buy and harder to leave.

NVIDIA's model lineage long predates ChatGPT:

The strategy in one sentence

NVIDIA's models are razors sold to move blades the other way round: the models, datasets, reward models and frameworks are (mostly) free and openly licensed, because their entire purpose is to increase the number of GPU-hours the world wants to buy. NVIDIA wins whether you train your model or theirs — as long as you train it on their silicon.

That posture is why NVIDIA has become one of the most prolific publishers of openly-licensed models, datasets and tooling — it has no incentive to keep them closed, and a strong incentive to make the open ecosystem thrive on CUDA.

03

The Nemotron Line & the People Behind It

Nemotron is NVIDIA's family of openly-licensed language models. The defining release was Nemotron-4 340B (June 2024) — shipped as a trio of base, instruct and reward models under the permissive NVIDIA Open Model License. Its headline idea was not raw capability but synthetic data generation: the alignment data was overwhelmingly model-generated, and NVIDIA released the pipeline so others could generate their own training data with it.

WhenReleaseWhat it added
2019Megatron-LMTensor/pipeline parallelism — the training method, not a product.
2021Megatron-Turing NLG 530BJoint with Microsoft; a stack demonstration at extreme scale.
2024Nemotron-4 15B / 340BOpen base + instruct + reward models; synthetic-data pipeline as the headline.
2025Llama-Nemotron (Nano / Super / Ultra)Reasoning-tuned models built on Llama bases; tiered for edge-to-datacentre.
2025Nemotron-HHybrid Mamba–Transformer line, trading attention for throughput on long context.
JH

Jensen Huang — co-founder & CEO, NVIDIA

Co-founded NVIDIA in 1993; CEO ever since

Taiwanese-American. The strategist behind treating AI software — CUDA, libraries, frameworks, and now models — as the moat around the hardware. NVIDIA's model releases are an expression of his long-running thesis that the company sells a platform, not a chip.

BC

Bryan Catanzaro — VP, Applied Deep Learning Research

Leads much of NVIDIA's model and applied-research work

A Megatron-LM co-author and long-time leader of NVIDIA's applied deep-learning research. His org produces the Nemotron models, the NeMo framework, and a steady stream of training and inference techniques that ship back into the wider community.

Why a reward model was the interesting part

Most labs guard their reward models — they encode the lab's taste and are central to RLHF. NVIDIA released Nemotron-4 340B's reward model and topped reward-model leaderboards with it, precisely because giving everyone a good reward model and a synthetic-data generator lowers the barrier to training models on NVIDIA hardware. It is the shovel strategy applied to the post-training stack.

04

AI2 / Allen Institute — Truly-Open Science

The Allen Institute for AI (AI2) was founded in Seattle in 2014 by Microsoft co-founder Paul Allen, with computer scientist Oren Etzioni as founding CEO. It is a non-profit research institute, not a company — which frees it to do the one thing the commercial labs structurally cannot: release everything.

PA

Paul Allen — founder (1953–2018)

Microsoft co-founder; funded AI2 as a non-profit

Endowed AI2 (alongside the Allen Institute for Brain Science and others) as part of a long-running philanthropic bet on open, foundational research. He died in 2018; the institute continues under his foundation.

AF

Ali Farhadi — CEO, AI2

CEO since 2023; computer-vision researcher (University of Washington)

Returned to lead AI2 in 2023 and made fully-open foundation models the institute's flagship mission — the OLMo programme is the direct result. Known earlier for YOLO-era object detection and for XNOR-Net (binary neural networks).

HH

Hanna Hajishirzi — research lead

Senior director of NLP research, AI2; professor, University of Washington

A driving force behind OLMo and the Tülu post-training work. With colleagues such as Noah Smith and Luca Soldaini, she anchors AI2's "release the data and the recipe, not just the weights" philosophy.

What "non-profit" buys you

AI2 has no API revenue to protect and no investors demanding a closed moat. That is exactly why it can publish the training data, the data-processing code, the training code, and the checkpoints — the things a commercial lab treats as trade secrets. The output is less a product than a public good for the research community: a model you can actually study, not just call.

05

The OLMo Family & the "How Open?" Spectrum

AI2's OLMo (Open Language Model), first released in February 2024, was the first genuinely competitive LLM shipped with the entire production stack open. The point was reproducibility: a model that researchers could take apart, retrain, and audit end to end.

The AI2 open stack

  • OLMo — the base and instruct models (and OLMo 2; OLMoE, a mixture-of-experts variant).
  • Dolma — the open multi-trillion-token pretraining dataset, released with its curation tooling.
  • Tülu — open post-training recipes (SFT + preference tuning + RL), through Tülu 3.
  • Molmo — open multimodal (vision-language) models.

What ships with OLMo

  • Final weights and hundreds of intermediate checkpoints.
  • The full training dataset (Dolma).
  • Training, data-processing and evaluation code.
  • Training logs, metrics and ablations — the "lab notebook".

The openness spectrum

LevelWhat you getExamples
ClosedAPI access only; no weights.GPT-5, Claude, Gemini
Open weightsDownloadable weights; data & recipe private; licence may restrict use.Llama, Mistral, Nemotron, DBRX, Falcon, Gemma
Open weights + permissive licenceAs above, under Apache-2.0/MIT — no usage strings.Granite, Arctic, OLMo, many Falcon/Qwen releases
Fully open / reproducibleWeights + data + code + checkpoints + logs.OLMo, Pythia, BLOOM
Why the distinction is load-bearing

You cannot do real science on a model whose training data you cannot see. Questions like "did it memorise this benchmark?", "where did this bias come from?", and "how do capabilities emerge during training?" are only answerable with the data and the checkpoints. The fully-open labs — AI2, EleutherAI, BigScience — exist largely to keep that kind of question answerable.

06

EleutherAI — The Grassroots Open-Source Era

When GPT-3 appeared in 2020 it was closed, and there was no open equivalent. EleutherAI started in July 2020 as a Discord collective — Connor Leahy, Sid Black, Leo Gao and a rotating cast of volunteers — with the explicit goal of building an open GPT-3. For two years it was the centre of gravity of open LLM research, before the corporate open-weight labs existed.

WhenReleaseSignificance
2020The PileAn 825 GiB curated, diverse text dataset — the open pretraining corpus an entire generation of models was trained on.
2021GPT-Neo / GPT-J-6BThe first widely-used open GPT-3-style models; GPT-J-6B became a community default.
2022GPT-NeoX-20BThe largest open dense model at release; its codebase trained many later models.
2023PythiaA suite of models across scales with full checkpoints — built specifically for interpretability and training-dynamics research.
SB

Stella Biderman — Executive Director, EleutherAI

Led the transition to a non-profit research institute

A central figure in EleutherAI's research output (the Pile, Pythia) and in turning a volunteer Discord into a sustained non-profit lab. Pythia in particular has become a standard substrate for studying how models learn.

Legacy

EleutherAI's lasting contribution is less any single model than a proof of concept and a toolchain: it showed a distributed group of volunteers could train serious models in the open, and it produced the datasets (the Pile) and code (GPT-NeoX) that later open efforts — including some commercial ones — were built on. Several alumni went on to found or staff labs covered elsewhere in this series.

07

BigScience / BLOOM & Hugging Face

BigScience & BLOOM — the public-supercomputer model

BigScience was a year-long open research workshop (2021–2022) coordinated through Hugging Face, bringing together more than a thousand researchers from 60+ countries. Its flagship output, BLOOM (July 2022), was a 176-billion-parameter multilingual model covering 46 natural and 13 programming languages — trained on the Jean Zay public supercomputer in France under a national compute grant, and released openly under a Responsible AI License.

Why BLOOM mattered

BLOOM proved a frontier-scale model could be built by an international academic consortium on public compute, with full transparency about data and governance, rather than by a single well-funded company. It was a deliberate counter-model to the closed-lab playbook — and an early demonstration of multilingual coverage as a first-class goal rather than an afterthought.

Hugging Face — the distribution layer

Founded in 2016 by Clément Delangue, Julien Chaumond and Thomas Wolf (originally as a chatbot app), Hugging Face pivoted into infrastructure and became indispensable through two assets: the Transformers library and the Hub. Nearly every open model in this deck is downloaded from, and benchmarked on, Hugging Face.

The quiet kingmaker

Hugging Face ships relatively few frontier models of its own, yet it may be the most structurally important company in this deck: it owns the distribution and tooling layer the entire open ecosystem runs on. When a new open model lands, "it's on Hugging Face within the hour" is the measure of whether it exists.

08

Enterprise Open-Weight Builders

A cluster of data and cloud companies ship open-weight models aimed not at consumers but at enterprises that need permissive licences, on-prem deployment, indemnification and governance. The model is a feature of the platform, not the product.

Databricks / MosaicML — DBRX

MosaicML (founded 2021 by Naveen Rao & Hanlin Tang) shipped the open MPT models in 2023, then was acquired by Databricks for around $1.3 B. The result was DBRX (March 2024), a 132B mixture-of-experts model — among the strongest open models at release — sold as a way to build custom models on your own data inside Databricks.

IBM — Granite

IBM's Granite family (language, code, time-series, guardrails) ships under Apache 2.0 with IP indemnification, as part of the watsonx platform. The pitch is squarely enterprise: transparent training-data provenance, legal cover, and governance for regulated industries.

Snowflake — Arctic

Arctic (April 2024) is an Apache-2.0 mixture-of-experts model from Snowflake's AI research group, tuned for enterprise tasks (SQL, coding, instruction-following) and designed to be cheap to train — a model that makes Snowflake's data cloud more useful.

AI21 Labs — Jamba

Israeli lab (founded 2017 by Amnon Shashua, Yoav Shoham & Ori Goshen). After the Jurassic series, Jamba (2024) was the first production-grade Mamba–Transformer hybrid with open weights and very long context — an architectural bet, not a chatbot.

Adjacent to these sit Salesforce (CodeGen, xGen, the SFR embedding/agent models) and ServiceNow (co-founder of BigCode/StarCoder) — software vendors building open models to strengthen their own platforms rather than to compete in chat.

09

Microsoft Phi & the Small-Model / Data-Quality School

Separate from Microsoft's frontier ambitions (its OpenAI stake and the Inflection-derived Microsoft AI group, both covered in deck 08), Microsoft Research runs the Phi programme — a deliberate counter-bet to scale. The thesis, captured in the 2023 paper title "Textbooks Are All You Need", is that data quality beats data quantity: carefully curated and synthetically-generated "textbook-quality" data can make a small model punch far above its parameter count.

WhenModelIdea
2023Phi-1 / Phi-1.5~1.3B; "textbook-quality" code and reasoning data — small but surprisingly capable.
2023Phi-22.7B; competitive with much larger models on reasoning benchmarks.
2024Phi-3 (mini/small/medium)Small models good enough to run on a phone; MIT-licensed open weights.
2024–25Phi-4 (& reasoning variants)~14B; pushed the data-quality recipe toward strong reasoning at small scale.
Why small models are a category, not a consolation prize

Small, openly-licensed models matter for reasons the leaderboards understate: they run on-device (privacy, latency, no per-token cost), they are cheap to fine-tune, and they are the natural fit for edge and embedded deployment. The data-quality school — Phi, AI2's smaller OLMo variants, Hugging Face's SmolLM, and the small ends of Qwen, Gemma and Llama — is where a lot of the field's practical deployment actually happens.

Sébastien Bubeck and colleagues drove much of the early Phi work at MSR (Bubeck later moved to OpenAI), establishing curated/synthetic data as a first-class lever rather than a shortcut.

10

Sovereign & Hyperscaler Builders

Sovereign — TII Falcon

The Technology Innovation Institute (TII) in Abu Dhabi is a UAE government research body, and its Falcon models are a national-capability play. Falcon 40B (2023) briefly topped the open leaderboards; Falcon 180B (Sept 2023) was the largest openly-available model at release. TII also released the RefinedWeb dataset and has continued with Falcon 2, Falcon 3 and hybrid (Falcon-H1) lines. The motivation is sovereignty: a region building frontier-class AI in the open rather than renting it from US labs.

Hyperscalers — building in-house

The big clouds increasingly build their own foundation models — usually closed, offered only through their platforms, to avoid depending on a rival:

Two different bets on "open"

Sovereign builders go open on purpose — the whole point is broad, unrestricted access to a strategic capability. Hyperscalers mostly go closed for their own flagship models — the model is a way to keep customers on the cloud. Same act (build your own model), opposite licensing logic, driven by opposite incentives.

11

The Strategic Map — Why These Builders Exist

None of the producers in this deck are trying to win the consumer-assistant market. Sorting them by motivation rather than capability makes the landscape legible:

Sell the shovels

  • NVIDIA — Nemotron, Megatron, NeMo; models to sell GPUs.
  • Amazon / Snowflake / Databricks — models to sell cloud & data platforms.

Do the science

  • AI2 — OLMo, Dolma, Tülu, Molmo; fully reproducible.
  • EleutherAI — the Pile, Pythia; open research toolchain.
  • BigScience — BLOOM; public-compute consortium.

Own / serve the platform

  • Hugging Face — the distribution & tooling layer.
  • IBM — Granite; enterprise governance & indemnity.
  • Salesforce / ServiceNow — models inside their software.

Specialise

  • Microsoft Phi — small models, data quality.
  • AI21 — SSM/Transformer hybrids (Jamba).
  • TII Falcon — sovereign, open, regional.

The geographic spread

The US dominates (NVIDIA in Santa Clara; AI2 and Amazon in Seattle; Databricks in San Francisco; IBM and Microsoft Research; ServiceNow and Salesforce). But this tier is the most internationally distributed in the series: France (BLOOM, on Jean Zay; Hugging Face's Paris roots), Israel (AI21), the UAE (TII Falcon), and EleutherAI's borderless volunteer base. Open and sovereign motivations travel further than the consumer-chat race does.

12

Cheat Sheet

ProducerTypeFlagship modelsOpennessWhy it ships models
NVIDIAChip vendorMegatron-LM, Nemotron-4 340B, Llama-NemotronOpen weights (permissive)Drive demand for GPUs
AI2 (Allen Institute)Non-profitOLMo, OLMo 2, Dolma, Tülu, MolmoFully open / reproducibleOpen science
EleutherAINon-profitThe Pile, GPT-J, GPT-NeoX, PythiaFully open / reproducibleOpen research toolchain
BigScienceConsortiumBLOOM 176BFully open (RAIL)Public-compute science
Hugging FacePlatformHub, Transformers; SmolLM, StarCoder*OpenOwn the distribution layer
Databricks / MosaicMLData platformMPT, DBRXOpen weightsBuild-on-your-data platform
IBMEnterprise ITGraniteOpen weights (Apache-2.0)Enterprise governance / watsonx
SnowflakeData cloudArcticOpen weights (Apache-2.0)Make the data cloud useful
AI21 LabsIndependentJurassic, JambaOpen weightsArchitectural differentiation
Microsoft (Phi)Research labPhi-1 → Phi-4Open weights (MIT)Small models / data quality
TII (UAE)SovereignFalcon 40B/180B, Falcon 3Open weights (permissive)National AI capability
AmazonHyperscalerTitan, NovaClosed (Bedrock-only)Keep customers on the cloud

*StarCoder/BigCode is a Hugging Face & ServiceNow collaboration. Licences and model versions evolve quickly — check each producer's model cards for the current terms.

The one-line takeaway

These are the producers building models for reasons other than winning chat — and together they supply most of the open weights, data and tooling the rest of the field runs on.

Cross-references

  • 05–07 — OpenAI, Anthropic, DeepMind/Google (incl. Gemma).
  • 08 — Meta, Mistral, xAI, Microsoft AI.
  • 09 — Chinese labs (Qwen, DeepSeek).

Where this leads

  • 10 — Future directions: the open-weight ecosystem these builders created.