Browser LLM Providers, One API: WebLLM, Transformers.js v4, wllama, LiteRT-LM, and Chrome AI

Running a large language model inside the browser used to mean picking a single runtime and coupling your entire application to it. If you chose WebLLM, you were locked into MLC-compiled models and WebGPU-only hardware. If you went with llama.cpp WASM, you accepted slower inference on CPU. And switching later meant rewriting every call site.

LocalMode eliminates that trade-off. The @localmode/core package defines a single LanguageModel interface. Five separate provider packages -- @localmode/webllm, @localmode/transformers, @localmode/wllama, @localmode/litert, and @localmode/chrome-ai -- each implement that interface using a different engine. Your application code calls generateText() or streamText() exactly the same way no matter which provider is active. Swap a single import, and the engine changes. Your UI, your prompts, your streaming logic -- none of it moves.

This post breaks down how each provider works, what it is best at, and how to choose between them. We also show how to set up automatic fallback so your app works on every browser without a single if statement in your components. Four of the five providers download and run a model your application chooses. The fifth, Chrome AI, is different in kind: it borrows the Gemini Nano model Chrome itself ships, so your application never fetches model files -- at the cost of running only on Chrome. LiteRT-LM remains a new early-preview addition, covered in its own section below.

The Shared Interface

Every provider implements this interface from @localmode/core:

interface LanguageModel {
  readonly modelId: string;
  readonly provider: string;
  readonly contextLength: number;
  readonly supportsVision?: boolean;

  doGenerate(options: DoGenerateOptions): Promise<DoGenerateResult>;
  doStream?(options: DoStreamOptions): AsyncIterable<StreamChunk>;
}

The core functions generateText() and streamText() accept any LanguageModel. They handle retries, abort signals, usage tracking, and response metadata. The provider handles model loading, tokenization, and inference. Clean separation.

Here is the same streaming call with all five providers:

import { streamText } from '@localmode/core';
import { webllm } from '@localmode/webllm';
import { transformers } from '@localmode/transformers';
import { wllama } from '@localmode/wllama';
import { litert } from '@localmode/litert';
import { chromeAI } from '@localmode/chrome-ai';

// Option A: WebLLM (WebGPU, MLC-compiled)
const resultA = await streamText({
  model: webllm.languageModel('Llama-3.2-3B-Instruct-q4f16_1-MLC'),
  prompt: 'Explain quantum tunneling in plain English',
  maxTokens: 300,
});

// Option B: Transformers.js v4 (ONNX, WebGPU + WASM fallback)
const resultB = await streamText({
  model: transformers.languageModel('onnx-community/Qwen3.5-0.8B-ONNX'),
  prompt: 'Explain quantum tunneling in plain English',
  maxTokens: 300,
});

// Option C: wllama (GGUF, llama.cpp WASM)
const resultC = await streamText({
  model: wllama.languageModel('Llama-3.2-1B-Instruct-Q4_K_M'),
  prompt: 'Explain quantum tunneling in plain English',
  maxTokens: 300,
});

// Option D: LiteRT-LM (.litertlm, Google's on-device engine -- early preview)
const resultD = await streamText({
  model: litert.languageModel('qwen3-0.6B'),
  prompt: 'Explain quantum tunneling in plain English',
  maxTokens: 300,
});

// Option E: Chrome AI (built-in Gemini Nano -- no model files to ship)
const resultE = await streamText({
  model: chromeAI.languageModel(),
  prompt: 'Explain quantum tunneling in plain English',
  maxTokens: 300,
});

// All five return the same StreamTextResult shape:
for await (const chunk of resultA.stream) {
  process.stdout.write(chunk.text); // identical API
}

The return type is always StreamTextResult with .stream, .text, .usage, and .response. The chunk shape is always StreamChunk with .text, .done, and optional .finishReason and .usage on the final chunk. No provider-specific handling required.

Provider 1: WebLLM (Fastest Inference)

Package: @localmode/webllm | Engine: MLC-AI WebLLM | Runtime: WebGPU

WebLLM compiles models through the MLC (Machine Learning Compilation) pipeline into WebGPU-optimized shaders. The result is the fastest in-browser LLM inference available today -- published benchmarks show Phi 3.5 Mini at 71 tokens/second and Llama 3.1 8B at 41 tokens/second on an M3 Max.

How it works: Models are pre-compiled into MLC format with 4-bit quantization (q4f16). At runtime, WebLLM loads the compiled model, creates WebGPU compute pipelines, and runs inference entirely on the GPU. The JavaScript API is OpenAI-compatible under the hood, which LocalMode wraps into the standard LanguageModel interface.

Curated catalog: 32 models ranging from SmolLM2 135M (78MB) to Qwen 3.5 9B (5.06GB). Includes 1 vision model (Phi 3.5 Vision). Model families include Qwen 2.5/3/3.5, Llama 3.1/3.2, Phi 3/3.5, Gemma 2, DeepSeek R1, Mistral, Ministral, and SmolLM2.

Best for: Applications where inference speed matters most -- interactive chat, real-time code completion, streaming agents. If your users have Chrome 113+, Edge 113+, or Safari 26+ with a discrete or integrated GPU, WebLLM delivers the best experience.

Limitation: Requires WebGPU. Firefox ships WebGPU in stable releases: 141+ on Windows, 147+ on macOS Apple Silicon. Older devices without GPU support cannot run WebLLM at all. The model catalog is fixed to MLC-compiled variants -- you cannot bring an arbitrary model file.

Provider 2: Transformers.js v4 (Broadest Model Selection)

Package: @localmode/transformers | Engine: HuggingFace Transformers.js v4 | Runtime: WebGPU with WASM fallback

Transformers.js v4 runs ONNX-format models using ONNX Runtime Web. It prefers WebGPU when available but automatically falls back to WASM, meaning it works on every modern browser. All @localmode/transformers implementations - embeddings, classification, speech-to-text, text generation, and more - use a unified @huggingface/transformers@^4.2.0 dependency.

How it works: Models are loaded from HuggingFace Hub in ONNX format with q4 quantization. For standard text models (SmolLM2, Phi, Qwen3, Granite), TJS v4 uses its text-generation pipeline. For multimodal models (Qwen3.5, Gemma 4), it uses dedicated model classes with a split architecture (embed_tokens, vision_encoder, decoder). The loading strategy is auto-detected from the model ID.

Curated catalog: 16 recommended ONNX models from 120MB (Granite 4.0 350M) to 3GB (Gemma 4 E4B). Includes 5 vision-capable models (Qwen3.5-0.8B, Qwen3.5-2B, Qwen3.5-4B, Gemma 4 E2B, Gemma 4 E4B). But any ONNX model on HuggingFace Hub can be loaded -- you are not restricted to the curated list.

Best for: Applications that need vision input (image + text), broad model selection, or the ability to load custom ONNX models. Also the right choice when you want WebGPU speed where available but need guaranteed WASM fallback on older hardware.

Limitation: ONNX model availability on HuggingFace lags behind GGUF. Slightly slower than WebLLM on WebGPU due to the ONNX Runtime overhead (40-60 tok/s vs 60-100 tok/s).

Provider 3: wllama (Universal Browser Support)

Package: @localmode/wllama | Engine: wllama v3 (llama.cpp compiled to WASM) | Runtime: WASM (CPU) with optional WebGPU acceleration

wllama is llama.cpp compiled to WebAssembly. It runs any standard GGUF model file on the CPU without requiring WebGPU, WebNN, or any GPU API at all. With wllama v3, optional WebGPU acceleration is available via useWebGPU and nGpuLayers for faster inference on capable devices. This makes it the most universally compatible provider -- if the browser supports WASM (which is every modern browser since 2017), wllama works.

How it works: The wllama library loads GGUF files directly from HuggingFace URLs, initializes a llama.cpp WASM instance, and runs inference token by token. Multi-threaded execution is available when the page has Cross-Origin Isolation headers (Cross-Origin-Opener-Policy: same-origin and Cross-Origin-Embedder-Policy: require-corp), which enables SharedArrayBuffer. Without CORS headers, wllama falls back to single-threaded mode -- functional but 2-4x slower.

Curated catalog: 30 curated models (25 language + 3 embedding + 2 reranker) from SmolLM2 135M (70MB) to Gemma 4 E4B (5.41GB), spanning Q4_K_M text models, four vision-language models (Holo2 4B/8B for UI grounding, Gemma 4 E2B/E4B for vision + tool calling), and three GGUF embedding models. wllama v3 adds embeddings via wllama.embedding(), optional WebGPU acceleration, OAI-compatible tool calling, vision input via mmprojUrl, and native Jinja chat templates. But the real story is the open ecosystem: wllama accepts any GGUF URL, which means access to the 160,000+ GGUF models on HuggingFace Hub. Any model that bartowski, TheBloke, or the original authors have quantized to GGUF is one URL away.

Best for: Maximum compatibility, maximum model selection, and environments where WebGPU is unavailable (Firefox stable, older Safari, corporate locked-down browsers). Also the right choice when you need a specific fine-tuned or quantized model that only exists in GGUF format.

Limitation: CPU-only inference is inherently slower (5-15 tok/s), though WebGPU acceleration closes the gap on supported browsers. The 2GB ArrayBuffer limit in browsers means large models must be split into chunks (wllama handles this automatically for catalog models). Single-threaded mode without CORS headers is noticeably slow.

Provider 4: LiteRT-LM (Google On-Device Engine -- Early Preview)

Package: @localmode/litert | Engine: Google LiteRT-LM via @litert-lm/core | Runtime: WebGPU with CPU WASM fallback

LiteRT-LM is the on-device inference engine Google uses to ship Gemini Nano-class models to Android and ChromeOS. The @litert-lm/core package compiles it to WebAssembly, and @localmode/litert wraps it behind the standard LanguageModel interface. It loads models in the .litertlm format -- a single artifact bundling tokenizer, weights, and metadata.

How it works: The provider downloads and caches the .litertlm file, creates a LiteRT-LM Engine (WebGPU backend by default), and runs a Conversation for generation. The Gemma 4 builds are GPU-compiled, so the provider checks WebGPU availability up front and fails fast with a clear error if it is missing. For portable models (Qwen3 0.6B), if GPU streaming load is unsupported it retries once on the CPU backend automatically.

Curated catalog: Three models -- gemma-4-E2B (Gemma 4 E2B, 2.0GB), gemma-4-E4B (Gemma 4 E4B, 3.0GB), and qwen3-0.6B (Qwen3 0.6B, 614MB) -- all verified end-to-end in real Chrome. The Gemma 4 entries use the web-optimized *-it-web.litertlm builds Google publishes as the models officially supported by the LiteRT-LM JS API; those builds are WebGPU-only. Qwen3 0.6B is a portable build that also runs on the CPU backend. Any other .litertlm file loads via a HuggingFace shorthand or full URL; gated Google models (Gemma 3n, Gemma 3 1B, FunctionGemma) load via a resolved modelUrl.

Best for: Running Gemma 4 in the browser via Google's first-party engine (on WebGPU-capable hardware), or wiring LiteRT-LM into a provider-fallback chain. The other three providers are longer-established if you need a wider catalog or vision input.

Limitation: Early preview. The @litert-lm/core API surface may still change, the JS API is text-in / text-out (no vision or audio input), Gemma 4 is WebGPU-only, stopSequences is unsupported (LiteRT-LM uses token IDs), and token usage counts are estimated from text length. See the dedicated post, Google's LiteRT-LM in the Browser, for the full details.

Provider 5: Chrome AI (No Model Files to Ship)

Package: @localmode/chrome-ai | Engine: Chrome's built-in Prompt API (Gemini Nano) | Runtime: Chrome-managed

The other four providers all answer the same question -- which model do I download, and in what format? Chrome AI does not answer it. Chrome 148+ exposes a window.LanguageModel global backed by Gemini Nano, a model the browser itself provides. @localmode/chrome-ai wraps it behind the standard LanguageModel interface, so it drops into the same generateText() and streamText() calls as everything else.

How it works: chromeAI.languageModel() is cheap and lazy -- it does not touch LanguageModel.create() at construction time. The underlying Gemini Nano session is created on the first doGenerate(), doStream(), or warmUp() call. That laziness has a consequence worth internalizing: construction never throws on an unsupported browser. You must gate it with the synchronous isPromptAPISupported() export before you build the model, rather than wrapping it in a try/catch and expecting a rejection.

Catalog: Exactly one model, chrome-ai:gemini-nano. There is nothing to choose. contextLength defaults to a documentation value of 6144; temperature and topK forward to LanguageModel.create().

On "zero download": Your application ships and fetches no model files -- but Chrome may still download Gemini Nano once, browser-wide, on first use. If availability() reports downloadable or downloading and you have not opted in, the call fails fast with chrome-ai-download-required. Opt in with providerOptions: { chromeAI: { allowDownload: true } }, or trigger creation after a user gesture. The download is the browser's, shared across every site, and happens once -- not per-app, and not per-page-load.

Best for: Shaving the model download out of your bundle story entirely on Chrome desktop, and as the top tier of a fallback chain that degrades to a downloadable provider everywhere else.

Limitation: Chrome 148+ on desktop only -- this is the narrowest browser support of the five by a wide margin, and it is why Chrome AI belongs above a fallback chain rather than as a sole dependency. Text-only: passing an ImagePart is rejected with chrome-ai-multimodal-not-supported. Availability is Chrome's call, not yours; the model can be unavailable on a supported browser (chrome-ai-model-not-available), and oversized inputs raise chrome-ai-quota-exceeded. Token usage is estimated from text length. The same package also wraps Chrome's Summarizer and Translator APIs, which are separate interfaces beyond the scope of this post.

Master Comparison Table

Decision Flowchart

Use this text-based flowchart to pick the right provider for your use case:

START
  |
  v
Do you need vision, a specific model, or non-Chrome support?
  |
  +-- NO --> isPromptAPISupported()?          (Chrome 148+ desktop)
  |            |
  |            +-- YES --> Use @localmode/chrome-ai
  |            |           (text-only; your app ships no model files)
  |            |
  |            +-- NO --> fall through to the download path below
  |
  +-- YES -> v
Does the user's browser support WebGPU?
  |
  +-- YES --> Do you need vision (image input)?
  |             |
  |             +-- YES --> Use @localmode/transformers
  |             |           (Qwen3.5 / Gemma 4 ONNX models)
  |             |
  |             +-- NO --> Is inference speed the top priority?
  |                          |
  |                          +-- YES --> Use @localmode/webllm
  |                          |           (fastest: 60-100 tok/s)
  |                          |
  |                          +-- NO --> Do you need a specific
  |                                     fine-tuned GGUF model?
  |                                       |
  |                                       +-- YES --> Use @localmode/wllama
  |                                       +-- NO  --> Use @localmode/webllm
  |
  +-- NO --> Does the app need LLM at all on this device?
               |
               +-- YES --> Use @localmode/wllama
               |           (works everywhere with WASM)
               |
               +-- NO --> Skip LLM, use lighter models
                          (embeddings, classification, etc.)

Chrome AI sits at the top of the chart because when it applies it removes the download question altogether -- but it applies narrowly, so every branch below it is the answer for the other browsers. @localmode/litert is intentionally left off the default decision path while it is in early preview -- opt into it explicitly when you want to experiment with Google's engine, or add it to a fallback chain so your app picks it up once its catalog grows.

Automatic Fallback: Zero Decision Logic in Your App

You can detect capabilities and route to the best available provider at runtime:

import { isWebGPUSupported } from '@localmode/core';
import { chromeAI, isPromptAPISupported } from '@localmode/chrome-ai';

// Chrome AI first: when the Prompt API exists, nothing needs downloading.
// isPromptAPISupported() is synchronous, and it is the ONLY correct gate --
// chromeAI.languageModel() constructs lazily and never throws here.
if (isPromptAPISupported()) {
  return chromeAI.languageModel();
}

const hasGPU = await isWebGPUSupported();

const model = hasGPU
  ? webllm.languageModel('Qwen3-4B-q4f16_1-MLC')
  : wllama.languageModel('Qwen2.5-3B-Instruct-Q4_K_M');

When to Use Each Provider

Use WebLLM when you are building a chat application, coding assistant, or any interactive feature where response latency directly impacts user experience. Your users have modern hardware with WebGPU, and you are satisfied with the 32 curated model options. This is the default choice for most consumer-facing applications in 2026.

Use Transformers.js v4 when you need multimodal input (image + text), want to load custom ONNX models, or need the safety net of automatic WASM fallback without maintaining two separate code paths. The Qwen3.5 vision models are currently the best sub-4B multimodal option for the browser.

Use wllama when you need to reach every user regardless of hardware, want access to a specific GGUF model from the 160K+ available on HuggingFace, or are targeting environments where WebGPU is blocked. Corporate intranets, kiosk browsers, and older devices are where wllama shines. It also carries four curated vision-language models: Holo2 4B/8B for UI grounding and Gemma 4 E2B/E4B for vision + tool calling.

Use LiteRT-LM when you want Gemma 4 in the browser via Google's first-party on-device engine, or want the provider wired into a fallback chain. Treat it as early preview -- the other providers are longer-established if you need a wider catalog or vision input.

Use Chrome AI when you are on Chrome desktop and the download is the thing you are trying to avoid: a text-only feature where waiting on a multi-hundred-megabyte model would sink adoption, and where you are content to let Chrome pick the model. Because it needs Chrome 148+ -- where the Prompt API finally shipped for web pages -- it is almost always the first tier of a fallback chain rather than the only provider you ship.

Use the fallback chain when you do not control the deployment environment and want the best possible experience on every device. This is the recommended approach for open-ended web applications where you cannot predict your users' hardware.

What They All Share

Regardless of which provider you choose, you get the same core capabilities through @localmode/core:

• generateText() and streamText() with identical options and return types
• generateObject() and streamObject() for structured JSON output with Zod schema validation
• LanguageModelMiddleware for logging, caching, guardrails, and semantic cache
• AbortSignal support for cancellation on every call
• Usage tracking with input tokens, output tokens, and duration on every response
• React hooks via @localmode/react: useGenerateText(), useChat()

The provider is an implementation detail. The interface is the contract. Build against the interface, and your application stays portable across all five engines -- today and as new providers emerge.

Methodology

All model counts, model IDs, size figures, and API shapes were verified directly against the LocalMode codebase (packages/webllm/src/models.ts, packages/wllama/src/models.ts, packages/transformers/src/models.ts, packages/litert/src/models.ts, packages/chrome-ai/src/implementations/language-model.ts, packages/chrome-ai/src/utils.ts, and packages/core/src/generation/types.ts). The Chrome AI model ID, contextLength default (6144), lazy-construction behaviour, allowDownload opt-in, and error codes were read from that provider's source rather than inferred. WebLLM performance figures (Phi 3.5 Mini at 71.1 tok/s, Llama 3.1 8B at 41.1 tok/s on M3 Max) were verified against Table 1 of the arxiv paper (2412.15803). The FOSDEM 2025 wllama talk URL was verified against the FOSDEM archive. No figures were sourced from unverified secondary summaries.

Sources

• MLC-AI WebLLM -- WebGPU LLM inference engine
• WebLLM: A High-Performance In-Browser LLM Inference Engine -- primary source for Phi 3.5 Mini (71.1 tok/s) and Llama 3.1 8B (41.1 tok/s) M3 Max benchmarks (Table 1)
• wllama -- llama.cpp WebAssembly binding for browser inference
• FOSDEM 2025: wllama -- bringing llama.cpp to the web
• Transformers.js -- HuggingFace ML framework for the browser
• Transformers.js v3: WebGPU Support -- WebGPU integration announcement
• LocalMode Transformers Text Generation Docs -- curated ONNX model catalog
• LocalMode WebLLM Docs -- curated MLC model catalog
• LocalMode wllama Docs -- curated GGUF model catalog and GGUF metadata parser
• Chrome Built-in AI: Prompt API -- the window.LanguageModel / Gemini Nano API wrapped by @localmode/chrome-ai
• LocalMode Chrome AI Docs -- provider setup, availability gating, and the Summarizer / Translator APIs
• Google LiteRT-LM -- Google's on-device LLM inference engine
• LocalMode LiteRT Docs -- .litertlm model catalog and early-preview status