Migration Guide

Migrate existing LangChain.js applications from cloud providers to 100% local inference. Only provider instantiation changes — all chain, retriever, and agent code stays the same.

See it in action

Try the Knowledge Base block for a fully migrated RAG pipeline — its engine toggle runs the same ingest, search, and grounded answers through the LocalMode adapters and the core pipeline behind identical UI.

Component Mapping

Component	Cloud (Before)	Local (After)
Embeddings	`OpenAIEmbeddings`	`LocalModeEmbeddings`
Chat Model	`ChatOpenAI`	`ChatLocalMode`
Vector Store	`PineconeStore` / `ChromaStore`	`LocalModeVectorStore`
Reranker	`CohereRerank`	`LocalModeReranker`

Full RAG Chain Migration

Before — Cloud

import { ChatOpenAI } from '@langchain/openai';
import { OpenAIEmbeddings } from '@langchain/openai';
import { PineconeStore } from '@langchain/pinecone';
import { Pinecone } from '@pinecone-database/pinecone';

// Requires: OPENAI_API_KEY, PINECONE_API_KEY
const llm = new ChatOpenAI({ modelName: 'gpt-4o-mini', temperature: 0.7 });
const embeddings = new OpenAIEmbeddings({ modelName: 'text-embedding-3-small' });

const pinecone = new Pinecone();
const index = pinecone.Index('my-docs');
const store = await PineconeStore.fromExistingIndex(embeddings, { pineconeIndex: index });

const retriever = store.asRetriever({ k: 5 });

After — Local

import { ChatLocalMode, LocalModeEmbeddings, LocalModeVectorStore } from '@localmode/langchain';
import { transformers } from '@localmode/transformers';
import { webllm } from '@localmode/webllm';
import { createVectorDB } from '@localmode/core';

// No API keys needed
const llm = new ChatLocalMode({
  model: webllm.languageModel('Qwen3-1.7B-q4f16_1-MLC'),
  temperature: 0.7,
});
const embeddings = new LocalModeEmbeddings({
  model: transformers.embedding('Xenova/bge-small-en-v1.5'),
});

const db = await createVectorDB({ name: 'my-docs', dimensions: 384 });
const store = new LocalModeVectorStore(embeddings, { db });

const retriever = store.asRetriever({ k: 5 });

Everything after retriever — chains, prompts, output parsers — stays identical.

Individual Adapter Migrations

Embeddings

- import { OpenAIEmbeddings } from '@langchain/openai';
+ import { LocalModeEmbeddings } from '@localmode/langchain';
+ import { transformers } from '@localmode/transformers';

- const embeddings = new OpenAIEmbeddings({ modelName: 'text-embedding-3-small' });
+ const embeddings = new LocalModeEmbeddings({
+   model: transformers.embedding('Xenova/bge-small-en-v1.5'),
+ });

Chat Model

- import { ChatOpenAI } from '@langchain/openai';
+ import { ChatLocalMode } from '@localmode/langchain';
+ import { webllm } from '@localmode/webllm';

- const llm = new ChatOpenAI({ modelName: 'gpt-4o-mini', temperature: 0.7 });
+ const llm = new ChatLocalMode({
+   model: webllm.languageModel('Qwen3-1.7B-q4f16_1-MLC'),
+   temperature: 0.7,
+ });

Vector Store

- import { PineconeStore } from '@langchain/pinecone';
+ import { LocalModeVectorStore } from '@localmode/langchain';
+ import { createVectorDB } from '@localmode/core';

- const store = await PineconeStore.fromExistingIndex(embeddings, { pineconeIndex });
+ const db = await createVectorDB({ name: 'docs', dimensions: 384 });
+ const store = new LocalModeVectorStore(embeddings, { db });

Reranker

- import { CohereRerank } from '@langchain/cohere';
+ import { LocalModeReranker } from '@localmode/langchain';
+ import { transformers } from '@localmode/transformers';

- const reranker = new CohereRerank({ model: 'rerank-english-v3.0', topN: 5 });
+ const reranker = new LocalModeReranker({
+   model: transformers.reranker('Xenova/ms-marco-MiniLM-L-6-v2'),
+   topK: 5,
+ });

The LocalModeReranker implements BaseDocumentCompressor, so it works with LangChain's ContextualCompressionRetriever:

import { ContextualCompressionRetriever } from 'langchain/retrievers/contextual_compression';

const compressedRetriever = new ContextualCompressionRetriever({
  baseCompressor: reranker,
  baseRetriever: store.asRetriever({ k: 20 }),
});

// Returns top 5 most relevant docs from the initial 20
const results = await compressedRetriever.invoke('search query');

What Changes, What Doesn't

Aspect	Changes?	Details
Chain/agent code	No	Same LangChain chains, retrievers, prompts
Provider imports	Yes	3 import line changes
API keys	Removed	No `OPENAI_API_KEY`, `PINECONE_API_KEY` needed
Monthly cost	$0	All inference is local, unlimited usage
Data privacy	Improved	Documents and embeddings never leave the device
First-run time	Slower	One-time model download (33MB-2GB depending on models)
Model quality	Different	Smaller models — excellent for embeddings/reranking, adequate for simple generation
Streaming	Supported	`ChatLocalMode` streams via `doStream()` when available
Tool calling	Not supported	Local models don't support LangChain's tool/function calling
Structured output	Not supported	Use `generateObject()` from `@localmode/core` directly instead

Hybrid approach: Use LocalModeEmbeddings + LocalModeVectorStore + LocalModeReranker for the retrieval pipeline (runs locally at $0 cost), and keep ChatOpenAI for the generation step when you need frontier model quality. This gives you 90% cost savings while maintaining answer quality.

Composed Block

Block	Description	Links
RAG Chat	Streaming grounded RAG over your own text/PDF corpus with inline citations, on a core ⇄ LangChain engine toggle	Live · Install: `npx shadcn add @localmode/ui/blocks/knowledge/rag-chat`

Migration Guide

On this page