Core
Middleware
Extend functionality with caching, logging, retry, and validation.
Middleware lets you extend and modify the behavior of embedding models and vector databases.
Embedding Model Middleware
Wrap embedding models with middleware:
import { wrapEmbeddingModel, cachingMiddleware, loggingMiddleware } from '@localmode/core';
import { transformers } from '@localmode/transformers';
const baseModel = transformers.embedding('Xenova/all-MiniLM-L6-v2');
const model = wrapEmbeddingModel(baseModel, [
cachingMiddleware({ maxSize: 1000 }),
loggingMiddleware({ logger: console.log }),
]);Available Middleware
Combining Middleware
Stack multiple middleware:
const model = wrapEmbeddingModel(baseModel, [
validationMiddleware({ maxLength: 8192 }),
piiRedactionMiddleware({ patterns: ['email', 'phone'] }),
cachingMiddleware({ maxSize: 1000 }),
retryMiddleware({ maxRetries: 3 }),
loggingMiddleware({ logger: console.log }),
]);Middleware Order
Middleware executes in order. Place validation first, caching before expensive operations, and logging last.
Vector DB Middleware
Wrap vector databases:
import { wrapVectorDB } from '@localmode/core';
const baseDB = await createVectorDB({ name: 'db', dimensions: 384 });
const db = wrapVectorDB(baseDB, {
beforeAdd: async (docs) => {
console.log('Adding', docs.length, 'documents');
return docs;
},
afterAdd: async (docs) => {
console.log('Added', docs.length, 'documents');
},
beforeSearch: async (vector, options) => {
console.log('Searching with k =', options.k);
return { vector, options };
},
afterSearch: async (results) => {
console.log('Found', results.length, 'results');
return results;
},
beforeDelete: async (id) => {
console.log('Deleting', id);
return id;
},
afterDelete: async () => {
console.log('Deleted');
},
});Vector DB Middleware Interface
interface VectorDBMiddleware {
beforeAdd?: (docs: Document[]) => Promise<Document[]>;
afterAdd?: (docs: Document[]) => Promise<void>;
beforeSearch?: (
vector: Float32Array,
options: SearchOptions
) => Promise<{ vector: Float32Array; options: SearchOptions }>;
afterSearch?: (results: SearchResult[]) => Promise<SearchResult[]>;
beforeDelete?: (id: string) => Promise<string>;
afterDelete?: () => Promise<void>;
beforeClear?: () => Promise<void>;
afterClear?: () => Promise<void>;
}Custom Middleware
Create your own middleware:
import type { EmbeddingModelMiddleware } from '@localmode/core';
function myCustomMiddleware(options: { threshold: number }): EmbeddingModelMiddleware {
return {
transformParams: async ({ values }) => {
// Transform input values
const filtered = values.filter((v) => v.length > options.threshold);
return { values: filtered };
},
wrapEmbed: async ({ doEmbed, values, model }) => {
const start = Date.now();
// Call the actual embedding function
const result = await doEmbed({ values });
const duration = Date.now() - start;
console.log(`Embedded ${values.length} values in ${duration}ms`);
return result;
},
};
}
const model = wrapEmbeddingModel(baseModel, [myCustomMiddleware({ threshold: 10 })]);Best Practices
Middleware Tips
- Order matters - Validation first, caching early, logging last
- Keep middleware focused - One concern per middleware
- Handle errors - Middleware can throw; handle gracefully
- Consider performance - Each middleware adds overhead
- Use composition - Stack simple middleware for complex behavior