Question Answering

Extract precise answers from a given context using extractive question answering models. The model identifies the span of text in the context that best answers the question.

See it in action

Try QA Bot for a working demo of these APIs.

answerQuestion()

Answer a question given a context:

import { answerQuestion } from '@localmode/core';
import { transformers } from '@localmode/transformers';

const model = transformers.questionAnswering('Xenova/distilbert-base-cased-distilled-squad');

const { answer, score, start, end } = await answerQuestion({
  model,
  question: 'What is the capital of France?',
  context: 'France is a country in Europe. The capital of France is Paris. It is known for the Eiffel Tower.',
});

console.log(answer); // 'Paris'
console.log(score);  // 0.98
console.log(start);  // 51
console.log(end);    // 56

const controller = new AbortController();

setTimeout(() => controller.abort(), 5000);

const { answer } = await answerQuestion({
  model,
  question: 'What is the capital of France?',
  context: 'The capital of France is Paris.',
  abortSignal: controller.signal,
});

AnswerQuestionOptions

Prop

Type

AnswerQuestionResult

Prop

Type

answerQuestionMany()

Answer multiple questions:

import { answerQuestionMany } from '@localmode/core';

const { answers } = await answerQuestionMany({
  model,
  questions: [
    { question: 'What is the capital?', context: 'The capital of France is Paris.' },
    { question: 'What is it known for?', context: 'Paris is known for the Eiffel Tower.' },
  ],
});

answers.forEach((answerSet) => {
  answerSet.forEach((a) => console.log(`${a.answer} (${a.score.toFixed(2)})`));
});

Custom Provider

Implement the QuestionAnsweringModel interface:

import type {
  QuestionAnsweringModel,
  DoAnswerQuestionOptions,
  DoAnswerQuestionResult,
} from '@localmode/core';

class MyQAModel implements QuestionAnsweringModel {
  readonly modelId = 'custom:my-qa';
  readonly provider = 'custom';

  async doAnswer(options: DoAnswerQuestionOptions): Promise<DoAnswerQuestionResult> {
    const { questions } = options;

    // Your QA logic here
    const answers = questions.map(({ question, context }) => [
      { answer: 'example', score: 0.9, start: 0, end: 7 },
    ]);

    return {
      answers,
      usage: { inputTokens: 0, durationMs: 0 },
    };
  }
}