Backend

Overview

Backend is an umbrella module that encapsulates a unified way to work with the following functionalities:

• Chat Models (ChatModel class)
• Embedding Models (EmbeddingModel class)
• Audio Models (coming soon)
• Image Models (coming soon)

BeeAI framework’s backend is designed with a provider-based architecture, allowing you to switch between different AI service providers while maintaining a consistent API.

Note

Supported in Python and TypeScript.

---

Supported providers

The following table depicts supported providers. Each provider requires specific configuration through environment variables. Ensure all required variables are set before initializing a provider.

Name	Chat	Embedding	Environment Variables
Ollama	✅	✅	OLLAMA_CHAT_MODEL OLLAMA_BASE_URL
OpenAI	✅	✅	OPENAI_CHAT_MODEL OPENAI_EMBEDDING_MODEL OPENAI_API_BASE OPENAI_API_KEY OPENAI_ORGANIZATION OPENAI_API_HEADERS
IBM watsonx.ai	✅	✅	WATSONX_CHAT_MODEL WATSONX_API_KEY WATSONX_PROJECT_ID WATSONX_SPACE_ID WATSONX_TOKEN WATSONX_ZENAPIKEY WATSONX_URL WATSONX_REGION
Anthropic	✅	✅	ANTHROPIC_CHAT_MODEL ANTHROPIC_API_KEY ANTHROPIC_API_HEADERS
Groq	✅	❌	GROQ_CHAT_MODEL GROQ_API_KEY
Amazon Bedrock	✅	✅	AWS_CHAT_MODEL AWS_BEDROCK_API_KEY AWS_ACCESS_KEY_ID AWS_SECRET_ACCESS_KEY AWS_REGION AWS_API_HEADERS
Google Vertex	✅	✅	GOOGLE_VERTEX_CHAT_MODEL GOOGLE_VERTEX_PROJECT GOOGLE_VERTEX_LOCATION GOOGLE_APPLICATION_CREDENTIALS GOOGLE_APPLICATION_CREDENTIALS_JSON GOOGLE_CREDENTIALS GOOGLE_VERTEX_API_HEADERS
Azure OpenAI	✅	✅	AZURE_OPENAI_CHAT_MODEL AZURE_OPENAI_API_KEY AZURE_OPENAI_API_BASE AZURE_OPENAI_API_VERSION AZURE_AD_TOKEN AZURE_API_TYPE AZURE_API_HEADERS
xAI	✅	✅	XAI_CHAT_MODEL XAI_API_KEY
Google Gemini	✅	✅	GEMINI_CHAT_MODEL GEMINI_API_KEY GEMINI_API_HEADERS
MistralAI	✅	✅	MISTRALAI_CHAT_MODEL MISTRALAI_EMBEDDING_MODEL MISTRALAI_API_KEY MISTRALAI_API_BASE
Transformers	✅	✅	TRANSFORMERS_CHAT_MODEL HF_TOKEN
MiniMax	✅	❌	MINIMAX_CHAT_MODEL MINIMAX_API_KEY MINIMAX_API_BASE MINIMAX_API_HEADERS

Tip

If you don’t see your provider raise an issue here. Meanwhile, you can use the Ollama for local models in Python or TypeScript or the Langchain adapter for hosted providers.

Note

Google Gemini, MistralAI, and Transformers are supported in Python only. The Transformers chat model does not support tool calling.

---

Backend initialization

The Backend class serves as a central entry point to access models from your chosen provider. This example illustrate how to leverage the framework’s unified interface for different provider model operations by showcasing various interaction patterns including:

• Basic chat completion
• Streaming responses with abort functionality
• Structured output generation
• Real-time response parsing
• Tool calling with external APIs
• Text embedding generation

Tip

Explore more provider examples in Python or TypeScript

Python

import asyncio
import datetime
import sys
import traceback

from pydantic import BaseModel, Field

from beeai_framework.adapters.openai import OpenAIChatModel, OpenAIEmbeddingModel
from beeai_framework.backend import (
    ChatModel,
    ChatModelNewTokenEvent,
    ChatModelParameters,
    MessageToolResultContent,
    ToolMessage,
    UserMessage,
)
from beeai_framework.emitter import EventMeta
from beeai_framework.errors import AbortError, FrameworkError
from beeai_framework.parsers.field import ParserField
from beeai_framework.parsers.line_prefix import LinePrefixParser, LinePrefixParserNode
from beeai_framework.tools.weather import OpenMeteoTool, OpenMeteoToolInput
from beeai_framework.utils import AbortSignal


async def openai_from_name() -> None:
    llm = ChatModel.from_name("openai:gpt-4.1-mini")
    user_message = UserMessage("what states are part of New England?")
    response = await llm.run([user_message])
    print(response.get_text_content())


async def openai_granite_from_name() -> None:
    llm = ChatModel.from_name("openai:gpt-4.1-mini")
    user_message = UserMessage("what states are part of New England?")
    response = await llm.run([user_message])
    print(response.get_text_content())


async def openai_sync() -> None:
    llm = OpenAIChatModel("gpt-4.1-mini")
    user_message = UserMessage("what is the capital of Massachusetts?")
    response = await llm.run([user_message])
    print(response.get_text_content())


async def openai_stream() -> None:
    llm = OpenAIChatModel("gpt-4.1-mini")
    user_message = UserMessage("How many islands make up the country of Cape Verde?")
    response = await llm.run([user_message], stream=True)
    print(response.get_text_content())


async def openai_stream_abort() -> None:
    llm = OpenAIChatModel("gpt-4.1-mini")
    user_message = UserMessage("What is the smallest of the Cape Verde islands?")

    try:
        response = await llm.run([user_message], stream=True, signal=AbortSignal.timeout(0.5))

        if response is not None:
            print(response.get_text_content())
        else:
            print("No response returned.")
    except AbortError as err:
        print(f"Aborted: {err}")


async def openai_structure() -> None:
    class TestSchema(BaseModel):
        answer: str = Field(description="your final answer")

    llm = OpenAIChatModel("gpt-4.1-mini")
    user_message = UserMessage("How many islands make up the country of Cape Verde?")
    response = await llm.run([user_message], response_format=TestSchema, stream=True)
    print(response.output_structured)


async def openai_stream_parser() -> None:
    llm = OpenAIChatModel("gpt-4.1-mini")

    parser = LinePrefixParser(
        nodes={
            "test": LinePrefixParserNode(
                prefix="Prefix: ", field=ParserField.from_type(str), is_start=True, is_end=True
            )
        }
    )

    async def on_new_token(data: ChatModelNewTokenEvent, event: EventMeta) -> None:
        await parser.add(data.value.get_text_content())

    user_message = UserMessage("Produce 3 lines each starting with 'Prefix: ' followed by a sentence and a new line.")
    await llm.run([user_message], stream=True).observe(lambda emitter: emitter.on("new_token", on_new_token))
    result = await parser.end()
    print(result)


async def openai_tool_calling() -> None:
    llm = ChatModel.from_name("openai:gpt-4.1-mini", ChatModelParameters(stream=True, temperature=0))
    user_message = UserMessage(f"What is the current weather in Boston? Current date is {datetime.datetime.today()}.")
    weather_tool = OpenMeteoTool()
    response = await llm.run([user_message], tools=[weather_tool])
    tool_call_msg = response.get_tool_calls()[0]
    print(tool_call_msg.model_dump())
    tool_response = await weather_tool.run(OpenMeteoToolInput(location_name="Boston"))
    tool_response_msg = ToolMessage(
        MessageToolResultContent(
            result=tool_response.get_text_content(),
            tool_name=weather_tool.name,
            tool_call_id=response.get_tool_calls()[0].id,
        )
    )
    print(tool_response_msg.to_plain())
    final_response = await llm.run([user_message, *response.output, tool_response_msg], tools=[])
    print(final_response.get_text_content())


async def openai_embedding() -> None:
    embedding_llm = OpenAIEmbeddingModel()

    response = await embedding_llm.create(["Text", "to", "embed"])

    for row in response.embeddings:
        print(*row)


async def openai_cloning() -> None:
    llm = OpenAIChatModel("gpt-4.1-mini")
    await llm.clone()

    embedding_llm = OpenAIEmbeddingModel()
    await embedding_llm.clone()


async def openai_file_example() -> None:
    llm = ChatModel.from_name("openai:gpt-4.1-mini")
    data_uri = "data:application/pdf;base64,JVBERi0xLjQKMSAwIG9iago8PC9UeXBlIC9DYXRhbG9nCi9QYWdlcyAyIDAgUgo+PgplbmRvYmoKMiAwIG9iago8PC9UeXBlIC9QYWdlcwovS2lkcyBbMyAwIFJdCi9Db3VudCAxCj4+CmVuZG9iagozIDAgb2JqCjw8L1R5cGUgL1BhZ2UKL1BhcmVudCAyIDAgUgovTWVkaWFCb3ggWzAgMCA1OTUgODQyXQovQ29udGVudHMgNSAwIFIKL1Jlc291cmNlcyA8PC9Qcm9jU2V0IFsvUERGIC9UZXh0XQovRm9udCA8PC9GMSA0IDAgUj4+Cj4+Cj4+CmVuZG9iago0IDAgb2JqCjw8L1R5cGUgL0ZvbnQKL1N1YnR5cGUgL1R5cGUxCi9OYW1lIC9GMQovQmFzZUZvbnQgL0hlbHZldGljYQovRW5jb2RpbmcgL01hY1JvbWFuRW5jb2RpbmcKPj4KZW5kb2JqCjUgMCBvYmoKPDwvTGVuZ3RoIDUzCj4+CnN0cmVhbQpCVAovRjEgMjAgVGYKMjIwIDQwMCBUZAooRHVtbXkgUERGKSBUagpFVAplbmRzdHJlYW0KZW5kb2JqCnhyZWYKMCA2CjAwMDAwMDAwMDAgNjU1MzUgZgowMDAwMDAwMDA5IDAwMDAwIG4KMDAwMDAwMDA2MyAwMDAwMCBuCjAwMDAwMDAxMjQgMDAwMDAgbgowMDAwMDAwMjc3IDAwMDAwIG4KMDAwMDAwMDM5MiAwMDAwMCBuCnRyYWlsZXIKPDwvU2l6ZSA2Ci9Sb290IDEgMCBSCj4+CnN0YXJ0eHJlZgo0OTUKJSVFT0YK"

    file_message = UserMessage.from_file(file_data=data_uri, format="text")
    print(file_message.to_plain())
    response = await llm.run([UserMessage("Read content of the file."), file_message])
    print(response.get_text_content())


async def main() -> None:
    print("*" * 10, "openai_from_name")
    await openai_from_name()
    print("*" * 10, "openai_granite_from_name")
    await openai_granite_from_name()
    print("*" * 10, "openai_sync")
    await openai_sync()
    print("*" * 10, "openai_stream")
    await openai_stream()
    print("*" * 10, "openai_stream_abort")
    await openai_stream_abort()
    print("*" * 10, "openai_structure")
    await openai_structure()
    print("*" * 10, "openai_stream_parser")
    await openai_stream_parser()
    print("*" * 10, "openai_tool_calling")
    await openai_tool_calling()
    print("*" * 10, "openai_embedding")
    await openai_embedding()
    print("*" * 10, "openai_cloning")
    await openai_cloning()


if __name__ == "__main__":
    try:
        asyncio.run(main())
    except FrameworkError as e:
        traceback.print_exc()
        sys.exit(e.explain())

TypeScript

import "dotenv/config.js";
import { OpenAIChatModel } from "beeai-framework/adapters/openai/backend/chat";
import { ToolMessage, UserMessage } from "beeai-framework/backend/message";
import { ChatModel } from "beeai-framework/backend/chat";
import { z } from "zod";
import { ChatModelError } from "beeai-framework/backend/errors";
import { OpenMeteoTool } from "beeai-framework/tools/weather/openMeteo";

const llm = new OpenAIChatModel(
  "gpt-5-nano",
  {},
  // {
  //   baseURL: "OPENAI_BASE_URL",
  //   apiKey: "OPENAI_API_KEY",
  //   organization: "OPENAI_ORGANIZATION",
  //   project: "OPENAI_PROJECT",
  // },
);

llm.config({
  parameters: {
    maxTokens: 2048,
  },
});

async function openaiFromName() {
  const openaiLLM = await ChatModel.fromName("openai:gpt-5-nano");
  const response = await openaiLLM.create({
    messages: [new UserMessage("what states are part of New England?")],
  });
  console.info(response.getTextContent());
}

async function openaiSync() {
  const response = await llm.create({
    messages: [new UserMessage("what is the capital of Massachusetts?")],
  });
  console.info(response.getTextContent());
}

async function openaiStream() {
  const response = await llm.create({
    messages: [new UserMessage("How many islands make up the country of Cape Verde?")],
    stream: true,
  });
  console.info(response.getTextContent());
}

async function openaiAbort() {
  try {
    const response = await llm.create({
      messages: [new UserMessage("What is the smallest of the Cape Verde islands?")],
      stream: true,
      abortSignal: AbortSignal.timeout(1 * 500),
    });
    console.info(response.getTextContent());
  } catch (err) {
    if (err instanceof ChatModelError) {
      console.log("Aborted", { err });
    }
  }
}

async function openaiStructure() {
  const response = await llm.createStructure({
    schema: z.object({
      answer: z.string({ description: "your final answer" }),
    }),
    messages: [new UserMessage("How many islands make up the country of Cape Verde?")],
  });
  console.info(response.object);
}

async function openaiToolCalling() {
  const userMessage = new UserMessage(
    `What is the current weather in Boston? Current date is ${new Date().toISOString().split("T")[0]}.`,
  );
  const weatherTool = new OpenMeteoTool({ retryOptions: { maxRetries: 3 } });
  const response = await llm.create({
    messages: [userMessage],
    tools: [weatherTool],
    toolChoice: weatherTool,
  });
  const toolCallMsg = response.getToolCalls()[0];
  console.debug(JSON.stringify(toolCallMsg));
  const toolResponse = await weatherTool.run(toolCallMsg.input as any);
  const toolResponseMsg = new ToolMessage({
    type: "tool-result",
    output: { type: "text", value: toolResponse.getTextContent() },
    toolName: toolCallMsg.toolName,
    toolCallId: toolCallMsg.toolCallId,
  });
  console.info(toolResponseMsg.toPlain());
  const finalResponse = await llm.create({
    messages: [userMessage, ...response.messages, toolResponseMsg],
    tools: [],
  });
  console.info(finalResponse.getTextContent());
}

async function openaiDebug() {
  // Log every request
  llm.emitter.match("*", (value, event) =>
    console.debug(
      `Time: ${event.createdAt.toISOString()}`,
      `Event: ${event.name}`,
      `Data: ${JSON.stringify(value)}`,
    ),
  );

  const response = await llm.create({
    messages: [new UserMessage("Hello world!")],
  });
  console.info(response.messages[0].toPlain());
}

console.info(" openaiFromName".padStart(25, "*"));
await openaiFromName();
console.info(" openaiSync".padStart(25, "*"));
await openaiSync();
console.info(" openaiStream".padStart(25, "*"));
await openaiStream();
console.info(" openaiAbort".padStart(25, "*"));
await openaiAbort();
console.info(" openaiStructure".padStart(25, "*"));
await openaiStructure();
console.info(" openaiToolCalling".padStart(25, "*"));
await openaiToolCalling();
console.info(" openaiDebug".padStart(25, "*"));
await openaiDebug();

Note

See the events documentation for more information on standard emitter events.

---

Chat model

The ChatModel class represents a Chat Language Model and provides methods for text generation, streaming responses, and more. You can initialize a chat model in multiple ways:

Method 1: Using the from_name method

Python

from beeai_framework.backend.chat import ChatModel

model = ChatModel.from_name("ollama:llama3.1")

TypeScript

import { ChatModel } from "beeai-framework/backend/chat";

const model = await ChatModel.fromName("ollama:granite3.3:8b");

Method 2: Directly specifying the provider class

Python

from beeai_framework.adapters.ollama.backend.chat import OllamaChatModel

model = OllamaChatModel("llama3.1")

TypeScript

import { OllamaChatModel } from "beeai-framework/adapters/ollama/backend/chat";

const model = new OllamaChatModel("llama3.1");

File / Document Inputs (PDF etc.)

You can attach files (e.g. PDFs) to a UserMessage using the MessageFileContent part or the convenience factory UserMessage.from_file. Provide either a remote file_id/URL or an inline base64 data URI (file_data). Optionally specify a MIME format.

from beeai_framework.backend import UserMessage, MessageFileContent

# Using a remote / previously uploaded file URL or id (flattened API)
msg_with_file_id = UserMessage([
  MessageFileContent(
    file_id="https://www.w3.org/WAI/ER/tests/xhtml/testfiles/resources/pdf/dummy.pdf",
    format="application/pdf",
  ),
  "What's this file about?",
])

# Same using the factory helper
msg_with_file_id_factory = UserMessage.from_file(
  file_id="https://www.w3.org/WAI/ER/tests/xhtml/testfiles/resources/pdf/dummy.pdf",
  format="application/pdf",
)

# Using inline base64 data (shortened example)
msg_with_file_data = UserMessage([
  MessageFileContent(
    file_data="data:application/pdf;base64,AAA...",
    format="application/pdf",
  ),
  "Summarize the document",
])

# Inline base64 with factory
msg_with_file_data_factory = UserMessage.from_file(
  file_data="data:application/pdf;base64,AAA...",
  format="application/pdf",
)

These content parts serialize to the flattened schema (legacy nested { "file": {...} } removed):

{
  "type": "file",
  "file_id": "https://www.w3.org/WAI/ER/tests/xhtml/testfiles/resources/pdf/dummy.pdf",
  "format": "application/pdf"
}

If neither file_id nor file_data is supplied a validation error is raised.

Chat model configuration

You can configure various parameters for your chat model.

Python

import asyncio
import sys
import traceback

from beeai_framework.adapters.ollama import OllamaChatModel
from beeai_framework.backend import UserMessage
from beeai_framework.errors import FrameworkError
from examples.helpers.io import ConsoleReader


async def main() -> None:
    llm = OllamaChatModel("llama3.1")

    #  Optionally one may set llm parameters
    llm.parameters.max_tokens = 10000  # high number yields longer potential output
    llm.parameters.top_p = 0.1  # higher number yields more complex vocabulary, recommend only changing p or k
    llm.parameters.frequency_penalty = 0  # higher number yields reduction in word repetition
    llm.parameters.temperature = 0  # higher number yields greater randomness and variation
    llm.parameters.top_k = 0  # higher number yields more variance, recommend only changing p or k
    llm.parameters.n = 1  # higher number yields more choices
    llm.parameters.presence_penalty = 0  # higher number yields reduction in repetition of words
    llm.parameters.seed = 10  # can help produce similar responses if prompt and seed are always the same
    llm.parameters.stop_sequences = ["q", "quit", "ahhhhhhhhh"]  # stops the model on input of any of these strings
    llm.parameters.stream = False  # determines whether or not to use streaming to receive incremental data

    reader = ConsoleReader()

    for prompt in reader:
        response = await llm.run([UserMessage(prompt)])
        reader.write("LLM 🤖 (txt) : ", response.get_text_content())
        reader.write("LLM 🤖 (raw) : ", "\n".join([str(msg.to_plain()) for msg in response.output]))


if __name__ == "__main__":
    try:
        asyncio.run(main())
    except FrameworkError as e:
        traceback.print_exc()
        sys.exit(e.explain())

TypeScript

import "dotenv/config.js";
import { createConsoleReader } from "examples/helpers/io.js";
import { UserMessage } from "beeai-framework/backend/message";
import { OllamaChatModel } from "beeai-framework/adapters/ollama/backend/chat";

const llm = new OllamaChatModel("granite4:micro");

//  Optionally one may set llm parameters
llm.parameters.maxTokens = 10000; // high number yields longer potential output
llm.parameters.topP = 0; // higher number yields more complex vocabulary, recommend only changing p or k
llm.parameters.frequencyPenalty = 0; // higher number yields reduction in word repetition
llm.parameters.temperature = 0; // higher number yields greater randomness and variation
llm.parameters.topK = 0; // higher number yields more variance, recommend only changing p or k
llm.parameters.n = 1; // higher number yields more choices
llm.parameters.presencePenalty = 0; // higher number yields reduction in repetition of words
llm.parameters.seed = 10; // can help produce similar responses if prompt and seed are always the same
llm.parameters.stopSequences = ["q", "quit", "ahhhhhhhhh"]; // stops the model on input of any of these strings

// alternatively
llm.config({
  parameters: {
    maxTokens: 10000,
    // other parameters
  },
});

const reader = createConsoleReader();

for await (const { prompt } of reader) {
  const response = await llm.create({
    messages: [new UserMessage(prompt)],
  });
  reader.write(`LLM 🤖 (txt) : `, response.getTextContent());
  reader.write(`LLM 🤖 (raw) : `, JSON.stringify(response.messages));
}

Text generation

The most basic usage is to generate text responses:

Python

from beeai_framework.adapters.ollama.backend.chat import OllamaChatModel
from beeai_framework.backend.message import UserMessage

model = OllamaChatModel("llama3.1")
response = await model.create(
    messages=[UserMessage("what states are part of New England?")]
)

print(response.get_text_content())

TypeScript

import { UserMessage } from "beeai-framework/backend/message";
import { OllamaChatModel } from "beeai-framework/adapters/ollama/backend/chat";

const llm = new OllamaChatModel("llama3.1");

const response = await llm.create({
  messages: [new UserMessage("what states are part of New England?")],
});

console.log(response.getTextContent());

Note

Execution parameters (those passed to model.create({...})) take precedent over ones defined via config.

Streaming responses

For applications requiring real-time responses:

Python

from beeai_framework.adapters.ollama.backend.chat import OllamaChatModel
from beeai_framework.backend.message import UserMessage

llm = OllamaChatModel("llama3.1")
user_message = UserMessage("How many islands make up the country of Cape Verde?")
response = await llm.create(messages=[user_message], stream=True)
  .on(
    "new_token",
    lambda data, event: print(data.value.get_text_content()))
  )
)
print("Full response", response.get_text_content())

TypeScript

import "dotenv/config.js";
import { createConsoleReader } from "examples/helpers/io.js";
import { UserMessage } from "beeai-framework/backend/message";
import { OllamaChatModel } from "beeai-framework/adapters/ollama/backend/chat";

const llm = new OllamaChatModel("granite4:micro");

const reader = createConsoleReader();

for await (const { prompt } of reader) {
  const response = await llm
    .create({
      messages: [new UserMessage(prompt)],
    })
    .observe((emitter) =>
      emitter.match("*", (data, event) => {
        reader.write(`LLM 🤖 (event: ${event.name})`, JSON.stringify(data));

        // if you want to close the stream prematurely, just uncomment the following line
        // callbacks.abort()
      }),
    );

  reader.write(`LLM 🤖 (txt) : `, response.getTextContent());
  reader.write(`LLM 🤖 (raw) : `, JSON.stringify(response.messages));
}

Structured generation

Generate structured data according to a schema:

Python

import asyncio
import json
import sys
import traceback

from pydantic import BaseModel, Field

from beeai_framework.backend import ChatModel, UserMessage
from beeai_framework.errors import FrameworkError


async def main() -> None:
    model = ChatModel.from_name("ollama:granite4:micro")

    class ProfileSchema(BaseModel):
        first_name: str = Field(..., min_length=1)
        last_name: str = Field(..., min_length=1)
        address: str
        age: int
        hobby: str

    response = await model.run(
        [UserMessage("Generate a profile of a citizen of Europe.")], response_format=ProfileSchema
    )
    assert isinstance(response.output_structured, ProfileSchema)
    print(json.dumps(response.output_structured.model_dump(), indent=4))


if __name__ == "__main__":
    try:
        asyncio.run(main())
    except FrameworkError as e:
        traceback.print_exc()
        sys.exit(e.explain())

TypeScript

import { ChatModel, UserMessage } from "beeai-framework/backend/core";
import { z } from "zod";

const model = await ChatModel.fromName("ollama:granite4:micro");
const response = await model.createStructure({
  schema: z.union([
    z.object({
      firstName: z.string().min(1),
      lastName: z.string().min(1),
      address: z.string(),
      age: z.number().int().min(1),
      hobby: z.string(),
    }),
    z.object({
      error: z.string(),
    }),
  ]),
  messages: [new UserMessage("Generate a profile of a citizen of Europe.")],
});
console.log(response.object);

Tool calling

Integrate external tools with your AI model:

Python

import asyncio
import json
import re
import sys
import traceback

from beeai_framework.backend import (
    AnyMessage,
    ChatModel,
    ChatModelParameters,
    MessageToolResultContent,
    SystemMessage,
    ToolMessage,
    UserMessage,
)
from beeai_framework.errors import FrameworkError
from beeai_framework.tools import AnyTool, ToolOutput
from beeai_framework.tools.search.duckduckgo import DuckDuckGoSearchTool
from beeai_framework.tools.weather.openmeteo import OpenMeteoTool


async def main() -> None:
    model = ChatModel.from_name("ollama:llama3.1", ChatModelParameters(temperature=0))
    tools: list[AnyTool] = [DuckDuckGoSearchTool(), OpenMeteoTool()]
    messages: list[AnyMessage] = [
        SystemMessage("You are a helpful assistant. Use tools to provide a correct answer."),
        UserMessage("What's the fastest marathon time?"),
    ]

    while True:
        response = await model.run(
            messages,
            tools=tools,
        )

        tool_calls = response.get_tool_calls()
        messages.extend(response.output)

        tool_results: list[ToolMessage] = []

        for tool_call in tool_calls:
            print(f"-> running '{tool_call.tool_name}' tool with {tool_call.args}")
            tool: AnyTool = next(tool for tool in tools if tool.name == tool_call.tool_name)
            assert tool is not None
            res: ToolOutput = await tool.run(json.loads(tool_call.args))
            result = res.get_text_content()
            print(f"<- got response from '{tool_call.tool_name}'", re.sub(r"\s+", " ", result)[:256] + " (truncated)")
            tool_results.append(
                ToolMessage(
                    MessageToolResultContent(
                        result=result,
                        tool_name=tool_call.tool_name,
                        tool_call_id=tool_call.id,
                    )
                )
            )

        messages.extend(tool_results)

        answer = response.get_text_content()

        if answer:
            print(f"Agent: {answer}")
            break


if __name__ == "__main__":
    try:
        asyncio.run(main())
    except FrameworkError as e:
        traceback.print_exc()
        sys.exit(e.explain())

TypeScript

import "dotenv/config";
import {
  ChatModel,
  Message,
  SystemMessage,
  ToolMessage,
  UserMessage,
} from "beeai-framework/backend/core";
import { DuckDuckGoSearchTool } from "beeai-framework/tools/search/duckDuckGoSearch";
import { OpenMeteoTool } from "beeai-framework/tools/weather/openMeteo";
import { AnyTool, ToolOutput } from "beeai-framework/tools/base";

const model = await ChatModel.fromName("ollama:granite4:micro");
const tools: AnyTool[] = [new DuckDuckGoSearchTool(), new OpenMeteoTool()];
const messages: Message[] = [
  new SystemMessage("You are a helpful assistant. Use tools to provide a correct answer."),
  new UserMessage("What's the fastest marathon time?"),
];

while (true) {
  const response = await model.create({
    messages,
    tools,
  });
  messages.push(...response.messages);

  const toolCalls = response.getToolCalls();
  const toolResults = await Promise.all(
    toolCalls.map(async ({ input, toolName, toolCallId }) => {
      console.log(`-> running '${toolName}' tool with ${JSON.stringify(input)}`);
      const tool = tools.find((tool) => tool.name === toolName)!;
      const response: ToolOutput = await tool.run(input as any);
      const result = response.getTextContent();
      console.log(
        `<- got response from '${toolName}'`,
        result.replaceAll(/\s+/g, " ").substring(0, 90).concat(" (truncated)"),
      );
      return new ToolMessage({
        type: "tool-result",
        output: { type: "text", value: result },
        toolName,
        toolCallId,
      });
    }),
  );
  messages.push(...toolResults);

  const answer = response.getTextContent();
  if (answer) {
    console.info(`Agent: ${answer}`);
    break;
  }
}

---

Embedding model

The EmbedingModel class provides functionality for generating vector embeddings from text.

Embedding model initialization

You can initialize an embedding model in multiple ways:

Method 1: Using the from_name method

Python

from beeai_framework.backend.embedding import EmbeddingModel

model = EmbeddingModel.from_name("ollama:nomic-embed-text")

TypeScript

import { EmbeddingModel } from "beeai-framework/backend/embedding";

const model = await EmbeddingModel.fromName("ollama:nomic-embed-text");

Method 2: Directly specifying the provider class

Python

from beeai_framework.adapters.ollama.backend import OllamaEmbeddingModel

model = OllamaEmbeddingModel("nomic-embed-text")

TypeScript

import { OpenAIEmbeddingModel } from "beeai-framework/adapters/openai/embedding";

const model = new OpenAIEmbeddingModel(
  "text-embedding-3-large",
  {
    dimensions: 512,
    maxEmbeddingsPerCall: 5,
  },
  {
    baseURL: "your_custom_endpoint",
    compatibility: "compatible",
    headers: {
      CUSTOM_HEADER: "...",
    },
  },
);

Embedding model usage

Generate embeddings for one or more text strings:

Python

from beeai_framework.backend.embedding import EmbeddingModel

model = EmbeddingModel.from_name("ollama:nomic-embed-text")

response = await model.create(["Hello world!", "Hello Bee!"])
console.log(response.values)
console.log(response.embeddings)

TypeScript

import { EmbeddingModel } from "beeai-framework/backend/embedding";

const model = await EmbeddingModel.fromName("ollama:nomic-embed-text");

const response = await model.create({
  values: ["Hello world!", "Hello Bee!"],
});
console.log(response.values);
console.log(response.embeddings);

---

Adding a Provider Using the LangChain Adapter

If your preferred provider isn’t directly supported, you can use the LangChain adapter as a bridge as long as that provider has LangChain compatibility.

Python

import asyncio
import json
import sys
import traceback
from datetime import UTC, datetime

from pydantic import BaseModel, Field

from beeai_framework.adapters.langchain.backend.chat import LangChainChatModel
from beeai_framework.backend import (
    AnyMessage,
    ChatModelNewTokenEvent,
    MessageToolResultContent,
    SystemMessage,
    ToolMessage,
    UserMessage,
)
from beeai_framework.emitter import EventMeta
from beeai_framework.errors import AbortError, FrameworkError
from beeai_framework.parsers.field import ParserField
from beeai_framework.parsers.line_prefix import LinePrefixParser, LinePrefixParserNode
from beeai_framework.tools.weather import OpenMeteoTool
from beeai_framework.utils import AbortSignal

# prevent import error for langchain_ollama (only needed in this context)
cur_dir = sys.path.pop(0)
while cur_dir in sys.path:
    sys.path.remove(cur_dir)

from langchain_ollama.chat_models import ChatOllama as LangChainOllamaChat  # noqa: E402


async def langchain_ollama_from_name() -> None:
    langchain_llm = LangChainOllamaChat(model="granite4:micro")
    llm = LangChainChatModel(langchain_llm)
    user_message = UserMessage("what states are part of New England?")
    response = await llm.run([user_message])
    print(response.get_text_content())


async def langchain_ollama_granite_from_name() -> None:
    langchain_llm = LangChainOllamaChat(model="granite4:micro")
    llm = LangChainChatModel(langchain_llm)
    user_message = UserMessage("what states are part of New England?")
    response = await llm.run([user_message])
    print(response.get_text_content())


async def langchain_ollama_sync() -> None:
    langchain_llm = LangChainOllamaChat(model="granite4:micro")
    llm = LangChainChatModel(langchain_llm)
    user_message = UserMessage("what is the capital of Massachusetts?")
    response = await llm.run([user_message])
    print(response.get_text_content())


async def langchain_ollama_stream() -> None:
    langchain_llm = LangChainOllamaChat(model="granite4:micro")
    llm = LangChainChatModel(langchain_llm)
    user_message = UserMessage("How many islands make up the country of Cape Verde?")
    response = await llm.run([user_message], stream=True)
    print(response.get_text_content())


async def langchain_ollama_stream_abort() -> None:
    langchain_llm = LangChainOllamaChat(model="granite4:micro")
    llm = LangChainChatModel(langchain_llm)
    user_message = UserMessage("What is the smallest of the Cape Verde islands?")

    try:
        response = await llm.run([user_message], stream=True, signal=AbortSignal.timeout(0.5))

        if response is not None:
            print(response.get_text_content())
        else:
            print("No response returned.")
    except AbortError as err:
        print(f"Aborted: {err}")


async def langchain_ollama_structure() -> None:
    class TestSchema(BaseModel):
        answer: str = Field(description="your final answer")

    langchain_llm = LangChainOllamaChat(model="granite4:micro")
    llm = LangChainChatModel(langchain_llm)
    user_message = UserMessage("How many islands make up the country of Cape Verde?")
    response = await llm.run([user_message], response_format=TestSchema)
    print(response.output_structured)


async def langchain_ollama_stream_parser() -> None:
    langchain_llm = LangChainOllamaChat(model="granite4:micro")
    llm = LangChainChatModel(langchain_llm)

    parser = LinePrefixParser(
        nodes={
            "test": LinePrefixParserNode(
                prefix="Prefix: ", field=ParserField.from_type(str), is_start=True, is_end=True
            )
        }
    )

    async def on_new_token(data: ChatModelNewTokenEvent, event: EventMeta) -> None:
        await parser.add(data.value.get_text_content())

    user_message = UserMessage("Produce 3 lines each starting with 'Prefix: ' followed by a sentence and a new line.")
    await llm.run([user_message], stream=True).observe(lambda emitter: emitter.on("new_token", on_new_token))
    result = await parser.end()
    print(result)


async def langchain_ollama_tool_calling() -> None:
    langchain_llm = LangChainOllamaChat(model="granite4:micro")
    llm = LangChainChatModel(langchain_llm)
    llm.parameters.stream = True
    weather_tool = OpenMeteoTool()
    messages: list[AnyMessage] = [
        SystemMessage(
            f"""You are a helpful assistant that uses tools to provide answers.
Current date is {datetime.now(tz=UTC).date()!s}
"""
        ),
        UserMessage("What is the current weather in Berlin?"),
    ]
    response = await llm.run(messages, tools=[weather_tool], tool_choice="required")
    messages.extend(response.output)
    tool_call_msg = response.get_tool_calls()[0]
    print(tool_call_msg.model_dump())
    tool_response = await weather_tool.run(json.loads(tool_call_msg.args))
    tool_response_msg = ToolMessage(
        MessageToolResultContent(
            result=tool_response.get_text_content(), tool_name=tool_call_msg.tool_name, tool_call_id=tool_call_msg.id
        )
    )
    print(tool_response_msg.to_plain())
    final_response = await llm.run([*messages, tool_response_msg], tools=[])
    print(final_response.get_text_content())


async def langchain_ollama_cloning() -> None:
    langchain_llm = LangChainOllamaChat(model="granite4:micro")
    llm = LangChainChatModel(langchain_llm)
    await llm.clone()


async def main() -> None:
    print("*" * 10, "langchain_ollama_from_name")
    await langchain_ollama_from_name()
    print("*" * 10, "langchain_ollama_granite_from_name")
    await langchain_ollama_granite_from_name()
    print("*" * 10, "langchain_ollama_sync")
    await langchain_ollama_sync()
    print("*" * 10, "langchain_ollama_stream")
    await langchain_ollama_stream()
    print("*" * 10, "langchain_ollama_stream_abort")
    await langchain_ollama_stream_abort()
    print("*" * 10, "langchain_ollama_structure")
    await langchain_ollama_structure()
    print("*" * 10, "langchain_ollama_stream_parser")
    await langchain_ollama_stream_parser()
    print("*" * 10, "langchain_ollama_tool_calling")
    await langchain_ollama_tool_calling()
    print("*" * 10, "langchain_ollama_cloning")
    await langchain_ollama_cloning()


if __name__ == "__main__":
    try:
        asyncio.run(main())
    except FrameworkError as e:
        traceback.print_exc()
        sys.exit(e.explain())

TypeScript

// NOTE: ensure you have installed following packages
// - @langchain/core
// - @langchain/cohere (or any other provider related package that you would like to use)
// List of available providers: https://js.langchain.com/v0.2/docs/integrations/chat/

import { LangChainChatModel } from "beeai-framework/adapters/langchain/backend/chat";
// @ts-expect-error package not installed
import { ChatCohere } from "@langchain/cohere";
import "dotenv/config.js";
import { ToolMessage, UserMessage } from "beeai-framework/backend/message";
import { z } from "zod";
import { ChatModelError } from "beeai-framework/backend/errors";
import { OpenMeteoTool } from "beeai-framework/tools/weather/openMeteo";

const llm = new LangChainChatModel(
  new ChatCohere({
    model: "command-r-plus",
    temperature: 0,
  }),
);

async function langchainSync() {
  const response = await llm.create({
    messages: [new UserMessage("what is the capital of Massachusetts?")],
  });
  console.info(response.getTextContent());
}

async function langchainStream() {
  const response = await llm.create({
    messages: [new UserMessage("How many islands make up the country of Cape Verde?")],
    stream: true,
  });
  console.info(response.getTextContent());
}

async function langchainAbort() {
  try {
    const response = await llm.create({
      messages: [new UserMessage("What is the smallest of the Cape Verde islands?")],
      stream: true,
      abortSignal: AbortSignal.timeout(1 * 1000),
    });
    console.info(response.getTextContent());
  } catch (err) {
    if (err instanceof ChatModelError) {
      console.log("Aborted", { err });
    }
  }
}

async function langchainStructure() {
  const response = await llm.createStructure({
    schema: z.object({
      answer: z.string({ description: "your final answer" }),
    }),
    messages: [new UserMessage("How many islands make up the country of Cape Verde?")],
  });
  console.info(response.object);
}

async function langchainToolCalling() {
  const userMessage = new UserMessage(
    `What is the current weather in Boston? Current date is ${new Date().toISOString().split("T")[0]}.`,
  );
  const weatherTool = new OpenMeteoTool({ retryOptions: { maxRetries: 3 } });
  const response = await llm.create({ messages: [userMessage], tools: [weatherTool] });
  const toolCallMsg = response.getToolCalls()[0];
  console.debug(JSON.stringify(toolCallMsg));
  const toolResponse = await weatherTool.run(toolCallMsg.input as any);
  const toolResponseMsg = new ToolMessage({
    type: "tool-result",
    output: { type: "text", value: toolResponse.getTextContent() },
    toolName: toolCallMsg.toolName,
    toolCallId: toolCallMsg.toolCallId,
  });
  console.info(toolResponseMsg.toPlain());
  const finalResponse = await llm.create({
    messages: [userMessage, ...response.messages, toolResponseMsg],
    tools: [],
  });
  console.info(finalResponse.getTextContent());
}

async function langchainDebug() {
  // Log every request
  llm.emitter.match("*", (value, event) =>
    console.debug(
      `Time: ${event.createdAt.toISOString()}`,
      `Event: ${event.name}`,
      `Data: ${JSON.stringify(value)}`,
    ),
  );

  const response = await llm.create({
    messages: [new UserMessage("Hello world!")],
  });
  console.info(response.messages[0].toPlain());
}

console.info(" langchainSync".padStart(25, "*"));
await langchainSync();
console.info(" langchainStream".padStart(25, "*"));
await langchainStream();
console.info(" langchainAbort".padStart(25, "*"));
await langchainAbort();
console.info(" langchainStructure".padStart(25, "*"));
await langchainStructure();
console.info(" langchainToolCalling".padStart(25, "*"));
await langchainToolCalling();
console.info(" langchainDebug".padStart(25, "*"));
await langchainDebug();

---

Troubleshooting

Common issues and their solutions:

1. Authentication errors: Ensure all required environment variables are set correctly
2. Model not found: Verify that the model ID is correct and available for the selected provider

---

Examples

Python

Explore reference backend implementations in Python

TypeScript

Explore reference backend implementations in TypeScript