3d3bd3c9f9
Files sent as m.file Matrix messages are now downloaded, MIME-detected, and forwarded to LLM providers alongside the conversation context, similar to how m.image is already handled. - OpenAI provider: sends files inline as base64 data URLs - Anthropic provider: skips files with a warning (library limitation) - OpenAI-compat provider: skips files with a warning (library limitation) Controller routing respects the existing prefix requirement setting. MIME detection expanded to cover PDF, text, code, and document formats. Docs updated to reflect file support and known limitations. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2.0 KiB
🤝 Handlers
Introduction
You can use different models in different rooms (e.g. OpenAI GPT-4o alongside Llama running on Groq, etc.)
You can also use different models within the same room (e.g. 💬 text-generation handled by one agent, 🦻 speech-to-text handled by another, 🗣️ text-to-speech by a 3rd, etc.)
The bot supports the following use-purposes:
- 💬 text-generation: communicating with you via text (though certain models may also process images and files)
- 🦻 speech-to-text: turning your voice messages into text
- 🗣️ text-to-speech: turning bot or users text messages into voice messages
- 🖌️ image-generation: generating images based on instructions
In a given room, each different purpose can be served by a different provider and model. This combination of provider and model configuration is called an 🤖 agent. Each purpose can be served by a different handler agent.
See a 🖼️ Screenshot of an example room configuration.
Configuring
Handlers can be configured dynamically:
- either per-room (e.g.
!bai config room set-handler text-generation room-local/openai-gpt-4o) - or globally (e.g.
!bai config global set-handler text-generation global/openai-gpt-4o)
The per-room configuration takes priority over the global configuration.
There's also a catch-all purpose that can be used as a fallback handler for messages that don't match any other handler.
💡 It's a good idea to globally-configure a powerful agent as a catch-all handler, so that the bot can always handle messages of any kind. You can then override individual handlers per room or globally.