8f86289373
This is a huge patch which does some major refactoring like: - renaming "Image Generation" to "Image Creation" in most places, to better match its new command (`!bai image create`) - relocating image creation command (`!bai image` -> `!bai image create`), so it wouldn't conflict with the new image editing command (`!bai image edit`) - introducing a new image editing command (`!bai image edit`), which is meant to work only with the OpenAI provider, but doesn't fully work yet due to https://github.com/64bit/async-openai/issues/364, though a next patch will fix it - adding support for reading images off of Matrix conversations and forwarding them to text conversations. Works for OpenAI, but not for Anthropic yet (requires custom patches) and not for OpenAI-Compat (no support for images there) - relocating some utils around (base64, mime)
2.0 KiB
🤝 Handlers
Introduction
You can use different models in different rooms (e.g. OpenAI GPT-4o alongside Llama running on Groq, etc.)
You can also use different models within the same room (e.g. 💬 text-generation handled by one agent, 🦻 speech-to-text handled by another, 🗣️ text-to-speech by a 3rd, etc.)
The bot supports the following use-purposes:
- 💬 text-generation: communicating with you via text
- 🦻 speech-to-text: turning your voice messages into text
- 🗣️ text-to-speech: turning bot or users text messages into voice messages
- 🖌️ image-generation: generating images based on instructions
In a given room, each different purpose can be served by a different provider and model. This combination of provider and model configuration is called an 🤖 agent. Each purpose can be served by a different handler agent.
See a 🖼️ Screenshot of an example room configuration.
Configuring
Handlers can be configured dynamically:
- either per-room (e.g.
!bai config room set-handler text-generation room-local/openai-gpt-4o) - or globally (e.g.
!bai config global set-handler text-generation global/openai-gpt-4o)
The per-room configuration takes priority over the global configuration.
There's also a catch-all purpose that can be used as a fallback handler for messages that don't match any other handler.
💡 It's a good idea to globally-configure a powerful agent as a catch-all handler, so that the bot can always handle messages of any kind. You can then override individual handlers per room or globally.