Add support for file attachments (m.file) in conversations

Files sent as m.file Matrix messages are now downloaded, MIME-detected,
and forwarded to LLM providers alongside the conversation context,
similar to how m.image is already handled.

- OpenAI provider: sends files inline as base64 data URLs
- Anthropic provider: skips files with a warning (library limitation)
- OpenAI-compat provider: skips files with a warning (library limitation)

Controller routing respects the existing prefix requirement setting.
MIME detection expanded to cover PDF, text, code, and document formats.
Docs updated to reflect file support and known limitations.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
Slavi Pantaleev
2026-03-20 12:07:15 +02:00
parent a25845e89e
commit 3d3bd3c9f9
15 changed files with 163 additions and 11 deletions

View File

@@ -8,7 +8,7 @@ You can also use **different models within the same room** (e.g. [💬 text-gene
The bot supports the following use-purposes:
- [💬 text-generation](../features.md#-text-generation): communicating with you via text (though certain models may "see" images as well)
- [💬 text-generation](../features.md#-text-generation): communicating with you via text (though certain models may also process images and files)
- [🦻 speech-to-text](../features.md#-speech-to-text): turning your voice messages into text
- [🗣️ text-to-speech](../features.md#️-text-to-speech): turning bot or users text messages into voice messages
- [🖌️ image-generation](../features.md#image-generation): generating images based on instructions

View File

@@ -8,7 +8,7 @@ You can also use **different models within the same room** (e.g. [💬 text-gene
The bot supports the following use-purposes:
- [💬 text-generation](#-text-generation): communicating with you via text (though certain models may "see" images as well)
- [💬 text-generation](#-text-generation): communicating with you via text (though certain models may also process images and files)
- [🦻 speech-to-text](#-speech-to-text): turning your voice messages into text
- [🗣️ text-to-speech](#%EF%B8%8F-text-to-speech): turning bot or users text messages into voice messages
- [🖌️ image-generation](#%EF%B8%8F-image-generation): generating images based on instructions
@@ -26,7 +26,7 @@ Text Generation is the bot's ability to **respond to users' messages with text**
![Screenshot of Text Generation - a user sends a message and the bot replies in a new conversation thread](./screenshots/text-generation.webp)
Some models also support vision, so you may be able to mix text and images in the same conversation.
Some models also support vision and document understanding, so you may be able to mix text, images, and files (PDFs, text documents, etc.) in the same conversation. Note that certain providers may not support all file types or may have issues with specific files (e.g. scanned/image-based PDFs). If a file is rejected by the provider, the conversation thread may become unusable — start a new thread to work around this.
In multi-user (group) rooms, to avoid disturbing the normal conversation between people, the bot is auto-configured to only respond to messages starting with the command prefix (`!bai`) or direct mentions via the [💬 Text Generation / 🗟 Prefix Requirement Type](./configuration/text-generation.md#-prefix-requirement-type) setting.

View File

@@ -11,7 +11,7 @@ This is related to the [💬 Text Generation](./features.md#-text-generation) fe
If there's a text-generation handler agent configured, the bot **may** respond to messages sent in the room.
Some models also support vision, so you may be able to mix text and images in the same conversation.
Some models also support vision and document understanding, so you may be able to mix text, images, and files (PDFs, text documents, etc.) in the same conversation.
See screenshots of: