Add a per-room/global `sender_context_mode` setting that optionally
prefixes conversation messages with sender metadata before sending them
to the model provider.
This helps models distinguish between participants in multi-user rooms.
Three modes are supported:
- `disabled` (default, no change)
- `matrix_user_id` (prefixes with `[sender=@user:server]`)
- `matrix_user_id_and_timestamp` (adds `send_at`; Example: `[sender=@user:server sent_at=<ISO 8601>]`)
Sender context is applied to user and assistant text messages only,
skipping system prompts and non-text content.
Mixed-sender merged turns (something we intentionally do for Anthropic)
have their `sender_id` cleared to avoid misattribution.
async-openai 0.34 adds a required phase field to EasyInputMessage,
which broke our Responses API request construction.
Set phase to None because baibot does not currently model assistant commentary vs final-answer turns,
so omitting phase preserves the previous behavior while keeping the request compatible with the new crate.
Files with unrecognized MIME types (application/octet-stream) are
not supported by any LLM provider and would cause errors that
permanently break the conversation thread. Instead, represent them
as a text message describing the attachment so the LLM is still
aware a file was sent without the thread becoming unusable.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace the hand-maintained extension-to-MIME mapping with the
mime_guess crate, which was already in the dependency tree.
This covers hundreds of file extensions out of the box.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Files sent as m.file Matrix messages are now downloaded, MIME-detected,
and forwarded to LLM providers alongside the conversation context,
similar to how m.image is already handled.
- OpenAI provider: sends files inline as base64 data URLs
- Anthropic provider: skips files with a warning (library limitation)
- OpenAI-compat provider: skips files with a warning (library limitation)
Controller routing respects the existing prefix requirement setting.
MIME detection expanded to cover PDF, text, code, and document formats.
Docs updated to reflect file support and known limitations.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Add support for access tokens using MAS
* use 1.13.0
* Update dependencies
* Harden auth credential selection in matrix link init
Use the same non-empty access-token criterion for auth mode selection and bind the token directly from the branch condition.
Return explicit configuration errors for missing or empty `device_id`/`password` instead of panicking, so invalid auth config fails gracefully.
* Centralize and harden user auth config handling
Move authentication-mode resolution into typed config parsing with ConfigUserAuth,
so downstream login setup consumes validated credentials instead of re-checking raw optional fields.
Enforce explicit password-vs-token selection, validate token/device/user-id requirements in one place,
and normalize empty auth env overrides to unset values for consistent behavior across YAML and environment input.
* Add auth config unit tests
Move auth_config tests into a dedicated cfg test module file to keep production config code compact while preserving behavior coverage. The tests cover password/token mode selection, missing/both auth method rejection, missing device_id, and empty-value handling.
* Use conventional mxlink version requirement
Replace the unconventional wildcard lower-bound expression with a standard semver lower bound for readability and tooling consistency.
---------
Co-authored-by: Slavi Pantaleev <slavi@devture.com>
serde_yaml is deprecated and unmaintained. serde_yaml_ng is the community
fork with a compatible API, so this is a straightforward rename across the
codebase.
- Add mise.toml (prek 0.3.2) and .pre-commit-config.yaml with hooks for
trailing whitespace, end-of-file, YAML check, merge conflicts, large files,
cargo fmt, cargo clippy (-D warnings), and unit tests
- Add prek/mise recipes to justfile
- Run cargo fmt to fix formatting issues
- Fix all clippy warnings: collapse nested if statements, derive Default for Avatar
Document the new tools feature in README, features.md, and providers.md.
Update the `!bai providers` command output to show vision/tools support
consistently for all providers.
Based on #62 by @yeslayla which migrated the OpenAI provider to the
Responses API.
See: https://github.com/etkecc/baibot/pull/62
Sticker generation was failing when using newer GPT image models
(gpt-image-1, gpt-image-1-mini, gpt-image-1.5). The issue occurred
because stickers requested 256x256 size, but these models only support
1024x1024, 1536x1024, 1024x1536, and auto.
To reproduce, send `!bai sticker Something` to an agent configured
with a GPT image model. The error was:
invalid_request_error: Invalid value: '256x256'. Supported values
are: '1024x1024', '1024x1536', '1536x1024', and 'auto'. (param: size)
(code: invalid_value)
The fix replaces the hardcoded 256x256 size override with a
`smallest_size_possible` flag, letting each provider determine the
appropriate sticker size based on the model being used.
The `openai_compat` provider still defaults to requesting 256x256 in all cases
(regardless of model name).
This is based on the work done in https://github.com/etkecc/baibot/pull/60 by https://github.com/Fmstrat (Ben Curtis),
with various changes on top to make the code more idiomatic and flexible.
This commit squashes the following patches (newest first):
- Improve handling of `user.avatar` configuration (null & empty string being the same now) and add support for a special `keep` value
- Minor import reordering
- Simplify avatar configuration (`user.avatar.source` -> `user.avatar`)
- Combine `logo_bytes` and `mime_type` determination logic and do not fall back to default avatar if reading the custom avatar file fails
- Switch from deprecated `mime_guess::guess_mime_type(avatar_path)` to `mime_guess::from_path(avatar_path).first_or_octet_stream()`
- Relax `mime_guess` constraint and order alphabetically
- Use `mime` from `mxlink`
- Add mime-type support and switch to user.avatar.source
- (Original work by Fmstrat) Add support for custom avatars
Co-authored-by: Fmstrat <nospam@nowsci.com>
Switches `async-openai` from our own etkecc fork (0.28.1-patched) to the
official crates.io version 0.31.1.
We adapt to async-openai's types reorganization and making use of crate
features to only enable what we need.
This is a huge patch which does some major refactoring like:
- renaming "Image Generation" to "Image Creation" in most places,
to better match its new command (`!bai image create`)
- relocating image creation command (`!bai image` -> `!bai image create`),
so it wouldn't conflict with the new image editing command (`!bai image edit`)
- introducing a new image editing command (`!bai image edit`), which
is meant to work only with the OpenAI provider, but doesn't fully work yet
due to https://github.com/64bit/async-openai/issues/364, though a next patch will fix it
- adding support for reading images off of Matrix conversations and forwarding them to
text conversations. Works for OpenAI, but not for Anthropic yet
(requires custom patches) and not for OpenAI-Compat (no support for
images there)
- relocating some utils around (base64, mime)
The API reference for `response_format` says:
> This parameter isn't supported for gpt-image-1 which will always return base64-encoded images.
Related to https://github.com/etkecc/baibot/issues/40
What previously was a generic error message like:
> ⚠️ Error: An error occurred while processing your message. Please try again.
.. now becomes a much more helpful error message like:
> ⚠️ Error: There was a problem performing image-generation via the room-local/my-openai-agent agent:
>
> invalid_request_error: Invalid value: 'standard'. Supported values are: 'low', 'medium', 'high', and 'auto'. (param: quality) (code: invalid_value)
Related to https://github.com/etkecc/baibot/issues/40
🦻 is used for another purpose - to denote that a message is one coming
from speech-to-text, by:
- having the bot react to its own speech-to-text transcription message
with the 🦻 emoji when it's posted in a non-thread
- having the bot prefix its speech-to-text transcription messages with
`> 🦻` when it's posted in a thread
⏳ is already used as a progress indicator for other features, so it
makes sense to use it for indicating that speech-to-text is happening for a given audio message as well.
While 🦻 was an even more descriptive illustration of what's actually happening to the audio message
("it's being heard by the bot"), us using the 🦻 emoji for different things didn't seem good.
When speech-to-text/flow-type = `only_transcribe`, the bot will now send
text messages by default, not notices.
While notice messages may be less desirable with other bots in the room,
it's probably a better default for most people who enable "transcribe-only" mode.
This is an improvement related to https://github.com/etkecc/baibot/issues/14
Various clients (including newer versions of Element Web), do not like
it when the `body` field of the attachment is not a file name.
For images, a preview may not be shown and downloading the attachment
may suggest that the whole long text is used as a filename (which is odd).
There is value (improved accessibility, etc.)
in adding better descriptions (especially to generated images),
but given that it's currently problematic, I'm getting rid of it.
It's better and safer if we stick to using filenames.
