This is a huge patch which does some major refactoring like:
- renaming "Image Generation" to "Image Creation" in most places,
to better match its new command (`!bai image create`)
- relocating image creation command (`!bai image` -> `!bai image create`),
so it wouldn't conflict with the new image editing command (`!bai image edit`)
- introducing a new image editing command (`!bai image edit`), which
is meant to work only with the OpenAI provider, but doesn't fully work yet
due to https://github.com/64bit/async-openai/issues/364, though a next patch will fix it
- adding support for reading images off of Matrix conversations and forwarding them to
text conversations. Works for OpenAI, but not for Anthropic yet
(requires custom patches) and not for OpenAI-Compat (no support for
images there)
- relocating some utils around (base64, mime)
The API reference for `response_format` says:
> This parameter isn't supported for gpt-image-1 which will always return base64-encoded images.
Related to https://github.com/etkecc/baibot/issues/40
What previously was a generic error message like:
> ⚠️ Error: An error occurred while processing your message. Please try again.
.. now becomes a much more helpful error message like:
> ⚠️ Error: There was a problem performing image-generation via the room-local/my-openai-agent agent:
>
> invalid_request_error: Invalid value: 'standard'. Supported values are: 'low', 'medium', 'high', and 'auto'. (param: quality) (code: invalid_value)
Related to https://github.com/etkecc/baibot/issues/40
🦻 is used for another purpose - to denote that a message is one coming
from speech-to-text, by:
- having the bot react to its own speech-to-text transcription message
with the 🦻 emoji when it's posted in a non-thread
- having the bot prefix its speech-to-text transcription messages with
`> 🦻` when it's posted in a thread
⏳ is already used as a progress indicator for other features, so it
makes sense to use it for indicating that speech-to-text is happening for a given audio message as well.
While 🦻 was an even more descriptive illustration of what's actually happening to the audio message
("it's being heard by the bot"), us using the 🦻 emoji for different things didn't seem good.
When speech-to-text/flow-type = `only_transcribe`, the bot will now send
text messages by default, not notices.
While notice messages may be less desirable with other bots in the room,
it's probably a better default for most people who enable "transcribe-only" mode.
This is an improvement related to https://github.com/etkecc/baibot/issues/14
Various clients (including newer versions of Element Web), do not like
it when the `body` field of the attachment is not a file name.
For images, a preview may not be shown and downloading the attachment
may suggest that the whole long text is used as a filename (which is odd).
There is value (improved accessibility, etc.)
in adding better descriptions (especially to generated images),
but given that it's currently problematic, I'm getting rid of it.
It's better and safer if we stick to using filenames.
