Initial work on Vision support in text conversations and Image Editing
This is a huge patch which does some major refactoring like: - renaming "Image Generation" to "Image Creation" in most places, to better match its new command (`!bai image create`) - relocating image creation command (`!bai image` -> `!bai image create`), so it wouldn't conflict with the new image editing command (`!bai image edit`) - introducing a new image editing command (`!bai image edit`), which is meant to work only with the OpenAI provider, but doesn't fully work yet due to https://github.com/64bit/async-openai/issues/364, though a next patch will fix it - adding support for reading images off of Matrix conversations and forwarding them to text conversations. Works for OpenAI, but not for Anthropic yet (requires custom patches) and not for OpenAI-Compat (no support for images there) - relocating some utils around (base64, mime)
This commit is contained in:
@@ -1,5 +1,5 @@
|
||||
use mxlink::matrix_sdk::ruma::events::room::message::{
|
||||
AudioMessageEventContent, MessageType, TextMessageEventContent,
|
||||
AudioMessageEventContent, ImageMessageEventContent, MessageType, TextMessageEventContent,
|
||||
};
|
||||
use mxlink::matrix_sdk::ruma::{OwnedEventId, OwnedUserId};
|
||||
|
||||
@@ -29,6 +29,7 @@ pub enum MessagePayload {
|
||||
|
||||
Text(TextMessageEventContent),
|
||||
Audio(AudioMessageEventContent),
|
||||
Image(ImageMessageEventContent),
|
||||
|
||||
Reaction {
|
||||
key: String,
|
||||
@@ -55,6 +56,7 @@ impl TryInto<MessagePayload> for MessageType {
|
||||
// For this reason, we handle all audio.
|
||||
MessagePayload::Audio(audio_content)
|
||||
}
|
||||
MessageType::Image(image_content) => MessagePayload::Image(image_content),
|
||||
other => {
|
||||
return Err(format!("Unsupported message type: {:?}", other));
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user