Initial work on Vision support in text conversations and Image Editing
This is a huge patch which does some major refactoring like: - renaming "Image Generation" to "Image Creation" in most places, to better match its new command (`!bai image create`) - relocating image creation command (`!bai image` -> `!bai image create`), so it wouldn't conflict with the new image editing command (`!bai image edit`) - introducing a new image editing command (`!bai image edit`), which is meant to work only with the OpenAI provider, but doesn't fully work yet due to https://github.com/64bit/async-openai/issues/364, though a next patch will fix it - adding support for reading images off of Matrix conversations and forwarding them to text conversations. Works for OpenAI, but not for Anthropic yet (requires custom patches) and not for OpenAI-Compat (no support for images there) - relocating some utils around (base64, mime)
This commit is contained in:
@@ -2,20 +2,22 @@ use chrono::{DateTime, Utc};
|
||||
use regex::Regex;
|
||||
|
||||
use mxlink::matrix_sdk::ruma::OwnedUserId;
|
||||
use mxlink::matrix_sdk::ruma::events::room::message::ImageMessageEventContent;
|
||||
use mxlink::mime::Mime;
|
||||
|
||||
#[derive(Clone)]
|
||||
pub struct MatrixMessage {
|
||||
pub sender_id: OwnedUserId,
|
||||
pub message_type: MatrixMessageType,
|
||||
pub message_text: String,
|
||||
pub content: MatrixMessageContent,
|
||||
pub mentioned_users: Vec<OwnedUserId>,
|
||||
pub timestamp: DateTime<Utc>,
|
||||
}
|
||||
|
||||
#[derive(Clone)]
|
||||
pub enum MatrixMessageType {
|
||||
Text,
|
||||
Notice,
|
||||
pub enum MatrixMessageContent {
|
||||
Text(String),
|
||||
Notice(String),
|
||||
Image(ImageMessageEventContent, Mime, Vec<u8>),
|
||||
}
|
||||
|
||||
#[derive(Clone)]
|
||||
|
||||
Reference in New Issue
Block a user