Initial work on Vision support in text conversations and Image Editing
This is a huge patch which does some major refactoring like: - renaming "Image Generation" to "Image Creation" in most places, to better match its new command (`!bai image create`) - relocating image creation command (`!bai image` -> `!bai image create`), so it wouldn't conflict with the new image editing command (`!bai image edit`) - introducing a new image editing command (`!bai image edit`), which is meant to work only with the OpenAI provider, but doesn't fully work yet due to https://github.com/64bit/async-openai/issues/364, though a next patch will fix it - adding support for reading images off of Matrix conversations and forwarding them to text conversations. Works for OpenAI, but not for Anthropic yet (requires custom patches) and not for OpenAI-Compat (no support for images there) - relocating some utils around (base64, mime)
This commit is contained in:
@@ -3,5 +3,5 @@ pub fn heading() -> &'static str {
|
||||
}
|
||||
|
||||
pub fn intro() -> &'static str {
|
||||
"The bot can perform various tasks, such as 💬 Text Generation, 🗣️ Text-to-Speech, 🦻 Speech-to-Text, 🖌️ Image Generation, and more."
|
||||
"The bot can perform various tasks, such as 💬 Text Generation, 🗣️ Text-to-Speech, 🦻 Speech-to-Text, 🖌️ Image Creation, and more."
|
||||
}
|
||||
|
||||
15
src/strings/image_edit.rs
Normal file
15
src/strings/image_edit.rs
Normal file
@@ -0,0 +1,15 @@
|
||||
pub fn guide_how_to_proceed() -> String {
|
||||
let mut message = String::new();
|
||||
|
||||
message.push_str("💡 Respond in this thread (in any order) with:\n");
|
||||
message.push_str("- one or more images: to use the given images for creating an edit\n");
|
||||
message.push_str("- more messages: to expand on your original prompt\n");
|
||||
message.push_str(
|
||||
"- a message saying `go`: to generate an edit with the current prompt\n",
|
||||
);
|
||||
message.push_str(
|
||||
"- a message saying `again`: to generate one more image edit with the current prompt\n",
|
||||
);
|
||||
|
||||
message
|
||||
}
|
||||
@@ -8,7 +8,7 @@ pub fn guide_how_to_proceed() -> String {
|
||||
message.push_str("💡 Respond in this thread with:\n");
|
||||
message.push_str("- more messages: to expand on your original prompt\n");
|
||||
message.push_str(
|
||||
"- a message saying `again`: to generate one more image with the current prompt.\n",
|
||||
"- a message saying `again`: to generate one more image with the current prompt\n",
|
||||
);
|
||||
|
||||
message
|
||||
|
||||
@@ -5,6 +5,7 @@ pub mod error;
|
||||
pub mod global_config;
|
||||
pub mod help;
|
||||
pub mod image_generation;
|
||||
pub mod image_edit;
|
||||
pub mod introduction;
|
||||
pub mod provider;
|
||||
pub mod room_config;
|
||||
|
||||
@@ -34,26 +34,26 @@ By default, the bot will also perform 💬 Text Generation on the text. This is
|
||||
If all your messages are in the same language, you can improve accuracy & latency by configuring the language via the **🦻 Speech-to-Text / 🔤 Language** setting.
|
||||
|
||||
|
||||
### 🖌️ Image Generation
|
||||
### 🖌️ Image Creation
|
||||
|
||||
#### Generating images
|
||||
#### Creating images
|
||||
|
||||
Simply send a command like `%command_prefix% image A beautiful sunset over the ocean` and the bot will start a threaded conversation and post an image based on your prompt.
|
||||
Simply send a command like `%command_prefix% image create A beautiful sunset over the ocean` and the bot will start a threaded conversation and post an image based on your prompt.
|
||||
|
||||
You can then, respond in the same message thread with:
|
||||
|
||||
- more messages, to add more criteria to your prompt.
|
||||
- a message saying `again`, to generate one more image with the current prompt.
|
||||
|
||||
#### Generating stickers
|
||||
#### Creating stickers
|
||||
|
||||
A variation of **generating images** is to generate "sticker images".
|
||||
A variation of **creating images** is to create "sticker images".
|
||||
|
||||
To generate a sticker, send a command like `%command_prefix% sticker A huge bowl of steaming ramen with a mountain of beansprouts on top`.
|
||||
To create a sticker, send a command like `%command_prefix% sticker A huge bowl of steaming ramen with a mountain of beansprouts on top`.
|
||||
|
||||
The difference from **generating images** is that the bot will:
|
||||
The difference from **creating images** is that the bot will:
|
||||
|
||||
- generate a smaller-resolution image (`256x256`) - smaller/quicker, but still good enough for a sticker
|
||||
- create a smaller-resolution image (`256x256`) - smaller/quicker, but still good enough for a sticker
|
||||
- potentially switch to a different (cheaper or otherwise more suitable) model, if available
|
||||
- post the image directly to the room (as a reply to your message), without starting a threaded conversation
|
||||
"#;
|
||||
|
||||
Reference in New Issue
Block a user