8f86289373
This is a huge patch which does some major refactoring like: - renaming "Image Generation" to "Image Creation" in most places, to better match its new command (`!bai image create`) - relocating image creation command (`!bai image` -> `!bai image create`), so it wouldn't conflict with the new image editing command (`!bai image edit`) - introducing a new image editing command (`!bai image edit`), which is meant to work only with the OpenAI provider, but doesn't fully work yet due to https://github.com/64bit/async-openai/issues/364, though a next patch will fix it - adding support for reading images off of Matrix conversations and forwarding them to text conversations. Works for OpenAI, but not for Anthropic yet (requires custom patches) and not for OpenAI-Compat (no support for images there) - relocating some utils around (base64, mime)
3.6 KiB
🤖 Agents
An agent is an instantiation and configuration of some ☁️ provider. It can support different capabilities (text-generation, speech-to-text, etc.) depending on the provider used and on the configuration of the agent.
Agents can be set as 🤝 handlers for various purposes (text-generation, speech-to-text, etc.) globally or in specific rooms. Send a !bai config status command to see the current configuration.
Agents can be defined statically (in the server configuration) or dynamically (via commands sent to the bot).
When creating agents dynamically, you can do it per-room or globally. Globally-defined agents can be used by any authorized bot user in any room, while room-local agents can only be used in the room where they were defined.
Agent configuration (like all other configuration) is stored in the Matrix Account Data of the bot user and is potentially encrypted (if enabled in the configuration), so that your configuration data is safe even on untrusted homeservers.
Listing agents
To list all available agents: !bai agent list
Creating agents
See a 🖼️ Screenshot of the agent creation process.
To create a new agent, you need to specify the provider and an agent id of your choosing.
- Create a new agent:
- (Accessible in this room only)
!bai agent create-room-local PROVIDER_ID AGENT_ID - (Accessible in all rooms)
!bai agent create-global PROVIDER AGENT_ID - Example:
!bai agent create-room-local openai my-openai-agent
- (Accessible in this room only)
The AGENT_ID is a unique identifier for the agent. It can be any string which doesn't contain spaces and /.
Depending on where the agent is defined (within a room, globally, or statically), this id will get a prefix (e.g. room-local/, global/ or static/). The combined id (prefix + agent id) makes the full agent identifier (refered to as FULL_AGENT_IDENTIFIER in commands below).
When creating an agent, you will be given some sample YAML configuration which you can use to customize the agent's behavior.
This configuration varies depending on the ☁️ provider used and the capabilities of the agent. Based on the configuration keys you pass, certain features will be enabled or disabled. For example, if you skip the image_generation key for an OpenAI agent, it won't be able to generate images (see 🖌️ Image Creation, 🎨 Image Editing, 🫵 Sticker Creation).
After making your modifications to the sample YAML, you submit it back to the bot and the new agent will be created.
To make use of the agent, you need to 🤝 configure it as a handler for a given purpose.
Showing agent details
To show full details for a given agent: !bai agent details FULL_AGENT_IDENTIFIER
This command requires a full agent identifier (e.g. room-local/agent-id).
Deleting agents
To delete an agent: !bai agent delete FULL_AGENT_IDENTIFIER
This command requires a full agent identifier (e.g. room-local/agent-id).
Updating agents
To update a given agent's configuration: show the agent's details (current configuration), then delete it and finally re-create it.