Compare commits

...

20 Commits

Author SHA1 Message Date
Slavi Pantaleev
d32380e56b Release 1.1.0 2024-09-21 17:38:28 +03:00
Slavi Pantaleev
c8c5e0e540 Split build-container-image into build-container-image-{debug,release}
This allows `run-in-container` to default to using the much faster
`build-container-image-debug`.
2024-09-21 14:28:38 +00:00
Slavi Pantaleev
2a5a2d6a4d Add support for prompt variables (bot name, date/time, model id)
Fixes https://github.com/etkecc/baibot/issues/10

This also includes them in the default prompts (for newly-created agents),
so that people can get a better experience out of the box.
2024-09-21 14:28:38 +00:00
Slavi Pantaleev
0ee663ee92 Add missing recipe description for run-in-container 2024-09-21 14:28:38 +00:00
Slavi Pantaleev
5de7559ed6 Explicitly set up QEMU to try and work around CI trouble
Related to https://github.com/etkecc/baibot/issues/2

We've had a few more instances of the same issue since then.
2024-09-21 14:28:38 +00:00
Slavi Pantaleev
bba5b7996b Make run-in-container explicitly run the :latest container image 2024-09-19 18:21:42 +03:00
Slavi Pantaleev
354063abb7 Shave off ~20MB from resulting container image by purging apt cache 2024-09-19 18:18:56 +03:00
Slavi Pantaleev
d59e6b59c2 Upgrade Rust compiler in container image (1.80.1 -> 1.81.0) 2024-09-19 18:18:21 +03:00
Slavi Pantaleev
e0ae874e3a Improve Access documentation page
[skip ci]
2024-09-19 16:22:59 +03:00
Slavi Pantaleev
3c91b50d60 Improve "Room-local agent managers" section 2024-09-19 16:11:55 +03:00
Slavi Pantaleev
5f7b1c9e38 Remove useless use of format!()
[skip ci]
2024-09-19 14:07:35 +03:00
Slavi Pantaleev
2dbd600d05 Release 1.0.6 2024-09-19 14:03:00 +03:00
Slavi Pantaleev
f6cc8363d1 Allow regular (unprivileged) users to see providers help and adapt it to them 2024-09-19 13:39:52 +03:00
Slavi Pantaleev
e3b07aa291 Improve bot make-use-of-me steps in introduction message
This especially improves the introduction for when the bot has no
configured agents.
2024-09-19 12:24:03 +03:00
Slavi Pantaleev
324c8a976f Relocate some access check code 2024-09-19 10:02:23 +03:00
Slavi Pantaleev
fb1f16aa40 Add missing new line before closing code-block 2024-09-18 09:15:23 +03:00
Slavi Pantaleev
012069891d Adjust incorrect comment
[skip ci]
2024-09-18 08:53:33 +03:00
Slavi Pantaleev
3b7c28a55e Release 1.0.5 2024-09-14 10:47:56 +03:00
Slavi Pantaleev
3b25b92a81 Implement more fine-grained typing notices sending
The previous approach (implemented in dd1dd78312) was simple
(send typing notices for as long as the "controller" is running),
but this proved to be overly simplistic and unable to handle edge-cases:

- in multi-user rooms (or rooms with a prefix requirement), the bot
  used to send a typing notice while "working", but its work consisted
  of ignoring the message. So it then sent a "not typing" notice.
  This is wasteful and otherwise problematic - certain clients (like nheko)
  do not handle this "race" well.

- certain reactions (anything other than 🗣️ right now) are meant to be
  ignored. There's no point in doing the same "typing / not typing"
  dance

- there are other instances where the bot may do work, but doesn't (due
  to configuration or lack of capabilities)

This new more fine-grained implementation of typing notices aims to:

- only send a typing notice if actual "slow work" will be done

- avoid stopping & restarting typing notices (wasteful) if a chain of work is to
  be performed (processing voice messages and doing speech-to-text +
  text-generation + ...). Rather, maintaining typing notice sending
  throughout
2024-09-14 10:39:20 +03:00
Slavi Pantaleev
509f683365 Fix typo 2024-09-14 09:30:58 +03:00
55 changed files with 516 additions and 249 deletions

View File

@@ -24,6 +24,10 @@ jobs:
name: Build and Publish name: Build and Publish
runs-on: self-hosted runs-on: self-hosted
steps: steps:
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
with:
platforms: arm64
- name: Set up Docker Buildx - name: Set up Docker Buildx
uses: docker/setup-buildx-action@v1 uses: docker/setup-buildx-action@v1
- name: Login to ghcr.io - name: Login to ghcr.io

View File

@@ -1,3 +1,27 @@
# (2024-09-21) Version 1.1.0
- (**Feature**) Adds support for [prompt variables](./docs/configuration/text-generation.md#️-prompt-override) (date/time, bot name, model id) ([2a5a2d6a4d](https://github.com/etkecc/baibot/commit/2a5a2d6a4dbf5fd7cb504ac07d4187fdc32ae395))
- (**Improvement**) [Dockerfile](./Dockerfile) changes to produce ~20MB smaller container images ([354063abb7](https://github.com/etkecc/baibot/commit/354063abb79035069bd3b26c53214874e9cdd95d))
- (**Improvement**) [Dockerfile](./Dockerfile) changes to optimize local (debug) runs in a container ([c8c5e0e540](https://github.com/etkecc/baibot/commit/c8c5e0e540ab981e849452eb3ddb0378105e1fc6))
- (**Improvement**) CI changes to try and work around multi-arch image issues like [this one](https://github.com/etkecc/baibot/issues/2) ([5de7559ed6](https://github.com/etkecc/baibot/commit/5de7559ed685a41c22dfc12283681f02f4c2ee00))
# (2024-09-19) Version 1.0.6
Improvements to:
- messages sent by the bot - better onboarding flow, especially when no agents have been created yet
- documentation pages
# (2024-09-14) Version 1.0.5
Further [improves](https://github.com/etkecc/baibot/commit/3b25b92a81a05ebaf1c6dbabf675fbfbe6c9f418) the typing notification logic, so that it tolerates edge cases better.
# (2024-09-14) Version 1.0.4 # (2024-09-14) Version 1.0.4
[Improves](https://github.com/etkecc/baibot/commit/dd1dd78312e3db7f92b37fb3b4750fbe35de7115) the typing notification logic. [Improves](https://github.com/etkecc/baibot/commit/dd1dd78312e3db7f92b37fb3b4750fbe35de7115) the typing notification logic.

16
Cargo.lock generated
View File

@@ -297,12 +297,13 @@ dependencies = [
[[package]] [[package]]
name = "baibot" name = "baibot"
version = "1.0.4" version = "1.1.0"
dependencies = [ dependencies = [
"anthropic-rs", "anthropic-rs",
"anyhow", "anyhow",
"async-openai 0.24.0", "async-openai 0.24.0",
"base64 0.22.1", "base64 0.22.1",
"chrono",
"etke_openai_api_rust", "etke_openai_api_rust",
"matrix-sdk", "matrix-sdk",
"mxidwc", "mxidwc",
@@ -506,6 +507,15 @@ dependencies = [
"zeroize", "zeroize",
] ]
[[package]]
name = "chrono"
version = "0.4.38"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a21f936df1771bf62b77f047b726c4625ff2e8aa607c01ec06e5a05bd8463401"
dependencies = [
"num-traits",
]
[[package]] [[package]]
name = "cipher" name = "cipher"
version = "0.4.4" version = "0.4.4"
@@ -2122,9 +2132,9 @@ dependencies = [
[[package]] [[package]]
name = "mxlink" name = "mxlink"
version = "1.2.0" version = "1.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "195f0bd9764068de3e3e446bbac81a54489ff51674d80e387057b805c7dab96c" checksum = "f1aa513435471af3e1131dc18a1f4eb2e6fb42c11308f1be9b4f0316b3809e32"
dependencies = [ dependencies = [
"base64 0.22.1", "base64 0.22.1",
"chacha20poly1305", "chacha20poly1305",

View File

@@ -7,7 +7,7 @@ license = "AGPL-3.0-or-later"
readme = "README.md" readme = "README.md"
keywords = ["matrix", "chat", "bot", "AI", "LLM"] keywords = ["matrix", "chat", "bot", "AI", "LLM"]
include = ["/etc/assets/baibot-torso-768.png", "/src", "/README.md", "/CHANGELOG.md", "/LICENSE"] include = ["/etc/assets/baibot-torso-768.png", "/src", "/README.md", "/CHANGELOG.md", "/LICENSE"]
version = "1.0.4" version = "1.1.0"
edition = "2021" edition = "2021"
[lib] [lib]
@@ -19,10 +19,11 @@ anthropic-rs = "0.1.*"
anyhow = "1.0.*" anyhow = "1.0.*"
async-openai = "0.24.*" async-openai = "0.24.*"
base64 = "0.22.*" base64 = "0.22.*"
chrono = { version = "0.4.*", default-features = false, features = ["std", "now"] }
# We'd rather not depend on this, but we cannot use the ruma-events EventContent macro without it. # We'd rather not depend on this, but we cannot use the ruma-events EventContent macro without it.
matrix-sdk = { version = "0.7.1", default-features = false } matrix-sdk = { version = "0.7.1", default-features = false }
mxidwc = "1.0.*" mxidwc = "1.0.*"
mxlink = "1.2.*" mxlink = ">=1.2.1"
etke_openai_api_rust = "0.1.*" etke_openai_api_rust = "0.1.*"
quick_cache = "0.6.*" quick_cache = "0.6.*"
regex = "1.10.*" regex = "1.10.*"

View File

@@ -4,7 +4,7 @@
# # # #
####################################### #######################################
FROM docker.io/rust:1.80.1-slim-bookworm AS build FROM docker.io/rust:1.81.0-slim-bookworm AS build
RUN apt-get update && apt-get install -y build-essential pkg-config libssl-dev libsqlite3-dev RUN apt-get update && apt-get install -y build-essential pkg-config libssl-dev libsqlite3-dev
@@ -15,14 +15,23 @@ WORKDIR /app
COPY . /app COPY . /app
ARG RELEASE_BUILD=true
RUN --mount=type=cache,target=/cargo,sharing=locked \ RUN --mount=type=cache,target=/cargo,sharing=locked \
--mount=type=cache,target=/target,sharing=locked \ --mount=type=cache,target=/target,sharing=locked \
cargo build --release if [ "$RELEASE_BUILD" = "true" ]; then \
cargo build --release; \
else \
cargo build; \
fi
# Move it out of the mounted cache, so we can copy it in the next stage. # Move it out of the mounted cache, so we can copy it in the next stage.
RUN --mount=type=cache,target=/target,sharing=locked \ RUN --mount=type=cache,target=/target,sharing=locked \
cp /target/release/baibot /baibot if [ "$RELEASE_BUILD" = "true" ]; then \
cp /target/release/baibot /baibot; \
else \
cp /target/debug/baibot /baibot; \
fi
####################################### #######################################
# # # #
@@ -32,7 +41,9 @@ RUN --mount=type=cache,target=/target,sharing=locked \
FROM docker.io/debian:bookworm-slim FROM docker.io/debian:bookworm-slim
RUN apt-get update && apt-get install -y ca-certificates sqlite3 RUN apt-get update && apt-get install -y ca-certificates sqlite3 && \
apt-get clean && \
rm -rf /var/lib/apt/lists/*
WORKDIR /app WORKDIR /app

View File

@@ -5,17 +5,21 @@ This bot employs access control to decide who can use its services and manage it
### 👋 Joining rooms ### 👋 Joining rooms
The bot automatically joins rooms when invited by someone considered a bot [user](#-users). The bot automatically joins rooms only when invited by someone considered a bot [👥 user](#-users).
### 👥 Users ### 👥 Users
The bot will ignore messages (and room invitations) from unallowed users.
Users can **use all the bot's [features](./features.md)** ([💬 Text Generation](./features.md#-text-generation), [🦻 Speech-to-Text](./features.md#-speech-to-text), etc.), but **cannot manage the bot's configuration**.
The bot can be used by users that match some [dynamically](./configuration/README.md#dynamic-configuration) configured [Matrix user id](https://spec.matrix.org/v1.11/#users) patterns. The bot can be used by users that match some [dynamically](./configuration/README.md#dynamic-configuration) configured [Matrix user id](https://spec.matrix.org/v1.11/#users) patterns.
Users:
- ✅ can **invite the bot to rooms**
- ✅ can **use all the bot's [features](./features.md)** ([💬 Text Generation](./features.md#-text-generation), [🦻 Speech-to-Text](./features.md#-speech-to-text), etc.) by sending room messages
- ✅ can **change the bot's configuration in a room** (e.g. `!bai config room ...` commands)
- ❌ cannot **change the bot's global configuration** (e.g. `!bai config global ...` commands)
- ❌ cannot **create new [🤖 Agents](./agents.md)** (neither in rooms, nor globally). See [💼 Room-local agent managers](#-room-local-agent-managers) for controlling which users can create agents.
The following commands are available: The following commands are available:
- **Show** the currently allowed users: `!bai access users` - **Show** the currently allowed users: `!bai access users`
- **Set** the list of allowed users: `!bai access set-users SPACE_SEPARATED_PATTERNS` - **Set** the list of allowed users: `!bai access set-users SPACE_SEPARATED_PATTERNS`
@@ -27,6 +31,8 @@ Example patterns: `@*:example.com @*:another.com @someone:company.org`
Administrators can **manage the bot's configuration and access control**. Administrators can **manage the bot's configuration and access control**.
Administrators are [👥 Users](#-users) and [💼 Room-local agent managers](#-room-local-agent-managers) implicitly, so they inherit all their permissions.
The bot can be administrated by users that match some [statically](./configuration/README.md#static-configuration) configured [Matrix user id](https://spec.matrix.org/v1.11/#users) patterns. The bot can be administrated by users that match some [statically](./configuration/README.md#static-configuration) configured [Matrix user id](https://spec.matrix.org/v1.11/#users) patterns.
Administrators cannot be changed without adjusting the bot's configuration on the server. Administrators cannot be changed without adjusting the bot's configuration on the server.
@@ -35,12 +41,11 @@ Administrators cannot be changed without adjusting the bot's configuration on th
### 💼 Room-local agent managers ### 💼 Room-local agent managers
Room-local agent managers are users privileged to **create their own [agents](./agents.md)** (see `!bai agent`) in rooms. Room-local agent managers are users privileged to **create their own [agents](./agents.md)** (see `!bai agent`) in rooms.
Letting regular users create agents which contact arbitrary network services **may be a security issue**.
No room-local agent manager patterns are configured, so new agents can only be created by administrators. **⚠️ WARNING**: Letting regular users create agents which contact arbitrary network services **may be a security issue**.
The following commands are available: The following commands are available:
- **Show** the currently allowed users: `!bai access room-local-agent-managers` - **Show** the currently allowed users: `!bai access room-local-agent-managers`
- **Set** the list of allowed users: `!bai access set-room-local-agent-managers SPACE_SEPARATED_PATTERNS` - **Set** the list of allowed users: `!bai access set-room-local-agent-managers SPACE_SEPARATED_PATTERNS`
Example patterns: `@*:synapse.127.0.0.1.nip.io @*:another.com @someone:company.org` Example patterns: `@*:example.com @*:another.com @someone:company.org`

View File

@@ -68,6 +68,19 @@ Where appropriate, you'll mention best practices and common pitfalls.
A prompt override can also be set globally, see [🛠️ Room Settings](./README.md#room-settings). A prompt override can also be set globally, see [🛠️ Room Settings](./README.md#room-settings).
Prompts may contain the following **placeholder variables** which will be replaced *every time* the bot is interacted with:
| Placeholder | Description | Example |
|---------------------------|-------------|---------|
| `{{ baibot_name }}` | Name of the bot as configured in the `user.name` field in the [Static configuration](./README.md#static-configuration) | `Baibot` |
| `{{ baibot_model_id }}` | Text-Generation model ID as configured in the [🤖 agent](../agents.md)'s configuration | `gpt-4o` |
| `{{ baibot_now_utc }}` | Current date and time in UTC | `2024-09-20 (Friday), 14:26:42 UTC (local timezone/time: unknown)` |
Here's a prompt that combines some of the above variables:
> You are a brief, but helpful bot called {{ baibot_name }} powered by the {{ baibot_model_id }} model. The date/time now is: {{ baibot_now_utc }}."
### 🌡️ Temperature Override ### 🌡️ Temperature Override
You can override the [temperature](https://blogs.novita.ai/what-are-large-language-model-settings-temperature-top-p-and-max-tokens/#what-is-llm-temperature) (randomness / creativity) parameter configured at the [🤖 agent](../agents.md) level. You can override the [temperature](https://blogs.novita.ai/what-are-large-language-model-settings-temperature-top-p-and-max-tokens/#what-is-llm-temperature) (randomness / creativity) parameter configured at the [🤖 agent](../agents.md) level.

View File

@@ -15,8 +15,16 @@
We provide prebuilt container images for the `amd64` and `arm64` architectures, so **you don't necessarily need to build images yourself** and can jump to [Running in a container](#-running-in-a-container). We provide prebuilt container images for the `amd64` and `arm64` architectures, so **you don't necessarily need to build images yourself** and can jump to [Running in a container](#-running-in-a-container).
If you nevertheless wish to build a container image yourself, you can do so by running `just build-container-image`. If you nevertheless wish to build a container image yourself, you can do so by running:
This will build and tag your container image as `localhost/baibot:latest`.
- (recommended) `just build-container-image-release` to build a release version of the container image
- or `just build-container-image-debug` to build a debug version of the container image
Debug images are faster to build but are larger in size.
Release images are ~5x smaller in size, but are slower to build.
Both of these commands will build and tag your container image as `localhost/baibot:latest`.
### 🐋 Running in a container ### 🐋 Running in a container

View File

@@ -23,17 +23,20 @@ The list of supported providers is below.
### How to choose a provider ### How to choose a provider
If you're not sure which provider to start with, we **recommend [OpenAI](#openai)** as it's the most popular and has the **widest range of capabilities**: [💬 text-generation](./features.md#-text-generation), [🖌️ image-generation](./features.md#️-image-generation), [🦻 speech-to-text](./features.md#-speech-to-text), [🗣️ text-to-speech](./features.md#️-text-to-speech). If you're not sure which provider to start with, **we recommend [OpenAI](#openai)** as it's the most popular and has the **widest range of capabilities**: [💬 text-generation](./features.md#-text-generation), [🖌️ image-generation](./features.md#️-image-generation), [🦻 speech-to-text](./features.md#-speech-to-text), [🗣️ text-to-speech](./features.md#️-text-to-speech).
You don't need to choose just one though. The bot supports [mixing & matching models](./features.md#-mixing--matching-models), so you can use multiple providers at the same time. You don't need to choose just one though. The bot supports [mixing & matching models](./features.md#-mixing--matching-models), so you can use multiple providers at the same time.
### How to use a provider ### How to use a provider
- sign up for it 1. 📝 **Sign up for it**
- obtain an API key
- [create a new agent](./agents.md#creating-agents) 2. 🔑 **Obtain an API key**
- set it as a handler for some types of messages (see [Mixing & matching models](./features.md#-mixing--matching-models)) for a specific room or globally
3. 🤖 **Create one or more agents** in a given room or globally. Next to each provider in the [list below](#supported-providers) you'll see **🗲 Quick start** commands, but you may also refer to the [agent creation guide](./agents.md#creating-agents).
4. 🤝 **Set the new agent as a handler** for a given use-purpose like text-generation, image-generation, etc. The agent creation wizard will tell you how, but you may also refer to the [🤝 Handlers](./configuration/handlers.md) guide.
### Supported providers ### Supported providers

View File

@@ -2,7 +2,7 @@ base_url: https://api.anthropic.com/v1
api_key: YOUR_API_KEY_HERE api_key: YOUR_API_KEY_HERE
text_generation: text_generation:
model_id: claude-3-5-sonnet-20240620 model_id: claude-3-5-sonnet-20240620
prompt: You are a brief, but helpful bot. prompt: "You are a brief, but helpful bot called {{ baibot_name }} powered by the {{ baibot_model_id }} model. The date/time now is: {{ baibot_now_utc }}."
temperature: 1.0 temperature: 1.0
max_response_tokens: 8192 max_response_tokens: 8192
max_context_tokens: 204800 max_context_tokens: 204800

View File

@@ -2,7 +2,7 @@ base_url: https://api.groq.com/openai/v1
api_key: YOUR_API_KEY_HERE api_key: YOUR_API_KEY_HERE
text_generation: text_generation:
model_id: llama3-70b-8192 model_id: llama3-70b-8192
prompt: You are a brief, but helpful bot. prompt: "You are a brief, but helpful bot called {{ baibot_name }} powered by the {{ baibot_model_id }} model. The date/time now is: {{ baibot_now_utc }}."
temperature: 1.0 temperature: 1.0
max_response_tokens: 4096 max_response_tokens: 4096
max_context_tokens: 131072 max_context_tokens: 131072

View File

@@ -2,7 +2,7 @@ base_url: http://my-localai-self-hosted-service:8080/v1
api_key: YOUR_API_KEY_HERE api_key: YOUR_API_KEY_HERE
text_generation: text_generation:
model_id: gpt-4 model_id: gpt-4
prompt: You are a brief, but helpful bot. prompt: "You are a brief, but helpful bot called {{ baibot_name }} powered by the {{ baibot_model_id }} model. The date/time now is: {{ baibot_now_utc }}."
temperature: 1.0 temperature: 1.0
max_response_tokens: 4096 max_response_tokens: 4096
max_context_tokens: 128000 max_context_tokens: 128000

View File

@@ -2,7 +2,7 @@ base_url: https://api.mistral.ai/v1
api_key: YOUR_API_KEY_HERE api_key: YOUR_API_KEY_HERE
text_generation: text_generation:
model_id: mistral-large-latest model_id: mistral-large-latest
prompt: You are a brief, but helpful bot. prompt: "You are a brief, but helpful bot called {{ baibot_name }} powered by the {{ baibot_model_id }} model. The date/time now is: {{ baibot_now_utc }}."
temperature: 1.0 temperature: 1.0
max_response_tokens: 4096 max_response_tokens: 4096
max_context_tokens: 128000 max_context_tokens: 128000

View File

@@ -2,7 +2,7 @@ base_url: http://my-ollama-self-hosted-service:11434/v1
api_key: YOUR_API_KEY_HERE api_key: YOUR_API_KEY_HERE
text_generation: text_generation:
model_id: gemma2:2b model_id: gemma2:2b
prompt: You are a brief, but helpful bot. prompt: "You are a brief, but helpful bot called {{ baibot_name }} powered by the {{ baibot_model_id }} model. The date/time now is: {{ baibot_now_utc }}."
temperature: 1.0 temperature: 1.0
max_response_tokens: 4096 max_response_tokens: 4096
max_context_tokens: 128000 max_context_tokens: 128000

View File

@@ -2,7 +2,7 @@ base_url: ''
api_key: YOUR_API_KEY_HERE api_key: YOUR_API_KEY_HERE
text_generation: text_generation:
model_id: some-model model_id: some-model
prompt: You are a brief, but helpful bot. prompt: "You are a brief, but helpful bot called {{ baibot_name }} powered by the {{ baibot_model_id }} model. The date/time now is: {{ baibot_now_utc }}."
temperature: 1.0 temperature: 1.0
max_response_tokens: 4096 max_response_tokens: 4096
max_context_tokens: 128000 max_context_tokens: 128000

View File

@@ -2,7 +2,7 @@ base_url: https://api.openai.com/v1
api_key: YOUR_API_KEY_HERE api_key: YOUR_API_KEY_HERE
text_generation: text_generation:
model_id: gpt-4o-2024-08-06 model_id: gpt-4o-2024-08-06
prompt: You are a brief, but helpful bot. prompt: "You are a brief, but helpful bot called {{ baibot_name }} powered by the {{ baibot_model_id }} model. The date/time now is: {{ baibot_now_utc }}."
temperature: 1.0 temperature: 1.0
max_response_tokens: 16384 max_response_tokens: 16384
max_context_tokens: 128000 max_context_tokens: 128000

View File

@@ -2,7 +2,7 @@ base_url: https://openrouter.ai/api/v1
api_key: YOUR_API_KEY_HERE api_key: YOUR_API_KEY_HERE
text_generation: text_generation:
model_id: mattshumer/reflection-70b:free model_id: mattshumer/reflection-70b:free
prompt: You are a brief, but helpful bot. prompt: "You are a brief, but helpful bot called {{ baibot_name }} powered by the {{ baibot_model_id }} model. The date/time now is: {{ baibot_now_utc }}."
temperature: 1.0 temperature: 1.0
max_response_tokens: 2048 max_response_tokens: 2048
max_context_tokens: 8192 max_context_tokens: 8192

View File

@@ -2,7 +2,7 @@ base_url: https://api.together.xyz/v1
api_key: YOUR_API_KEY_HERE api_key: YOUR_API_KEY_HERE
text_generation: text_generation:
model_id: meta-llama/Meta-Llama-3.1-405B-Instruct-Turbo model_id: meta-llama/Meta-Llama-3.1-405B-Instruct-Turbo
prompt: You are a brief, but helpful bot. prompt: "You are a brief, but helpful bot called {{ baibot_name }} powered by the {{ baibot_model_id }} model. The date/time now is: {{ baibot_now_utc }}."
temperature: 1.0 temperature: 1.0
max_response_tokens: 2048 max_response_tokens: 2048
max_context_tokens: 8192 max_context_tokens: 8192

View File

@@ -73,7 +73,7 @@ agents:
# api_key: "" # api_key: ""
# text_generation: # text_generation:
# model_id: gpt-4o-2024-08-06 # model_id: gpt-4o-2024-08-06
# prompt: You are a brief, but helpful bot. # prompt: "You are a brief, but helpful bot called {{ baibot_name }} powered by the {{ baibot_model_id }} model. The date/time now is: {{ baibot_now_utc }}."
# temperature: 1.0 # temperature: 1.0
# max_response_tokens: 16384 # max_response_tokens: 16384
# max_context_tokens: 128000 # max_context_tokens: 128000
@@ -97,7 +97,7 @@ agents:
# api_key: null # api_key: null
# text_generation: # text_generation:
# model_id: gpt-4 # model_id: gpt-4
# prompt: You are a brief, but helpful bot. # prompt: "You are a brief, but helpful bot called {{ baibot_name }} powered by the {{ baibot_model_id }} model. The date/time now is: {{ baibot_now_utc }}."
# temperature: 1.0 # temperature: 1.0
# max_response_tokens: 16384 # max_response_tokens: 16384
# max_context_tokens: 128000 # max_context_tokens: 128000

View File

@@ -13,7 +13,8 @@ run-locally *extra_args: app-local-prepare
BAIBOT_PERSISTENCE_DATA_DIR_PATH={{ justfile_directory() }}/var/app/local/data \ BAIBOT_PERSISTENCE_DATA_DIR_PATH={{ justfile_directory() }}/var/app/local/data \
cargo run -- {{ extra_args }} cargo run -- {{ extra_args }}
run-in-container *extra_args: app-container-prepare build-container-image # Builds and runs the bot in a container
run-in-container *extra_args: app-container-prepare build-container-image-debug
/usr/bin/env docker run \ /usr/bin/env docker run \
-it \ -it \
--rm \ --rm \
@@ -25,7 +26,7 @@ run-in-container *extra_args: app-container-prepare build-container-image
--env BAIBOT_PERSISTENCE_DATA_DIR_PATH=/data \ --env BAIBOT_PERSISTENCE_DATA_DIR_PATH=/data \
--mount type=bind,src={{ justfile_directory() }}/var/app/container/config.yml,dst=/app/config.yml,ro \ --mount type=bind,src={{ justfile_directory() }}/var/app/container/config.yml,dst=/app/config.yml,ro \
--mount type=bind,src={{ justfile_directory() }}/var/app/container/data,dst=/data \ --mount type=bind,src={{ justfile_directory() }}/var/app/container/data,dst=/data \
{{ container_image_name }} {{ extra_args }} {{ container_image_name }}:latest {{ extra_args }}
# Runs tests # Runs tests
test *extra_args: test *extra_args:
@@ -38,9 +39,16 @@ build-debug *extra_args:
# Builds an optimized release binary (target/release/*) # Builds an optimized release binary (target/release/*)
build-release *extra_args: (build-debug "--release") build-release *extra_args: (build-debug "--release")
# Builds a container image # Builds a container image (debug mode)
build-container-image tag='latest': build-container-image-debug tag='latest': (_build-container-image "false" tag)
# Builds a container image (release mode)
build-container-image-release tag='latest': (_build-container-image "true" tag)
_build-container-image release_build tag:
/usr/bin/env docker build \ /usr/bin/env docker build \
--build-arg RELEASE_BUILD={{ release_build }} \
-f {{ justfile_directory() }}/Dockerfile \
-t {{ container_image_name }}:{{ tag }} \ -t {{ container_image_name }}:{{ tag }} \
. .

View File

@@ -19,3 +19,7 @@ pub use instantiation::Result as AgentInstantiationResult;
pub use provider::{AgentProvider, AgentProviderInfo, ControllerTrait}; pub use provider::{AgentProvider, AgentProviderInfo, ControllerTrait};
pub use purpose::AgentPurpose; pub use purpose::AgentPurpose;
pub(super) fn default_prompt() -> &'static str {
"You are a brief, but helpful bot called {{ baibot_name }} powered by the {{ baibot_model_id }} model. The date/time now is: {{ baibot_now_utc }}."
}

View File

@@ -2,7 +2,7 @@ use serde::{Deserialize, Serialize};
use anthropic_rs::models::claude::ClaudeModel; use anthropic_rs::models::claude::ClaudeModel;
use crate::agent::provider::ConfigTrait; use crate::agent::{default_prompt, provider::ConfigTrait};
#[derive(Debug, Clone, Serialize, Deserialize)] #[derive(Debug, Clone, Serialize, Deserialize)]
pub struct Config { pub struct Config {
@@ -58,7 +58,7 @@ impl Default for TextGenerationConfig {
fn default() -> Self { fn default() -> Self {
Self { Self {
model_id: default_text_model_id(), model_id: default_text_model_id(),
prompt: Some("You are a brief, but helpful bot.".to_owned()), prompt: Some(default_prompt().to_owned()),
temperature: super::super::default_temperature(), temperature: super::super::default_temperature(),
max_response_tokens: 8192, max_response_tokens: 8192,
max_context_tokens: 204_800, max_context_tokens: 204_800,

View File

@@ -95,11 +95,12 @@ impl ControllerTrait for Controller {
)); ));
}; };
let prompt_text = params let prompt_text = params.prompt_variables.format(
.prompt_override params
.unwrap_or(self.text_generation_prompt().unwrap_or("".to_owned())) .prompt_override
.trim() .unwrap_or(self.text_generation_prompt().unwrap_or("".to_owned()))
.to_owned(); .trim(),
);
let prompt_message = if prompt_text.is_empty() { let prompt_message = if prompt_text.is_empty() {
None None
@@ -225,20 +226,25 @@ impl ControllerTrait for Controller {
} }
} }
fn text_generation_prompt(&self) -> Option<String> { fn text_generation_model_id(&self) -> Option<String> {
let Some(text_generation_config) = &self.config.text_generation else { self.config
return None; .text_generation
}; .as_ref()
.map(|config| config.model_id.to_owned())
}
text_generation_config.prompt.clone() fn text_generation_prompt(&self) -> Option<String> {
self.config
.text_generation
.as_ref()
.and_then(|config| config.prompt.clone())
} }
fn text_generation_temperature(&self) -> Option<f32> { fn text_generation_temperature(&self) -> Option<f32> {
let Some(text_generation_config) = &self.config.text_generation else { self.config
return None; .text_generation
}; .as_ref()
.map(|config| config.temperature)
Some(text_generation_config.temperature)
} }
fn text_to_speech_voice(&self) -> Option<String> { fn text_to_speech_voice(&self) -> Option<String> {

View File

@@ -13,6 +13,8 @@ pub trait ControllerTrait {
fn ping(&self) -> impl std::future::Future<Output = anyhow::Result<PingResult>> + Send; fn ping(&self) -> impl std::future::Future<Output = anyhow::Result<PingResult>> + Send;
fn text_generation_model_id(&self) -> Option<String>;
fn text_generation_prompt(&self) -> Option<String>; fn text_generation_prompt(&self) -> Option<String>;
fn text_generation_temperature(&self) -> Option<f32>; fn text_generation_temperature(&self) -> Option<f32>;
@@ -63,6 +65,14 @@ impl ControllerTrait for ControllerType {
} }
} }
fn text_generation_model_id(&self) -> Option<String> {
match &self {
ControllerType::OpenAI(controller) => controller.text_generation_model_id(),
ControllerType::OpenAICompat(controller) => controller.text_generation_model_id(),
ControllerType::Anthropic(controller) => controller.text_generation_model_id(),
}
}
fn text_generation_prompt(&self) -> Option<String> { fn text_generation_prompt(&self) -> Option<String> {
match &self { match &self {
ControllerType::OpenAI(controller) => controller.text_generation_prompt(), ControllerType::OpenAI(controller) => controller.text_generation_prompt(),

View File

@@ -9,5 +9,7 @@ pub use agent_provider::{AgentProvider, AgentProviderInfo};
pub use image_generation::{ImageGenerationParams, ImageGenerationResult}; pub use image_generation::{ImageGenerationParams, ImageGenerationResult};
pub use ping::PingResult; pub use ping::PingResult;
pub use speech_to_text::{SpeechToTextParams, SpeechToTextResult}; pub use speech_to_text::{SpeechToTextParams, SpeechToTextResult};
pub use text_generation::{TextGenerationParams, TextGenerationResult}; pub use text_generation::{
TextGenerationParams, TextGenerationPromptVariables, TextGenerationResult,
};
pub use text_to_speech::{TextToSpeechParams, TextToSpeechResult}; pub use text_to_speech::{TextToSpeechParams, TextToSpeechResult};

View File

@@ -1,8 +1,13 @@
mod prompt_variables;
pub use prompt_variables::TextGenerationPromptVariables;
#[derive(Default)] #[derive(Default)]
pub struct TextGenerationParams { pub struct TextGenerationParams {
pub context_management_enabled: bool, pub context_management_enabled: bool,
pub prompt_override: Option<String>, pub prompt_override: Option<String>,
pub temperature_override: Option<f32>, pub temperature_override: Option<f32>,
pub prompt_variables: TextGenerationPromptVariables,
} }
pub struct TextGenerationResult { pub struct TextGenerationResult {

View File

@@ -0,0 +1,75 @@
use chrono::{DateTime, Utc};
use std::collections::HashMap;
pub struct TextGenerationPromptVariables {
map: HashMap<String, String>,
}
impl Default for TextGenerationPromptVariables {
fn default() -> Self {
Self::new("unnamed", "unknown-model", Utc::now())
}
}
impl TextGenerationPromptVariables {
pub fn new(bot_name: &str, model_id: &str, utc_time: DateTime<Utc>) -> Self {
let mut map = HashMap::new();
map.insert("baibot_name".to_string(), bot_name.to_string());
map.insert("baibot_model_id".to_string(), model_id.to_string());
map.insert("baibot_now_utc".to_string(), format_utc_time(utc_time));
Self { map }
}
pub fn format(&self, text: &str) -> String {
let mut formatted_text = text.to_string();
for (key, value) in &self.map {
let placeholder = format!("{{{{ {} }}}}", key);
formatted_text = formatted_text.replace(&placeholder, value);
}
formatted_text
}
}
fn format_utc_time(time: DateTime<Utc>) -> String {
time.format("%Y-%m-%d (%A), %H:%M:%S UTC").to_string()
}
#[cfg(test)]
mod tests {
use super::*;
use chrono::{TimeZone, Timelike};
#[test]
fn test_new() {
// Intentionally injecting some sub-seconds to ensure formatting would ignore them.
let now_utc = Utc
.with_ymd_and_hms(2024, 9, 20, 18, 34, 15)
.unwrap()
.with_nanosecond(250000000)
.unwrap();
let variables = TextGenerationPromptVariables::new("baibot", "gpt-4o", now_utc);
assert_eq!(
variables.map.get("baibot_name"),
Some(&"baibot".to_string())
);
assert_eq!(
variables.map.get("baibot_model_id"),
Some(&"gpt-4o".to_string())
);
assert_eq!(
variables.map.get("baibot_now_utc"),
Some(&format_utc_time(now_utc))
);
let prompt = "Hello, I'm {{ baibot_name }} using {{ baibot_model_id }}. The date/time now is {{ baibot_now_utc }}.";
let expected = "Hello, I'm baibot using gpt-4o. The date/time now is 2024-09-20 (Friday), 18:34:15 UTC.";
assert_eq!(variables.format(prompt), expected);
}
}

View File

@@ -1,6 +1,5 @@
// LocalAI is based on OpenAI (async-openai), because it seems to be fully compatible. // At the time of testing, LocalAI can be powered by `openai`, but we use `openai_compat` for better reliability
// Moreover, openai_api_rust does not support speech-to-text, so if we wish to use this feature // in the event of future updates to `async-openai`.
// we need to stick to async-openai.
use super::openai_compat::Config; use super::openai_compat::Config;

View File

@@ -21,5 +21,5 @@ pub use config::ConfigTrait;
pub use entity::{ pub use entity::{
AgentProvider, AgentProviderInfo, ImageGenerationParams, PingResult, SpeechToTextParams, AgentProvider, AgentProviderInfo, ImageGenerationParams, PingResult, SpeechToTextParams,
SpeechToTextResult, TextGenerationParams, TextToSpeechParams, SpeechToTextResult, TextGenerationParams, TextGenerationPromptVariables, TextToSpeechParams,
}; };

View File

@@ -1,6 +1,6 @@
use serde::{Deserialize, Serialize}; use serde::{Deserialize, Serialize};
use crate::agent::provider::ConfigTrait; use crate::agent::{default_prompt, provider::ConfigTrait};
#[derive(Debug, Clone, Serialize, Deserialize)] #[derive(Debug, Clone, Serialize, Deserialize)]
pub struct Config { pub struct Config {
@@ -66,7 +66,7 @@ impl Default for TextGenerationConfig {
fn default() -> Self { fn default() -> Self {
Self { Self {
model_id: default_text_model_id(), model_id: default_text_model_id(),
prompt: Some("You are a brief, but helpful bot.".to_owned()), prompt: Some(default_prompt().to_owned()),
temperature: super::super::default_temperature(), temperature: super::super::default_temperature(),
max_response_tokens: 16_384, max_response_tokens: 16_384,
max_context_tokens: 128_000, max_context_tokens: 128_000,

View File

@@ -86,11 +86,12 @@ impl ControllerTrait for Controller {
)); ));
}; };
let prompt_text = params let prompt_text = params.prompt_variables.format(
.prompt_override params
.unwrap_or(self.text_generation_prompt().unwrap_or("".to_owned())) .prompt_override
.trim() .unwrap_or(self.text_generation_prompt().unwrap_or("".to_owned()))
.to_owned(); .trim(),
);
let prompt_message = if prompt_text.is_empty() { let prompt_message = if prompt_text.is_empty() {
None None
@@ -391,20 +392,25 @@ impl ControllerTrait for Controller {
} }
} }
fn text_generation_prompt(&self) -> Option<String> { fn text_generation_model_id(&self) -> Option<String> {
let Some(text_generation_config) = &self.config.text_generation else { self.config
return None; .text_generation
}; .as_ref()
.map(|config| config.model_id.to_owned())
}
text_generation_config.prompt.clone() fn text_generation_prompt(&self) -> Option<String> {
self.config
.text_generation
.as_ref()
.and_then(|config| config.prompt.clone())
} }
fn text_generation_temperature(&self) -> Option<f32> { fn text_generation_temperature(&self) -> Option<f32> {
let Some(text_generation_config) = &self.config.text_generation else { self.config
return None; .text_generation
}; .as_ref()
.map(|config| config.temperature)
Some(text_generation_config.temperature)
} }
fn text_to_speech_voice(&self) -> Option<String> { fn text_to_speech_voice(&self) -> Option<String> {

View File

@@ -1,5 +1,6 @@
use serde::{Deserialize, Serialize}; use serde::{Deserialize, Serialize};
use crate::agent::default_prompt;
use crate::agent::provider::openai::{ use crate::agent::provider::openai::{
ImageGenerationConfig as OpenAIImageGenerationConfig, ImageGenerationConfig as OpenAIImageGenerationConfig,
SpeechToTextConfig as OpenAISpeechToTextConfig, SpeechToTextConfig as OpenAISpeechToTextConfig,
@@ -75,7 +76,7 @@ impl Default for TextGenerationConfig {
fn default() -> Self { fn default() -> Self {
Self { Self {
model_id: default_text_model_id(), model_id: default_text_model_id(),
prompt: Some("You are a brief, but helpful bot.".to_owned()), prompt: Some(default_prompt().to_owned()),
temperature: super::super::default_temperature(), temperature: super::super::default_temperature(),
max_response_tokens: 4096, max_response_tokens: 4096,
max_context_tokens: 128_000, max_context_tokens: 128_000,

View File

@@ -84,11 +84,12 @@ impl ControllerTrait for Controller {
)); ));
}; };
let prompt_text = params let prompt_text = params.prompt_variables.format(
.prompt_override params
.unwrap_or(self.text_generation_prompt().unwrap_or("".to_owned())) .prompt_override
.trim() .unwrap_or(self.text_generation_prompt().unwrap_or("".to_owned()))
.to_owned(); .trim(),
);
let prompt_message = if prompt_text.is_empty() { let prompt_message = if prompt_text.is_empty() {
None None
@@ -409,20 +410,25 @@ impl ControllerTrait for Controller {
} }
} }
fn text_generation_prompt(&self) -> Option<String> { fn text_generation_model_id(&self) -> Option<String> {
let Some(text_generation_config) = &self.config.text_generation else { self.config
return None; .text_generation
}; .as_ref()
.map(|config| config.model_id.to_owned())
}
text_generation_config.prompt.clone() fn text_generation_prompt(&self) -> Option<String> {
self.config
.text_generation
.as_ref()
.and_then(|config| config.prompt.clone())
} }
fn text_generation_temperature(&self) -> Option<f32> { fn text_generation_temperature(&self) -> Option<f32> {
let Some(text_generation_config) = &self.config.text_generation else { self.config
return None; .text_generation
}; .as_ref()
.map(|config| config.temperature)
Some(text_generation_config.temperature)
} }
fn text_to_speech_voice(&self) -> Option<String> { fn text_to_speech_voice(&self) -> Option<String> {

View File

@@ -9,6 +9,7 @@ use mxlink::matrix_sdk::Room;
use mxlink::{ use mxlink::{
InitConfig, LoginConfig, LoginCredentials, LoginEncryption, MatrixLink, PersistenceConfig, InitConfig, LoginConfig, LoginCredentials, LoginEncryption, MatrixLink, PersistenceConfig,
TypingNoticeGuard,
}; };
use mxlink::helpers::account_data_config::{ use mxlink::helpers::account_data_config::{
@@ -210,6 +211,14 @@ impl Bot {
.await .await
} }
pub(crate) async fn start_typing_notice(&self, room: &Room) -> TypingNoticeGuard {
self.inner
.matrix_link
.rooms()
.start_typing_notice(room)
.await
}
pub async fn start(&self) -> anyhow::Result<()> { pub async fn start(&self) -> anyhow::Result<()> {
self.rooms().attach_event_handlers().await; self.rooms().attach_event_handlers().await;
self.messaging().attach_event_handlers().await; self.messaging().attach_event_handlers().await;

View File

@@ -317,13 +317,6 @@ impl Messaging {
let start_time = std::time::Instant::now(); let start_time = std::time::Instant::now();
let _typing_notice_guard = self
.bot
.matrix_link()
.rooms()
.start_typing_notice(message_context.room())
.await;
let event_span = tracing::error_span!("message_controller", ?controller_type); let event_span = tracing::error_span!("message_controller", ?controller_type);
crate::controller::dispatch_controller(&controller_type, &message_context, &self.bot) crate::controller::dispatch_controller(&controller_type, &message_context, &self.bot)

View File

@@ -244,13 +244,6 @@ impl Reacting {
tracing::info!("Handling reaction via reaction controller"); tracing::info!("Handling reaction via reaction controller");
let _typing_notice_guard = self
.bot
.matrix_link()
.rooms()
.start_typing_notice(message_context.room())
.await;
let event_span = tracing::error_span!("reaction_controller"); let event_span = tracing::error_span!("reaction_controller");
crate::controller::reaction::handle( crate::controller::reaction::handle(

View File

@@ -13,7 +13,7 @@ pub async fn dispatch_controller(
match handler { match handler {
AccessControllerType::Help => {} AccessControllerType::Help => {}
_ => { _ => {
if !message_context.sender_can_manage_global_config()? { if !message_context.sender_can_manage_global_config() {
bot.messaging() bot.messaging()
.send_error_markdown_no_fail( .send_error_markdown_no_fail(
message_context.room(), message_context.room(),

View File

@@ -80,24 +80,21 @@ fn build_section_users(
message.push_str(&strings::access::users_no_patterns()); message.push_str(&strings::access::users_no_patterns());
} }
let can_manage_global_config = message_context.sender_can_manage_global_config(); if message_context.sender_can_manage_global_config() {
if let Ok(can_manage_global_config) = can_manage_global_config { message.push_str("\n\n");
if can_manage_global_config {
message.push_str("\n\n");
message.push_str(strings::the_following_commands_are_available()); message.push_str(strings::the_following_commands_are_available());
message.push('\n'); message.push('\n');
message.push_str(&strings::help::access::users_command_get(command_prefix)); message.push_str(&strings::help::access::users_command_get(command_prefix));
message.push('\n'); message.push('\n');
message.push_str(&strings::help::access::users_command_set(command_prefix)); message.push_str(&strings::help::access::users_command_set(command_prefix));
message.push_str("\n\n"); message.push_str("\n\n");
message.push_str(&strings::help::access::example_user_patterns( message.push_str(&strings::help::access::example_user_patterns(
homeserver_name, homeserver_name,
)); ));
}
} }
message message
@@ -156,27 +153,24 @@ fn build_section_room_local_agent_managers(
message.push_str(&strings::access::room_local_agent_managers_no_patterns()); message.push_str(&strings::access::room_local_agent_managers_no_patterns());
} }
let can_manage_global_config = message_context.sender_can_manage_global_config(); if message_context.sender_can_manage_global_config() {
if let Ok(can_manage_global_config) = can_manage_global_config { message.push_str("\n\n");
if can_manage_global_config { message.push_str(strings::the_following_commands_are_available());
message.push_str("\n\n"); message.push('\n');
message.push_str(strings::the_following_commands_are_available());
message.push('\n');
message.push_str( message.push_str(
&strings::help::access::room_local_agent_managers_command_get(command_prefix), &strings::help::access::room_local_agent_managers_command_get(command_prefix),
); );
message.push('\n'); message.push('\n');
message.push_str( message.push_str(
&strings::help::access::room_local_agent_managers_command_set(command_prefix), &strings::help::access::room_local_agent_managers_command_set(command_prefix),
); );
message.push_str("\n\n"); message.push_str("\n\n");
message.push_str(&strings::help::access::example_user_patterns( message.push_str(&strings::help::access::example_user_patterns(
homeserver_name, homeserver_name,
)); ));
}
} }
message message

View File

@@ -138,7 +138,7 @@ pub async fn handle_global(
provider: &str, provider: &str,
agent_id_prefixless: &str, agent_id_prefixless: &str,
) -> anyhow::Result<()> { ) -> anyhow::Result<()> {
if !message_context.sender_can_manage_global_config()? { if !message_context.sender_can_manage_global_config() {
bot.messaging() bot.messaging()
.send_error_markdown_no_fail( .send_error_markdown_no_fail(
message_context.room(), message_context.room(),

View File

@@ -50,7 +50,7 @@ pub async fn handle(
.await .await
} }
PublicIdentifier::DynamicGlobal(_) => { PublicIdentifier::DynamicGlobal(_) => {
if !message_context.sender_can_manage_global_config()? { if !message_context.sender_can_manage_global_config() {
bot.messaging() bot.messaging()
.send_error_markdown_no_fail( .send_error_markdown_no_fail(
message_context.room(), message_context.room(),

View File

@@ -47,7 +47,7 @@ pub async fn handle(
} }
} }
PublicIdentifier::DynamicGlobal(_) => { PublicIdentifier::DynamicGlobal(_) => {
if !message_context.sender_can_manage_global_config()? { if !message_context.sender_can_manage_global_config() {
bot.messaging() bot.messaging()
.send_error_markdown_no_fail( .send_error_markdown_no_fail(
message_context.room(), message_context.room(),

View File

@@ -12,10 +12,7 @@ pub async fn handle(bot: &Bot, message_context: &MessageContext) -> anyhow::Resu
message.push_str(&format!("## {}", strings::help::agent::heading())); message.push_str(&format!("## {}", strings::help::agent::heading()));
message.push_str("\n\n"); message.push_str("\n\n");
message.push_str(&strings::help::agent::intro( message.push_str(&strings::help::agent::intro(bot.command_prefix()));
bot.command_prefix(),
can_manage_agents,
));
message.push('\n'); message.push('\n');
message.push_str(&strings::help::agent::intro_capabilities()); message.push_str(&strings::help::agent::intro_capabilities());
message.push_str("\n\n"); message.push_str("\n\n");
@@ -39,7 +36,7 @@ pub async fn handle(bot: &Bot, message_context: &MessageContext) -> anyhow::Resu
)); ));
message.push('\n'); message.push('\n');
if message_context.sender_can_manage_global_config()? { if message_context.sender_can_manage_global_config() {
message.push_str(&strings::help::agent::create_agent_global( message.push_str(&strings::help::agent::create_agent_global(
bot.command_prefix(), bot.command_prefix(),
)); ));

View File

@@ -40,7 +40,7 @@ async fn dispatch_config_related_handler(
bot: &Bot, bot: &Bot,
) -> anyhow::Result<()> { ) -> anyhow::Result<()> {
if let SettingsStorageSource::Global = config_type { if let SettingsStorageSource::Global = config_type {
if !message_context.sender_can_manage_global_config()? { if !message_context.sender_can_manage_global_config() {
bot.messaging() bot.messaging()
.send_error_markdown_no_fail( .send_error_markdown_no_fail(
message_context.room(), message_context.room(),

View File

@@ -4,7 +4,9 @@ use mxlink::{MatrixLink, MessageResponseType};
use tracing::Instrument; use tracing::Instrument;
use crate::agent::provider::{SpeechToTextParams, TextGenerationParams}; use crate::agent::provider::{
SpeechToTextParams, TextGenerationParams, TextGenerationPromptVariables,
};
use crate::agent::AgentInstance; use crate::agent::AgentInstance;
use crate::agent::AgentPurpose; use crate::agent::AgentPurpose;
use crate::agent::ControllerTrait; use crate::agent::ControllerTrait;
@@ -43,6 +45,8 @@ pub async fn handle(
) -> anyhow::Result<()> { ) -> anyhow::Result<()> {
let mut original_message_is_audio = false; let mut original_message_is_audio = false;
let mut _typing_notice_guard: Option<mxlink::TypingNoticeGuard> = None;
let speech_to_text_flow_type = message_context let speech_to_text_flow_type = message_context
.room_config_context() .room_config_context()
.speech_to_text_flow_type(); .speech_to_text_flow_type();
@@ -58,11 +62,11 @@ pub async fn handle(
return Ok(()); return Ok(());
} }
SpeechToTextFlowType::TranscribeAndGenerateText => { SpeechToTextFlowType::TranscribeAndGenerateText => {
tracing::debug!("Will be trascribing and possibly generating text.."); tracing::debug!("Will be transcribing and possibly generating text..");
MessageResponseType::InThread(message_context.thread_info().clone()) MessageResponseType::InThread(message_context.thread_info().clone())
} }
SpeechToTextFlowType::OnlyTranscribe => { SpeechToTextFlowType::OnlyTranscribe => {
tracing::debug!("Will only be trascribing audio to text.."); tracing::debug!("Will only be transcribing audio to text..");
if message_context.thread_info().is_thread_root_only() { if message_context.thread_info().is_thread_root_only() {
MessageResponseType::Reply(message_context.thread_info().root_event_id.clone()) MessageResponseType::Reply(message_context.thread_info().root_event_id.clone())
} else { } else {
@@ -71,6 +75,10 @@ pub async fn handle(
} }
}; };
if _typing_notice_guard.is_none() {
_typing_notice_guard = Some(bot.start_typing_notice(message_context.room()).await);
}
let Some(speech_to_text_created_event_id_result) = let Some(speech_to_text_created_event_id_result) =
handle_stage_speech_to_text(bot, message_context, audio_content, response_type).await handle_stage_speech_to_text(bot, message_context, audio_content, response_type).await
else { else {
@@ -96,6 +104,10 @@ pub async fn handle(
.room_config_context() .room_config_context()
.should_auto_text_generate(original_message_is_audio) .should_auto_text_generate(original_message_is_audio)
{ {
if _typing_notice_guard.is_none() {
_typing_notice_guard = Some(bot.start_typing_notice(message_context.room()).await);
}
let speech_to_text_created_event_id_reaction_event_id = let speech_to_text_created_event_id_reaction_event_id =
if let Some(speech_to_text_created_event_id) = speech_to_text_created_event_id { if let Some(speech_to_text_created_event_id) = speech_to_text_created_event_id {
let reaction_event_response = bot let reaction_event_response = bot
@@ -213,6 +225,10 @@ pub async fn handle(
match text_to_speech_stage_params { match text_to_speech_stage_params {
Some(TextToSpeechParams::Perform(text_to_speech_eligible_payload, response_type)) => { Some(TextToSpeechParams::Perform(text_to_speech_eligible_payload, response_type)) => {
if _typing_notice_guard.is_none() {
_typing_notice_guard = Some(bot.start_typing_notice(message_context.room()).await);
}
let _tts_result = generate_and_send_tts_for_message( let _tts_result = generate_and_send_tts_for_message(
bot, bot,
matrix_link.clone(), matrix_link.clone(),
@@ -391,6 +407,16 @@ async fn handle_stage_text_generation(
let start_time = std::time::Instant::now(); let start_time = std::time::Instant::now();
let controller = agent.controller();
let prompt_variables = TextGenerationPromptVariables::new(
bot.name(),
&controller
.text_generation_model_id()
.unwrap_or("unknown-model".to_owned()),
chrono::Utc::now(),
);
let params = TextGenerationParams { let params = TextGenerationParams {
context_management_enabled: message_context context_management_enabled: message_context
.room_config_context() .room_config_context()
@@ -403,10 +429,11 @@ async fn handle_stage_text_generation(
temperature_override: message_context temperature_override: message_context
.room_config_context() .room_config_context()
.text_generation_temperature_override(), .text_generation_temperature_override(),
prompt_variables,
}; };
let result = agent let result = controller
.controller()
.generate_text(conversation, params) .generate_text(conversation, params)
.instrument(span) .instrument(span)
.await; .await;

View File

@@ -3,7 +3,7 @@ use mxlink::MessageResponseType;
use crate::{entity::MessageContext, strings, Bot}; use crate::{entity::MessageContext, strings, Bot};
pub async fn handle(bot: &Bot, message_context: &MessageContext) -> anyhow::Result<()> { pub async fn handle(bot: &Bot, message_context: &MessageContext) -> anyhow::Result<()> {
let sender_can_manage_global_config = message_context.sender_can_manage_global_config()?; let sender_can_manage_global_config = message_context.sender_can_manage_global_config();
let sender_can_manage_room_local_agents = let sender_can_manage_room_local_agents =
message_context.sender_can_manage_room_local_agents()?; message_context.sender_can_manage_room_local_agents()?;
@@ -18,10 +18,7 @@ pub async fn handle(bot: &Bot, message_context: &MessageContext) -> anyhow::Resu
// Agents // Agents
message.push_str(&format!("## {}", strings::help::agent::heading())); message.push_str(&format!("## {}", strings::help::agent::heading()));
message.push_str("\n\n"); message.push_str("\n\n");
message.push_str(&strings::help::agent::intro( message.push_str(&strings::help::agent::intro(bot.command_prefix()));
bot.command_prefix(),
sender_can_manage_room_local_agents,
));
message.push_str("\n\n"); message.push_str("\n\n");
message.push_str(&strings::help::agent::intro_handler_relation( message.push_str(&strings::help::agent::intro_handler_relation(
bot.command_prefix(), bot.command_prefix(),

View File

@@ -137,6 +137,8 @@ pub async fn handle_sticker(
return Ok(()); return Ok(());
}; };
let _typing_notice_guard = bot.start_typing_notice(message_context.room()).await;
let span = tracing::debug_span!( let span = tracing::debug_span!(
"sticker_generation", "sticker_generation",
agent_id = agent.identifier().as_string() agent_id = agent.identifier().as_string()

View File

@@ -9,17 +9,8 @@ pub fn determine_controller(_text: &str) -> ControllerType {
} }
pub async fn handle_help(message_context: &MessageContext, bot: &Bot) -> anyhow::Result<()> { pub async fn handle_help(message_context: &MessageContext, bot: &Bot) -> anyhow::Result<()> {
if !message_context.sender_can_manage_room_local_agents()? { let can_create_global_agents = message_context.sender_can_manage_global_config();
bot.messaging() let can_create_room_local_agents = message_context.sender_can_manage_room_local_agents()?;
.send_error_markdown_no_fail(
message_context.room(),
&strings::provider::not_allowed(),
MessageResponseType::Reply(message_context.thread_info().root_event_id.clone()),
)
.await;
return Ok(());
}
let mut message = String::new(); let mut message = String::new();
message.push_str(&format!("## {}", strings::help::provider::heading())); message.push_str(&format!("## {}", strings::help::provider::heading()));
@@ -69,18 +60,26 @@ pub async fn handle_help(message_context: &MessageContext, bot: &Bot) -> anyhow:
&provider_info, &provider_info,
)); ));
message.push_str("- 🗲 Quick start:\n"); // We always show a "Quick start" section (even to unprivileged users),
message.push_str(&format!( // because we're talking about it in a previous message.
"\t- create a room-local agent: `{command_prefix} agent create-room-local {provider_id} my-{provider_id}-agent`", message.push_str("- 🗲 Quick start:");
command_prefix = bot.command_prefix(), if can_create_room_local_agents {
provider_id = provider.to_static_str(), message.push_str(&format!(
)); "\n\t- create a room-local agent: `{command_prefix} agent create-room-local {provider_id} my-{provider_id}-agent`",
message.push('\n'); command_prefix = bot.command_prefix(),
message.push_str(&format!( provider_id = provider.to_static_str(),
"\t- create a global agent: `{command_prefix} agent create-global {provider_id} my-{provider_id}-agent`", ));
command_prefix = bot.command_prefix(), }
provider_id = provider.to_static_str(), if can_create_global_agents {
)); message.push_str(&format!(
"\n\t- create a global agent: `{command_prefix} agent create-global {provider_id} my-{provider_id}-agent`",
command_prefix = bot.command_prefix(),
provider_id = provider.to_static_str(),
));
}
if !can_create_room_local_agents && !can_create_global_agents {
message.push_str(" ask an administrator to create an agent for you (you lack permissions to do so yourself)");
}
message.push_str("\n\n"); message.push_str("\n\n");
} }

View File

@@ -52,6 +52,8 @@ pub(super) async fn handle(
return Ok(()); return Ok(());
}; };
let _typing_notice_guard = bot.start_typing_notice(message_context.room()).await;
crate::controller::utils::text_to_speech::generate_and_send_tts_for_message( crate::controller::utils::text_to_speech::generate_and_send_tts_for_message(
bot, bot,
matrix_link, matrix_link,

View File

@@ -70,12 +70,12 @@ impl MessageContext {
&self.thread_info &self.thread_info
} }
pub fn sender_can_manage_global_config(&self) -> anyhow::Result<bool> { pub fn sender_can_manage_global_config(&self) -> bool {
Ok(self.trigger_event_info.sender_is_admin) self.trigger_event_info.sender_is_admin
} }
pub fn sender_can_manage_room_local_agents(&self) -> anyhow::Result<bool> { pub fn sender_can_manage_room_local_agents(&self) -> mxidwc::Result<bool> {
Ok(self.sender_can_manage_global_config()? Ok(self.sender_can_manage_global_config()
|| self.sender_is_allowed_room_local_agent_manager()?) || self.sender_is_allowed_room_local_agent_manager()?)
} }
@@ -102,21 +102,8 @@ impl MessageContext {
combined combined
} }
fn sender_is_allowed_room_local_agent_manager(&self) -> anyhow::Result<bool> { fn sender_is_allowed_room_local_agent_manager(&self) -> mxidwc::Result<bool> {
match &self self.room_config_context()
.global_config() .is_user_allowed_room_local_agent_manager(self.sender_id().clone())
.access
.room_local_agent_manager_patterns
{
None => Ok(false),
Some(patterns) => {
let allowed_regexes = mxidwc::parse_patterns_vector(patterns)?;
Ok(mxidwc::match_user_id(
self.sender_id().as_str(),
&allowed_regexes,
))
}
}
} }
} }

View File

@@ -1,3 +1,5 @@
use mxlink::matrix_sdk::ruma::OwnedUserId;
use super::globalconfig::GlobalConfig; use super::globalconfig::GlobalConfig;
use super::roomconfig::RoomConfig; use super::roomconfig::RoomConfig;
@@ -180,4 +182,18 @@ impl RoomConfigContext {
.clone() .clone()
}) })
} }
pub fn is_user_allowed_room_local_agent_manager(
&self,
user_id: OwnedUserId,
) -> mxidwc::Result<bool> {
match &self.global_config.access.room_local_agent_manager_patterns {
None => Ok(false),
Some(patterns) => {
let allowed_regexes = mxidwc::parse_patterns_vector(patterns)?;
Ok(mxidwc::match_user_id(user_id.as_str(), &allowed_regexes))
}
}
}
} }

View File

@@ -44,7 +44,7 @@ pub fn not_allowed_to_manage_static_agents() -> String {
pub fn configuration_does_not_result_in_a_working_agent(err: anyhow::Error) -> String { pub fn configuration_does_not_result_in_a_working_agent(err: anyhow::Error) -> String {
format!( format!(
"The provided configuration does not result in a working agent. The following error was encountered when trying to talk to the agent API:\n```\n{}```", "The provided configuration does not result in a working agent. The following error was encountered when trying to talk to the agent API:\n```\n{}\n```",
err, err,
) )
} }

View File

@@ -2,15 +2,8 @@ pub fn heading() -> String {
"🤖 Agents".to_owned() "🤖 Agents".to_owned()
} }
pub fn intro(command_prefix: &str, can_see_providers: bool) -> String { pub fn intro(command_prefix: &str) -> String {
format!( format!("An agent is an instantiation and configuration of some **☁️ provider** (see `{command_prefix} provider`).")
"An agent is an instantiation and configuration of some **☁️ provider**{}.",
if can_see_providers {
format!(" (see `{command_prefix} provider`)")
} else {
"".to_owned()
}
)
} }
pub fn intro_handler_relation(command_prefix: &str) -> String { pub fn intro_handler_relation(command_prefix: &str) -> String {
@@ -24,7 +17,7 @@ pub fn intro_capabilities() -> String {
} }
pub fn no_permission_to_create_agents() -> &'static str { pub fn no_permission_to_create_agents() -> &'static str {
"You are neither an administrator, nor a room-local agent manager, so **you cannot create new agents by yourself**." "⚠️ You are neither a bot administrator, nor a room-local agent manager, so **you cannot create new agents by yourself**."
} }
pub fn list_agents(command_prefix: &str) -> String { pub fn list_agents(command_prefix: &str) -> String {

View File

@@ -23,20 +23,6 @@ fn purposes_intro() -> &'static str {
"I can typically be used for the following purposes:" "I can typically be used for the following purposes:"
} }
fn no_text_generation_handler_agent() -> String {
format!(
"There is no configured handler agent which supports {} {}.",
AgentPurpose::TextGeneration.emoji(),
AgentPurpose::TextGeneration
)
}
fn introduction_outro(command_prefix: &str) -> String {
format!(
"You may also send a `{command_prefix} help` command message in this room for more information."
)
}
pub async fn create_on_join_introduction( pub async fn create_on_join_introduction(
name: &str, name: &str,
command_prefix: &str, command_prefix: &str,
@@ -111,18 +97,17 @@ pub async fn create_on_join_introduction(
message.push_str("\n\n"); message.push_str("\n\n");
if got_text_generation_agent { if got_text_generation_agent {
message.push_str(&simply_send_a_message( message.push_str(&make_use_of_me_simply_send_a_message(
command_prefix, command_prefix,
room_config_context.text_generation_prefix_requirement_type(), room_config_context.text_generation_prefix_requirement_type(),
)); ));
} else { } else {
message.push_str(&no_text_generation_handler_agent()); message.push_str(&make_use_of_me_agent_creation(
command_prefix,
room_config_context.text_generation_prefix_requirement_type(),
));
} }
message.push_str("\n\n");
message.push_str(&introduction_outro(command_prefix));
message message
} }
@@ -130,17 +115,85 @@ pub fn create_short_introduction(name: &str) -> String {
its_me(name) its_me(name)
} }
fn simply_send_a_message( fn make_use_of_me_simply_send_a_message(
command_prefix: &str,
prefix_requirement_type: TextGenerationPrefixRequirementType,
) -> String {
let message = r#"**To make use of me**:
1. 👋 %send_a_message%
2. 📖 %learn_more%
"#;
message
.replace("%command_prefix%", command_prefix)
.replace(
"%send_a_message%",
&send_a_text_message(command_prefix, prefix_requirement_type),
)
.replace(
"%learn_more%",
&learn_more_from_usage_or_help(command_prefix),
)
}
fn make_use_of_me_agent_creation(
command_prefix: &str,
prefix_requirement_type: TextGenerationPrefixRequirementType,
) -> String {
let message = r#"**To make use of me**:
1. ☁️ **Choose an agent provider** (e.g. OpenAI, Mistral, etc). Send a `%command_prefix% provider` command to see the list.
2. 🤖 %create_one_or_more_agents%
3. 🤝 %set_new_agent_as_handler%
4. 👋 %send_a_message%
5. 📖 %learn_more%
"#;
message
.replace("%command_prefix%", command_prefix)
.replace(
"%send_a_message%",
&send_a_text_message(command_prefix, prefix_requirement_type),
)
.replace(
"%learn_more%",
&learn_more_from_usage_or_help(command_prefix),
)
.replace(
"%create_one_or_more_agents%",
&create_one_or_more_agents(command_prefix),
)
.replace(
"%set_new_agent_as_handler%",
&set_new_agent_as_handler(command_prefix),
)
}
fn send_a_text_message(
command_prefix: &str, command_prefix: &str,
prefix_requirement_type: TextGenerationPrefixRequirementType, prefix_requirement_type: TextGenerationPrefixRequirementType,
) -> String { ) -> String {
match prefix_requirement_type { match prefix_requirement_type {
TextGenerationPrefixRequirementType::No => { TextGenerationPrefixRequirementType::No => {
"**To make use of me, send a message** in this room (e.g. `Hello!`) and see me reply." "**Send a text message** in this room (e.g. `Hello!`) and see me reply.".to_owned()
.to_owned()
} }
TextGenerationPrefixRequirementType::CommandPrefix => { TextGenerationPrefixRequirementType::CommandPrefix => {
format!("In this room, I'm configured to require the command prefix (`{command_prefix}`) for text messages.\n**To make use of me, send a prefixed text message** (e.g. `{command_prefix} Hello!`) and see me reply.") format!("In this room, I'm configured to require the command prefix (`{command_prefix}`) for text messages. **Send a prefixed text message** (e.g. `{command_prefix} Hello!`) and see me reply.")
} }
} }
} }
fn learn_more_from_usage_or_help(command_prefix: &str) -> String {
format!(
"**Learn more** by sending a `{command_prefix} usage` or `{command_prefix} help` command."
)
}
pub fn create_one_or_more_agents(command_prefix: &str) -> String {
format!("**Create one or more agents** in this room or globally. The provider help message will show you **🗲 Quick start** commands, but you may also send a `{command_prefix} agent` command to see the guide.")
}
pub fn set_new_agent_as_handler(command_prefix: &str) -> String {
format!("**Set the new agent as a handler** for a given use-purpose like text-generation, image-generation, etc. The agent-creation wizard will tell you how, but you may also send a `{command_prefix} config` command to see the guide (in the 🤖 *Handler Agents* section).")
}

View File

@@ -25,10 +25,6 @@ pub fn invalid_configuration_for_provider(
) )
} }
pub fn not_allowed() -> String {
"You are not allowed to see the providers list.".to_owned()
}
pub fn providers_list_intro() -> String { pub fn providers_list_intro() -> String {
"The list of supported providers is below.".to_owned() "The list of supported providers is below.".to_owned()
} }
@@ -39,7 +35,7 @@ pub fn help_how_to_choose_heading() -> String {
pub fn help_how_to_choose_description(command_prefix: &str) -> String { pub fn help_how_to_choose_description(command_prefix: &str) -> String {
let str = r#" let str = r#"
If you're not sure which provider to start with, we **recommend OpenAI** as it's the most popular and has the **widest range of capabilities**. If you're not sure which provider to start with, **we recommend OpenAI** as it's the most popular and has the **widest range of capabilities**.
You don't need to choose just one though. The bot supports **mixing & matching models** (by setting different handlers for different types of messages - see `%command_prefix% config`), so you can use multiple providers at the same time. You don't need to choose just one though. The bot supports **mixing & matching models** (by setting different handlers for different types of messages - see `%command_prefix% config`), so you can use multiple providers at the same time.
"#; "#;
@@ -55,13 +51,21 @@ pub fn help_how_to_use_heading() -> String {
pub fn help_how_to_use_description(command_prefix: &str) -> String { pub fn help_how_to_use_description(command_prefix: &str) -> String {
let str = r#" let str = r#"
- sign up for it 1. 📝 **Sign up for it**
- obtain an API key 2. 🔑 **Obtain an API key**
- create a new agent (see `%command_prefix% agent`) 3. 🤖 %create_one_or_more_agents%
- set the new agent as a handler for some types of messages (see `%command_prefix% config`) 4. 🤝 %set_new_agent_as_handler%
"#; "#;
str.replace("%command_prefix%", command_prefix) str.replace("%command_prefix%", command_prefix)
.replace(
"%create_one_or_more_agents%",
&super::introduction::create_one_or_more_agents(command_prefix),
)
.replace(
"%set_new_agent_as_handler%",
&super::introduction::set_new_agent_as_handler(command_prefix),
)
.trim() .trim()
.to_owned() .to_owned()
} }

View File

@@ -12,7 +12,7 @@ If there's a text-generation handler agent configured (see `%command_prefix% con
Whether the bot responds depends on the **💬 Text Generation / 🗟 Prefix Requirement** setting (see `%command_prefix% config status`). Whether the bot responds depends on the **💬 Text Generation / 🗟 Prefix Requirement** setting (see `%command_prefix% config status`).
Sometimes, a prefix (e.g. `%command_prefix%`) is required in front of messages sent to the room for the bot to respond. Sometimes, a prefix (e.g. `%command_prefix%`) is required in front of messages sent to the room for the bot to respond.
For multi-user rooms, this setting defaults to "required" For multi-user rooms, this setting defaults to "required".
Room messages start a threaded conversation where you can continue back-and-forth communication with the bot. Room messages start a threaded conversation where you can continue back-and-forth communication with the bot.