unfunny/baibot-withmcp
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Allow for prompt caching to work by using baibot_conversation_start_time_utc instead of baibot_now_utc

Browse Source 
This patch introduces a new `baibot_conversation_start_time_utc`
variable which indicates the time the conversation got started.

Using `baibot_now_utc` is still possible, but given that the current
time is a moving target, its use is in conflict with prompt caching.

Because the new `baibot_conversation_start_time_utc` prompt variable
is a more reasonable default, we're now using it in all sample configs.
This commit is contained in:
Slavi Pantaleev
2024-10-03 11:48:14 +03:00
parent db9422740c
commit 85e66406dc
25 changed files with 159 additions and 25 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
docs/sample-provider-configs/localai.yml
View File

@@ -2,7 +2,7 @@ base_url: http://my-localai-self-hosted-service:8080/v1
api_key: YOUR_API_KEY_HERE
text_generation:
model_id: gpt-4
prompt: "You are a brief, but helpful bot called {{ baibot_name }} powered by the {{ baibot_model_id }} model. The date/time now is: {{ baibot_now_utc }}."
prompt: "You are a brief, but helpful bot called {{ baibot_name }} powered by the {{ baibot_model_id }} model. The date/time of this conversation's start is: {{ baibot_conversation_start_time_utc }}."
temperature: 1.0
max_response_tokens: 4096
max_context_tokens: 128000