docs/sample-provider-configs/localai.yml

base_url: http://my-localai-self-hosted-service:8080/v1
api_key: YOUR_API_KEY_HERE
text_generation:
  model_id: gpt-4
  prompt: "You are a brief, but helpful bot called {{ baibot_name }} powered by the {{ baibot_model_id }} model. The date/time of this conversation's start is: {{ baibot_conversation_start_time_utc }}."
  temperature: 1.0
  max_response_tokens: 4096
  max_context_tokens: 128000
speech_to_text:
  model_id: whisper-1
text_to_speech:
  model_id: tts-1
  voice: onyx
  speed: 1.0
  response_format: opus
image_generation:
  model_id: stablediffusion
  style: vivid
  size: 1024x1024
  quality: standard
Initial commit 2024-09-12 13:44:06 +03:00			`base_url: http://my-localai-self-hosted-service:8080/v1`
			`api_key: YOUR_API_KEY_HERE`
			`text_generation:`
			`model_id: gpt-4`
Allow for prompt caching to work by using `baibot_conversation_start_time_utc` instead of `baibot_now_utc` This patch introduces a new `baibot_conversation_start_time_utc` variable which indicates the time the conversation got started. Using `baibot_now_utc` is still possible, but given that the current time is a moving target, its use is in conflict with prompt caching. Because the new `baibot_conversation_start_time_utc` prompt variable is a more reasonable default, we're now using it in all sample configs. 2024-10-03 11:48:14 +03:00			`prompt: "You are a brief, but helpful bot called {{ baibot_name }} powered by the {{ baibot_model_id }} model. The date/time of this conversation's start is: {{ baibot_conversation_start_time_utc }}."`
Initial commit 2024-09-12 13:44:06 +03:00			`temperature: 1.0`
			`max_response_tokens: 4096`
			`max_context_tokens: 128000`
			`speech_to_text:`
			`model_id: whisper-1`
			`text_to_speech:`
			`model_id: tts-1`
			`voice: onyx`
			`speed: 1.0`
			`response_format: opus`
			`image_generation:`
			`model_id: stablediffusion`
			`style: vivid`
			`size: 1024x1024`
			`quality: standard`