docs/sample-provider-configs/together-ai.yml

base_url: https://api.together.xyz/v1
api_key: YOUR_API_KEY_HERE
text_generation:
  model_id: meta-llama/Meta-Llama-3.1-405B-Instruct-Turbo
  prompt: "You are a brief, but helpful bot called {{ baibot_name }} powered by the {{ baibot_model_id }} model. The date/time of this conversation's start is: {{ baibot_conversation_start_time_utc }}."
  temperature: 1.0
  max_response_tokens: 2048
  max_context_tokens: 8192
Initial commit 2024-09-12 13:44:06 +03:00			`base_url: https://api.together.xyz/v1`
			`api_key: YOUR_API_KEY_HERE`
			`text_generation:`
			`model_id: meta-llama/Meta-Llama-3.1-405B-Instruct-Turbo`
Allow for prompt caching to work by using `baibot_conversation_start_time_utc` instead of `baibot_now_utc` This patch introduces a new `baibot_conversation_start_time_utc` variable which indicates the time the conversation got started. Using `baibot_now_utc` is still possible, but given that the current time is a moving target, its use is in conflict with prompt caching. Because the new `baibot_conversation_start_time_utc` prompt variable is a more reasonable default, we're now using it in all sample configs. 2024-10-03 11:48:14 +03:00			`prompt: "You are a brief, but helpful bot called {{ baibot_name }} powered by the {{ baibot_model_id }} model. The date/time of this conversation's start is: {{ baibot_conversation_start_time_utc }}."`
Initial commit 2024-09-12 13:44:06 +03:00			`temperature: 1.0`
			`max_response_tokens: 2048`
			`max_context_tokens: 8192`