Add support for OpenAI's o1 models by making max_response_tokens optional
The other prerequisite seems to be not using a `prompt` (`prompt: null`), but we already supported this. It'd be nice to add an optional `max_completion_tokens` parameter as well, for the benefit of the o1 models, but this is not yet supported by async-openai. Possibly tracked here: https://github.com/64bit/async-openai/issues/272
This commit is contained in:
@@ -14,7 +14,7 @@ pub fn default_config() -> Config {
|
||||
if let Some(ref mut config) = config.text_generation.as_mut() {
|
||||
config.model_id = "mattshumer/reflection-70b:free".to_owned();
|
||||
config.max_context_tokens = 8192;
|
||||
config.max_response_tokens = 2048;
|
||||
config.max_response_tokens = Some(2048);
|
||||
}
|
||||
|
||||
config
|
||||
|
||||
Reference in New Issue
Block a user