Add support for OpenAI's o1 models by making max_response_tokens optional
The other prerequisite seems to be not using a `prompt` (`prompt: null`), but we already supported this. It'd be nice to add an optional `max_completion_tokens` parameter as well, for the benefit of the o1 models, but this is not yet supported by async-openai. Possibly tracked here: https://github.com/64bit/async-openai/issues/272
This commit is contained in:
@@ -131,12 +131,15 @@ impl ControllerTrait for Controller {
|
||||
|
||||
let max_tokens = text_generation_config
|
||||
.max_response_tokens
|
||||
.try_into()
|
||||
.expect("Failed converting max_response_tokens from u32 to i32");
|
||||
.map(|max_response_tokens| {
|
||||
max_response_tokens
|
||||
.try_into()
|
||||
.expect("Failed converting max_response_tokens from u32 to i32")
|
||||
});
|
||||
|
||||
let request = ChatBody {
|
||||
model: text_generation_config.model_id.clone(),
|
||||
max_tokens: Some(max_tokens),
|
||||
max_tokens,
|
||||
temperature: Some(temperature),
|
||||
top_p: None,
|
||||
n: Some(1),
|
||||
|
||||
Reference in New Issue
Block a user