fix: parse reasoning output from LM Studio reasoning models#5584
fix: parse reasoning output from LM Studio reasoning models#5584angelplusultra wants to merge 3 commits into
Conversation
shatfield4
left a comment
There was a problem hiding this comment.
Tested using deepseek/deepseek-r1-0528-qwen3-8b and everything worked as it should. LGTM.
…n-stream parse | preserve todo comment
|
should the i'm experiencing this problem with the LocalAI provider, and i'm sure Ollama and other providers have the same problem because they all use OpenAI-compatible endpoints, requiring the same fix. i think this would be a good opportunity to do that |
Yes, there will be some sort of unified refactor for all these identical |
Pull Request Type
Relevant Issues
resolves #5583
Description
The LM Studio provider was using the generic
handleDefaultStreamResponseV2stream handler, which only readschoices[0].delta.content. LM Studio's OpenAI-compatible endpoint emits reasoning tokens for reasoning models in a separatereasoning_contentfield (same shape as DeepSeek), so those tokens were silently dropped on both the streaming and non-streaming paths and the UI never showed the model's thinking output.Streaming (
handleStream) — replaced the default handler with a bespoke implementation modeled after the existing DeepSeek/Foundry handlers:delta.contentanddelta.reasoning_contentper chunk.<think>tag and streams subsequent reasoning tokens through verbatim.</think>tag, flushes the buffered reasoning intofullText, and resumes normal content streaming.finish_reasontermination logic from the default handler.Non-streaming (
getChatCompletion) — added a#parseReasoningFromResponsehelper that, whenmessage.reasoning_contentis present, prepends<think>{reasoning}</think>to the response content before returning it astextResponse. Same shape the streaming path produces, so downstream rendering is consistent across both modes.Visuals (if applicable)
Additional Information
Developer Validations
yarn lintfrom the root of the repo & committed changes