feat(litellm): handle reasoning_content and reasoning fields in streaming

[daewoongoh]

Related GitHub Issue

Closes: #447

Description

LiteLLMHandler.createMessage overrides the streaming loop independently from BaseOpenAiCompatibleProvider, but was missing the logic to extract reasoning_content / reasoning fields from stream deltas. As a result, thinking output from reasoning models (e.g. DeepSeek-R1, QwQ-32B) routed through a LiteLLM proxy was silently dropped and never surfaced in the UI.

How it was fixed:
In the streaming loop of lite-llm.ts, added the same reasoning extraction pattern already present in BaseOpenAiCompatibleProvider:

• Check delta.reasoning_content first, fall back to delta.reasoning
• Skip empty / whitespace-only values
• Yield a { type: "reasoning", text } chunk when a non-empty value is found

No changes to types, configuration, or other providers were needed.

Test Procedure

Unit tests (src/api/providers/__tests__/lite-llm.spec.ts):

Added a "reasoning field handling" describe block with 4 cases:

1. reasoning_content delta yields a reasoning chunk
2. reasoning delta yields a reasoning chunk (fallback)
3. When both fields are present, reasoning_content takes precedence (only one chunk emitted)
4. Empty / whitespace-only value does not yield a chunk

Run locally with:

node_modules/.bin/vitest run --root . api/providers/__tests__/lite-llm.spec.ts 
All 26 tests pass.

Pre-Submission Checklist

• Issue Linked: This PR is linked to an approved GitHub Issue (see "Related GitHub Issue" above).
• Scope: My changes are focused on the linked issue (one major feature/fix per PR).
• Self-Review: I have performed a thorough self-review of my code.
• Testing: New and/or updated tests have been added to cover my changes (if applicable).
• Documentation Impact: I have considered if my changes require documentation updates (see "Documentation Updates" section below).
• Contribution Guidelines: I have read and agree to the Contributor Guidelines.

Screenshots / Videos

N/A

Documentation Updates

• No documentation updates are required.

Additional Notes

The fix is intentionally minimal — only lite-llm.ts is touched on the implementation side. The pattern is identical to what BaseOpenAiCompatibleProvider already does, so behavior is consistent across providers.

Get in Touch

hehegwk_23849

Summary by CodeRabbit

• New Features
  • Streams now extract and emit model-provided reasoning content as distinct "reasoning" chunks when present and non-blank.
• Tests
  • Added tests validating extraction behavior: prefers explicit reasoning content when both fields appear, and suppresses empty or whitespace-only reasoning.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: e236b010-0ff9-488e-b948-ad013e4067c0

📥 Commits

Reviewing files that changed from the base of the PR and between dd0ff5b and 6314c15.

📒 Files selected for processing (1)

• src/api/providers/__tests__/lite-llm.spec.ts

🚧 Files skipped from review as they are similar to previous changes (1)

• src/api/providers/tests/lite-llm.spec.ts

---

📝 Walkthrough

Walkthrough

LiteLLMHandler's streaming message creation now processes reasoning fields (reasoning_content or reasoning) from model deltas, emitting them as reasoning chunks. The implementation adds twelve lines of detection and yielding logic, and the companion test suite validates field extraction, precedence, and blank-text suppression across six test cases.

Changes

Reasoning Field Support

Layer / File(s)	Summary
Reasoning field extraction and emission src/api/providers/lite-llm.ts, src/api/providers/__tests__/lite-llm.spec.ts	Streaming loop now checks each delta for reasoning_content (preferred) or reasoning fields; when present and non-blank, emits type: "reasoning" chunks. Tests verify extraction from both fields, correct precedence when both are present, single emission behavior, and suppression of empty/whitespace-only reasoning.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Poem

🐰 I nibble on bytes where the model thinks deep,
I pull out the musings it used not to keep.
From whitespace and silence I gently refrain,
Now reasons and thoughts hop back into the frame. 🥕

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly and specifically summarizes the main change: adding reasoning field handling to the LiteLLM provider's streaming implementation.
Description check	✅ Passed	The description includes all required sections: linked issue (#447), detailed implementation explanation, comprehensive test procedure with verification command, completed pre-submission checklist, and additional context.
Linked Issues check	✅ Passed	The PR directly addresses issue #447 by implementing reasoning field extraction in LiteLLMHandler's streaming loop, matching BaseOpenAiCompatibleProvider's pattern and resolving the dropped reasoning output bug.
Out of Scope Changes check	✅ Passed	All changes are within scope: only lite-llm.ts and its test file were modified to add reasoning field handling, with no unrelated alterations to types, configuration, or other providers.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/api/providers/lite-llm.ts`:
- Around line 238-248: The current logic in `src/api/providers/lite-llm.ts`
inside the loop over keys "reasoning_content" and "reasoning" breaks after
finding the key even if its value is empty or null, preventing fallback to the
other key. To fix this, modify the code to only break the loop if the reasoning
text is non-empty after trimming. This ensures if `reasoning_content` is present
but empty or null, it falls back properly to `reasoning`. Apply the same fix
pattern in `src/api/providers/base-openai-compatible-provider.ts`. Additionally,
add a test case in `lite-llm.spec.ts` covering when `reasoning_content` is null
but `reasoning` contains valid text to verify the fallback behavior.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: aba7faab-689b-45fe-92bf-f0c10079900f

📥 Commits

Reviewing files that changed from the base of the PR and between f1f7cb4 and dd0ff5b.

📒 Files selected for processing (2)

• src/api/providers/__tests__/lite-llm.spec.ts
• src/api/providers/lite-llm.ts

[codecov]

Codecov Report

❌ Patch coverage is 90.90909% with 1 line in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
src/api/providers/lite-llm.ts	90.90%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!