Skip to content

[bot] Cohere: v1 Client.generate() / generate_stream() text generation not instrumented #488

@braintrust-bot

Description

@braintrust-bot

Summary

The existing Cohere integration instruments chat, chat_stream, embed, and rerank for both v1 (BaseCohere) and v2 (V2Client), plus audio.transcriptions.create.

The v1 text generation endpointClient.generate() and Client.generate_stream() — is entirely absent from the integration:

Method Description
cohere.Client.generate() Synchronous free-form text generation (non-chat)
cohere.Client.generate_stream() Streaming variant
cohere.AsyncClient.generate() Async variant
cohere.AsyncClient.generate_stream() Async streaming variant

These methods are part of BaseCohere (the shared base for Client and AsyncClient) and are still present in cohere==7.0.2 (the current matrix latest).

What's missing

  • Wrapper function(s) for generate / generate_stream in py/src/braintrust/integrations/cohere/tracing.py
  • FunctionWrapperPatcher classes targeting BaseCohere.generate and BaseCohere.generate_stream (sync and async) in py/src/braintrust/integrations/cohere/patchers.py
  • Wiring into the composite patcher / CoherePatcher and the auto_instrument() path
  • Cassette-backed tests under py/src/braintrust/integrations/cohere/cassettes/<version>/

Upstream sources

  • cohere.base_client.BaseCohere.generate — shared sync surface
  • cohere.base_client.BaseCohere.generate_stream — shared streaming surface
  • Cohere Generate API docs: https://docs.cohere.com/reference/generate
  • Current matrix version: cohere==7.0.2 ([tool.braintrust.matrix] in py/pyproject.toml)

Braintrust docs

The Braintrust integrations docs cover LLM call tracing with input (prompt), output (generated text), metadata (model, temperature, max_tokens, stop_sequences), and metrics (prompt_tokens, completion_tokens, total_tokens). The generate endpoint maps cleanly to this shape.

Local files inspected

  • py/src/braintrust/integrations/cohere/patchers.py — current patcher list (chat, embed, rerank, audio — no generate)
  • py/src/braintrust/integrations/cohere/tracing.py — wrapper implementations
  • py/pyproject.toml — matrix version cohere==7.0.2

Metadata

Metadata

Assignees

No one assigned
    No fields configured for Feature.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions