Skip to content

[Refactor] Add FloatElem trait and DataReader<T> #1339

@0lai0

Description

@0lai0

What

Generalize DataReader from Vec<f64> to Vec<T> by introducing a FloatElem trait, so that dtype=f32 data from file sources can stay as f32 through the entire read path.

Why

DataReader::read_batch returns Vec<f64> only (qdp-core/src/reader.rs; the comment at line 100 marks this as structural debt). Until this is fixed, dtype=f32 file sources are silently cast to f64, preventing a true f32 pipeline.

How

  • Add FloatElem marker trait with f32 / f64 impls (or reuse existing Dtype mapping) in qdp-core/src/types.rs or a new qdp-core/src/float_elem.rs
  • Change DataReaderDataReader<T: FloatElem>, read_batch -> Result<(Vec<T>, usize, usize)>
  • Update StreamingDataReader<T> the same way
  • Fix call sites that no longer compile (minimal — non-Parquet readers may still produce f64 for now)
  • No behavior change on existing f64 paths; trait docs explain the f32 end-to-end goal

Out of scope: Parquet schema/cast logic, pipeline_runner wiring
Acceptance criteria:

  • cargo test -p qdp-core passes on Linux
  • Trait docs explain f32 end-to-end goal
  • No behavior change on existing f64 paths

Metadata

Metadata

Assignees

No one assigned

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions