What
Generalize DataReader from Vec<f64> to Vec<T> by introducing a FloatElem trait, so that dtype=f32 data from file sources can stay as f32 through the entire read path.
Why
DataReader::read_batch returns Vec<f64> only (qdp-core/src/reader.rs; the comment at line 100 marks this as structural debt). Until this is fixed, dtype=f32 file sources are silently cast to f64, preventing a true f32 pipeline.
How
- Add
FloatElem marker trait with f32 / f64 impls (or reuse existing Dtype mapping) in qdp-core/src/types.rs or a new qdp-core/src/float_elem.rs
- Change
DataReader → DataReader<T: FloatElem>, read_batch -> Result<(Vec<T>, usize, usize)>
- Update
StreamingDataReader<T> the same way
- Fix call sites that no longer compile (minimal — non-Parquet readers may still produce
f64 for now)
- No behavior change on existing
f64 paths; trait docs explain the f32 end-to-end goal
Out of scope: Parquet schema/cast logic, pipeline_runner wiring
Acceptance criteria:
What
Generalize
DataReaderfromVec<f64>toVec<T>by introducing aFloatElemtrait, so thatdtype=f32data from file sources can stay asf32through the entire read path.Why
DataReader::read_batchreturnsVec<f64>only (qdp-core/src/reader.rs; the comment at line 100 marks this as structural debt). Until this is fixed,dtype=f32file sources are silently cast to f64, preventing a true f32 pipeline.How
FloatElemmarker trait withf32/f64impls (or reuse existingDtypemapping) inqdp-core/src/types.rsor a newqdp-core/src/float_elem.rsDataReader→DataReader<T: FloatElem>,read_batch -> Result<(Vec<T>, usize, usize)>StreamingDataReader<T>the same wayf64for now)f64paths; trait docs explain the f32 end-to-end goalOut of scope: Parquet schema/cast logic, pipeline_runner wiring
Acceptance criteria:
cargo test -p qdp-corepasses on Linux