What
Wire PipelineConfig.dtype through read_file_by_extension and the producer layer so that f32-configured pipelines actually reach the f32 encoder kernel when loading from file sources.
Why
read_file_by_extension always returns Vec<f64> (pipeline_runner.rs). InMemoryProducer and StreamingProducer populate BatchData::F64 even when config.dtype == Float32, so the f32 encoder is never reached from file sources. The user's dtype choice is silently ignored.
How
read_file_by_extension (and streaming path) selects the f32 vs f64 reader based on PipelineConfig.dtypeInMemoryProducer / StreamingProducer use the matching BatchData variant- Python API (
QuantumDataLoader.source_file) unchanged at the surface
- Integration test: f32 Parquet +
dtype=f32 config → encode_batch_f32_for_pipeline path is exercised
Acceptance criteria:
What
Wire
PipelineConfig.dtypethroughread_file_by_extensionand the producer layer so that f32-configured pipelines actually reach the f32 encoder kernel when loading from file sources.Why
read_file_by_extensionalways returnsVec<f64>(pipeline_runner.rs).InMemoryProducerandStreamingProducerpopulateBatchData::F64even whenconfig.dtype == Float32, so the f32 encoder is never reached from file sources. The user's dtype choice is silently ignored.How
read_file_by_extension(and streaming path) selects thef32vsf64reader based onPipelineConfig.dtypeInMemoryProducer/StreamingProduceruse the matchingBatchDatavariantQuantumDataLoader.source_file) unchanged at the surfacedtype=f32config →encode_batch_f32_for_pipelinepath is exercisedAcceptance criteria:
dtype=f32config →encode_batch_f32_for_pipelinepath is used