AI GitHub
River for learning from data streams
Update models and metrics one observation at a time in streaming applications.
Key Takeaways
- River estimators expose incremental prediction and learning.
- Choose the ordering and feature availability of observations before constructing the pipeline.
- Predict and score each item before updating the model on its label.
- Using future data in preprocessing invalidates the stream evaluation.
How it works
River estimators expose incremental prediction and learning. Transformers and metrics can participate in the same stream without fitting a complete batch first.
What you need
Choose the ordering and feature availability of observations before constructing the pipeline. If labels arrive later, make that delay part of the evaluation protocol.
What to try first
Predict and score each item before updating the model on its label. Simulate delayed labels and a shift in the input distribution.
Track performance over consecutive time windows rather than one cumulative score. A recent failure can disappear inside a long history of good predictions.
Where to look for failures
Using future data in preprocessing invalidates the stream evaluation. Track label arrival time and memory growth as well as accuracy.
Related Franklin pages
A random train-test split can hide time-dependent failures. Use rolling or sequential evaluation and match the feature information available at each prediction time.
- Build and evaluate forecasting and other time-series models with consistent interfaces. sktime for composable time-series learning
- Build predictive pipelines with consistent preprocessing, validation and estimator APIs. scikit-learn for reproducible machine-learning baselines
For a primer on learning from data and evaluating predictions, see Franklin’s glossary. Machine learning
Project documentation
Try it for your workflow
Explore the project’s documentation for setup and usage, or save this page to a collection for later.
Read the project documentation ↗