Recently came across two live modelling challenges relevant to real-time / outbreak modelling. They are extremely different (development, audience) and thought it was kind of an interesting contrast, so sharing below.
(1) Uni Bristol SPHERE-PPL challenge: https://github.com/SPHERE-PPL/NHS-EAD-forecast
- Forecasting daily capacity strain in terms of “avoidable deaths” (based on bed demand from emergency visits)
- Challenge fully designed with the Bristol NHS trust; for the winner - with all evaluation scored by lowest overall MSE - code will be handed over to the NHS trust for implementation (if I understand correctly).
(2) MIDAS scenario hub “cryptic” challenge
- Global pandemic simulation, aimed at scenario modelling for (?) policy intervention. Extremely fancy website (the simulation extends to full NYT news coverage…).
- Would be interested to see a more specific evaluation plan; what I can find heavily focuses on consistency with data/ with other models rather than use-case.
For pathogen-specific parameters such as the case fatality rate and infection fatality rate, we will evaluate the accuracy of model estimates against the ground truth synthetic data. For parameters that are model-dependent such as Rt, we will compare estimates across different models and methods as well as against the ground truth