Next epinowcast meeting (2022-10-20)

Next week Thursday (20 October) is the third Thursday of the month and I have just discussed with Sam that we would like to keep this date for our regular monthly meetings. Previously we had scheduled the meetings for 1pm GMT, but I saw that there was some discussion about the best time in connection with the last meeting. Since I don’t know exactly where everyone is located who would like to attend the meeting, I wanted to ask if this time is okay or if we can agree on something better? I will post the invitation and a zoom link here as soon as we figure it out!

Looking forward to see you next week!

1 Like

This is a good time for the eastern time zone in the US (9:00AM). By the way, 18th is Tuesday… is the meeting on the 20th?

1 Like

Great! And thanks for the correction, I adjusted the date now!

As there were no objections and some positive feedback, we will meet on Thursday at 1pm (UK time) for the next Epinowcast meeting. We can use this Zoom channel for our meeting:

Felix Günther invites you to a planned Zoom meeting.

Topic: Epinowcast meeting
Time: Oct 20, 2022 01:00 PM London

Join the Zoom meeting

Looking forward to seeing you there!

I have a question… Google tells me GMT is different from UK time, which is GMT+1. It seems like this because of daylight summer time (BST)… So it’s actually 1pm UK time (GMT+1)?

1 Like

Yes, I think your right, UK is currently GMT+1!
So our meeting will be at 1pm London, and 2pm NOR/SWE/GER/CHE, 8 am Eastern Time (ET).

1 Like

The UK - always so helpful!

Does anyone have any key points to add to the agenda today + Would anyone like to volunteer to run the meeting?

Our last meeting minutes are here: Short-dated epinowcast meeting, 2022-09-30 - #6 by alison

As far as I am aware no great progress has been made on the issues discussed so far.

I have had a few interesting discussions recently about some pain points people have been having and types of analysis so could summarise those.

I have also been chatting with @sangwoopark about estimation of truncated delays (which are of course extremely relevant to nowcasting). We have discussed adding support for external estimation (perhaps in another package etc) so worth talking a bit about which methods work or not. @sangwoopark would you be interested in giving us 10 minutes summary of this?

Finally, It might be useful to discuss the shape of these meetings, and their frequency. One idea that has been floated is that we could have multiple kinds of meetings (for example a talk based meeting every month and a general meeting also every month (so a meeting every two weeks). Alternatively, we could have a regular informal meeting and then a more formal monthly meeting. What do people think?

Since I took notes last time, I’d be glad if someone else would do it. But I can run the meeting if no one else is keen :wink:

I am happy to take notes.

Just a note about agenda it might be useful to go through blockers in the epinowcast package itself though not much has changed since last meeting sadly.

It might also be useful for us to agree to allocate someone to run the next meeting and take notes at the end of each meeting. The person running the meeting would then make the forum post for the next etc and build the agenda.

Can give a summary if there’s nothing else to talk about.


Hate to miss this. I’m at IDWeek this week but look forward to reading the minutes!

Minutes epinowcast meeting: 2022-10-20

This meeting was led by @adrianlison and minutes were taken by @samabbott. Note: If anyone with edit access thinks anything was missed please adapt these minutes as you see fit.

See the minutes from last time here if missing context: Short-dated epinowcast meeting, 2022-09-30 - #6 by alison

Introductions for new community members

As usual we had a round of introductions for those attending their first meeting. Today these were @sangwoopark (read more about his work here: Sang Woo Park - intro) and @kathsherratt.

Organisational points

Recap from last time: no decisions in meetings. Rough consensus from discussion and then give people a chance to reply on the forum.

We decided that at the end of each meeting someone should volunteer to run the next one. This includes making an open post for the meeting on the day of the last meeting, making an agenda based on peoples suggestions, and running the meeting itself. The volunteer to take minutes will be found on the day.

We discussed how it would be useful to have an organisation account for hosting video calls and discussed the potential for small sources of funding to cover this (as well as the general forum costs which are currently out of pocket).

Finally, we discussed running a seminar series where presenters are invited to talk about real-time analysis issues, we lead workshops on potential use cases, and do deep dives on areas on the code/model to stimulate research and motivate contributions. The thought was this would be every month and be slightly more formal than our current meetings which would continue to be monthly (so we would overall have a meeting of some kind every two weeks)

@samabbott will open a thread on this and lead organising the first round of speakers.

Updates on known users

Various contributors have been contacted by users working at multiple public health agencies on a range of tasks. Most recently these have been people responding to novel outbreaks. Much of this work is not in the public domain but the synthesis of requirements is similar to that discussed here: Nowcasting in a real-time analysis pipeline to estimate the effective reproduction number with missing data and reporting delay

In terms of meeting these requirements once all proposed packages changes are merged in from pull requests and the development version is released we will only be missing non-parametric delays and the ability to forecast without custom code (though @Gunnar has a workaround for this discussed here: Forecasting in epinowcast - #7 by samabbott).

Another frequently raised point was issues with fully understanding all of the options given the time available to deploy the package. To help with this it was suggested that more feedback could be given - for example, if the number of days of data is much greater than the maximum delay (as this will take a long time to run and is unlikely to bring huge amounts of additional information depending on the task at hand) . Generally, we thought it was a good idea as long as it could be turned off from experienced users. One option that was floated was having a default verbose mode and an optional non-verbose mode.

A final common thread was the presence of reporting schedules (i.e by day of the week). This was similar to @teojcryan recent case study (Nowcasting COVID-19 cases by specimen date in England) which suggested we need to be able to easily specify constant priors for days with no reporting.

Epidemic phase bias and right truncation of delay distributions

@sangwoopark very kindly gave us an introduction into some of the issues estimating delay distributions in real-time.

The critical bias he flagged as an example was that when an epidemic is growing quickly we are more likely to see shorter serial intervals due to truncation. One of the questions he is looking at is whether we need to account for this using transmission dynamics, truncation adjustment, or both.
His current results suggest that either work, though dynamic adjustment requires modelling the underlying transmission process, they should not be used together.

Work on this is ongoing and there should be a paper giving more details in the near term (exciting).

Another useful example that was given is estimating the incubation period where left truncation can be an issue due to recruiting people based on if their onset has been observed. Similar methods can be used to correct for this and again the message was only to use one.

We discussed the relationship between this work and nowcasting. In particular, we highlighted how being able to estimate these delays is crucial when using latent reporting delays which we will shortly support in the development version of the package.

From there we moved on to discussing the role of censoring and especially censoring of onsets which are generally only known down to daily resolution and often not even with that granularity. @sangwoopark is also exploring this and interaction with truncation. I have also being playing around for methods to do this with the thought of providing tools for epinowcast users (See here (WIP):

We flagged the fact that epinowcast has all the tools to deal with this censoring in a robust fashion and so thinking about supporting this in the future would make sense.

Data package for nowcasting

Based on: Create a collection of benchmark data sets - #5 by nickreich

We discussed how useful it would be to have an easy way to find example data sets both for our own work and for other researchers. We also discussed that this could be a good second package for the organisation and be a nice entry point for contributors.

The general design we settled on was to have a remote store for the data (like Zenodo) and provide tools in R to download this data along with metadata. For this, to work we would need some kind of schema for how the data should be uploaded and the kind of metadata we would need alongside.

In the first instance, the suggestion was to focus on data sets that could be considered “traditional” for nowcasting and then to expand to include other kinds of data sources (for example to estimate delays).

The first actions we need to take are to lay out the design in the related thread, come up with a prototype name. From there we can set up a skeleton repository in the organisation and move development into the issues there.

@kathsherratt expressed some interest (no pressure!) in taking a leading role in some of this work.

How can we make the model easier to understand?

We discussed a range of ways we can make the model, and the vision for the future, easier to understand. Something that got some traction was writing up a mission + motivation paper so that others can understand our planned development path and how we want to get there (i.e as a community project). @FelixGuenther is going to make a post for further discussion on this.

The current roadmap is nearly documented in math in various PRs but this needs to be expanded on and cleaned up. We also discussed how useful a model schematic would be (@teojcryan schematic was mentioned here).

Some issues were also raised with the fact some parts of the current code base may be hard to understand. We concluded we could take a two pronged approach to this.

  1. Use the wiki functionality of GitHub to map out the connections between the various models, pre and post processing

  2. Highlight areas of the code that are currently hard to understand as GitHub issues and then flag these to contributors. Hopefully this will mean we can improve the commenting over time.

See here for post discussing mission statement: Mission statement/Model outline (manuscript/document)

Progress from last meeting

At the close of the meeting we briefly touched on progress from last meeting. The main issue that was raised is the regression in performance introduced with the switch to the renewal equation based approach ( ). Both @adrianlison and @samabbott have looked at this and the conclusion is that it is likely bug free but that the log based convolutions may be just slower than natural scale ones. Hopefully @samabbott will have time in the coming week to test further. @Gunnar also flagged that he needs this functionality for his surveillance use case so ideally it would be ready sooner rather than later.

We also raised the lack of recovery of simulated missingness in the missing reference date model. Since the meeting @sbfnk has made some progress here by simulating data using the model and then fitting to it. This potentially suggests an identifiability problem with the current parameterization but it is still being looked into (Bug: Missing data proportion recovery · Issue #165 · epinowcast/epinowcast · GitHub)

Planning the next meeting

  • The next meeting is at 1pm UK time on the 17th of November.
  • @FelixGuenther is running the next meeting and will make an open agenda post shortly.
  • @samabbott will make a permanent meeting link.