compare models that are using backfill vs those that aren't

[nickreich]

Which models are accounting for backfill? @brookslogan and @d-osthus are any of your models explicitly accounting for the backfill uncertainty?

[lcbrooks]

We added backfill modeling back in during the season, so for Delphi models:
Retrospective forecasts: no
Earlier 2017/2018 forecasts: no
Later 2017/2018 forecasts, uniform and empirical trajectories models: no
Later 2017/2018 forecasts, all models but uniform and empirical trajectories: yes
I can provide retrospective forecasts files with the backfill modeling if this will help.

[lcbrooks]

We added backfill modeling back in during the season, so for Delphi models:
Retrospective forecasts: no
Earlier 2017/2018 forecasts: no
Later 2017/2018 forecasts, uniform and empirical trajectories models: no
Later 2017/2018 forecasts, all models but uniform and empirical trajectories: yes
I can provide retrospective forecasts files with the backfill modeling if this will help.

[d-osthus]

Hi Nick, The DBM model does account for backfill. Dave From: Nicholas G Reich <notifications@github.com<mailto:notifications@github.com>> Reply-To: FluSightNetwork/cdc-flusight-ensemble <reply@reply.github.com<mailto:reply@reply.github.com>> Date: Thursday, March 22, 2018 at 8:04 PM To: FluSightNetwork/cdc-flusight-ensemble <cdc-flusight-ensemble@noreply.github.com<mailto:cdc-flusight-ensemble@noreply.github.com>> Cc: Dave Osthus <dosthus@lanl.gov<mailto:dosthus@lanl.gov>>, Mention <mention@noreply.github.com<mailto:mention@noreply.github.com>> Subject: [FluSightNetwork/cdc-flusight-ensemble] compare models that are using backfill vs those that aren't (#120) Which models are accounting for backfill? @brookslogan<https://github.com/brookslogan> and @d-osthus<https://github.com/d-osthus> are any of your models explicitly accounting for the backfill uncertainty? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#120>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AfiVSqb_q8Iqp6BfIuSeaDVYw8_6ebwNks5thFhIgaJpZM4S4D5_>.

[nickreich]

Thanks @d-osthus !

[nickreich]

@brookslogan It would be useful I think to have the retrospective forecasts for Delphi models that do account for backfill. For now, I think we should treat these as completely separate models with their own folders. Would that work ok for you? I've included a analysis in the draft manuscript (coming to you hopefully in the next week or so) that attempts to show the impact of backfill on forecast skill, although in some ways the most clear comparison might be to do a comparison with delphi models with vs without backfill accounting...