# Do your evaluations have enough power? **URL:** \ **Category:** Uncategorized\ **Created:** [15 December 2025 17:03 UTC](https://community.epinowcast.org/t/do-your-evaluations-have-enough-power/366 "2025-12-15T17:03:46Z")\ **Posts on this page:** 1\ **Showing post:** 17
**Author:** ![samabbott](https://dub1.discourse-cdn.com/flex005/user_avatar/community.epinowcast.org/samabbott/32/5_2.png) [@samabbott](https://community.epinowcast.org/u/samabbott)\ **Post date:** [27 January 2026 14:24 UTC](https://community.epinowcast.org/t/do-your-evaluations-have-enough-power/366/17 "2026-01-27T14:24:33Z")
> [@nickreich](#): > > Overall, I’m saying YES we should build a model. But, man, it’s going to take a lot of convincing for me to trust that the inferences that come out of it are valid. Yes, I totally agree with this but I also don’t trust the vast majority of evaluation that is done at the moment that looks at lots of different strata in the data in an ad-hoc way so my threshold for thinking we should use a model is much lower. I need to engage with the MCS literature more but on a first pass I assume it can also be represented in a model framework which would be handy as again it means you have just a single set of tools to learn/develop best practices for. > [@johannes](#): > > One issue with regression models I’ve been pondering for a while is how it interacts with propriety. Yup this is the big problem right - depending on how you set them up and as you say one of the arguments @nikosbosse gave for why a transformed score is nice. Again something, I wonder is how often we have similar problems when we are reasoning about a model’s performance by i.e. location and horizon using graphs etc. It would be interesting to try and unpick if the model setup just makes a more common problem obvious. --- _[View the full topic](https://community.epinowcast.org/t/do-your-evaluations-have-enough-power/366)._