I originally did not expect the time intensity involved with creating and maintaining models.  At first, I thought some clever tricks could find patterns and quickly “crack the code”.  Unfortunately, I learned there are huge time sinks. 

For many followers, the Subvertadown project has been a story of continual improvement. The models require continuous maintenance, to consistently perform well.  I recently read the Superforecasters book, which reaffirmed by belief: for the best forecasters, everything is in “perpetual beta”. There never comes a point of feeling it’s finished product! Every new season has seen a new modeling perspective: not just new variables, but new considerations around how to segment situational aspects.

And in-season, I have spent a lot of time analyzing the model results, so I could identify and explain what is causing an observed trend.  (Why is team X rated lower than they were expected last week?)  That has been automated and turned into a new feature on the site, which is another body of work to make improvements to features.

The Subvertadown project has always been about more than the models themselves: it has often been about trying to make sense of things— generating new ideas, a lot of trial-and-error, and then incorporating new learnings. It will always be a Work in Progress.