For reference, here were my earlier accuracy reports: last year 2022, 1st month 2023, 2nd month 2023, 3rd month 2023.

TL;DR-- D/STs good, Kickers = a bad year for all rankers, QB good, Survivor meh, Betting lines prettah prettah prettah good.

Hi Everyone, and Welcome to Another Accuracy Round-Up!

If you’re new to these: Examining predictive accuracy has been a long-standing tradition, underpinning the purpose of Subvertadown. We want to understand "Was this season more or less predictable than normal?" And we want to know whether Subvertadown models are keeping up with some consistently-top-accuracy sources.

How does this help us?

1. With the context of all the randomness we’re playing in, we can understand if it's reasonable to expect better reliability in our fantasy selections.

2. It's a chance to reflect on whether there are possible model improvements, and

3. We hope to validate that it's meaningful to use future-week forecasting from the models.

Compared to Other Seasons

So as usual, here’s a look at how each individual model is doing, compared to other seasons.

This does not tell "how good the models are". It only tells us how predictable the current season is, compared to the historical norm.

The final 5-week stretch was the least predictable period, compared to weeks 1-12. But overall, it appears this season was more predictable than average-- the big exception being kickers. See below for more about this. I've talked all season long about how all rankers are having more trouble than usual with kickers.

After terrible D/ST and kicker predictivity last year, I predicted they would rebound from 2022. I was correct for D/ST, but I was wrong for kickers.

Tiered Flex, but OK

An extra note this time, for those of you who used my Rest-Of-Season charts for WR/RB/TE. The accuracy of these models was apparently pretty good this year. I hope that helped some of you with your lineups.

Comparative Accuracy Assessment

Reminder / for newbies: My goal is to make sure models are performing at a similar level to others, specifically sources that have been consistently good for at least a few years. Many sources are great one year but then poor the next. My chosen experts aren't like that; and most of them you would surely recognize. Anyway, the purpose is not so much to "beat" all of them. Rather, the main goal is to check that my models perform at least at a "similar" level to top sources. It builds confidence and gives reason to trust forecasting, when we extrapolate models to future weeks.

Defensive Maneuvers

D/ST accuracy for "ESPN" scoring

I’m again happy to say that two things have remained true of D/STs, this season, for the case of "ESPN" scoring settings (when "yards allowed" is penalized):

1. D/STs have been more predictable this year for all sources (all rankings appear more "accurate"), which has made a much better streaming experience, compared to the terrible 2022 season.

2. My own D/ST model (ESPN setting) has been #1 in accuracy this season. After measuring myself at #3 in week 4, I have managed to stay at #1 each next period. I don't always expect this, but it's a nice bonus.

Reflections: It’s hard to ask for more from my ESPN D/ST model. I've got a creative idea for improvement, but basically it's doing what I ask of it. Based on my last 6 years doing this, I do feel my models provide the most reliable resource for projections at this scoring setting.

D/ST accuracy for "Yahoo" scoring

For league scoring settings that do NOT penalize D/STs for "yards-allowed", my season accuracy finishes at #3. I think that's good, because this is what I aim for.

However, what is not so great is...

1. This scoring setting always gives more randomness of fantasy points, compared to the "ESPN" D/ST setting, and

2. Unlike most fantasy positions in 2023-- including the ESPN model-- the "Yahoo"-scored D/STs were even less predictable in 2023, than most years. All rankers did worse this year.

Reflections: For the Yahoo D/ST model, I think it's worth revisiting and the model development. There might be possible room for better prediction of turnovers, and my method of in-season adjustments (such as facing a 2nd string QB). Still, by comparing accuracy, it's still evident that the models are robust enough to help with future-week planning.

Here's the Kicker

It's been the same story all year: Kickers in 2023 were way less predictable than usual. For all rankers.

I'm pretty sad about this. Not only had I gotten used to being #1 in kicker accuracy but I was also really enjoying to show to the world that it's possible for kickers to be a legitimately strategic fantasy position. Just 2 years ago, 2021, kickers were looking GREAT, and I was riding an accuracy wave.

The accuracy assessment shows 2 things:

1. Everyone was bad at predicting kickers this year, even best "top 10" sources as measured by FantasyPros. It's at least comforting to know that there was no "good source" out there, this year.

2. It's unclear whether my model was really "better" than other top sources I track. By one measurement, I'm technically #1; but measuring by correlation coefficients, I was lower than 3 others.

By the way, looking at FantasyPros website accuracy rankings, I can see that I'm tracking (1) a top 10 kicker source, (2) a top 10-20ish source, and (3) a top 20-30ish source. So I've got it well covered enough to conclude: nobody had good kicker accuracy this season.

Reflections: My current belief is that, just when I had got kickers all figured out (2018-2021)..., the rules for kickers changed. I suspect the equations are different now-- meaning that Kickers are now used differently by real-world NFL teams, compared to how they were used even 2 years ago. In particular, “going for it on the 4th” is probably the main culprit. This will be my off-season project, to make sure we're ready for 2024.

Two Cents for a Quarterback

I find myself at #2 for QBs. I'm more than okay with that!

The bar-chart at the top of this post shows that QBs have been more predictable than average this year. It's not just my models that think so, as top-rankers have pretty much kept pace.

Reflections: The QB model has been doing better than ever this year, also relative to others sources. My lesson early on this year was to pay more attention to rookies.

Survivor

The below chart shows the failures and success to the 3 pathways that I show on the website.

Looking back, I should have provided more alternatives in week 14; next year I will. The overall failure rate this year was 31% (16 out of 51). That corresponds to an average betting spread of 6. I would normally expect better.

Line Betting

Line betting was overall a success this year, even if we ended up below target. We ended at +32% ROI (relative to the weekly betting amount). While we would naturally hope for +85%, it's still statistically impressive to reach 0%-- let alone our 32%.

For people new to this, remember that by default you should expect to lose 5% per week-- therefore expect to lose 85% over 17 weeks. So a loss of -85% is the baseline expectation, if all bets are a 50-50 chance and if each win returns 190%.

Here is the full plot of bet amounts and gains, for each week. We experienced an undulating pattern this year:

Although I could start the plot at "100" instead of "200", this is a stylistic choice to indicate "plan on the contingency that you might need another 100 at some later point, before the end of the season." (It could as well accomplished by continually adding an extra "5" each week.)

Reflections: You guys have no idea how much I worked on the off-season 2023, to improve this-- and it seems like it paid off. Not only are my game score estimates better, but the strategies for calculating bet amounts-- and timing-- were improved. However, I still believe we can improve and reach the target. I will be playing with several ideas in the off-season, to identify better picks and to reduce weekly risk.

Below you can see the list of actual wagers that have been suggested each week. I posted these each month, and this time I need to issue a correction: I previously posted 3 bets as wins when they were actually losses (weeks 4 and 6). Decimal error, fixed in the below and corrected in the above earnings graph.

TL;DR-- D/ST good, kicker bad, QB good, Survivor meh, Betting lines prettah prettah prettah good.

Thanks to all who followed the season, and I'm looking forward to kicking off advancements with you when the 2024 season kicks-off!

/Subvertadown