Dec 6th: Click here for the week 14 community discussion

December 4th, 2022

Welcome to my 3rd monthly "accuracy review" for 2022. The same post appears here.

As always to start, here's the chart of all positions, with some notes below:

Notes:

• This year 2022 really has been a significantly less predictable season than any year I've seen before. Even Vegas has had a tougher time. This circumstance makes it hard to draw conclusions about the models' fidelity this season.

• RB and Kicker appear to be "not working" -- but we also know these specific positions have been more fickle this year. So it's best to check how other sources are doing (see below, for kicker), when trying to infer whether the models would be somehow “unsuitable” for 2023. (I don't think so.)

• As noted before, the other noticeably struggling position is D/ST. As for kicker, comparison to other sources is interesting. (Spoiler: It's all sources whiffing.)

• QB is the one thing that has been absolutely rocking. Accuracy is exceeding any other year. I'm pleased with this one.

Comparative Accuracy Assessment

Reminder / for newbies: My goal is to make sure models are performing at a similar level to top ranking sources, to trust that future week projections have meaning, and thereby improve our strategy. And as long as they hit similar high marks, then we can maintain confidence in the underlying statistical trends backed by hard data.

Although I don't label the other sources' names this year, they are the dependable year-after-year top-tier ranking sources. You would surely recognize them. While it's great if I can "beat" them (for a few years it looked like that at D/ST), I know it can’t be expected every year.

Defensive Maneuvers

There have been more complaints than usual this year, but the fact remains: The very best ranking sources in the world have all struggled at D/ST, in 2022. Remember that 2021 was also tough, but at least each week’s top 2 picks were averaging 9-10 fantasy points. This year, no source has managed to have their top 2 picks average above 8 points! (not shown).

Here's the Kicker

As reported last time, I'm taking a different view on Kicker assessment, because correlation coefficients seem to make a poor measurement: (1) Correlations have been "negative" for multiple weeks this year. (2) Even on a "good week" (meaning when top-ranked kickers do okay), the measurement is disturbed by surprise booms among bottom-ranked kickers.

We can at least look at the average point profiles, for now. Showing this graph is something I've done the last few years, and I also reported it last month. For each data point, it shows "what was the average fantasy score of the top N kickers, for the given ranking source?"

This is a little like reading tea leaves... but here's what I see:

• I just want to say that the #4 source (orange) is widely seen as a very dependable one, despite his seemingly mediocre performance.

• Last year's #1 kicker comes out the best when measuring all top 7+ kickers--- However, it is has been very poor at recommending the top 3-4. Tough to conclude on reliability.

• Last year's #3 is pretty good at choosing the #1 kicker each week, but really bad with the #2 kicker.

• Although kicker selection has been tragic this year for everyone, it appears my model has been somehow not been doing worse, in comparison to the others.

Two Cents for a Quarterback

QBs have had a great year for predictability. I'm overall really glad the model is keeping up with external experts, who are famously top-notch at it. Again, "just keeping up" with these guys gives good reason to trust model projection forecasts for future week planning (holding).

It is interesting that last year's #1 and #2 sources have apparently switched places with #3 / #4.

So that's it! As I wrote before "this will not be a main focus of mine this year. I mainly use the top graph so I can check if my new model revisions are keeping pace. But it also helps psychologically just to understand if we're in an easy or tough season. The answer is 2022 is (and I think will continue) to be tougher than other seasons of the last 5 years. Hold tight!"

The story is the same 8 weeks later. See you again in week 16, and good luck in the meantime.

EDIT: UPDATING WITH BETTING LINES INFO.

Sorry I actually missed this before, but here is the betting line results as of week 14.

Because it looks mostly negative, I wanted to include a couple comments of optimism:

• I made some important updates during the season, to address issues I saw this season. I know it doesn’t help now, but it’s been a learning process. The main updates are: (1) adjusting for league-wide point totals (the season started off particularly low-scoring) and (2) improved calibration for games outside the average range— accounting for changes in deviation from the median.

• I view essentially the losses as coming from the week 10 upsets. That is not fair to single out one week, statistically speaking, but it was a particularly shocking week that I made an extra comment about. All other weeks would have left us slightly above break-even. The adjustment #2 mentioned above would have lessened that impact.

• Finally, I have actually been doing better than shown, because the below graph only represents the bets on Closing lines. I personally make at least half my bets at the week start— some of those disappear soon, or they become riskier. I do not have a log of opening line bets (I haven’t been recording it), but I think it’s worth mentioning.

Feeling better as we close out. Fingers crossed!

/Subvertadown