Here’s a mathy post to improve your understanding of the game, with a stats breakdown that I had searched for but never found. The goal is to give some perspective on how predictable / unpredictable fantasy is, in order to hopefully give a more intuitive feel for the game you're playing. This post is approaching a dry "Introduction to Statistics", but I think some of you will enjoy.

Motivation

Mostly, I always wondered how reliable fantasy projections are. Not just how good a certain ranker is, but how good a ranker can possibly become. I'm going to try to describe statistics and results in an accessible way for everyone: I will dissect correlation coefficients and distributions, which might even help your understanding of basic stats.

Correlation coefficients, by position

The following bar graph shows how the fantasy point outcomes correlated with projected scores (in 2018, standard, ESPN/NFL default). Correlation coefficients theoretically range from -1 to +1.

When I first created this analysis, I chose to analyze the score projections from fantasyfootballanalytics.com, for several reasons: (1) it is a source that generates "average best" projections by combining top sources, which means (1a) it is unbiased for that reason, and (1b) it represents sources you might have used. Furthermore (2) the past season projections are readily available and not too terrible to scrape from the website. (2b) Also: the website owner cares about stats, and the weekly projection feature of the site is free access.

Note the categories in the bar graphs: for each week, I went through the extra step of classifying each player as "the WR1/RB1" or "the WR2/RB2" on a team, so they could be evaluated separately. (Yes it's a big pain to account for injuries). For the remainder of this post, I will leave out WR2 and RB2. Finally, you can see I've also included Vegas' accuracy for predicting NFL team scores. The Kicker position, which stands out as unpredictable for most ranking sources.

Can you see the correlation?

For more intuitive understanding, here's the scatter-plot for Projected QB Scores vs. Results. Remember, these are your so-called trusty, dependable quarterbacks. It’s a blur! Whenever you're trying to understand the game or set your expectation levels, I find this image always helps to remember how much randomness there is, even with the best projections.

_{Actual vs. Projected weekly fantasy points for QBs}

Point Variance levels from different positions

Correlation, alone, does not convey everything about how each position impacts the randomness in your fantasy roster. Yes, Kickers are difficult to rank in advance… but their range of points is generally narrowest.

The conclusion is: even a terrible correlation can possibly have minimal consequence. Example: Imagine if, in a given week, all the kickers expected for 7 points actually turn out to score 8 points— and vice versa. That kind of correlation is awful (negative!), but the consequence would be minimal for your team: just a single point (in this imaginary case).

Fantasy point distributions — vs. distributions of errors

In a related post, I try to give a sense of how much error is usual, for the different fantasy positions. Check out that analysis to see the shapes of point distributions— and to understand how much of that variance is removed from rankings.

For now, here is the bar chart that summarizes the projection errors, for all positions. In the next sections I will compare this pattern of error levels against the “predictable range”.

How much is predictable

Keen skeptics might brush off the low error of kicker projections and instead ask how much "control" each position gives you. ("So what? I know about the randomness. What I care about is increasing the total expectation value of my team's score.") That's true; when our choices can create more points (on average), it becomes a game of skill. If projections have no correlation with outcome, then you're just flipping coins, which is no fun.

So we’d like to know: how much of a controllable/predictable range is provided by each position? ("How much point difference can it make, by picking good players over bad players?")

I propose the following measure, under the assumptions of normal distributions: Multiply the correlation coefficient by the standard deviation of score results. (Equivalently, and maybe more intuitive: multiply the slope (outcomes vs. projections) by the standard deviation of projections.) This re-scales the projection range to "controllable" outcome range. (Bonus 3rd equation: divide covariance by the standard deviation of projected scores.)

This range can also be approximated from how much the distributions narrow after projections are subtracted. In my post about this topic, we saw that the range for QB decreased from 8.0 to 7.5. This means QB projections provide a “control” of Sqrt(82 - 7.52)— which is just under 3 points. (Not 0.5.)

Here's a bar chart showing what the first calculation yields for each position:

As you can see, RB and QB give the widest range of predictable scores. Over many games, you can influence a higher point difference above your opponent. Kicker projections help least in controlling points, which means most kicker rankings don’t let you influence your roster’s score by much.

Putting them together: Reward/Risk

I’ve just shown two bar charts above— one showing the point error made for each position, and one showing the controllable/predictable range each position allows. You might have noticed that those bar graphs actually look similar! It turns out that positions with more point randomness also tend to have more controllable points as well.

This fact makes it interesting to compare: how much is predictable vs. how much is random error? Let’s divide predictability by total error. The result is the left-hand plot. For comparison, the right-hand chart is a repeat of the correlation coefficients shown at the beginning.

You shouldn't be surprised, but there it is: the graph looks the same! We're right back where we started. (The measures are very closely related mathematically, except correlation coefficients are divided by total deviation.)

The point is: in fantasy football, the randomness remains significantly large— even after subtracting projections, and this is the reason that these two measures look similar.

I also hope that you understand the meaning behind the correlation coefficient just a little bit better. Correlation coefficients are representative of the skill/luck ratio.

But let’s re-cap the meaning of the left-hand chart: Most positions (individually) let you predictably control roughly 35%-45% of the points you get, relative to the amount of randomness they introduce. (Kickers are the exception, giving even more relative randomness.) If you're like me, a game is more fun when there's more strategy than randomness-- say, playing Risk rather than Candy Land. Combining the risk and reward factors, you could say that this ratio and correlation coefficient indicate how enjoyable the game can be, in this sense. I would not want to play a game gambling only on a single kicker!

But we will see, in another related post, that this conclusion looks very different for a team roster situation.

Trying to conclude on kickers

Obviously this whole post was about a lot more than kickers. But here are my concluding thoughts on that one issue. Ultimately, whether to include kickers in your league comes down to personal opinion and your feelings of "what fantasy means" to you. Maybe you feel that special teams does not represent the main action/play-making element of the game. Point is: math is unlikely to change that. You could make arguments from any of the above angles.

Personally, I still stick to the opinion that (1) kickers are part of overall NFL point generation while affecting game flow, (2) individual kicker error is relatively small, even after accounting for the small predictability of score, and (3) kicker randomness is even smaller after factoring in the reduction in variance when combined in a total team score-- so removing them has very little effect.