Finding Upside in Fantasy Drafts

Welcome to a fresh look at Upside—a rather murky aspect of draft strategy!

This analysis is about a topic a lot of us seem to think is important: finding Upside potential in your draft.  My goal is to bring some fresh insight into what historical numbers have to say. As usual, I will guide you through my assumptions and methodology, in the hopes you’ll enjoy following along while finding inspiration to for thoughts of your own. And in the end, hopefully we can better wrap our heads around this aspect of fantasy strategy.

Today’s Topic is Upside

People talk about it. People believe it’s important. Some people cite Talladega nights (“If you ain’t first, you’re last”) like it’s the only priority. 

But how to make sense of it?

I haven’t yet seen a good assessment showing when Upside potential can give a boost. What’s the best time during the draft? Near the start?  

Upside", as I think Some People see it

To begin, here's the way I think "some" people imagine Upside. And my goal was to figure out that the picture should REALLY look like instead, if we use historical data.

One of the reasons I think people assume Upside can be so big is that they have judged what they call "Upside" by sorting players by their season outcomes-- after the results are known.

Unfortunately I see people assuming too much-- for example claiming that the top-rated players have more upside. They’ll point to top players last year and say things like “You need to take an RB, because looks at Barkley last year— his points above baseline beat anyone else!”  The problem with that logic is: it always looks like top-performing players achieved amazingly great upside, after you see them do it.  But we didn’t know that before it happened!  

This happens all too commonly: casual managers —or analysts without good stats backgrounds— tend to rate player outcomes, after the fact, which is always misleading.  In fact, it’s a fallacy. Barkley was not ranked the RB1 before the season started. He came out the previous year ranked around #13 by half-PPR, and yes after his trade his ADP was maybe up to RB6. But he was not ranked as the RB1 or RB. Meanwhile, how did last year’s RB1 do? Well we don’t need to go into that (hint, he played 4 games and averaged 11 ppg.)

If we actually want to learn something, we need to treat problems ex ante— before the fact of knowing the result.

My goal today is to take a look at one way we might actually quantify “Upside” in a useful way. I acknowledge that my chosen method gives only a rough approximation.  But even without the most precise answers, I think you’ll see that it helps bring a useful perspective to the arguments around Upside. 

And I’m sure the methodology can inspire further work along the same lines of thinking!

Which Rounds in the Draft Give the Most Upside Potential to Different Positions?

In my approach, my main assumption is that we can get meaningful results by looking at positional ranks combined across different years.  What I mean is: let’s look at the RB1s from previous years, as they were ranked before the season started, and we treat them as one group. And let’s do the same for all RB6s for that matter.   Then let’s see how things actually turned out— what were the distributions of point outcomes for each group.

To approximate this, I’m using the assumption that “the previous year’s #1 ranking RB probably represents a reasonable choice to approximate the next seasons #1 ranked RB”, before knowing the results. It’s not perfect, and I can do better in a follow-up. But this will give us a reasonable approximation for finding upside over each player rank. I think it’s fine, except the most obvious weakness that it will exclude rookies. But let’s come back to that later.

To get anywhere with this analysis, I also need an additional assumption: that the MODELED score is a good representation of each year’s actual score projections for the given rank-position.  This is not too uncommon an approximation, and we can expect small errors to cancel out later.

How to get a modeled score prediction? You all can skip this part— it’s more of an aside. My personally preferred method was to perform a multivariate regression on season scoring outcomes, as a function of (a) positional rank for each year and on (b) the rank of sorted outcomes over multiple years. In other words, I take the distribution of “RB1” outcomes for all 12 years used, and I create a formula that can approximate that same distribution— simultaneously for RB2s, RB3s, etc. Here is an example of the fit of distributions:

Revisiting VORP — and Why We Look Beyond it to Find Upside

We need to make the distinction that Upside is not the same as base value. Value-based drafting (my article linked here) assigns value to players as value-over-baseline (Value Over Replacement). You draft top-tier players because you expect them to score a lot; and value-over-replacement measures a player’s relative worth. . But we we say "Upside", we mean whatever they score BEYOND their expected draft value. BEYOND their projected point potential. So let's remember what base value looks like. Here it is

Here’s how the Value-over-replacement then looks (averaged over 12 years), for the 4 fantasy positions:, where the blue parts represent high ranked players who have positive value:

For this particular article, I’m using the term “VORP” generically— I don’t specifically mean the Value over Waivers players, but just whoever we consider “Replacement” players. In fact, we want to represent something closer to “VOLS”, value over last starter: I’ve chosen illustrative baselines at QB13, TE13, RB30, and WR30 to try and capture the value of the strongest starters. (In these examples, I have used 0.5 PPR scoring.)

If we have a good metric to prioritize players, then why are we spending today talking about Upside?  Where does that come in?  The problem we face is that VBD ranking has become widespread over the years.  If you’re in a league where nobody else drafts by VBD principles, then by all means focus purely on Value!  You have a statistical advantage even without considering Upside.  However, more and more teams find they have eked out the value they can.

So, the relevance of upside comes when we’ve leveled the playing field with VBD Value. If everyone in the league has already optimized, then they’re all the same— all average. Then how can a team differentiate itself?     

The obvious answer is you need to be lucky.  But the “Upside Theory” also exists and implies that you need to gamble.  You need some players who have a boom-bust characteristic because:

• If you just have average players, you’ll finish #6 in your league, which means you lose.

• If you’re unlucky with your gamble, it doesn’t really matter, because you still lose anyway (#12 in your league isn’t different from an average #6)

• If your gamble on Upside  pays off—speaking only of the type of Upside that was rational to consider in advance—then you have a strategy to probabilistically propel yourself above average.

This is all to preface: Upside is not "just" variance or deviation. Deviation would include Downside, too. But when we talk about Upside, we're deliberately treating downside as if it's the same as ZERO. There's a reason for this-- it's the philosophy that an average team can't place first, and in that sense it's equivalent to a below-average team. Here's a mock grid of this kind of decision making, to get the point across:

In other words, there’s a “new” game, on top of the value-over-baseline game. You need the base value, of course, but there’s an extra layered game of identifying where you might find upside.  And that’s what I want to try to do now.

Upside: Definition and Analysis

So, in this “new game in town”, the simplification we’re going with is this: when your team scores below average (i.e. when your upside bet doesn’t pay off), the resulting loss isn’t different from returning 0 value.  Instead, what matters is performing above average, and only above.  Everything below average is almost irrelevant: because if you lose out on your gamble, it’s the same as getting a zero anyway (zero above average). 

Let’s use this concept to invoke a definition for calculating “Upside”, that conveniently describes something we can calculate: we measure how much a player’s score goes above Valuation, and we count negative numbers the same as zero. (Oh, and one small correction that lower-ranked players get measured from 0. You shouldn't measure them from their already-negative values.) Some people refer to “beating ADP”, but instead we want to talk about “beating projections”.

(Note that beating ADP is kind of useless to compare. For top-ranked players at their position, beating ADP doesn’t matter. The RB1 can never beat his ADP— Then he’d be what, RB0, RB-Zero? That doesn’t make sense for treating an ordinal distribution. And a low-ranked player beating ADP is a poor measure, because if the WR57 performs as the WR42, does that 15 point difference matter? No. We need to talk in measurements of fantasy points. So we’re making it numerical.)

Stop and think about what this Upside definition implies: we’re moving from using a VBD baseline (that used to be a baseline representing replacement players), and switching to an Upside baseline—which means setting the baseline exactly at the players own original Valuation.  This is something of a simplification, but it let’s us form a picture.

All that leads to one nice combined graph. (1) We can get VORP visualized. (2) We can see the historical scatter at each rank. (3) We can see the deviation from expectations. (4) And FINALLY we can visualize the Upside calculation (bright green dots)-- taking an average while flooring deviations at zero:

This final metric— the green dots—represent the upside calculation. More specifically, the graph is displaying Upside above the baseline point expectations, to make it easier to follow.

The main points are:

• You can see how VBD Value (“VORP”) comes from setting a baseline, by looking at the Blue shading.

• You can see how the scatter of historical results goes above and below modeled point-expectations.

• You can focus on the results that we consider Upside: Highlighted by the yellow lines. These are points that went above expectation OR above the zero-value threshold (for lower ranked players).

• Finally, I have additionally marked with green-ish dots what the AVERAGE UPSIDE has been, for each positional rank.

Is the Average Sufficient?

Maybe Not. There’s an interesting assumption going into this computation, so let’s make sure you and I can roughly agree, using this example.  Imagine that we have two players, both with valuations of “10” ppg above replacement.  The left-hand player  scores +50ppg once during a 10-year period, and all other years are at-Value or negative.  The right-hand player scores +10ppg 5 times over the same 10 year period.

Which is better?  To make conclusions in this study, I’m assuming both players are valued identically: at “on average” a +5 extra points of upside in a given year.  This is flawed and it is also debatable —many of us feel consistency is more valuable—but many others argue that the extreme upside is more worth the swing.  But differentiating between these two options is a more sophisticated discussion, for some other  day.  And I know some of you care about prioritizing those big upside gambles, so I’ll assume “average upside”, to tie up the conclusions.

Now the procedure should be clear:  Players are already “Valued” over their positional replacement, to get a baseline value.  And now on top of that amount, we want to add their “average Upside at the given positional rank”.  The calculation of “upside” is simply :

Finding Trends in the Upside Calculations

To complete this study, I then took these calculated, average Upsides for all positions— the Green dots in the above plot— and I searched for rough trends with dependence on player VORP. VORP is preferable as a reference (instead of using player rank), because then we can compare the positions against each other, and — you’ll see after a couple more steps— we can even get a feeling for how the different fantasy positions are “distributed when ordered by value”— i.e. what a typical draft rankings list would look like. To show simple, uncluttered versions, here are the fits for TE and QB:

There’s the usual amount of scatter I would expect, but I think the trends are apparent.

Now let’s re-plot these examples and put all the positions together with color coding. Here’s the picture of Upside dependency that emerges:

That’s pretty cool! I think this result is fascinating, even though we know it’s based on some loose interpretation of player pre-season rank.  It’s not the result I expected to see.  There’s surely some scatter, but it’s also fairly easy to pick out some trends. 

• FOR EACH POSITION; THERE ARE DIFFERENT TIMES IN THE DRAFT WHERE UPSIDE SEEMS TO OCCUR MORE STRONGLY, RELATIVE TO PROJECTION.

• A normal VBD draft ranking will sort players by VORP. So what we can see here is a visual of the draft progressing from right to left. You can see that the different fantasy positions have different Upside potential from each other, at different stages of the draft.

• The magnitude of upside doesn't get much larger then 1.5ppg. I think that's a lot lower than some people imagine. This is a consequence of looking at things appropriately ex ante. It means that Upside is distributed with uncertainty across the ranks. (The magnitude would of course be higher if we excluded the negative numbers instead of treating them as zeros.)   

• RBs have highest Upside, and it seems uniform across the draft. You can get lucky Upside from RBs at most stages; it’s distributed over most draft pick ranges and not concentrated only at the top.

• THE SCALE OF UPSIDE, when averaged, is much lower than I think people expect. In the above example, it seems reasonable that you should not expect more than 1.5 points of upside.

• TE1s have surprisingly held up by this measurement, despite bust years which every position has.

Relative Upside — Guidance for Choosing Upside Among Different Positions

There’s one final step.  We need to be able to compare the positions against each other. We can't just leave the analysis saying "Always take RBs", because -- sorry to say-- you need to fill your roster with the other positions besides RBs, too! What we REALLY want to know is "When is it RELATIVELY better to take an RB during the draft, compared to other stages?" We want to know when you might find more upside from a position, relative to other times in the draft.

And to accomplish that, we set the bottom of each curve to zero (counting only positive VORP-- though I'm taking a special liberty with the RB1.) Just for visualization, I'm redistributing the scattered points back onto the curves. Don't over-interpret it. This is not to pretend the scatter isn't there, it's just to represent the approximate draft picks, along the fit curves.

I’ve also labeled the points on the graph, from right to left, to indicate the approximate Draft Pick. Naturally those would depend on the scoring setting (I’m using 0.5 PPR), and it will depend on the baseline assumption that I made. But I think the above picture is indicative.

I think the best way to understand the "surprise" of the results is that Upside is SPREAD OUT over all positions in the draft. The season might end with RB8 having the most surprise upside. Or it might be the RB12 or RB25. A truer ADP-based study may change this outcome, and I'll be working towards that.

The final picture

So that’s about it guys!  That’s the picture I’ve been working towards.  Before we blindly draw conclusions from it, let’s remember that there are assumption to revisit: Using actual player ADPs, using actual projections, and finding more sophisticated ways to treat the consistency of upside.  And the big missing pool of players we excluded by doing it this way: The rookies!  By the method I chose, they’re not even represented here.

So yes, it will be worth repeating this exercise with better data, but I still think we can see some valuable observations:

The overall point is that “Upside” is not relatively concentrated at the top of the draft.

That means, before the season starts, the possibility for upside is “SPREAD OUT” over all positions in the draft.

And don’t forget the important point that we have measured Upside relative to VORP. So what you might have previously considered the “Upside” of the top players is actually just their baked-in value, and the whole reason for drafting them high in the first place. You want a solid base in your first rounds. And then the “Upside” is just anything that happens to go over those large amount of expected points.

Now let’s look at the individual positions.

• QB The top 1 or 2 QBs offer some limited upside, but otherwise mid-late round QBs give that opportunity.

• RB Upside is not only concentrated among top RBs—many 2nd and 3rd round RBs offer at least as much potential.  RB2 and RB3 have been giving more upside than RB1.

• WR WRs offer appear to offer the most “relative upside” later in the draft of starting rosters.

• TE The TE1 has been a reasonable gamble for upside (relatively speaking), otherwise the later round TEs are better for that gamble.

Finally don’t forget that you can’t just value upside alone.  The whole point is that you need to grab Value-over-replacement first, and then consider upside as an extra.  

Total potential = Value + Upside

Thanks if you followed along!  I hope this helps you as you approach your draft strategy.  For a lot of us—myself included—it has been enormously difficult to find a way to judge draft order positional value, and I hope this paves the way to better analysis and understanding, in the future.

/Subvertadown