Finding Upside in Fantasy Drafts

Welcome to a fresh look at Upside—a rather murky aspect of draft strategy!

This analysis is about a topic a lot of us seem to think is important: finding Upside potential in your draft.  My goal is to bring some fresh insight into what historical numbers have to say. As usual, I will guide you through my assumptions and methodology, in the hopes you’ll follow along. And in the end, hopefully we can better wrap our heads around this aspect of fantasy strategy.

Today’s Topic is Upside

People talk about it. People believe it’s important. Some people cite Talladega nights (“If you ain’t first, you’re last”) like it’s the only priority. 

But how to make sense of it?

I haven’t yet seen a good assessment showing when Upside potential can give a boost. What’s the best time during the draft? Near the start?  Unfortunately I see people assuming too much-- for example claiming that the top-rated players have more upside. They’ll point to top players last year and say things like “You need to take an RB, because looks at Barkley last year— his points above baseline beat anyone else!”  The problem with that logic is: it always looks like top-performing players achieved amazingly great upside, after you see them do it.  But we didn’t know that before it happened!  

This happens all too commonly: casual managers —or inept analysts— tend to rate player outcomes, after the fact, which is always misleading.  In fact, it’s a fallacy. Barkley was not ranked the RB1 before the season started. He came out the previous year ranked around #13 by half-PPR, and yes after his trade his ADP was maybe up to RB6. But he was not ranked as the RB1 or RB. Meanwhile, how did last year’s RB1 do? Well we don’t need to go into that (hint, he played 4 games and averaged 11 ppg.)

If we actually want to learn something, we need to treat problems ex ante— before the fact of knowing the result.

My goal today is to take a look at one way we might actually quantify “Upside” in a useful way. I acknowledge that my chosen method gives only a rough approximation— In fact, this might be one of the less rigorous analyses I’ve done, because I lack a complete ADP data set.  But even without the most precise answers, I think you’ll see that it helps bring a useful perspective to the arguments around Upside. 

And I’m sure the methodology can inspire further work along the same lines of thinking!

Which Rounds in the Draft Give the Most Upside Potential to Different Positions ? 

In my approach, my main assumption is that we can get meaningful results by looking at positional ranks combined across different years.  What I mean is: let’s look at the RB1s from previous years, as they were ranked before the season started, and we treat them as one group. And let’s do the same for all RB6s for that matter.   Then let’s see how things actually turned out— what were the distributions of point outcomes for each group.

To approximate this, I’m using the assumption that “the previous year’s #1 ranking RB probably represents a reasonable choice to approximate the next seasons #1 ranked RB”, before knowing the results. It’s not perfect, and I can do better in a follow-up. But this will give us a reasonable approximation for finding upside over each player rank. I think it’s fine, except the most obvious weakness that it will exclude rookies. But let’s come back to that later.

To get anywhere with this analysis, I also need an additional assumption: that the MODELED score is a good representation of each year’s actual score projections for the given rank-position.  This is not too uncommon an approximation, and we can expect small errors to cancel out later.

How to get a modeled score prediction? You all can skip this part— it’s more of an aside. My personally preferred method was to perform a multivariate regression on season scoring outcomes, as a function of (a) positional rank for each year and on (b) the rank of sorted outcomes over multiple years. In other words, I take the distribution of “RB1” outcomes for all 12 years used, and I create a formula that can approximate that same distribution— simultaneously for RB2s, RB3s, etc. Here is an example of the fit of distributions:

Revisiting VORP — and Why We Look Beyond it to Find Upside

This is a good a time to remind ourselves about Value Based Drafting (my article linked here): value-over-replacement measures a player’s relative worth.  We can optimize our rosters for a “probably high score” by maximizing these “added values” each player lends to his position.  Here’s how the Value-over-replacement then looks (averaged over 12 years), where the blue parts represent high ranked players who have positive value:

For this particular article, I’m using the term “VORP” generically— I don’t specifically mean the Value over Waivers players, but just whoever we consider “Replacement” players. In fact, we want to represent something closer to “VOLS”, value over last starter: I’ve chosen illustrative baselines at QB13, TE13, RB30, and WR30 to try and capture the value of the strongest starters. (In these examples, I have used 0.5 PPR scoring.)

If we have a good metric to prioritize players, then why are we spending today talking about Upside?  Where does that come in?  The problem we face is that VBD ranking has become widespread over the years.  If you’re in a league where nobody else drafts by VBD principles, then by all means focus purely on Value!  You have a statistical advantage even without considering Upside.  However, more and more teams find they have eked out the value they can.

So, the relevance of upside comes when we’ve leveled the playing field with VBD Value. If everyone in the league has already optimized, then they’re all the same— all average. Then how can a team differentiate itself?     

The obvious answer is you need to be lucky.  But the “Upside Theory” also exists and implies that you need to gamble.  You need some players who have a boom-bust characteristic because:

• If you just have average players, you’ll finish #6 in your league, which means you lose.

• If you’re unlucky with your gamble, it doesn’t really matter, because you still lose anyway (#12 in your league isn’t different from an average #6)

• If your gamble on Upside  pays off—speaking only of the type of Upside that was rational to consider in advance—then you have a strategy to probabilistically propel yourself above average.

In other words, there’s a “new” game on top of the value-over-baseline game. You need the base value, of course, but there’s an extra layered game of identifying where you might find upside.  And that’s what I want to try to do now.

Upside: Definition and Analysis

So, in this “new game in town”, the simplification we’re going with is this: when your team scores below average (i.e. when your upside bet doesn’t pay off), the resulting loss isn’t different from returning 0 value.  Instead, what matters is performing above average, and only above.  Everything below average is almost irrelevant: because if you lose out on your gamble, it’s the same as getting a zero anyway (zero above average). 

Let’s turn this into a convenient definition for calculating “Upside”: we measure how much a player’s score goes above Valuation, and we count negative numbers the same as zero. Some people refer to “beating ADP”, but instead we want to talk about “beating projections”.

(Note that beating ADP is kind of useless to compare. For top-ranked players at their position, beating ADP doesn’t matter. The RB1 can never beat his ADP— Then he’d be what, RB0, RB-Zero? That doesn’t make sense for treating an ordinal distribution. And a low-ranked player beating ADP is a poor measure, because if the WR57 performs as the WR42, does that 15 point difference matter? No. We need to talk in measurements of fantasy points. So we’re making it numerical.)

Stop and think about what this Upside definition implies: we’re moving from using a VBD baseline (that used to be a baseline representing replacement players), and switching to an Upside baseline—which means setting the baseline exactly at the players own original Valuation.  This is something of a simplification, but it let’s us form a picture.

In the below diagram, I’m showing a lot of information for RBs, so you can see how all the numbers come together.

The main points are:

• You can see how VBD Value (“VORP”) comes from setting a baseline, by looking at the Blue shading.

• You can see how the scatter of historical results goes above and below modeled point-expectations.

• You can focus on the results that we consider Upside: Highlighted by the yellow lines. These are points that went above expectation OR above the zero-value threshold (for lower ranked players).

• Finally, I have additionally marked with green-ish dots what the AVERAGE UPSIDE has been, for each positional rank.

This final metric— the green dots— is what I claim we’re really after! THAT’s UPSIDE VISUALIZED.

Is the Average Sufficient?

Maybe Not. There’s an interesting assumption going into this computation, so let’s make sure you and I can roughly agree, using this example.  Imagine that we have two players, both with valuations of “10” ppg above replacement.  The left-hand player  scores +50ppg once during a 10-year period, and all other years are at-Value or negative.  The right-hand player scores +10ppg 5 times over the same 10 year period.

Which is better?  To make conclusions in this study, I’m assuming both players are valued identically: at “on average” a +5 extra points of upside in a given year.  This is flawed and it is also debatable —many of us feel consistency is more valuable—but many others argue that the extreme upside is more worth the swing.  But differentiating between these two options is a more sophisticated discussion, for some other  day.  And I know some of you care about prioritizing those big upside gambles, so I’ll assume “average upside”, to tie up the conclusions.

Now the procedure should be clear:  Players are already “Valued” over their positional replacement, to get a baseline value.  And now on top of that amount, we want to add their “average Upside at the given positional rank”.  The calculation of “upside” is simply :

Finding Trends in the Upside Calculations

To complete this study, I then took these calculated, average Upsides for all positions— the Green dots in the above plot— and I searched for rough trends with dependence on player VORP. VORP is preferable as a reference (instead of using player rank), because then we can compare the positions against each other, and — you’ll see after a couple more steps— we can even get a feeling for how the different fantasy positions are “distributed when ordered by value”— i.e. what a typical draft rankings list would look like. To show simple, uncluttered versions, here are the fits for TE and QB:

There’s the usual amount of scatter I would expect, but I think the trends are apparent.

Now let’s re-plot these examples and put all the positions together with color coding. Here’s the picture of Upside dependency that emerges:

That’s pretty cool! I think this result is fascinating, even though we know it’s based on some loose interpretation of player pre-season rank.  It’s not the result I expected to see.  There’s surely some scatter, but it’s also fairly easy to pick out some trends. 

• FOR EACH POSITION; THERE ARE DIFFERENT TIMES IN THE DRAFT WHERE UPSIDE SEEMS TO OCCUR MORE STRONGLY, RELATIVE TO PROJECTION.   

• Running backs overall have the highest upside potential, but it is distributed over most draft pick ranges and not concentrated only at the top.

• THE SCALE OF UPSIDE, when averaged, is much lower than I think people expect. In the above example, it seems reasonable that you should not expect more than 1.5 points of upside.

• To my surprise, the TE1 trend remains pretty clear, despite years when that selection busted.

Relative Upside — Guidance for Choosing Upside Among Different Positions

There’s one final step.  We need to be able to compare the positions against each other. The graph above looks like “Running Backs always offer more upside” and you might start to conclude “you should always just draft Running backs”, but of course that’s nonsense— You will not form a roster with a solid VORP baseline.  We want to know when you might find more upside from a position, relative to other times in the draft.

So we need to subtract the “constant” upside that applies the whole way.  In other words, we calibrate all the graphs so they lie on the axis (I’m assuming the polynomial fits can be used as the reference point for troughs):

I’ve also labeled the points on the graph, from right to left, to indicate the approximate Draft Pick. Naturally those would depend on the scoring setting (I’m using 0.5 PPR), and it will depend on the baseline assumption that I made. But I think the above picture is indicative.

The final picture

So that’s about it guys!  That’s the picture I’ve been working towards.  Before we blindly draw conclusions from it, let’s remember that there are assumption to revisit: Using actual player ADPs, using actual projections, and finding more sophisticated ways to treat the consistency of upside.  And the big missing pool of players we excluded by doing it this way: The rookies!  By the method I chose, they’re not even represented here.

So yes, it will be worth repeating this exercise with better data, but I still think we can see some valuable observations:

The overall point is that “Upside” is not relatively concentrated at the top of the draft.

That means, before the season starts, the possibility for upside is “SPREAD OUT” over all positions in the draft.

And don’t forget the important point that we have measured Upside relative to VORP. So what you might have previously considered the “Upside” of the top players is actually just their baked-in value, and the whole reason for drafting them first in the first place. You want a solid base in your first rounds. And then the “Upside” is just anything that happens to go over those large amount of expected points.

Now let’s look at the individual positions.

• QB The top 1 or 2 QBs offer some limited upside, but otherwise mid-late round QBs give that opportunity.

• RB Upside is not only concentrated among top RBs—many 2nd and 3rd round RBs offer at least as much potential.  RB2 and RB3 have been giving more upside than RB1.

• WR WRs offer appear to offer the most “relative upside” later in the draft of starting rosters.

• TE The TE1 has been a reasonable gamble for upside (relatively speaking), otherwise the later round TEs are better for that gamble.

Finally don’t forget that you can’t just value upside alone.  The whole point is that you need to grab Value-over-replacement first, and then consider upside as an extra.  

Total potential = Value + Upside

Thanks if you followed along!  I hope this helps you as you approach your draft strategy.  For a lot of us—myself included—it has been enormously difficult to find a way to judge draft order positional value, and I hope this paves the way to better analysis and understanding, in the future.

/Subveretadown