The real meaning behind “stars matter”

Today marks the beginning of what some consider the college football silly season.

The introduction of the early signing period has already had a profound effect on the way teams do business. Coaches have been fired sooner than ever before. Boards have been revamped as teams have gone from pro-style to spread offenses or 4-3 to 3-4 defenses. And likely there is pressure on recruits who would prefer to wait until February but may fear not signing will allow them to get squeezed out.

Judging by my Twitter feed, there is one thing hasn’t changed. There is still an intense argument about whether star rankings from recruiting services actually matter. On one side, you have people who have seen what Nick Saban has done at Alabama and point to that as definitive proof that stars matter. On the other side, you have people pointing to Baker Mayfield, a former 3-star recruit who walked on at Texas Tech and just won the Heisman Trophy.

If you believe that stars don’t matter, let me say that I understand your stance. You see players like Mayfield, Louisville’s Lamar Jackson and Oregon’s Marcus Mariota – all 3-star recruits – and rightly wonder how they could have been missed by the recruiting services.

Likewise, you remember the intense battle between Auburn and Florida for 5-star defensive end Byron Cowart who struggled to get playing time at Auburn. Or perhaps you’re still scarred – like me – from watching 5-star QB recruit Jeff Driskel play for the Gators.

After all, the recruiting services can’t even agree on their rankings of players. That must mean they don’t know what they’re talking about, right?

Looking into the numbers

What seems to be missing from this argument is a discussion about probabilities. Probabilities are defined as the likelihood of something happening or being the case. Just pointing at Alabama or just pointing at Mayfield doesn’t convince anyone.

You’ll hear me cite sample size on this site a lot, and that is because there will always be single examples we can point to that support our case. But if we look at a large enough sample size, the data should point us in the right direction.

So how are recruits ranked anyway? For this, I highly recommend taking some time to listen to Bill Sikes’ excellent analysis regarding star ranking and how it translates to the NFL Draft.

About 3500 players are ranked by recruiting services each year. 247Sports combines the rankings of the major services and comes up with a composite, giving them not only a star ranking but an overall national ranking as well.

Number and percentage of recruits by star-ranking according to 247Sports. (Will Miles/Read and Reaction)

From 2016-2018, only 86 of the approximately 10,500 players ranked achieved 5-star rankings. Only 10 percent of those players achieved either a 4 or 5-star ranking. That means that 90 percent of players who are ranked are 3-star or below.

This is important because it gives you an idea of how many of each star ranking are on the field at any given time. If stars truly don’t matter, then the same percentages should win major awards and show up on All-American lists.

That’s not what happens when the players get on the field.

Heisman winners by 247Sports composite star rankings. (Will Miles/Read and Reaction)

I examined the players who placed in the top-3 of the Heisman Trophy voting from 2004-2017 and looked at both their 247Sports star ranking and national overall ranking. The average of those candidates was a star ranking of 3.98 and an average national ranking of 240 (note, there were two players ranked 3rd whose rankings I could not find).

Yes, 12 3-star players have received enough votes to be ranked in the top-3 of the Heisman Trophy voting in that span, including 5 winners. But 28 4 or 5-star players have met that bar, a whopping 71 percent. And those players have won 64 percent of the trophies as well.

Remember, these players only make up 10 percent of the population and they are producing enough to place top-3 for the most elite prize in the sport more than 70 percent of the time.

The same story plays out when looking at first team All American teams (as rewarded by the AP).

First-team AP All Americans by 247Sports composite star rankings. (Will Miles/Read and Reaction)

In this case, 62 percent of the All Americans from 2015-2017 were 4 or 5-star recruits. The average star ranking for an All American was 3.71 and the average national ranking was 578.

It’s pretty clear. Elite play comes from elite recruits at a percentage much higher than their population. Ten percent of the players are accounting for 62 percent of the All Americans. That infers that these rankings are not just useful, but accurate at predicting which programs are going to succeed long term.

Breaking down the probabilities

Let me say right now. If you are a 3-star recruit – or perhaps a parent of a 3-star recruit – this does not suggest that your son will not win the Heisman Trophy or be a first-team All American.

What it means is that there were only 20 3-star first-team All Americans from 2015-2017, while there were approximately 5,500 recruits who had the 3-star designation. That equates to a 0.36 percent chance. There is still a chance, but it’s about a quarter less than the next son that you have will have a twin sibling (1.5%).

Do we all know someone with a twin? Sure. But I certainly don’t assume everyone that I meet has a twin brother.

Conversely, there were 16 5-star first-team All Americans from 2015-2017. With an average of 30 5-star recruits signed every year, that equates to a 17.7 percent chance. That’s slightly more than the NFL’s New York Giants winning percentage (14.3%). Now, the Giants winning this year has been rare, but you wouldn’t be shocked if the Giants win this weekend against the Cardinals.

The percentage goes down to 2.7 percent for 4-star recruits. It is true that there are more total 4-star recruits who have been first-team All Americans than 5-stars so it feels like they are outperforming the higher ranked players. But that’s just not the case on a percentage basis.

That’s why teams that recruit at an elite level win consistently. They are essentially getting more shots at that All American than their counterparts. And this shows up in the numbers.

Comparison of 247Sports recruiting rankings for programs that are traditional powers. (Will Miles/Read and Reaction)

Alabama has – by far – signed the most 5-stars and has also signed the most 4-stars. Based on the percentages I cited above, that would predict 5.5 first-time All Americans over that time-span. The Tide have had 11.

Ohio State is the next highest of actual All Americans with 7, but is behind Georgia in predicted All Americans. That’s because in that time-frame, Georgia has actually recruited more elite-level talent than Ohio State.

And this is where the development that people who distrust star rankings talk about becomes so important. What this analysis suggests is that Georgia definitely had a coaching issue with Mark Richt. You could see that with the substandard results on the field, but you can also see it here where his players were not playing up to their talent level.

This opposite is actually true for Florida. Until this season, Jim McElwain had actually done a good job with his in-game coaching strategy and probably had done more than what should have been expected record-wise.

But that was the problem. Florida isn’t a program that wants to exceed expectations based on substandard recruiting. Based on this analysis the Gators clearly had a recruiting problem. Dan Mullen may be a fantastic in-game coach. But none of that will matter if he doesn’t get better results on the recruiting trail.

What this means

I tweeted the following after Matt Corral decommitted from the Gators.

Stars matter in aggregate as part of a holistic view of a program's health. One commitment or decommitment should not change your view of a program/staff, especially one in transition.

— Will Miles (@WillMilesSEC) December 15, 2017

That’s the conclusion that this – or any other analysis of this subject – keeps coming to. When we say stars matter, we don’t mean that Baker Mayfield can’t win the Heisman Trophy. What we mean is that a team full of players with Baker Mayfield’s profile is going to struggle.

To pretend that these rankings don’t matter does two things. First, it lets substandard recruiters spin stories about how things are about to turn the corner when we should know better. Second, it devalues the accomplishments of players like Baker Mayfield or Lamar Jackson.

Mayfield and Jackson had a 0.36 percent chance of becoming a first-team All American and they both did it. They had a 0.05 percent chance of winning the Heisman Trophy and they both did it. The odds of them winning the Heisman in back-to-back years is 1 in 4,000,000.

No matter how you spin it, those events are just not very likely. Them winning doesn’t prove that stars don’t matter. It proves that stars aren’t the only thing that matters.

And I think that’s where I think people can get upset in this argument. The argument for stars mattering is not an argument that coaching, development, attitude and hard work don’t matter. If that were the case then we’d just count up the recruits at the end of the February and give out a trophy.

Instead, it’s an argument that stars are the foundation for everything that comes after it. If a coach can get 4 and 5-star recruits to study film, get better, play with a positive attitude and work hard, that team will be better than a team of 3-stars with the same attributes.

Emory Jones commitment to Florida today is really immaterial in the grand scheme of things. He’s one 4-star recruit, and even then the odds are stacked against him becoming a true star. But combine him with Kyle Pitts, Amari Burney, Dameon Pierce, Trey Dean, Iverson Clement and David Reese and you 20 percent chance that one of them is going to be a first team All American.

That foundation – more than any individual signing – is what Mullen has built today. One of these players who signed today may become a star. Or maybe none of these players will be the one who brings Florida back to prominence.

But the probability that one of them is goes up with each one that Mullen convinces to become a Gator.

FEATURED IMAGE USED UNDER CREATIVE COMMONS LICENSE FROM ARCTIC_WHIRLWIND