The Gut Check 410: Analyzing Training Camp Reports

Photo: Isaiah J. Downing, USA Today Sports

Five years ago, an undrafted free agent wide receiver was the talk of an NFL camp. Long-time beat reporters and ESPN writers (often the same thing as the newspaper industry felt the effects of online journalism) sang his praises.

One reporter said on August 4th that he was the best wide receiver in training camp, despite competing with three other drafted rookies at his position and a free agent with an early round pedigree and steady success elsewhere. 

Another for a national online outlet said he was fantasy football's top bang for your buck on August 21st.  It quoted the most respected (and often accurate) beat writer covering the team, who said eight days earlier that, the receiver remained one of the top storylines of the team's training camp.

He beat veteran corners on the team for touchdowns. He did the same against the organization his team hosted for a week of scrimmages. He flashed during preseason games. And he even had a game-winning reception with five seconds on the clock in early October of that year.

However, Kenbrell Thompkins has played for the Patriots (twice), Raiders, and Jets since that excellent training camp run as a rookie where he earned a starting role with the team. His career stats for the past five years (and he's only seen playing time in three of those): 53 catches, 728 yards, and 4 touchdowns. More than half of that production (including all four of his touchdowns) occurred during his rookie year. 

I was bullish on Thompkins, a player I identified as a sleeper while he played at Cincinnati. Antonio Brown's cousin performed well at every stop during his circuitous college career and carried that through his first mini camp, OTAs, and training camp. But football is a performance industry and there's a notable difference with the intensity of pressure between practice, dress rehearsals, and the actual touring season. As Ben Watson told me a couple of years ago, when you realize that you're playing with and against grown men who are supporting families and they behave as if this is the case, you realize how much is on the line every year, every week, every quarter, every series, and every play.  

At the same time, Tom Brady routinely makes mistakes in practice that would set off beat reporters if Brady didn't have the resume he's earned. These contrasting stories beg the question: What should we make of training camp reports? Are they worthwhile? If they are, how do we discern worthwhile information from the noise? 

I argue that training camps are one of the reasons why fantasy football is so much fun. The information is predictable but often unreliable because there's so much variation of expertise and wisdom among football media in attendance. I've seen this as a writer covering the Georgia Bulldogs during the era of Terrell Davis, Eric Zeier, Hines Ward, Garrison Hearst, and Robert Edwards. And I've seen it for the past 10 years of Senior Bowl's practice week. Here is one example of many that I could provide that should underscore the point well: 

There was a Big 12 running back that I couldn't wait to watch. He burst onto the scene after his former teammate lit up the NFL as a rookie and became one of the best backs of his generation. However, injuries derailed this young running back's opportunity to step into the shoes of this NFL start and fulfill a similar collegiate legacy. I was anxious to see him play because he displayed a greater maturity of decision-making and toughness as a senior while gutting through an injury. He also changed his running style in subtle ways that helped him preserve his body in ways that he didn't as an underclassman. 

This was a smart, tough, physical running back with good burst and enough speed to still breakaway against NFL-caliber opponents. During Senior Bowl practices, he displayed this skill repeatedly against a defensive front that routinely earned penetration into the backfield and forced this runner to make people miss and bounce plays to the outside. I thought he was one of the most consistent and impressive players in practice that week. 

However, a former NFL scout with multiple teams and a media personality for a football show whom I respect (and has successfully identified some players that few were onto), said after Day 2 of practices that this back cost himself 2-3 rounds after what he showed in Mobile, Alabama. He didn't like the player's stride and he thought the player was tentative. 

We even talked about that player after practice for a few minutes. He was convinced this player hurt himself. 

On Day 3, before the Senior Bowl allowed (and smartly so) ESPN and NFL Network to hog the real estate next to the fenceline along the field, two NFL running back coaches (who were former NFL running backs of note) walked over to where I was and flanked me on each side as scrimmages got underway. Those former backs were Earnest Byner and Sam Gash. Byner had Pro Bowl caliber years in Cleveland and Washington as a do-it-all runner. Gash was an excellent fullback and two-time Pro Bowler who paved the way for the likes of Curtis Martin and Jamal Lewis. 

As this running back prospect in question bounced a play outside two top prospects of note who had an angle on him, Byner leaned over the fence looking towards Gash and said, "He's a thoroughbred." Gash nodded as he finished writing something in his notebook. I got a short glimpse of the notebook and he referred to that player by his number as he was noting what he saw. 

That player was DeMarco Murray and I bet if the Saints, Cardinals, Patriots, Lions, and Dolphins had the benefit of hindsight, they would have preferred Murray to Mark Ingram, Ryan Williams, Shane Vereen, Mikel Leshoure, and Daniel Thomas. After Murray and Ingram, the next best backs in that 2011 class have arguably been Bilal Powell, Jacquizz Rodgers, and Dion Lewis who had 8, 12, and 13 backs picked between Murray and each of them, respectively. 

Practice coverage is fraught with errors of perception for analysts, reporters, and the audience. Gleaning accurate assessments about a player's future from practice are difficult because it's an incomplete picture.  Those delivering the analysis are often trying to see multiple events that are happening at the same time because practices are often fragmented until it's time for scrimmages. If you're close enough to some these reporters, you'll see one watching one section of the field asking another what happened in another section, which means if one's perception of the events is flawed, it infects the other. 

How should fantasy owners navigate these reports? Here are guidelines I recommend that will help you at least develop a balanced perspective—even if you're not getting simple answers (wouldn't life be nice if everything were simple?). Ultimately, you'll have to make some leaps with your analysis that you wouldn't want to make with your day job, but that's why for most of us this is a game and not a job. And when it is a job, many of us play the game at a high volume with variations of these decisions so they can enhance their chances of averaging a winning profit. 

1. Understand the common politics of most team environments when assessing players

When tracking the progress of rookies, it's important to know that there is a latent bias. My buddy Ryan Riddle, a former NFL player—and Cal's season sack record holder—wrote an excellent piece several years ago called The HIdden Advantage Of Being A High NFL Draft Pick. Riddle explains that in most NFL camps, high-round rookie picks earn more reps and more forgiveness for errors ("Oh, he's still acclimating, just look at that speed and size. He'll catch on with more work." vs. "He can't even hold onto the ball when he gets his chance.") during practice from team decision makers than late-round picks.

Browns UDFA Isaiah Crowell didn't become a relevant name in the news, much less a relevant player until the second half of the third preseason game of his rookie year in 2014. the Browns had Terance West, Dion Lewis, and Ben Tate. They also had veteran Chris Ogbonnaya, who they had a positive history with as a depth chart option. Training camp reps and playing time were hard to come by. Terrell Davis was playing special teams and it took a big hit on a kick off in Japan and lobbying from Shannon Sharpe and other players for the coaching staff to give Davis extended reps. It's the nature of the business for the more expensive rookies to earn developmental time and attention than the least expensive. However, the differences of talent among these players are statistically small, which leads to the next point...

2. Understand the Talent Gap by the numbers

Veterans with a strong track record earn a right to make a few more mistakes as do high-round rookies for a year (maybe two), but there is generally a low tolerance for mistakes in the NFL. Extremely low.  

NFL Agent Greg Linton shared this on Twitter in 2014. It reveals how less than 1 percent of seniors playing college football will ever earn a second contact in the NFL. 

Only the top 6.5 percent of all high school players compete at the college level. It means they are in the 93.5 percentile of all high school players. Likewise, only the top 1.6 percent of all college players enter the NFL–the 98.4 percentile. And that second NFL contract–the seal of approval that you’re a good NFL player–is reserved for less than one percent of all college players; the 99.06 percentile.

Viewing the numbers in this fashion, it doesn’t look like a big difference between the 93.5 percentile, the 98.4 percentile, and the 99.06 percentile, but you’d be mistaken.

This may be a stretch for some–and it certainly isn’t scientific–but for the sake of entertainment, let’s presume that these percentiles were a reflection of a player’s success rate executing plays on a per snap basis. I understand this is not exact, but I think there’s enough to this idea to suspend disbelief long enough to make an overall point that is worthwhile.

The table below shows a number of errors–or bad plays–that a player would commit over the course of a million plays based these percentiles that represent their standing as a college (93.5 percentile), NFL prospect (98.4 percentile), or NFL vet earning a second contract (99.07 percentile).

The difference between 65,000 errors and 16,000 errors is massive and that’s just the gap between a college player and NFL prospect who might last three years in the league. The NFL vet who earns a second deal commits 42.3 percent fewer errors than the prospect ad 86.6 fewer errors than the college player. And I’m talking about the average player on a team, which includes the best and the worst players on each squad–forget about the stars!

Even these numbers are a little harder to grasp because we’re looking at a million plays. We won’t see any player execute that many over the course of a career–as hard as Brett Favre, George Blanda and Bruce Matthews tried.

So let’s break it down to plays in a season. Let’s estimate a player sees 40 plays a game for 16 games. I know this isn’t completely accurate for the college game or certain players in the NFL. However, it’s a more understandable sample size of plays for a season that equates to 640 plays.

Now look at the differences in errors/bad plays–it’s a lot easier to grasp.

The difference between 10 and 6 egregious errors per season per player is staggering–and that’s the difference between a young NFL player and a veteran. Those 41.6 errors per season for the average college player just doesn’t cut it for the pro game. This chart hints at why NFL athleticism is a difference maker in the college game even if the NFL skill and understanding of football isn’t always present.

In contrast, the gap between a prospect and vet is much smaller from an athletic standpoint, but the differences in errors are still large. This is where knowledge of technique, strategy, and consistency of execution come into play. Again, this is hypothesizing that we’re discussing the average player at each level.

Now think about the top four players on each team–Pro-Bowl caliber players–that’s 128 players in the NFL. These players are in the 99.88 percentile of football–high school, college, and NFL. Using a sample of 640 plays in a 16-game season they would commit .75 egregious errors.

This seems hard to believe. In fact, you can see where this theory begins to crack at the seams because even All-Pros make mistakes multiple times in a season. However, how many of them are solely their fault and not something that can be explained by the error of a teammate? Not as many as you might think.

The numbers aren’t exactly right, but the point is still a good one: The gap in talent is about the consistency of execution and it requires knowledge, skill, and focus—especially as the gap in athleticism narrows.

It means that even a smaller number of mistakes like turnovers and missed assignments (e.g., as a blocker, reading pre-snap/post-snap coverage, hitting the correct crease on a running play, or taking the wrong lane as a special teamer) can hurt a player's chances at making the team or earning playing time. 

3. How do I discern errors that will hurt a player vs. forgivable mistakes?

On August 1, Evan Silva referenced this ESPN article on twitter with the heading: Samaje Perine has had issues with ball security and pass protection early..." Here are the tweet and some of the responses after it. 

Some think it's not a serious issue while others are wondering if they should even consider Perine at all this year? Another follower accurately notes that the ESPN article says Perine fumbled once and his pass protection issue was a missed rep in a one-on-one drill. Silva's tweet is an example of what I see from a multitude of excellent fantasy writers in August. How do we unpack something like this and get meaningful information? 

Reading beyond the headline assessment is always a good start. It's not that Evan is wrong to point out that Perine fumbled and missed a pass protection assignment. In fact, given the nature of the error rates I mentioned above, it's newsworthy. It's a layer of information that could lead to Perine not earning the starting job and being over-drafted.

However, the article also notes that Perine is still in contention for the starting job and his head coach praised Perine for running hard between the tackles and earning yards after contact. There's a point where every writer and fantasy owner has to make the call with the available information. Evan made that call as something to note. I might have waited a little longer before labeling this as an issue after examining the context of the errors and the player.

The issues happened in practice, the runner didn't fumble the ball multiple times, and the pass pro error was a drill. In addition, the player is a rookie and while the fourth round isn't a high round by draft standards, Perine was the only back the team drafted this year and they've stated from the beginning that he would immediately compete for the starting job. This context should help you determine that Perine will have a longer leash, but you should also watch for additional reports during the next two weeks to see if Perine has fumbled multiple times and/or if he is failing at pass protection during scrimmages and preseason appearances. This to me is where Evan's retweeting of the ESPN report will have the most value (as a layer of information, but not a definitive conclusion). 

In addition to noting the stage of the player's development, the setting for the mistake, and the team's investment and expectation of the player, the nature of the errors are also important. Dress rehearsal (preseason) errors carry more weight than practice errors because the pressure to perform is higher and teams need to see if the guy lighting up practices can do it on command in games. Some players screw this up early in their careers and mature when they earn second or third chances with different teams. Others never get it. Since we're looking at this in a re-draft context, that bit of information is not as important for now. 

4. the context of praise and positive news

A strong preseason game can benefit a player greatly. However, we have to consider the other factors mentioned above and ask the following questions: Has he been consistently good during practice? Are there more established veterans ahead of him who are healthy? Is the praise about athletic ability (which everyone starting in the NFL has the prerequisite amount) or about his execution of the technical and mental side of the game?

Look for practice reports that give play-by-play examples to support the praise. Value strong performances against starters or established veterans higher than performances against third and fourth-team options. Most of all, look for a steady drumbeat of positive news that indicates he's consistent, improving, and gaining opportunities to practice or play on a stage with the best players in practice and during games. Keep in mind that consistency and improvement don't mean "doing it right every time," as much as it is a positive trend. If the player has a bad day in practice, does he bounce back? Do the errors return two days later or we never read about them again?

Takeaways From this post

1. High-round draft picks and/or expected competitors for starting jobs earn more reps and opportunities to grow without early judgment that they can/can't cut it.
2. Read beyond the headlines to determine the context of mistakes.
3. Repeated mistakes in one area are a bigger red flag than multiple mistakes in different areas during the same sessions.
4. Preseason mistakes carry more weight than practice mistakes.
5. The first preseason mistake won't be dealt with as harshly if the player is consistently good during practice.
6. Subsequent mistakes in games after the first mistake will have consequences for that player's opportunities. 
7. Praise for athletic ability carries less weight than praise for technique and mental grasp of the game.  
8. Analysts perceive different things from the same practice, which can create a variation with analysis. 
9. Eventually, you'll have to project an outcome that's positive or negative about the player without complete information. 
10. Even if you make correct calls with all of these analysis points, there's still palpable chance that things won't work out as expected. 

As my buddy, Jene Bramel likes to say, Good luck everybody...