There's a lot of strong dynasty analysis out there, especially when compared to five or ten years ago. But most of it is so dang practical— Player X is undervalued, Player Y's workload is troubling, the market at this position is irrational, and take this specific action to win your league. Dynasty, in Theory is meant as a corrective, offering insights and takeaways into the strategic and structural nature of the game that might not lead to an immediate benefit but which should help us become better players over time.
Our Good Friend Goodhart
In a 1975 article on British monetary policy, economist Charles Goodhart wrote:
Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes.
This became known as "Goodhart's Law", one of the most important observations of our time, and is frequently reformulated as "when a measure becomes a target, it ceases to be a good measure". There are all manner of social ills that have arisen from one policy-maker or another ignoring Goodhart's Law.
The most common illustration would be to imagine a Soviet-era nail factory. The factory foreman wants to increase productivity, so he first decides how to measure productivity. In this case, he settles on "number of nails produced". Ordinarily, "number of nails produced" would be a great measure of a factory's productive output. But the manager then takes it a step further and, in an effort to boost productivity, makes that a target; he mandates that workers must produce a specific number of nails per shift.
Workers confronted with this mandate quickly adapt; they start making tiny nails that can be mass-produced quite quickly. Suddenly, the factory's "productivity" has quadrupled overnight, but the actual useful output has fallen through the floor. So the manager goes back to the drawing board and instead mandates that workers produce a certain number of pounds of nails every shift, with predictable consequences; workers pivot to making exceptionally large nails, which are especially heavy for the amount of work required to manufacture them. The net weight of nails goes up, but the quality of the output remains terrible. "Quantity of nails" or "weight of nails" used to be good measures until they became a target, at which point they ceased being good measures.
Or consider a tale from British colonial rule in India. The British governor saw many people being killed by poisonous cobra bites, so he wanted to reduce the cobra population. Ordinarily, a good measure of how well the population was being reduced would be "How many cobras are being killed?", so the governor naively set up a bounty for every dead cobra that was brought to him. Enterprising citizens saw an opportunity, so they started breeding cobras, killing them once they reached maturity, then turning them in for the bounty. The governor finally caught on to the scheme and ended the bounty program. At that point, all those industrious cobra farmers no longer had a use for the snakes and released them into the wild, causing an explosion in the cobra population.
(There's a similar story about rats in Vietnam. The government issued a bounty for rat tails, so rat catchers started catching rats, cutting off their tails, then letting them go so they could breed more rats and produce more tails the next season.)
Any Other Examples of Goodhart?
These stories are mostly morality plays of questionable authenticity, but Goodhart's Law extends beyond just-so parables. Here are a few modern examples, ranging from the frivolous to the disheartening to the outright macabre:
The IRS wanted to tax "large" gambling winnings while leaving "small" winnings untouched, so it started a mandatory reporting threshold past which all winnings must be reported. As a result, the number of winnings that fell just short of the threshold exploded, while the number of winnings that inched just past it plummeted.
If you graph reported heights by NBA players, you'll notice curious spikes at 6-foot-0, 6-foot-3, 6-foot-6, and 6-foot-9, a direct result of making those specific heights a desirable target.
In 2012, Northern Ireland created a "renewable energy" program that provided incentives to businesses that used renewable energy. Those incentives were often greater than the cost of the energy, so businesses started buying empty buildings and paying to heat them with renewable energy for an easy profit.
In 2016, Wells Fargo instituted quotas for its employees mandating that they open a certain number of new accounts or sell a certain number of financial products per quarter, then had to settle a $3 billion lawsuit after its employees started opening up fraudulent accounts in its customers' names.
Goodhart Rules from Cradle to Grave
Growing up in America, our childhood is governed by Goodhart. Based on the observation that schools that did the best job of imparting knowledge produced students who performed the best on tests, the American education system was rebuilt around a series of high-stakes standardized tests, with school funding frequently tied to student performance.
In response, schools began "teaching to the test", ceasing to pursue any knowledge that wasn't being tested and often forsaking genuine education in favor of rote memorization and test-taking strategies. Once they became a target, test scores stopped being a good measure of the quality of education provided.
Graduating from the educational system doesn't earn you any reprieve from Goodhart, though. For any given worker, there's a general positive relationship between the amount of time and effort spent on work and the total productivity. Seeing this trend, the dominant managerial philosophy shifted to prizing "butts in the chair", promoting those who stay late or arrive early, even if that extra time doesn't translate to extra productivity.
(Perversely, studies have shown for decades that reducing time away from work incurs cognitive debt and lowers overall productivity. Still, chasing a metric that has become a target, most bosses would prefer a worker who did a job adequately in 8 hours a day over one who did the same job exceptionally in 6 hours a day and then left early.)
When we retire, we still can't escape Goodhart; our entire healthcare system is a multi-trillion-dollar instance of good metrics that became bad targets. The point of the system is to produce positive health outcomes, but we measure productivity (and therefore compensation) by the amount of treatment provided. This encourages unnecessary treatment (that doesn't improve health outcomes) while discouraging preventative care (which does).
There are even horror stories of unethical dentists performing root canals on perfectly healthy teeth or replacing fillings that are still in good shape just to boost their revenues. (A Canadian study found that the more debt a dentist had, the more likely they were to recommend unnecessary procedures.)
As a result of such perverse incentives, biotech investors like Goldman Sachs issue reports on whether "curing patients [is] a sustainable business model" (Fun pull-quotes include: "While this proposition carries tremendous value for patients and society, it could represent a challenge for genome medicine developers looking for sustained cash flow" and "In the case of infectious diseases such as hepatitis C, curing existing patients also decreases the number of carriers able to transmit the virus to new patients, thus the incident pool also declines".)
Drunks and Keys
There's another phenomenon that doesn't get nearly as much discussion as Goodhart's Law. It's related to a concept sometimes known as the "Streetlight Effect" after a joke about a drunk who loses his keys.
A policeman sees a drunk man searching for something under a streetlight and asks what the drunk has lost. He says he lost his keys, and they both look under the streetlight together. After a few minutes, the policeman asks if he is sure he lost them here, and the drunk replies, no, and that he lost them in the park. The policeman asks why he is searching here, and the drunk replies, "This is where the light is".
Goodhart's Law says that when a measure becomes a target, it ceases to be a good measure. But it's important to add that the mere act of measuring something makes it likely that it will one day become a target. When we're looking for our keys, we like to start where the light is.
This is often a problem for analytics, which tackles the factors that are easiest to solve first and then often places undue weight on those factors simply because there's a solution. Quantifying overall running back play is hard, but quantifying a back's ability to break tackles is fairly easy; as a result, we have robust measures of tackle-breaking ability. As a result, many tend to overrate running backs who are good at breaking tackles relative to running backs who are good at other, less-measurable aspects of the position.
Quantifying coaching ability is hard, but quantifying in-game strategy is fairly easy, so it's tempting to produce rankings of coaches based on who is the best at in-game strategy. I think those lists tend to be fairly correct in what they're measuring. But the fact that so many of the greatest, most successful coaches (Andy Reid, Pete Carroll, Mike Tomlin, etc.) tend to rank so low on this measure suggests that it's measuring a thing that overall is not that important (relative to a coach's other responsibilities such as hiring, training, and delegating to staff, improving and motivating players, creating a long-term strategic vision for the franchise and weekly short-term tactical variations on it, and so on).
And yet, once we have the measure, it becomes easy to assume that since Mike Tomlin is bad at challenges and Pete Carroll calls too many running plays, they're not great coaches (despite the wealth of results that suggest they genuinely are).