the quarterback problem

A couple of weeks ago I found a copy of “What the Dog Saw” by Malcolm Gladwell for 2€ at a second-hand books stall. Not yet sure what the dog saw, but I did see something I wanted to share.

The issue that got my attention is what Gladwell calls the “Quarterback Problem”. Generalizing, it refers to a situation where it is extremely difficult to predict how good a person will perform in a new context they have never experienced before.

Why the “QuarterBack”? A QB is a key position in a football team, and teams in the major league (NFL) have scouts who constantly scan the minor leagues for the best prospects. The “problem” here is that there is no way to really know who will succeed once in the NFL; none of them ever played a NFL match! Sure, scouts have experience, but data show that when predicting QBs success they fail most of the time. One big reason for that, Gladwell says, is that QBs are the players who undergo the biggest change when transitioning to the NFL, so being good in a minor league does not necessarily translate to being good in the NFL.

The main reflections for me are two: - Realizing that if a change is big enough, it affects people in unpredictable ways, indeed. I never thought about that. As data practitioners or decision makers, we should be aware of that. - Faced with this unpredictability, how do we handle it? For example, NFL cannot tell who is going to be good “tomorrow” in “B”, so they simply pick who is good “today” in “A”, ignoring that A ≠ B. This method proved not to be that good. Also, this method clearly introduces a bias, since it does not give opportunities (or hope) to sub-par performers in A.

Finally, I want to close with an analogy I see in education: replace players drafts with students admissions; replace the dreams of an aspiring QB with those of an applying student. Isn’t it true that being accepted to a top university can be a change big enough it might trigger unpredictable outcomes? If it is, are we aware and conscious that picking who is a top performer today in high-school (A) is not the same as finding who is going to be successful in our own academic institution or in a professional environment tomorrow (B)?

What could be other strategies?