Luck

I recently read a book called Success and luck: good fortune and the myth of meritocracy by Robert H. Frank. The basic thesis is that those who are successful in getting a job, being accepted to a university, etc., are virtually always (perhaps always) luckier than those who failed to do so. However, due to cognitive bias, people tend to underestimate the role of luck in their success and attribute all of their favorable outcomes to their superior intelligence, work ethic, foresight, and the like. Enjoying the fruits of one's hard work is not necessarily a bad thing, but the troublesome corollary to this cognitive bias is that such people tend to attribute others' lack of success to a deficiency -- not smart enough, didn't work hard enough, etc. (I heard the term "weaponized self-help" recently that is along the same lines -- people don't fail because of racism, sexism, ageism, etc., they just didn't "believe" in themselves enough.) This can cause us to treat others as "inherently" lazy, unmotivated, or intellectually limited, and provides a convenient excuse to avoid uncomfortable questions of privilege (e.g., white, male) as factors in the increasingly unequal distribution of wealth in the society.

I don't want to get too far afield in the social/political aspects of the author's thesis. One thing that was interesting in the book is that the author mentioned results of a statistical simulation as part of his argument. The basic idea is to create a set of simulated job hunters, college applicants, etc. each with a random value for  "strength" (or intelligence, inherent capability, or whatever else you want to call it) and a random value for "luck". These values are drawn from a uniform distribution between 1 and 100, so any value in that range is equally probable. Next, a composite variable called "performance" is created, which is the weighted sum of the strength and luck variables. For example, if you believe that luck plays a very small role in performance, you could set performance = (strength * .95) + (luck * .05).

I decided to implement this with a simple R script and apply it to some data. A quick web search shows that there were 39,494 applicants to Harvard for the class of 2021, and that 2,056 applicants were accepted. I created a "strength" vector and a "luck" vector of 39,494 draws each (the applicants) from a uniform distribution between 1 and 100,  and then a "performance" vector of the same length (representing the overall "score" of the application package), with strength accounting for 98% and performance 2% of the score. These were all combined into a dataframe. I then ordered the dataframe by performance (high to low) and split the top 2056 records (the successful applicants) from the others (the unsuccessful applicants). Using R's summary() command, I captured the median "luck" score for each group, along with the minimum strength score in the successful group and the maximum strength score in the unsuccessful group. This process was repeated 1000 times.

My simulation showed that the median "luck" value for the "chosen" group was 55.33, while the median for the "not chosen" group was only 50.24. (A t-test confirms that these are statistically significantly different.) This suggests that, indeed, the chosen group were "luckier."

The average minimum "strength" score for the chosen group was 93.92, while the average *maximum* score for the "not chosen" group was  96.19. In other words, the chosen group were not "the best of the best," but rather "the luckiest of the best."

It is important to note that the takeaway from this is NOT that personal effort doesn't matter or that it is ONLY a matter of luck that people are successful. Obviously, the harder one works the more likely they are to be in a position to take advantage of a "lucky" break. The important point is that is a mistake to equate lack of success with lack of personal effort or ability. As the old saying goes "better to be lucky than good."