Analysis of the USMLE Percentile Rankings – USMLE Gunner (2024)

The USMLE recently released the percentile rankings (PDF) for scores on all three board exams taken between 2017 through 2019 (updated May 19, 2022). The USMLE does not publish percentile rankings with individual scores and only provides this kind of data to the public every 1-2 years.

In the tables below we have kept the data from previous reports, but the graphs and analysis are updated to reflect the latest data.

This information is very interesting because we can see the actual distribution of scores compared to a normalized distribution, or standard bell curve, that we typically assume when calculating a percentile rank for USMLE scores.

USMLE Step 1

Let us get right into the data. The following is the table showing the mean and standard deviation on USMLE Step 1 for US and Canadian students.

The are some questions that we will try to answer with this information:

1. What is the actual distribution of scores? Is it close to a normalized distribution or is it skewed?
2. What do the different distributions really tell us?
3. How accurate is our percentile calculator?

Using the data provided by the USMLE we can create a graph demonstrating the percentile rank corresponding to a given score (blue) versus the calculated percentile rank (red). The calculated rank is created with the score, mean, and standard deviation and assumes a normalized distribution. Below the actual percentile is in blue, the calculated (normalized) percentile is in red, and the deviation from the normalized distribution is shown in orange.

So to answer our first question, no, the actual scores do not conform perfectly to a normalized distribution but are pretty close. If you look at the graph you can see that at extremes values (very high or very low scores), the actual percentile values are higher than the calculated ones. Also, percentiles closer to the mean tend to be lower than the calculated values.

What this means is that the distribution of USMLE scores has a slightly negative skew, towards the lower scores on the left. To put this simply, more students did well on the exam, scoring above the mean than would be expected in a perfectly normal distribution. Fewer students than expected did poorly, scoring below the mean, but some of them did really poorly, scoring well below the mean, creating the skewed “tail” to the left on the next graph.

Another way of looking at the same data is to graph it as a normalized or “bell” curve. Visualizing both the actual distribution curve (blue) next to the calculated normalized curve (red) makes it easy to see that the actual distribution of scores has a negative skew.

So what does this mean? First, we know that more medical students taking Step 1 scored above than the mean than scored below than the mean. Notice that the mode of actual scores is to the right of where it is expected. However, fewer students did really well (above 260) and more students did really poorly (below 200) than would be expected from a perfectly normal distribution.

Therefore, if you used a percentile calculator to estimate your percentile rank and you did really well (scored above about 260), congratulations, your percentile rank is even higher than the calculated one! On the other hand, if you did poorly, you can be somewhat comforted by the fact that the calculator underestimated your rank. If, like most people, your score is somewhere in the middle, the calculator will have slightly over-estimated your percentile rank by approximately 5-10 points.

I talked a lot about the differences between the actual percentile rank and the calculated one to highlight that the score distribution is not a perfect bell curve. However, the actual percentiles are very close to a normalized distribution. The calculator estimates the percentile based on a perfectly normalized distribution and yet it does a pretty good job at estimating the percentile rank: at any given score, the highest deviation from the actual percentile was -12.6 points and the overall accuracy was -2.96 points.

In the future, I would not expect the actual distribution to change much, so for calculating percentile rank of scores after the date shown above, you can be confident the calculated percentile is accurate within a few points. If your score is on the extreme ends of the spectrum with either a very high or low score, the calculated percentile likely underestimates your actual ranking on Step 1.

USMLE Step 2

The following table is the overall data provided by the USMLE for Step 2.

Again we can graph the actual percentiles reported (blue) and our calculated percentile rank (red) for each of the given scores.

On USMLE Step 2, the distribution of scores also has a negative skew. Compared to Step 1 the differences are less obvious. Again, the calculated percentiles are slightly higher than the actual percentiles between 225 and 260. This shows us the negative skew towards the lower scores. Again this means that more people scored above the mean than scored below it.

The calculator does pretty well, with the significant deviations occurring near the mean. The largest difference is -7.84 percentile points at 250, but the overall accuracy is -1.14 points.

USMLE Step 3

The following table is the overall data provided by the USMLE for Step 3.

From the data presented in the table it is easy to see that the average score on Step 3 is lower compared to the first two exams, but the distribution of scores is also narrower (lower standard deviations).

On USMLE Step 3, there is no significant skewing of the score distribution. The actual and projected percentiles are nearly the same. The largest deviation from the normalized curve is at score 230 and is -4.66 percentile points. All other differences are less than that with an average accuracy of -0.91 points.