401(k) accounts are typically among the largest assets held by middle- and upper-middle-income households in the United States. So naturally they draw a lot of attention—in the marketplace, in the media, and in Washington. The government, for example, is proposing new rules on reporting fees and promoting impartiality in investment advice.

Meanwhile, in the marketplace, a few start-up companies are looking to profit from the trend. One is BrightScope, which provides a rating of many 401(k) plans in the U.S. If your employer plan is rated 90 or above, it’s stellar in their estimation. If it’s rated 50 or less, it’s not.

All of this seems like a useful exercise until you dig a bit further. BrightScope’s ranking system reveals one problem. The top of the scale is dominated by plans for doctors, money managers, and pilots; the bottom, by plans for retail workers. So at least from this perspective, it seems like the rating is less about your 401(k), and more about your income—whether you’re a pilot, a money manager, or a retail clerk.

Why this is happening is understandable. Rating companies use publicly available information from the government. That information tells us, for example, that in Plan A, the average employee is saving $10,000 a year, but in Plan B, the average employee is only saving $2,000. What it can’t do, however, is adjust those figures for the relative incomes of the workers—that information isn’t available. Higher-paid workers save more generally, not just because they are higher paid but also because they need to, since Social Security replaces a smaller part of their income. In the end, rating services give a better score to the strong savers in Plan A, as opposed to the cash-constrained in Plan B, even though the reason is that Plan A’s employees make a lot more money than Plan B’s employees.

(As a technical matter, these indicators are further skewed because they are averages, rather than medians. Income, wealth, contribution amounts, and similar statistics are skewed to higher-paid workers whenever averages rather than medians are used.)

Another tricky issue involves other retirement plans. For a couple of companies I know that offer traditional pension plans, I noticed that their 401(k) ratings were low. When a company offers a traditional pension plan, it may provide lower matching contributions in the 401(k) plan. That’s because the company is shouldering the cost of the traditional pension. Also, when a traditional pension is offered, employers tend to save less in the 401(k), knowing they have other retirement benefits. Public rating services can’t figure this out, and so some companies are misrated.

One final twist. Another company I noticed was rated as having below-average investment “quality.” It turns out that the company’s 401(k) uses a number of active managers, and in the rating system those managers’ recent performance was rated poorly. Now it’s hard to know exactly how the plan was penalized given that the ranking system is not fully disclosed. Nonetheless, my guess is that over the period the managers were rated, a specific metric of performance was below that of other plans. But saying “active money managers in the plan had below-average performance during a given period” is different than saying they are low quality.

These types of ratings can still be useful. For example, if you are a pilot, you can compare your employer’s plan with plans for other pilots. Same for workers in health care, manufacturing, oil, drugs, accounting, and so on. If you know the companies your firm competes with, a useful step is to see how your employer’s 401(k) stacks up against theirs. But I wouldn’t spend much time thinking about why your plan is worse than the pilots’ plan—unless of course you are a pilot!

Rating services of all kinds are hoping to replicate the success of Morningstar, whose star-rating system became the de facto industry standard for mutual funds. With the caveats I noted so far, there’s only so much an individual can do in using the latest 401(k) ratings. 401(k) plans are just complex, with plenty of moving parts.

What it seems we need is not just a ranking system, but an interpreter, or a user guide, providing us with all of the caveats and qualifications for a given ranking. It would be nice to have a simple metric for your 401(k) plan—is your plan one star or five stars? For the foreseeable future, that seems highly unlikely.


  • All investments are subject to risk.
  • The link to BrightScope.com will open a new browser window. Vanguard accepts no responsibility for content on third-party websites.