While I have been publishing research for many years, I consider myself new to the blogosphere. So I was a little surprised when my recent post on a new 401(k) rating service, Brightscope, elicited some response, including a critique from founders of the company. Frankly, I don’t think anyone has ever created an “open letter” to me before now. But I was glad to get a conversation started.

I stand by my original post. Any system designed to rate a complicated product, like a 401(k) plan, is bound to have some holes, especially when it relies on incomplete public data. Certainly, I encourage anyone with questions about a particular 401(k) plan to fiddle around with Brightscope’s service and see what they come up with. In my own small tests, I found a hospital company and online brokerage in the same peer group; in another peer group, a carpenters’ union and a boutique money manager.

The lesson is clear: It’s not just enough to process data and come up with a rating. Data can be dirty and often needs lots of scrubbing. And that presumes you have all the data you need.

In this age of information and social media, there will be opinions and ratings and an avalanche of information on all things in life—including your 401(k). How does the average web user make sense of it? By relying on the old Latin proverb: Caveat lector.

Let the reader beware.

Note: The links to Brightscope.com will open new browser windows. Vanguard accepts no responsibility for content on third-party websites.