Ratings that help you de-select funds

Assessing performance

When analyzing performance with an event study, it can be helpful to look at excess returns. What we want to study is whether the ratings produced by the Morningstar Quantitative Rating help select funds that are superior to their peers. In Exhibit 1, we examine the returns of Positive, Neutral, and Negative ratings in excess of the average returns of each fund's Morningstar Category. Here, Positive ratings are the Morningstar Medalist ratings, Bronze, Silver, and Gold.

Exhibit 1: Morningstar Quantitative Rating event study for returns ex-category average

Exhibit 1 shows the Neutral rating outperforms the Negative rating, and that the Positive ratings group outperforms the other two ratings. One initial observation is that the average excess return of the Positive rating group is slightly positive, which suggests that funds that the Morningstar Quantitative Rating rates positively may outperform their peers. Further, the biggest gap in performance is between the Negative and Neutral funds, which return 73 and 10 basis points below the category average, respectively, after 12 months. In addition, we find that the differences between Positive and Neutral, and Neutral and Negative, are statistically significant, with t-statistics of 14.29 and 27.67, respectively.

Because we are examining excess returns to the category average, we can interpret this as saying that Negative funds are likely to underperform peers, and Neutral funds are likely to perform much more in line with the peer group. This is the crucial piece for using the Morningstar Quantitative Rating for fund deselection, because even switching away from a Negative fund and into a Neutral fund offers the potential for improved outcomes for investors.

How stable are the ratings?

For a ratings system to be useful to investors, it must be stable as well as efficacious. Investors will get more value out of a ratings system if they know that a fund, once rated a certain way, is likely to be rated the same way in the future. There are several ways to examine the stability of ratings for funds, but the way we will analyze the Morningstar Quantitative Rating is with a series of transition matrixes. In general, a transition matrix shows how likely a fund is to move from one rating to another.

We will focus on a universe of funds that received Negative or Neutral ratings. A transition matrix showing maximum stability would show 100% in each cell down the diagonal from the top left to the bottom right. This would indicate that for the time period covered by the transition matrix, 100% of funds ended up in the same raw-rating quintile where they started. A totally unstable transition matrix would show 20% in each box, which would indicate that no matter where a fund started, it was equally likely to have any rating at the end of the transition period.

Exhibit 2 is a one-month transition matrix showing that 92% of funds which start in the first, or most negative, quintile end up in the same quintile one month later. This is encouraging, but as the Morningstar Quantitative Rating is produced on a monthly basis, this is not a particularly long time frame. However, it is a good baseline, and an important secondary observation is that funds are very unlikely to move beyond the adjacent quintile. A fund in the second quintile may move to the first or third but is very unlikely to skip to the fourth or fifth quintiles. If funds moved so far very often, that would be a murkier signal.

Exhibit 2: Quintile percent after one month

In Exhibit 3, we extend our transition period to six months. Knowing nothing else, we would expect the matrix to show somewhat less stability than the one-month matrix simply because there has been more time for conditions to change. However, we still see stability in the ratings. Interestingly, a fund in the first quintile is still very likely to remain there after six months. For the other quintiles, we find that the likelihood of a fund staying in the exact same quintile has meaningfully decreased. However, we still interpret the rating as stable so long as most funds that move quintiles only move to an adjacent quintile.

Funds in the first quintile are more than 94% likely to end up in either the first or second quintiles after six months. In the third quintile, funds are more than 89% likely to end up in the second, third, or fourth quintiles. Finally, for the fifth quintile, funds are more than 88% likely to end up in the fourth or fifth quintiles after six months. Funds are now more likely to shift between adjacent quintiles, but are still overwhelmingly unlikely to jump beyond adjacent quintiles.

There is an important point to recall here. These matrixes are focused just on funds receiving a Negative or Neutral rating, so we are examining a subset of all ratings. A fund moving between these quintiles may still show a Negative or Neutral rating. We are analyzing the stability of the Morningstar Quantitative Rating at a finer level of detail than the discrete ratings the system assigns, so some instability here may just reflect the reality of funds changing through time.

Exhibit 3: Quintile percent after six months

Finally, we extend our transition period to one year in Exhibit 4. At this time horizon we still find that the most common result in each quintile is that the fund ends up in the same quintile where it started. At the same time, funds are increasingly likely to move quintiles, especially when they start in the second, third, and fourth quintiles. As with the six-month matrix, this effect is not as visible in the first and fifth quintiles. This observation indicates that when a fund is in the bottom quintile of rated funds, it is likely to stay there; this is still true, but weaker, for the highest-rated funds in this universe.

The result of examining these transition matrixes is that the Morningstar Quantitative Rating's output is stable, even at a particularly granular level at the low end of the ratings spectrum. Further, funds in the lowest and highest fifths of this universe are likely to stay there, even after a year. Within the universe of poorly rated funds, investors can take the Morningstar Quantitative Rating as a reliable signal of the quality of the funds they are examining.

Exhibit 4: Quintile percent after 12 months

If an investor is examining multiple investment options, the Morningstar Quantitative Rating can help steer investors away from inferior funds. We examined the performance of the rating system from both a performance and stability perspective, and find that the rating system is successful. While Positive funds outperform, Negative funds significantly underperform.

We also evaluated the stability of the ratings that the Morningstar Quantitative Rating produces through transition matrixes. We find that the ratings are stable, even within the discrete ratings Morningstar publishes. This effect is present at both a short and a longer time horizon, showing that the Morningstar Quantitative Rating is stable over time. This stability can assure investors that the signals they see from the Morningstar Quantitative Rating are reliable. With these findings, we add depth to the potential use-cases of the system by showing how investors can avoid poor funds with the Morningstar Quantitative Rating.

This is an excerpt from the Morningstar Quantitative Research report “The Morningstar Quantitative Rating as a Fund Deselection Tool”.