Shining some light on the stars

The star rating gives a concise summary of how a fund has performed historically, after adjusting for risk, relative to its peers. (For a detailed explanation of the calculations involved in the Morningstar rating, click here.) It assesses how good a job a fund's management has done at balancing risk and return. One way to look at it is as an achievement test, not an aptitude test; or, put another way, as a report card, not an I.Q. test.

Why, then, is the star rating useful and of value?

Because often the factors that have made a fund an under- or over-performer in the past still exist. For example, low-cost funds tend to remain low-cost funds, and tend to provide above-average returns. Similarly, funds lacking diligent risk controls, a sound investment philosophy and/or consistent stock buying and selling disciplines may be, for those reasons, more likely to have been under-performers in the past and to continue to be under-performers in the future. Therefore, funds that have done a poor job in the past tend to be poor performers in the future—and historically superior funds tend to earn above-average ratings in the future.

How stable are the stars?

Morningstar conducted analysis to test the stability of star ratings over time.

There are 726 mutual funds in Canada that received a star rating both as of March 31, 2003 and five years earlier, as of March 31, 1998. We took a look at how stable star ratings tended to be between those two dates. Despite excluding funds that did not survive the whole period from 1998 to 2003, and thereby ignoring the higher extinction rate of poorer-performing funds, the results demonstrate a clear trend. Funds with above-average ratings of four or five stars in 1998 were more likely to retain above-average star ratings in 2003 than to receive average (three star) or below-average (one or two star) ratings. This is despite the fact that four- and five-star funds account for just about one-third of rated funds at any point in time. Similarly, funds with below-average ratings in 1998 were more likely to continue to have below-average ratings in 2003 than to have either average or above-average ratings.

TABLE I: Star ratings stability after five years

	March 1998 ratings
March 2003 Ratings	Above- average	Average	Below- average
Above-average	140	65	35
Average	70	131	66
Below-average	37	82	100

Comparing funds' star ratings at two specific points in time may not be entirely sufficient to make compelling conclusions, but it does offer a reasonable starting point. For a more comprehensive view on ratings stability, we also looked at all monthly star ratings across the full five-year period. To simplify things, we collated historical results over the full time period by way of funds' current star ratings and then separately by way of their star ratings five years earlier.

The next chart, which is based on over 80,000 observations on the nearly 2,400 funds that had a star rating as of March 31, 2003, shows the tendency for funds to have had historical star ratings typically identical to their current rating, or different by no more than one star. For example, funds with a current rating of five stars have (historically over the last five years) been rated as five or four stars 79% of the time and as two or one stars only 6% of the time. Conversely, current one-star funds have been rated five or four stars just 10% of the time in the past, compared with two- or one-star ratings 76% of the time.

Looking at star ratings from funds' positions five years ago might be even more compelling as, instead of looking backwards from the current situation, it provides a summary of what happened to funds following an initial position. It might therefore be more analogous to the way one could view the situation today in an attempt to establish expectations for the future. However, this analysis produced similar results.

The following chart is based on over 58,000 observations on the just over 1,000 funds that were assigned a rating as of March 31, 1998. Funds that had a four-star rating five years ago, for example, deviated from that rating by more than one star only 8% of the time, stayed at four stars 55% of the time and plus or minus one star the other 37% of the time.

A third, more comprehensive approach to studying star ratings stability was taken. For each and every fund that has ever received a star rating during the five-year period, we counted the number of months for which it received each rating. A hypothetical but somewhat typical example might be XYZ Fund's count of 15 months with a four-star rating, 30 months with a three-star rating, 10 months with a two-star rating, and no periods for which it was rated with either one or five stars.

By counting the 98,000 observations for the nearly 3,000 funds that have ever received a star rating from 1998 to 2003, we determined how many funds have consistently received one and only one rating over their lifetimes versus how many funds have experienced each and every rating at some point during their lives. The range of possibilities is therefore from only one to all five. As the following chart demonstrates, the latter has been much less common than the former. Over 60% of funds have received no more than two different ratings, while fewer than 13% of funds have received four or five different ratings at various times.

Detractors might find fault with using the results of one's model to test that very same model. There's no doubt that other measures can also be used to ascertain how well fund ratings have served as indicators of future performance. Other options include studying the relationship between funds' star ratings at one point in time with absolute returns (of one, three, five and 10 years) and risk-adjusted returns (Sharpe, Treynor and Sortino measures, alpha and beta, etc., over three, five and 10 years) over succeeding time-periods. Tractability almost demands that one or two measures for one or two periods be analyzed. But what then is the best measure, and what is the most appropriate period? The utility of the star rating is that not only is it a risk-adjusted performance metric, and not only are its results already relative, but that it is also a blended measure of results for multiple time periods.

Stars and returns

Nonetheless, we also made some preliminary examinations of relative fund performance based on absolute (i.e., not risk-adjusted) returns for the five-year period from March 31, 1998 through March 31, 2003. Within each fund category, all funds' five-year returns were calculated and ranked into quartiles. We then evaluated the results for that subset of funds within each category that had a star rating at the beginning of the five-year period.

As it turns out, despite the limitation of focusing on one very specific, end-date-sensitive period, the results based on absolute fund performance apparently confirm the results obtained from studying star ratings stability: above-average rated funds tended in most categories to outperform going forward and below-average rated funds tended to underperform.

We limited the analysis to just the nine fund categories with a minimum of 30 funds that had both a five-year return as of March 31, 2003 and a star rating as of March 31, 1998. The results were particularly conclusive in five of the nine groups, most tellingly in the Canadian Money Market and Canadian Bond categories, while the Canadian Balanced, Canadian Equity and Canadian Small-Cap Equity groups provided unambiguous results as well. Though the Canadian Large-Cap Equity, U.S. Equity, Global Equity and Foreign Bond results were rather less clear-cut, in no cases did above-average rated funds tend explicitly to under-perform going forward. Furthermore, in eight of the nine categories (Canadian Large Cap Equity being the lone exception) below-average rated funds clearly tended to underperform during the succeeding five years.

The results for all nine categories are shown below in tabular form. For instance, in the Money Market category, funds with an above-average rating (of four or five stars) in 1998 ranked predominantly (97%) in the category's top half over the succeeding five years. Funds with an average rating (of three stars) in 1998 were about equally likely to have a return over the next five years that ranked in the top or bottom halves (48% to 52%). Funds with below-average 1998 ratings (of two or one stars) had five-year returns through March 2003 that typically (84%) ranked below median in the category. You can also click on the names of each category to see the results in graphical form.

TABLE II: Percentage of funds whose five-year returns ranked in each category's top and bottom halves, sorted by their 1998 star ratings

Canadian Money Market
	March 1998 ratings
5-year rank
% above median	97%	48%	16%
% below median	3%	52%	84%
Canadian Bond
	March 1998 ratings
5-year rank
% above median	93%	50%	38%
% below median	7%	50%	62%
Canadian Balanced
	March 1998 ratings
5-year rank
% above median	63%	51%	43%
% below median	37%	49%	57%
Canadian Equity
	March 1998 ratings
5-year rank
% above median	71%	48%	29%
% below median	29%	53%	71%
Canadian Large Cap Equity
	March 1998 ratings
5-year rank
% above median	53%	47%	55%
% below median	47%	53%	45%
Canadian Small Cap Equity
	March 1998 ratings
5-year rank
% above median	80%	63%	22%
% below median	20%	38%	78%
U.S. Equity
	March 1998 ratings
5-year rank
% above median	44%	54%	40%
% below median	56%	46%	60%
Global Equity
	March 1998 ratings
5-year rank
% above median	44%	36%	36%
% below median	56%	64%	64%
Foreign Bond
	March 1998 ratings
5-year rank
% above median	50%	58%	25%
% below median	50%	42%	75%

The bottom line

The preceding analysis demonstrates that star ratings appear to be meaningful not just in the report card historical sense, but in the sense of establishing at least some degree of reasonable expectations for the future. Star ratings, like any fund-ranking service, cannot possibly be perfectly stable, nor can they remotely be expected to be perfect predictors of future performance. Nonetheless, fund ratings have demonstrated a material degree of stability in the past.

That said, a fund should certainly be neither bought nor sold simply on the basis of its star rating. While Morningstar's star ratings are a useful first step to examining the relative merits of different mutual funds, we are the first to admit, and in fact insist, that looking at star ratings is but one step of any conscientious, diligent investment appraisal.

For additional resources on the Star Rating and answers to frequently asked questions, click here. (This article was first published in May 2003.)

---

No statement in this article should be construed as a recommendation to buy or sell securities or to provide investment advice or individual financial planning. Morningstar Canada does not provide specific portfolio advice and recommends the use of a qualified financial planner when appropriate.

---