If you are a student that is in the middle of the college search process or you are a college administrator, college rankings play a BIG role in your life. Students rely upon these rankings to help them find the right school for their educational needs and college administrators either cower in fear or jump for joy depending on the results of the various rankings.
Given the widespread impact of these rankings…I really want to know… who is the watchdog for all of these college ranking entities? Who makes them accountable for the final results? Do they really know what they are doing?
I know that all the data and methods utilized to gather and sculpt the information are generally available for public review but at the end of the day it appears that most people really just take the rankings for face value and trust that someone applied the appropriate variables to the final calculations that ranked one institution higher (or lower) than another. This public trust of our college ranking systems put a lot of power at the hands of those doing the rankings. So… since the college rankings play such a big role in our lives, I think it is extremely reasonable for us to apply a similar ranking formula to the “College Rankers” and see how they fair.
I have devised the following formula as a starting point for our “ranking of the rankers”. As you can imagine, I can make it as complicated as humanly possible but I like to keep things simple. Besides, it allows a lot of room for subjective tweaking by me (which we know NEVER happens in college rankings). Each category/variable of the equation below can receive a score of 1 to 5 and I will try my best to work with round numbers but given the mathematical complexity of my equation, I can’t guarantee that some long stranded decimal point action won’t present itself. 😉 If my calculations are accurate, the highest score that can be achieved is 50 and the lowest is .40 (but that low number would take some real effort to reach).
Simple Equation for Ranking the College Rankers
Accuracy (plus) Legitimacy (divided by) Lack of Quantitative Intelligence (times) Method of Delivery (equals) Overall Ranking
-or-
(A+L) x MOD = OR
LQI
Now that we have an equation in place for our rankers. Let’s go ahead and start the Ranking of the Rankers… or ROTR for short…
U.S. News and World Report
U.S. News and World Report is to college rankings as Tiffany’s is to jewelry. If U.S. News and World Report were around when the first three colleges were formed, my guess is that they would have established the first college ranking system without haste. (By the way – if you are curious about which three were first, it all started with New College (known know as Harvard) in 1636, College of William and Mary in 1693, and Collegiate School (Yale) in 1701). U.S. News is probably the best known of all the college rankers. Every year people are chomping at the bit to see the release of their college ranking results. I would liken the interest to that of Groundhog’s day when everyone patiently awaits to see if Punxsutawny Phil catches a glimpse of his shadow. The only difference is that everyone leaves Groundhog’s day resolved on the prospect of snow for the coming weeks and it seems like absolutely no one is resolved after the release of the rankings by U.S. News. I wonder how the ROTR Report will treat them?
Accuracy: I am going to give U.S. News a 5 in this category. Over the years they have established a comprehensive systematic approach to how they gather data and the accuracy of this information doesn’t seem to draw too many questions
Legitimacy: They will get a 3 in this category. The resources that they utilize to gather their information are fairly reliable (at least the institutional figures). However, some of the rankings do take into account the opinions of high school counselors and academic peers (college presidents, provosts, and deans). The results on these surveys can definitely be skewed depending on a person’s knowledge (extensive or lack there of) and views of various schools. So that is why I ranked a 3 on this variable.
Quantitative Intelligence (Lack There Of): U.S. News has repeatedly tweaked the manner in which they calculate their numbers. A lot of time and effort seems to be emphasized in this area (since it garners the most attention?). Even so, I think they still have room to grow in this area so I am going to give them a ranking of 2 (remember a low number on this variable is better than a higher one because a larger number indicates that you excel in having a lack of quantitative intelligence)
Method of Delivery: U.S. News promotes their rankings in just about every media method available to mankind. They dominate the internet, the news stands, and also the reference section of bookstores with their statistical rankings. Therefore, I give them a 5 in this category.
The End Result: Using the ROTR calculation above, I see that U.S. News and World Report scores a 20 on the ROTR Report. Not too shabby all things considered.
When Forbes started doing college rankings, I had high hopes of the role they would play in the ranking system. If anyone was going to dissect and calculate the financial aspect and viability of colleges, you couldn’t get any better than Forbes… right? I mean.. seriously.. it is FORBES. Unfortunately, the financial wizards at Forbes let me down. I guess I must have had my expectations set too high. When they showed up to the college ranking dance floor, it looks like they left their “A game” at home and were prepared to make a mediocre appearance. Shame shame on Forbes… Let’s see how they fair on the ROTR Report.
Accuracy: Forbes is getting a 3 in this category. I would like to give them lower but they do a decent job of gathering the data from their various sources and compiling the data for their rankings.
Legitimacy: Going with a 1 on this category. Any ranking entity that utilizes RateMyProfessor.com as a resource for the data they are calculating, should get a zero but since zero is not an option in the ROTR Report, I am giving Forbes a 1. For those that don’t know, RateMyProfessor is open to the public and anyone can rate a professor regardless of whether they even had them for a class. Many times, professors even vote for themselves! I am also giving Forbes a 1 because they introduced this ill-conceived and unfounded ranking (be sure to check out the comments on this one). Lastly, they also use the Who’s Who in America series to measure post-graduate success. I really don’t see a connection there so I feel the legitimacy of the Forbes data and ranking system is sub par.
Quantitative Intelligence (Lack There Of): You would think that a bunch of financial wizards from Forbes Magazine would do a great job at crunching the numbers to come up with the most equitable results. However, their method is greatly flawed in that it doesn’t distinguish between school types so small private liberal arts colleges, large military academies, and research universities all get thrown into the same mix. I going to give Forbes a 5 on their lack of quantitative intelligence.
Method of Delivery: Forbes maximizes most media outlets to get their ranking out amongst the people. They have a great web tool that makes sorting out specific results a cinch. They provide appropriate coverage in their magazine but it appears they have not gone the route of book publication just yet. I give Forbes a 3 in this category.
The End Result: Using the ROTR calculation (above), Forbes comes out of the gate with a 2.4. This certainly leaves them room for improvement. Maybe they can adjust their ranking operations to perform better next year on the ROTR Report. 😉
You know… with a name like Princeton it just has to be good …right? The Princeton Review has certainly been around for a number of years and personally I would probably rank them right up there with U.S. News and World Report in regard to viability and usability by prospective students and families. However, at the end of the day, the ROTR Report decides all, so lets see how they compare.
Accuracy: Princeton Review (PR) used to collect all of their data by canvasing the nation and surveying students on each of the college campuses utilizing a paper form. However, in recent years they now offer an electronic survey that students can complete online 24 hours a day , 7 days a week. The information gathered in this survey is compiled in an efficient and effective manner that allows very little room for inaccuracy. Based upon this, I am giving them a 5 in this category.
Legitimacy: The unfortunate aspect of the Princeton Reviews data gathering process is that it is all based upon the opinion of students. However, since their rankings seem to be more personal in nature (the warm fuzzies of campus), the legitimacy of this data and how it impacts their ranking system is rather fitting. So, I give them a 4 in this category.
Quantitative Intelligence (Lack There Of): Princeton Review does a great job of simplifying how they calculate the data received. All the information they gather is warehoused in a database and each school is scored in a manner that is similar to how a student is graded in college (like a GPA). These scores alone decide which colleges make the 62 ranking lists. Nothing more.. nothing less. Based upon this simplistic approach, I am giving Princeton Review a 1 in this category (meaning, they have a high aptitude when it comes to quantitative intelligence).
Method of Delivery: Princeton Review has a fairly basic delivery method on their website. In addition, they also publish books each year with their findings. However, the greatest tool that they use to get the word out is their partnership with USA Today. Using the readership base of USA Today(online and paper subscriptions), they are able get their ranking results out among the masses. I give Princeton Review a 5 in this category.
The End Result: Dropping the figures above into the ROTR calculation, I show that Princeton Review scores a 45. Surprisingly, this allows them to blow past the competition and secure the ranking of 1st place in the 2010 ROTR Report (Ranking of the Rankers).
In Summary
College rankings serve as a great reference tool for students that are in the midst of the college search process. However, don’t let these rankings be the end all be all deciding factor in the college that you ultimately choose to hang your hat for four years. Take the time to visit each of the colleges you are interested in and ask the questions that are near and dear to what YOU are looking for in a college environment (academically, socially, politically, etc…). Then, and only then, can you make a decision about the best college for you. Don’t let these guys above (Forbes, U.S.News, and Princeton Review) make that choice for you. At the end of the day it is a personal choice and you will feel accomplished that you made it on your own.
