USDF Horse of the Year Rankings...do they need an overhaul?

[dressage_fan]

I'm new to the sport and certainly not intending to offend anybody, but something I read this morning about the results at Devon caused me to question the USDF Horse of the Year rankings. The article listed Endel Ots as winning the Intermediare I CDI. At the bottom of the article, he is quoted as saying he's first in the country at the Prix St. Georges and Intermediare I Freestyle. No offense intended to Endel, but my reaction was surprise because I had never heard of him.

After a quick score check on USDF, I found that he is indeed ranked first in both of these cateogories. I also noticed that he hadn't competed in a CDI or one of the USEF High Performance qualifiers all year until Devon. Taking it a step further, I compared his show schedule to the top 5 finishers at the Intermediare I Championships. This pre-Devon table lists the total number of events, CDI/HP events, and USDF-posted median at each level (or the calculated median if the rider hadn't qualified to be ranked):

Team Events CDI/HP PSG I1 I1 FREE
-------------------------- ------ ------ ------- ------- -------
Endel Ots-Bentley 5 0 71.875% 66.500% 74.500%
Michelle Gibson-Don Angelo 6 4 69.000% 70.292% 76.659%
Shawna Harding-Come On III 9 4 71.250% 71.000% 73.775%
Steffen Peters-Montango 6 5 69.917% 68.500% 72.534%
Elizabeth Ball-Orion 7 5 66.750% 70.017% 71.317%
Susan Jaccoma-Wadamur 8 4 68.125% 69.884% 69.650%
All of the other competitors were actively competing in CDI's and High Performance events. I looked through the rules and was surprised to notice that competing in a CDI or High Performance competition is not required to be ranked.

I also noticed that Endel didn't compete in the PSG yesterday at Devon. A quick calc revealed that he would have needed 71.250% to maintain his ranking in the PSG. If I were in his shoes and really wanted that first place ranking I certainly wouldn't have risked showing the PSG. The quick calc on the Intermediare I Freestyle reveals that he needs 73.049% today to maintain his I1 Freestyle ranking. I'm sure Endel is a good rider and wish him well today. If he hits the number to retain his ranking, that would be a great story.

Most likely these competitors are thinking grander thoughts about the Grand Prix at the International Level, so I doubt this is a big concern for them. For me though, as a fan of the sport, I wonder if who USDF is calling the "Horse of the Year" is really representative of the best team in the country. The Grand Prix rankings look about right...the top 4 were all at the Olympics. But, I look through the I1 and PSG rankings and they seem out of synch with who's competing and winning the big events.

I'm not the one to make policy, but maybe someone who is will read this and consider some recommendations. First, require the FEI-level awards to have at least one CDI or High Performance event to be ranked. Second, change from a median to an average score with one drop score. The median tends to discount exceptionally good or bad scores. Give each horse a drop score to get rid of that one hiccup ride and then average the rest to come up with a number that represents the horse's performance in all their shows during the year - not just their middle scores.

Taken a step further, why not have a system that calculates the riders ranking based on their score and the strength of their schedule. They could modify a riders score - used for ranking purposes only - based on the relative strength of the event and the "toughness" of the judge. It could be 100% statistical so there would be no question about whether it's easier or harder to score at a given event or with a given judge. Looking at the USDF website, it certainly seems like they have the data to do this and so much more.

[Equibrit]

I think he is a young rider - which would throw your theories out the window!
Besides - why should somebody have to take part in an INTERNATIONAL competition for their
score to count in a NATIONAL Championship? I would also imagine that he is a bit more concerned with his YH schedule than USDF HOY which is a bonus!

2008 Brentina Cup ride; http://video.aol.com/video-detail/picasso-and-endel-ots-yr-gp-test-brentina-cup-08/4183610366?icid=acvsv4

[FriesianX]

Well, these are the USDF HOY rankings. Which means they are based simply on USDF show results. Does the USDF HOY mean the horse is the best in the country at that level? No, there are just WAY too many variables to make that determination without a direct competition of the same horses at the same venue with the same panel of judges. Is it an honor to win USDF HOY? Yes, it means you are doing well, your horse has good training and riding, and you are successful in the show ring. Is there a way to overhaul the system? There are many ways to overhaul the system, but you'll always have inequities based on many different variables.

The one rule I DO wish they'd change is the requirement to show at so many different shows each year. Keep the number of judges, but for those who only make it to 2 or 3 different shows, but they are big, multi-day, multi-judge shows, it seems they should qualify if they achieve the number of scores and variety of judges needed.

[Anselcat]

I wonder if who USDF is calling the "Horse of the Year" is really representative of the best team in the country.

What FriesianX said.

I don't think USDF HOY is meant to find "the best team in the country". Though the best team in the country would certainly be able to win USDF HOY if they were actively trying to. Different people have different goals, and if all the year-end competitions were truly scored the same, you'd have one "best" horse and rider team winning everything at the end of the year.

[papony]

Hey! Don't spoil our fun! lol! My PSG average has me currently in 10th place for the open HOY standings, right behind Michele Gibson and Don Angelo..... now, do I really think that Michele would be quaking in her boots at the thought of competing against me? Hardly! lol!

I have to admit, when I saw that ranking I hit the print button so fast! I don't expect that ranking to last, but I am going to enjoy every minute its there.... realistic or not!

[YankeeLawyer]

I think this is kind of an odd post - -the guy did actually win at Devon, no? I think there might be reason for a raised eyebrow if he were ranked 1st but actually stunk.

[Atlantis]

This has come up before.

The first time I remember, was when Michelle Gibson (or maybe it was the owners of Peron, it was a very long time ago to remember details!) were very vocal the year the GP HOY was won by a relative unknown. It had something to do with most of the big names competing in Europe that year, because it was an Olympic year, and not having enough scores etc. at home (because international scores do not count for a national award.) Some felt the final results that year were misleading because of the noteable absense of some big names (like Peron.)

More recently, I think it was perhaps George Williams, who is known to compete in CDI's and in Florida, and he was making a case that competitions like that should be weighted because there is a consensus that scoring is tougher at competitions like Florida and CDI's. However, nothing ever came of that discussion, that I am aware of. It would have been a slippery slope to address that issue, for a number of reasons.

One of the best rule changes was to limit it to two scores per judge, because it cut down on judge shopping and score chasing.

It's not a perfect system, but it's a pretty good one. The requirements as far as shows, judges, classes, are pretty reasonable, and the cream will generally rise to the top.

There are always people who will try to play the system, but the current system seems as good as any, with the final results being about as accurate as can be expected, when you figure we're trying to rank results for an entire year and an entire country.

Congratulations to everyone who wins one!

[Maude]

Remember too, that there are many wonderful and talented rider/horse combinations out there that either don't have the time or money (or both) to compete and qualify for these awards. Not to take anything away from those who do win HOY awards-congratulations to them. Just saying that there is so much more to success than a title.

[dressage_fan]

A lot of good points in response to the original post. I guess my main point is that with the enormous wealth of data on rides - even compared to a few years ago - there may be an opportunity to substantially improve the ranking system where there never has been before.

For example, the data is there to determine the relative strength of a show based on the competitors and judges present. Putting a weighting on a ride based on the show and judge could make the score and ranking system more meaningful.

I would be willing to bet that every competitor has at some point finished a ride and wondered "Why did I get that score? I thought it was much (better/worse).". The potential is there to provide a lot of that explanation. It could be that everybody at the show was getting good or bad scores. Or, perhaps the judge of their ride generally scores that test higher or lower than their peers.

Beyond the ranking system, the data is there to substantially improve the entire sport. Imagine if someone at USDF ran a 5-second report off their database and discovered that competitors are generally receiving lower scores for a particular movement at a given level. Programs could be targeted toward that movement to generally improve the rider population.

Anyway...food for thought. I'm not in any way taking away from any rider, judge or the USDF. Just some ideas on how we might make use of all the data for the betterment of the sport.

[Equibrit]

So - how would you weight the fact that the judge had a bad lunch ? ;)

[yaya]

The USDF does not collect scores per movement, only the total for the test.

Shows only submit the total raw score and the percentage. If scores are done by adding machine instead of computer, the show doesn't even have a record of the individual movement scores.

[mjhco]

A lot of good points in response to the original post. I guess my main point is that with the enormous wealth of data on rides - even compared to a few years ago - there may be an opportunity to substantially improve the ranking system where there never has been before.

For example, the data is there to determine the relative strength of a show based on the competitors and judges present. Putting a weighting on a ride based on the show and judge could make the score and ranking system more meaningful.

I would be willing to bet that every competitor has at some point finished a ride and wondered "Why did I get that score? I thought it was much (better/worse).". The potential is there to provide a lot of that explanation. It could be that everybody at the show was getting good or bad scores. Or, perhaps the judge of their ride generally scores that test higher or lower than their peers.

Beyond the ranking system, the data is there to substantially improve the entire sport. Imagine if someone at USDF ran a 5-second report off their database and discovered that competitors are generally receiving lower scores for a particular movement at a given level. Programs could be targeted toward that movement to generally improve the rider population.

Anyway...food for thought. I'm not in any way taking away from any rider, judge or the USDF. Just some ideas on how we might make use of all the data for the betterment of the sport.

Have you considered that we as individuals go over our tests and note what movements need the most work? And attempt to improve them? And also note that some movements are x2 so someone apparently thinks they are of greater difficulty so we attempt to improve those areas?

Why would we need USDF to analyze tests for us when we have our own means of analysis as individuals?

[Pely]

Bentley is an awesome horse. He is also for sale. If he holds onto his HOY standings, i would bet that he would sell for a very nice price.

Bentley was shown sparingly this year. I do not know why. You only need 4 scores for FEI HOY awards. You can use 2 scores from the same judge, so in theory, you could show twice under two different judges and be able to be the HOY winner.

Mr. Ott is not a young rider. although he is still eligible for the Brentina cup.
Ages 25 and under ???? He is a very attractive rider. Beautiful equitation and able to produce the best this horse has to offer.

[Touchstone Farm]

More recently, I think it was perhaps George Williams, who is known to compete in CDI's and in Florida, and he was making a case that competitions like that should be weighted because there is a consensus that scoring is tougher at competitions like Florida and CDI's. However, nothing ever came of that discussion, that I am aware of. It would have been a slippery slope to address that issue, for a number of reasons.

You must have been at the USDF conference when this was discussed because it was Peter Lert who used the "slippery slope" phrase from their discussion in the judges meeting.

Some of the discussion was that a "6" should be a "6" no matter where it was awarded, so why should a CDI be weighted differently than another recognized show IF their is a judging standard.

I agree with previous posters that no system will ever be perfect or absolutely fair. Competitors will find a way to figure out any system. For some, they watch how the judge is scoring at a particular show and scratch if they think the judge is scoring too low (and hence, not affect their current standing). Others show no matter what. So I guess it is what it is -- hard to find a system that controls for everything.

[Equibrit]

Mr. Ott is not a young rider. although he is still eligible for the Brentina cup.
Ages 25 and under ???? He is a very attractive rider. Beautiful equitation and able to produce the best this horse has to offer.

Sorry - the correct term USEF use is "Young Adult" and they just raised the age limit to 28 for 2009.