The Chronicle of the Horse
MagazineNewsHorse SportsHorse CareCOTH StoreVoicesThe Chronicle UntackedMarketplaceDates & Results
 
Results 1 to 17 of 17
  1. #1
    Join Date
    Feb. 13, 2012
    Posts
    75

    Default Objective (statistical) Event Reviews complete for all areas/levels!

    I've posted before about a project I was working on using results data from events to provide an objective measure difficultly. All areas and levels are now complete! http://outsidethehorsebox.com/event-reviews/ Just use the map or the toggle boxes to find the area/level you are interested in looking at.

    The basic idea is that an objective way of assessing the difficulty of each event is to look at the % of competitors with jumping faults cross country and SJ and if an event has a lot more than the average for other horse trials at that level it's likely to be harder and a lot less, easier - thereby eliminating the subjective nature of reviews.

    The only thing I would say is that there are some upper level events where there really weren't enough competitors to give a reliable data. For instance there are a number of intermediates with <10 starters. But this data is on the charts so everyone can see for themselves.

    It's still very much a work in progress (for instance the more detailed data on dressage and time pens on each event is not up yet). I look forward to feedback and there are lots more aspects to investigate, weather, % of professional riders at an event etc etc.

    Hope it's helpful!


    2 members found this post helpful.

  2. #2
    Join Date
    Dec. 27, 1999
    Location
    Midland, NC, USA
    Posts
    7,201

    Default

    Thoughts, anyone? I was looking at the Intermediate stats and thinking 'hmmm, I've never been to Waredaca', then saw that The Fork also looked 'easy' going by the stats!

    Jennifer



  3. #3
    Join Date
    Jun. 9, 2005
    Location
    Unionville, PA
    Posts
    3,276

    Default

    Why are the BNs close to me so difficult? I can confirm as a fence judge that the BN at June Plantation Field last year was a trainwreck!

    Great resource. Thanks!
    Delaware Park Canter Volunteer
    http://www.canterusa.org/



  4. #4
    Join Date
    Apr. 17, 2003
    Location
    Area I
    Posts
    198

    Default

    I think this could work if you had the same horses / riders / weather conditions with no changing skill sets etc for each of the events per area but even that would be hard as first time out in the spring is different than mid-season. I get to watch a ton of XC as my full time job is as the official photographer at 10 USEA events in Area 1. I can guarantee you the skill sets on different horse / rider combos vary hugely from weekend to weekend - event to event. I am quite familiar w most of the courses in Area 1 and feel that the stats as they are shown do not fairly represent the various courses.



  5. #5
    Join Date
    Apr. 30, 2002
    Location
    Looking up
    Posts
    6,038

    Default

    I was just going to say....this is too basic. Weather and footing play a huge part especially in my area from spring to summer to fall. Having only one rail down in a monsoon is a great feat compared to summer when the footing is fine. Each event really is individual --- this is difficult for me based only on the parameters given.
    "Passion, though a bad regulator, is a powerful spring." -- Emerson
    www.eventhorse.wordpress.com



  6. #6
    Join Date
    Oct. 22, 2009
    Posts
    2,756

    Default

    It's a great idea and I'm impressed how much work you put in! But, I don't think you can rate an events difficulty like this. Just the fact that Stuart horse trials and is 'Average', while GVH and GVRDC are 'moderately difficult' shows it's not accurate. Stuarts is a HARD course, while GVH and GVRDC are average.
    Quote Originally Posted by pinecone View Post
    I can't decide if I should saddle up the drama llama, dust off the clue bat, or get out my soapbox.



  7. #7
    Join Date
    Nov. 7, 2006
    Location
    Knoxville TN
    Posts
    1,306

    Default

    OMG, this is absolutely Brilliant. I LOVE it. It's a most excellent start and I think as a community we can all help to build this into something Really great. Of course, it's an ongoing process, and it will evolve over the years but I absolutely love the platform you've set up, the graphics, the navigation, the data in the background. This is a great resource, thank you so much for putting in all the work for this. Well Done ! (I love it, by the way, can you tell ?)


    1 members found this post helpful.

  8. #8
    Join Date
    Jan. 14, 2006
    Location
    Nashville, TN
    Posts
    3,939



  9. #9
    Join Date
    Dec. 6, 2000
    Location
    SE Mass
    Posts
    4,065

    Default

    I think that this is very nicely done. Unfortunately, it suffers from something known as confounding by indication. Because horse-rider pairs are not randomly assigned to events, but choose which events to attend, that choice can confound the number of horses with penalties. For example, no well-informed green horse or rider would go to Stuart in Area 1, so you have more seasoned pairs there who are less likely to have problems XC. Unless you adjust for the number of prior successful XC completions by horse-rider pairs, the lovely graphs cannot be used to assess event difficulty. Sorry.


    1 members found this post helpful.

  10. #10
    Join Date
    Jul. 4, 2000
    Location
    Maryland
    Posts
    1,773

    Default

    Welcome to the Wonderful World of 'Statistical Analysis of Horse Sports'!

    As a member of the Nerd Herd that took on a similar effort with dressage scores, I say good on you for starting into this. The data can tell us all sorts of things about our chosen sports ... aspects that we may not otherwise consider or be aware of.

    We heard some of the same counter arguments as those above (the footing, the weather, the judges, access to proper instruction, etc) with the dressage studies. Yes, these can all impact the ranking system. But the point of studies like this is "To Ask The Questions". The answers will lead wherever they lead, which in many cases is to more questions.

    So Go For It! I look forward to studying all this in more detail ... but right now I'm going out to ride.

    *star* - Data Wrangler for the Dressage Nerd Herd
    "Avoid loud and aggressive persons, they are vexations to the spirit."
    - Desiderata, (c) Max Ehrman, 1926
    RIP Carleigh 1999-2011


    3 members found this post helpful.

  11. #11
    Join Date
    Apr. 14, 2001
    Location
    Fort Collins, CO
    Posts
    15,855

    Default

    I think this is an AWESOME way to look at the data. Sure, there are a LOT more variables, but what an excellent start!

    I imagine that if you could add the data for the last ...oh...10 years or so, you'd get a pretty darn accurate picture (barring course redesigns and such.)

    Thank you for putting this together! I really enjoyed reviewing it.



  12. #12
    Join Date
    Aug. 14, 2000
    Location
    Clarksdale, MS--the golden buckle on the cotton belt
    Posts
    17,547

    Default

    As previously pointed out, the competitors at each event are self selected. Where they have a plethora of events to choose from, they will probably make decisions based on their own competence. Where there are very few events in an area, if you want to compete, you go to the ones that are offered, regardless of your own competence (particularly at the lower levels) and the quality and difficulty of the courses.

    I guess what I'm saying is that where there are only four or five recognized events within a 400 miles radius, you're likely to go there even if you are a beginner rider, new to the level. And that will affect the results. Where you have oodles of choice, you will be likely to select a venue that is "appropriate" for your skills. And where you have oodles of opportunity, that will affect your skills.

    Quote Originally Posted by Simkie View Post
    I think this is an AWESOME way to look at the data. Sure, there are a LOT more variables, but what an excellent start!

    I imagine that if you could add the data for the last ...oh...10 years or so, you'd get a pretty darn accurate picture (barring course redesigns and such.)

    Thank you for putting this together! I really enjoyed reviewing it.
    "I'm a lumberjack, and I'm okay."
    Thread killer Extraordinaire



  13. #13
    Join Date
    Apr. 14, 2001
    Location
    Fort Collins, CO
    Posts
    15,855

    Default

    I live in an area where there just aren't that many events...if you want to be busy during the season, you go with what's available So very little "self selection", unless you're looking for something very specific for a horse's first time out or first time moving up in the levels.

    Take a look at our beginner novice offerings, for example:

    http://outsidethehorsebox.com/beginn...-area-ix-2012/

    Soooooooo different than area II:

    http://outsidethehorsebox.com/beginn...-area-ii-2012/



  14. #14
    Join Date
    Feb. 13, 2012
    Posts
    75

    Default

    Hi Everyone,

    Thanks very much for all of the feedback. As has been pointed out there are certainly loads of variables that have affected these results. I'm sure we could come up with even more factors that play into things. And hopefully down the track it would be possible to add more of these in. Weather and footing conditions could be straightforward to keep track of and It may even be possible to indicate the experience of riders/horses, for instance, how many were competing at that level for the first time.

    What would be the next most important factors people would want to see added in? Weather, footing, prior experience of horse/rider?

    The idea behind this is that it is just one piece of information for people to use in conjunction discussing events with friends, trainers, etc. If one event has 50% of starters with problems xc and another only has 10% the first is probably a tougher course than the second. Sure, competitors may self select but the more entrants the more likely that the pool of entrants will have an average representation. But the main thing is that the data is there for people to use at they like. And hopefully we can add information as time goes on.

    If anyone has any thoughts on where they think the numbers aren't what they saw in action then please send me your comments! It would be great to add that in for people to see as well!



  15. #15
    Join Date
    Apr. 14, 2001
    Location
    Fort Collins, CO
    Posts
    15,855

    Default

    I would be curious how difficultly rating changes over the course of the season (which might be out there already--I didn't spend a ton of time digging into it.)

    So for example, is an early event showing as more difficult because it's one of the first in the season and people are getting back into the swing of things, or is it truly more difficult than an event later in the season, that might be noted as easier, but it's truly the same or even harder, but riders and their horses simply just have more experience at that point?

    Something as simple as a listing of all of the events on the chart sorted by date and with their difficulty rating would fill that need for me, and be interesting to look at



  16. #16
    Join Date
    Jun. 6, 2007
    Posts
    251

    Default

    Quote Originally Posted by Simkie View Post
    I would be curious how difficultly rating changes over the course of the season (which might be out there already--I didn't spend a ton of time digging into it.)

    So for example, is an early event showing as more difficult because it's one of the first in the season and people are getting back into the swing of things, or is it truly more difficult than an event later in the season, that might be noted as easier, but it's truly the same or even harder, but riders and their horses simply just have more experience at that point?

    Or maybe late season events have more problems because people are trying to move up?

    I truly appreciate the effort being put into this. Great work.

    I think perhaps the over-simplified categories -"difficult -easy -etc" stray from the mission of objectivity, because these statistics do not tell the whole story. As a friend pointed out, Windridge HT May and October vary by 24%, and it was the precisely the same course, virtually the same conditions. (and it was the late season event that had twice as many problems).



  17. #17
    Join Date
    Nov. 7, 2006
    Location
    Knoxville TN
    Posts
    1,306

    Default

    I would guess that the next stage of development might be a comments section, which coule be a lot of work with moderation, and would definitely not help the 'objective' appeal of the current presentation, but it would be a useful little information-gathering tool for the researcher. I'm picturing a scenario where one person comments that "Oh, I rode this event and it was carnage out there in Novice because of jump 5 into the water, because of all the rain the night before" ... and when the researcher reads enough of the comments, over enough events, they get a feel for the idea that "weather forecast two days prior to the event" would be a suitable next bit of data to mine. i.e. I see "Comments" or "subjective feedback" as an important part of the iterative process of building an objective picture here.

    After that, I would say that ALL extra information is useful, and you should probably prioritise by picking "whatever's easiest". I would love to see a summary of competitors' prior experience for example. Is this something that you can develop an algorithm for ? Is the data easily accessible ? I would guess that it is since it's on the USEA site, but can you really do that analysis in real time ? If so, then I am bowled over by your talents and knowledge and you should go get a job in real time derivative pricing and earn a bunch of money and retire in 2017 with a nice farm and a little breeding operation.



Similar Threads

  1. Event Reviews Revisited?
    By EWim15 in forum Eventing
    Replies: 15
    Last Post: Feb. 13, 2013, 08:25 PM
  2. Event pet peeves--warm-up areas!
    By LookmaNohands in forum Eventing
    Replies: 21
    Last Post: May. 21, 2012, 07:28 PM
  3. How to be objective about PPE results...
    By CHT in forum Dressage
    Replies: 19
    Last Post: Jan. 16, 2012, 07:53 PM
  4. Dressage Statistical Analysis in JQAS
    By pluvinel in forum Dressage
    Replies: 19
    Last Post: Jul. 24, 2010, 07:36 AM
  5. Replies: 11
    Last Post: Jun. 11, 2010, 08:08 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
randomness