The Chronicle of the Horse
MagazineNewsHorse SportsHorse CareCOTH StoreVoicesThe Chronicle UntackedDirectoriesMarketplaceDates & Results
 
Page 1 of 2 12 LastLast
Results 1 to 20 of 36
  1. #1
    Join Date
    Feb. 3, 2000
    Location
    Nokesville, VA
    Posts
    35,538

    Default Nerd Herd

    I just skimmed it on line, but the article in today's Chronicle, about statistical analysis of dreessage scores, looks interesting.
    Janet

    chief feeder and mucker for Music, Spy, Belle and Tiara. Someone else is now feeding and mucking for Chief and Brain (both foxhunting now).



  2. #2
    Join Date
    Oct. 2, 2007
    Location
    Beyond the pale.
    Posts
    2,957

    Default

    TYVM

    Pity they did not also credit our riding/dressage credentials.

    The nerdherd has extensive experience in riding and showing as well as statistics and academia.
    "The Threat of Internet Ignorance: ... we are witnessing the rise of an age of equestrian disinformation, one where a trusting public can graze on nonsense packaged to look like fact."-LRG-AF



  3. #3
    Join Date
    Feb. 17, 2000
    Location
    PA
    Posts
    3,526

    Default

    Those people did some terrific number crunching...free of charge.



  4. #4
    Join Date
    Jul. 4, 2000
    Location
    Maryland
    Posts
    1,913

    Default

    They did get one key sentence wrong:

    There is no difference between the scores of 71.12% and 71.13%.

    *star*
    "Avoid loud and aggressive persons, they are vexations to the spirit."
    - Desiderata, (c) Max Ehrman, 1926



  5. #5
    Join Date
    Apr. 23, 1999
    Location
    Rosehill, TX
    Posts
    7,092

    Default

    Quote Originally Posted by ShotenStar View Post
    They did get one key sentence wrong:

    There is no difference between the scores of 71.12% and 71.13%.

    *star*
    um, yeah, seeing that in print (print version says "There’s no difference between the scores of 72.12 percent and 71.13 percent") did make the eyes cross
    Nothing says "I love you" like a tractor. (Clydejumper)

    The reports states, “Elizabeth reported that she accidently put down this pony, ........, at the show.”



  6. #6
    Join Date
    Feb. 3, 2000
    Location
    Nokesville, VA
    Posts
    35,538

    Default

    But is probably also true that the difference between 72% and 71% is not really meaningful, regardless of the digits after thd ecimal point.
    Janet

    chief feeder and mucker for Music, Spy, Belle and Tiara. Someone else is now feeding and mucking for Chief and Brain (both foxhunting now).



  7. #7
    Join Date
    Apr. 9, 2008
    Posts
    1,696

    Default

    Okay....'nother Nerd checking in.....so where is this article? The Pony Express gets my print copy in sometime during the week of the the edition....on a good day. I'm looking at the electronic version and can't find it.

    I have actually found academics studying this stuff for real. Been conversing with Tim Whitaker, Professor of Applied Sciences at Writtle College in UK.

    Deuel NR and Russek-Cohen E (1995). Scoring analysis of three world championship three-day events.
    Journal of Equine Veterinary Science 15: 11.

    Whitaker TC, Hill J and Shearman JP (2004). Analysis of pre-novice eventing horses completing six selected events.
    Equine & Comparative Exercise Physiology 1:185–192.

    Whitaker TC and Hill J (2004). An investigation into patterns associated with dressage scores at selected novice events.55th Annual Meeting EAAP, Bled, Slovenia, September 2004.



  8. #8
    Join Date
    Apr. 23, 1999
    Location
    Rosehill, TX
    Posts
    7,092

    Default

    below "In the Country"
    Dressage Issue June 5 - "It's Time to Modernize and Improve Judging and Scoring"
    Nothing says "I love you" like a tractor. (Clydejumper)

    The reports states, “Elizabeth reported that she accidently put down this pony, ........, at the show.”



  9. #9
    Join Date
    Apr. 9, 2008
    Posts
    1,696

    Default

    OK....found it.....

    I found Whitaker's work while googling to see what came up under "statistical analysis" and "dressage scores". Been having brief email discussion with him. I forwarded him a link to the Nerd Opus hosted on the PVDA web site. He was very complementary. Will forward his 2 papers to the Herd.....I just got them a couple of days ago.



  10. #10
    Join Date
    Dec. 4, 2002
    Location
    Dungeon of the Ivory Tower
    Posts
    20,393

    Default

    I don't have access to the article at this point...but glad it's out and hope it generates some additional and fruitful dialogue!
    www.specialhorses.org
    a 501(c)3 organization helping 501(c)3 equine rescues




  11. #11
    Join Date
    Dec. 4, 2002
    Location
    Dungeon of the Ivory Tower
    Posts
    20,393

    Default

    Quote Originally Posted by CatOnLap View Post
    TYVM

    Pity they did not also credit our riding/dressage credentials.

    The nerdherd has extensive experience in riding and showing as well as statistics and academia.
    Well, MOST of them do..hopefully being topheavy in the academia part makes up for my lightweight dressage credentials!
    www.specialhorses.org
    a 501(c)3 organization helping 501(c)3 equine rescues




  12. #12
    Join Date
    Feb. 3, 2000
    Location
    Nokesville, VA
    Posts
    35,538

    Default

    Quote Originally Posted by DressageGeek "Ribbon Ho" View Post
    I don't have access to the article at this point...but glad it's out and hope it generates some additional and fruitful dialogue!
    You should be able to get to it here
    http://www.chronofhorse.com/index.ph...10406093746904
    It did NOT ask for my password.
    Janet

    chief feeder and mucker for Music, Spy, Belle and Tiara. Someone else is now feeding and mucking for Chief and Brain (both foxhunting now).



  13. #13
    Join Date
    Jul. 27, 2007
    Location
    Behind the Orange Curtain
    Posts
    9,694

    Default

    Just some comments, although they are minor and probably have a lot to do with differing backgrounds.

    1) In my thinking, instrument is not the test. The instrument is the test and the specification for scoring the test (the actual instrument would be the test instructions, the judging criteria, and the scoring sheet together). Judging isn't entirely subjective, but is based on the assessment in total.

    2) I've always learned that assessments of this sort are NOT analogous to length/weight measures because a true score can never be found, only estimated. The inability to ever determine the "true score" is what makes validity and reliability analysis so crucial in these cases.

    3) Great point about sig figs.

    Once you do the reliability analysis and get the standard error of measurement, it becomes really obvious how much all of this stuff really matters... especially when someone can win or lose based on such small numbers.



  14. #14
    Join Date
    Apr. 9, 2008
    Posts
    1,696

    Default

    Quote Originally Posted by Ambrey View Post
    Just some comments, although they are minor and probably have a lot to do with differing backgrounds.

    1) In my thinking, instrument is not the test. The instrument is the test and the specification for scoring the test (the actual instrument would be the test instructions, the judging criteria, and the scoring sheet together). Judging isn't entirely subjective, but is based on the assessment in total.

    2) I've always learned that assessments of this sort are NOT analogous to length/weight measures because a true score can never be found, only estimated. The inability to ever determine the "true score" is what makes validity and reliability analysis so crucial in these cases.

    3) Great point about sig figs.

    Once you do the reliability analysis and get the standard error of measurement, it becomes really obvious how much all of this stuff really matters... especially when someone can win or lose based on such small numbers.
    Since I take some blame for making the analogy of the test as "the instrument"....want to say I agree wholeheartedly with Ambrey. I did not go into such detail, but the points brought out in Ambrey's post are key to use of "the instrument."

    The test is an instrument in the sense that it takes what is codified in the rules into a series of patterns that are then judged. How the judges are trained to interpret "the test" (eg., judging transitions into and out of movement) makes up the essence of "the instrument"....but I can be persuaded otherwise....sort of like "standard inspection procedures".

    The concept of accuracy....eg., the standard against which to judge horse and rider is something that really hasn't been discussed in the debate on dressage scoring. Part of the intent was to highlight the terms "precision" and "accuracy"....and to introduce theses concepts into the dialog. No point in worrying about judges all scoring the same if there is no reference back to the goal/target/ideal, eg., "the standard"....eg., the rules.

    So....as they say, "it is a journey"....so hopefully the article will be usefull to advance the discussion.



  15. #15
    Join Date
    Jul. 27, 2007
    Location
    Behind the Orange Curtain
    Posts
    9,694

    Default

    I understand. I just felt, personally, that understanding how such assessments did not really compare to the types of measurements people are used to, in which there is a "right" and "wrong" answer, was instrumental of my understanding of assessments and what they could and could not tell us about what we're measuring.

    But, indeed, how those judging criteria are written can/does have a huge impact on the reliability of the instrument. Basically, even two judges who are attempting to adhere to standards can come up with wildly different scores if the standards are written in such a way that they can be interpreted in very different ways, and those interpretations are not clarified in the training programs.



  16. #16
    Join Date
    Sep. 27, 2000
    Location
    Southern California - on a freeway someplace
    Posts
    10,125

    Default

    I have been waving the sig fig flag for years. In championships it's not unusual to see a "spread" of less than half a percentage point between a number of places and have several places statistically tied if you pay attention to sig figs. In a test with the total points earned more than 100., but less than 1000., you are allowed four sig figs (the ones place is known and the tenths place is uncertain). Thus reporting a score as 72.114% is a bit absurd.

    Never thought I'd see the accuracy/precision bulls-eye in a COTH article.

    There was something about the sig fig definition in the article that made it sound to me like the sig figs were the insig figs, but I only skimmed it so my recollection could be wrong. And I'm not a statistician.
    The Evil Chem Prof



  17. #17
    Join Date
    Sep. 27, 2000
    Location
    Southern California - on a freeway someplace
    Posts
    10,125

    Default

    In academia, a test or exam, is considered an instrument. Or in edu-speak a "testing instrument."
    The Evil Chem Prof



  18. #18
    Join Date
    Jul. 27, 2007
    Location
    Behind the Orange Curtain
    Posts
    9,694

    Default

    Quote Originally Posted by Peggy View Post
    In academia, a test or exam, is considered an instrument. Or in edu-speak a "testing instrument."
    I can understand that. In psycho educational testing, the instrument usually includes both the test itself and the test scoring mechanism (since each test has specific scoring instructions to maintain reliability/validity). So I was going from that angle.

    eta: And in psychometrics, reliability is tested on the whole package.
    Last edited by Ambrey; Jun. 6, 2009 at 12:52 PM.



  19. #19
    Join Date
    Oct. 16, 2002
    Posts
    1,291

    Thumbs up

    Thanks to the Nerd Herd for an excellent rebuttal piece. As always, I learn so much from your research efforts!



  20. #20
    Join Date
    Apr. 9, 2008
    Posts
    1,696

    Default

    Quote Originally Posted by Peggy View Post
    I have been waving the sig fig flag for years. In championships it's not unusual to see a "spread" of less than half a percentage point between a number of places and have several places statistically tied if you pay attention to sig figs. In a test with the total points earned more than 100., but less than 1000., you are allowed four sig figs (the ones place is known and the tenths place is uncertain). Thus reporting a score as 72.114% is a bit absurd.

    Never thought I'd see the accuracy/precision bulls-eye in a COTH article.
    That's sort of the point of our work....that if dressage scoring is going to improve, it might as well follow well-accepted standards for subjective judging...be it industrial, academic or psychometric....all disciplines that apply statistics to "non-quantitative" (eg., judgement-based) measures have some common fundamentals that can be applied to the subjective evaluation of a horse and rider. The equestrian/dressage community need not reinvent the wheel.

    I do believe the journey has just started. This first step is required "level setting" to get everyone on the same plane as dialog grows to include some of this fundamental terminology.

    And yes, I was also suprised to see the bullseye.....heh, hehe, heh....perhaps we are making a difference.

    Thanks to the COTH......



Similar Threads

  1. Nerd Herd needed on the Eventing forum
    By Janet in forum Dressage
    Replies: 3
    Last Post: Oct. 6, 2011, 09:44 PM
  2. I am a huge Nerd Alert!
    By twotrudoc in forum Off Course
    Replies: 0
    Last Post: Nov. 3, 2010, 08:56 PM
  3. News from the Nerd Herd - we're in press!
    By DressageGeek "Ribbon Ho" in forum Dressage
    Replies: 20
    Last Post: May. 29, 2010, 10:05 PM
  4. for Rebecca and the Nerd Herd
    By sm in forum Dressage
    Replies: 3
    Last Post: May. 23, 2010, 08:21 PM
  5. Am I the only nerd here? Reference thread above
    By CookiePony in forum Eventing
    Replies: 9
    Last Post: Jun. 23, 2009, 08:35 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •