Discussion: rankings

I would not have thought to go through these race by race, but you got me interested. I attended 21 events - all yours except for ROC plus the Boggs Mt and TSN A-meets. I had 20 actual results, because I missed a control at one of the Flying Pig sprints and got a 0. Otherwise, my results ranged from 52.51 to 91.81. Head to head, PG 15 - Charlie 1. Average % of PG where we both had a result was 78%. Average % on sprints was 87% and on other races 73%. For me, the distinction may be the simpler maps, but I think the critical factor is 1:5000. I pretty much can't see the 1:10000 maps without stopping.

I noticed that pattern with sprints (as have quite a few others) and raised it at the annual meeting a couple years ago. At the time, I was told that "something" was being discussed since skipping or excluding sprints is clearly in the best interest of your ranking. As nothing has happened yet, I just enter sprints as M45 and leave my Blue ranking for regular events. Then again, I'm not even ranked on Blue or Red this year, so it doesn't much matter for now. Maybe back to national competition next year.

I am certainly aware of the sprint/easy course ranking penalty and it is yet another reason I avoid ranked sprints. (Not that I care much about my ranking.) From the Attackpoint rankings you can see there is also a penalty for goat races where others can follow each other, and so the tail of the field doesn't do relatively as badly as it normally would, but this is not an issue for the USOF rankings where no goats or mass start races are ranked.

I have raised this kind of point in the discussions of how to convert WRE scores to USOF score-equivalents for team selection purposes. Nobody seems to worry too much.

The rankings committee - me, Valerie, JJ, a couple others I can't think of at the moment - considered this issue about two years ago and discussed whether there should be a separate sprint ranking. (We also discussed whether night-o events should be included.) We looked at rankings done with and without sprints, and if I recall correctly there was little, if any, shuffling amongst the top ranked folks and so we decided not to do anything at that time. But we can certainly revisit the issue. Perhaps there are enough ranked sprints these days for it to make more of a difference. But we probably won't do anything unless specifically asked to. Inertia and all that.

As for using the rankings for team selection purposes, if the teams would like rankings done by discipline or omitting certain events or whatever, we're happy to oblige. We already run a separate ranking taking out any night-o events at the request of the team.

I will admit to caring just enough about rankings to always tell Valerie before I run a sprint that I don't want it ranked. :)

That is an interesting subject, and one I thought a lot about, although in a different context.

For me, after the last US Champs in CO, I decided to never again race at altitude. My statistical analysis of a lifetime of my results indicated I performed significantly worse there, holding everything else constant. And, I simply did not enjoy it. Moreover, many times the terrain was just not that interesting. Having said that, if Wyoming was located at sea level, and everything else was the same, I am fairly sure I would be back all the time. One of my favorite places to orienteer is at Valley Forge, and I have had good results there, on terrain that, from a technical and running perspective, is quite similar.

So, am I gaming the system? Well, I guess so, but I don't care. I actually don't care that much about my own ranking. If I did, I think I could do a better job by not racing or dropping down when injured or sick, but why bother? I have never expected to make a WOC team, and every time I have, it was a thrill and surprise.

The first two times (I think) that I made the WOC team, I did it in spite of my ranking (in other words, my TT days contributed to my score, but the ranking didn't.) This year, however, I saw that a goodish ranking could be helpful, as I came into the TT well less than 100% physically or mentally. The ranking did make a difference.

Having said all that, I will confess to now gaming the DVOA rankings, something I care about, quixotically, more than the US Team. DVOA uses the same ranking system, and therefore has similar issues. I can't exaggerate how frustrated I have been to have a near perfect race (such as this one: http://www.dvoa.org/events/results/ev_show.php?eve... ) and gotten ~27% less points than I would have expected. I don't think anyone could have run that course in ~4 minutes per k, but they would have needed to to score 100 points. And on the DVOA scale, someone like Hubmann, who might be able to get that kind of time, should score like 130 points on average in DVOA-world, so obviously there is a distortion in places.

But, conceptually, examples like this are quite illustrative. Orienteering is an activity where excellence is predicated on running and navigating. The result function, if you will, is some sort of combination (linear?) of running and navigating. At an event like I cited, and some sprints, you are effectively eliminating (or setting to 0) the navigational component, and solving for the running one.

Anyway, as an upshot of all this, I've decide to focus on the gnarliest DVOA races. At local club races, in harder terrain, the spread seems even greater than nationally, and the payoff to having a good race at a tough venue is high. (Although conversely, that may explain why having a good race at an easy venue generates a poor score.)

The progression of ranked sprint races:
2005 - 2
2006 - 5
2007 - 7
2008 - 9 (including the one in Nov 08)

Sorry to go off on this tangent, but if we were to specify a generalized and parsimonious linear results function, I guess we need an interaction effect between the two independent variables (the running and navigating.)

And I know no one really cares about this, so I apologize again.

One other thing--the ingsight I casually take away from the DVOA local event result I cited may be that I have a greater (comparative) navigating advantage vis-a-vis the DVOA population than I do running. I think versus the national population, my spread to the very top is more balanced. I wonder if this is the correct conclusion?

One other thing--the ingsight I casually take away from the DVOA local event result I cited may be that I have a greater (comparative) navigating advantage vis-a-vis the DVOA population than I do running. I think versus the national population, my spread to the very top is more balanced. I wonder if this is the correct conclusion?

I think it is correct.

In my own case, the numbers show I clearly have an advantage when the navigating is as difficult as possible and (since we're talking older folks with vision issues on the Green course) when the weather conditions are as bad as possible. My best points in the last 12 months were at Quantico day 1 (thick, vague), West Point middle (rain and fog, complicated topography), and NA Champs middle and long (rain, gloomy, complicated topography).

I believe that if the Triple Crown that I was competing in was the Derby, Pimlico, and the Belmont (rather than the Billygoat, the Highlander, and the Traverse), than I would surely be called an excellent mudder.

Interesting discussion - the possibility of solving for both a navigational-difficulty, and running-difficulty, of a course, and correspondingly, the navigational-score, and running-score of the orienteers attempting it, has always been a bit out of reach.

In part, there's a problem as the basic input to the rankings are groups of names and times, and I'm not sure how you'd begin to tease out 'running' vs. 'navigation', as 2 independent dimensions, from a more 1-dimensional dataset. Maybe you could start with some random start conditions, and solve for the presumed two dimensions, iterating in such a way as to minimize the errors in the predicted times, and then perhaps a human could identify which component appeared to be the nav. score, and which appeared to be the running score. Not sure.

If the math really worked, at the end of all that, you'd have two separate scores for each person - a running score and a nav. score. How would we rank people? By running, or by nav., or by some recombination of the two, based on a course with a median nav. difficulty to running difficulty ratio??

On a more concrete note, both Angelica & I were quite concerned going into this week's DVOA Tyler event for the same reason as Clem cited above - it's a known 'runners course' park. We both had reasonably good runs, but I suspect we'll both get -'s - we're just hoping it's not too bad.
On pre-excusing myself, I've never done it, but I've certainly considered it. It feels a bit too much like cheating the system though. If you don't want to show up somewhere, fine, or if you otherwise wouldn't run the race at all, fine. But if you do show up, and are planning to compete as hard as you can, I figure it might as well count, good or bad.

The WRE scoring system tries to account for the spreading of results by calculating a standard deviation of the results on the course. I forget if it is raw times or PGVs that are used in the standard deviation.

It's hard to deal with universally though, I think .

Example 1: If you've got 10 people of similar running speed, and different nav. speeds on a course, the times will spread more on a hard nav. course, so things may work out okay in some kind of standard-deviation-based thing.

Example 2: But if you've got 10 people of different running speeds (say from fast to slow) and different nav. speeds (say from slow to fast), there's probably an average nav. difficulty course where you'd expect all times to be similar. At easy nav. courses, the faster running people would spread to the front, and at hard nav. courses, the faster nav. people would spread to the front. So in this case, the standard deviation gets quite small at 'average' navigation, and actually gets bigger with especially easy or especially hard navigation, which differs significantly from the first example.

I think the USOF population, esp. in M21 (just because I pay attention there - other categories may be similar) includes a significant mix of the latter - e.g. we've often had a mix of people who are fast runners, and slow navigators, and others who are slower runners and faster navigators.

A good framework question to ask is: "What is the fair way to rank a mixed population like this?" E.g. if I go to Wyoming, and Mark Everett beats me 5 times, by 10% on average, and then Mark comes to the east coast and I beat him 5 times, by 10% on average, who should have a higher ranking? Or if Erik Nystrom beats me by 10% at Sprints, and I beat him by 10% at Long's (wishful thinking here :), who should be ranked higher? That's probably a pretty subjective question, as to what kind of events you think are 'real' orienteering, and which aren't. E.g. are "Sprints" less of real orienteering than Middle or Long? Or are "Middle"s too nav.-heavy, compared to the Classic/Long, and the Classic should thus be the true stnadard judge of orienteering rankings? (Going back to the above - even if we could tease out 'nav. score' and 'running score' from the data, the subject of who gets the best overall score (what's the proper running/nav. weighting?) is subjective too.)

I think one reasonable way to deal with the multiple ways of sub-selecting events for the USOF rankings is what we do right now - let the Sanctioning committee decide what is a USOF event, and then rank them all in one pot. The Team Trials rankings, which remove Night O's, but retain Sprints, perhaps align the pot of events used for those rankings more closely with the WOC mix of events, since there is a Sprint, but no Night O'. But it's clearly still a mix.