Discussion: IOF Ranking question

What do you think of the formula of the OF ranking system? For me it looks weird there is such a huge penalty for not having full 4 races. Wouldn't it be better if you could get decent points event with one race? This way points would spread more evenly and faster around the globe without having to travel so much. And it would also motivate to travel even sometimes becouse running that one race it would mean a lot more than today.

Out of curiosity I made yesterday a little script calculating poinst a bit differently. There you see men's current IOF rankings with smaller penalty for not having full 4 races. A runner gets 88% if only one race, 92% of average of two is two races, 96% of average of three if three races, 100% if four or more. And poinst might get calculated using less races than runner has if points gets better that way (one has some really bad races compared to better ones).

Top positions didn't change, because all had full 4 races. But later on there is lots of changes. Take a look at the list. Would something like this make rankings more fair and more usable for counting who gets and how many places for race. Looks like there is runners from smaller O countries and some familar names in much higher places that in the original list. Here we go, men, all:

http://routegadget.awardspace.com/misc/IOFmen.htm

I like it. What it shows is that there are a lot of runners with less than 4 races, some with pretty good results (Jani Lakanen for 1). You are correct that many (most) of these are runners from distant/smaller countries.

The counter argument will be that the ranking system is inherently variable - people's points in a single race will vary a lot depending on some of the parameters; so by using a minimum of 4 races it is supposed to average out.

The advantage is certainly with the runners who do more races, though as once they do more than 4 only the best ones count. Therefore I don't think the counter argument holds much weight - you have demonstrated minimal change in the top positions with your system.

However, could you find a way to drop JJ Cote (1691) a few more places and me (1699) up a couple more?

I'm not sure of the point of changing it because for the people where the list matters, it doesn't make a difference.

Your idea seems to just serve to make the less-travelled (+ less good?) orienteer feel better!

Not that I'd be complaining, I go up 107 places ;)

Well, let's suppose I live in a small town called Forssa and I never travel more than 50km from home. Nevertheless, there being a lot of maps nearby with the best terrain in the world, I get very, very good at orienteering in local terrain. I'm really good at it, despite never training anywhere else.

As it happens, there is a WRE every year in Forssa. Occasionally orienteers from the south of France come and run this race. They do not run well. They say something about not being used to running in forest and marshes. I, on the other hand, always come top-ten in this race and get lots of WR-points, worth about 1200pts in the new Jagge-ranking-system.

Then one year I get invited to the south of France to run 6 days of Aveyron. I'm not very excited about the orienteering, because it is not Finnish, but I travel anyway because there will be sunshine without mosquitos and horseflies.

When I pick up the map at the start, I see lots of yellow so I think this must be easy. To my surprise, the orienteering gets quite technical as I have to weave my way amongst the patches of green and the rocks. I have to slow down a lot and still make a few mistakes. I finish the race a long way behind the French orienteers I always beat in Forssa. I only get 500 pts. My Jagge-ranking goes from 1200 down to under 900 by running a race.

Ah ha ha ha ha!
Yeah, dropping me further would probably be appropriate. As I've noted before, there are probably at least 1130 people in Stockholm alone who are better than I am.
The whole IOF ranking formula is so bizarre and arcane that I have trouble seeing the value of a small improvement like this.

My Jagge-ranking goes from 1200 down to under 900 by running a race.

No. It would stay at 1200. Running a bad race would not make overall points go down as I described above.

In 2008 Forssa Games 8. place gave 1235 points. Would that been If the only WRE race for the runner, he would have got 0.88 x 1235 =1086.8 points had he would be now ranked 206. overall. I can't see much wrong with that.

Anyway, the frame you described is so out of this world, there is no such home field experts....

As said, it would change nothing in top placings, but further behind there wuold be some changes and those would change national rankings planned to use for deciding how many runners each nation get to some races. I checked national rankings, this would mean at the moment RUS, UKR, POL, EST, LTU and BUL would go up, nations with strong runners but who does not travel much. And some other countries used to travel or located "better" would go down.

Previous thread anbout national rankngs:
http://www.attackpoint.org/discussionthread.jsp/me...

BorisGr lists some nations in his first coment, of those Russia would go 7 places up, Lithuania 3 places up, Germany 6 places down. Hungary, Austria wuold stay about where they are.

"Running a bad race would not make overall points go down as I described above. "
Oops, silly me, sorry.
But perhaps in the outlandish scenario described they should go down.

My point, badly made, is that having to run 4 races to get a good ranking ensures that you run in at least a couple of different terrain types (except perhaps for WOC/EOC competitors). Different orienteers are stronger in different terrain.

Other arguments against, just for the sake of it:
-Getting pulled around one race by a ThierryG could give you whole heaps of points not corresponding to your ability.
-It means you only have to rig one race to distort rankings, not 4.

Just playing devil's avocado...

btw, Jagge, any particular reason you chose 4% increments? not that switching to 2% or 6% would help Eoin's cause...

I like avocado. About %. No reason what so ever. I just picked a some wild % values out of nowhere just to see what would happen. Some other values might do better, for example 6% would make it more difficult to get superb ranking by following someone just once. And still it would help those nations/runners who don't have enough time and money to travel so much.

Expanding WREs would be cool. But there might be some drawbacks, like having to rank lots of recreational runners. Even I might get ranked, something like 25346th in the world :)

I'm not sure of the point of changing it because for the people where the list matters, it doesn't make a difference.

It wouldn't matter if we don't use list for anything and it is not puplished at all. But it matters if used for something. Like for ranking nations to give start slots for world cup. Or starting order. Or if there is 300 entries for a WRE race and we take only 60 best based on ranking (for example should we really pick a runner with four 700 pont races or a runner win two 1200 pint races?).

Also current ranking it terribly bad publicity for orienteering. Let's think we don't know much about orienteering but i have hears it's quite populatr in Swden and there is lots of runners there. So i go to www.orientering.org and I find international ranking. Ok, looks great, but I like to see how strong the field in Sweden actually is . Then I pick Swedish runnres and I get this list:

http://www.6prog.org/IOF/wre_aths$.asp?mw=M&FED=SW...

At first is looks like there is about 15 men in Sweden who take the sport seriously. Then I see there is some runners with only one race but they have got over 1200 points. They must be beginneers who havent your raced a lot yet or they are doing some other sports but sometimes run o races too. And I am sure all or at least most runners are listed, because there is lots of runners listed who hasn't raced for years and there is also lots of runners with poor points.

So, it looks like there is about 150 male orienteers in Sweden and not more than ten of them are taking it a bit more seriously, and 50th best orienteer in Sweden must be quite recreational runners who like to walk in the woods and be at one with the nature. And they sey that's the strongest nation of all.

Maybe a fun hobby, but hopefylly they don't take it to olympics.

haha

Although take tennis. Popular sport in the US I guess. Let's do the same:
http://www.atptennis.com/3/en/rankings/entrysystem...

Two good players with around 2000 points, ranked in the top 10.
Ten players in the top 100, of whom the tenth has only 457 points. Is that good?
And the thirtieth American has under a hundred ranking points. Must be an orienteer. Drop down to fiftieth and below and they have not more than 20 points... conclusion?

I agree there are problems when using the data to ascribe entries. Esp number of WC competitors. However WR is supposed to rank athletes competing at international level I guess, not rank all athletes in the world. Whom is a WRE supposed to cater for?

chili flavoured avocado reserves a jagged reply:
- points go down very steeply. It isn't so difficult to unerstand it's all about the pointing system. In O we have quite even top 15, then it drops. Why is that? Because in O we have only about 15 top athletes in Sweden or what?
- pick any athlete at the end of that tennis list. You can see age. Easy to understand they are young athletes starting their career. In O you pick someone at the end of tle list. Just some races occasionally during last 8 years. Can't be young athlete. Must be someone who has been competing for years but not too often. And click the race he got those points, you see he lost something like 25 min behind the winner and possibly not being last but almost. Can't be so good athelte. Whe is he listed? Because there is no one better in Sweden?
- tennis is well know TV sport, so I guess most people knows what it is about and have some idea how good you need to be to be a top athlete. What an average joe knows about orinteering and how serious sport it is? Not much I guess.

Good point - WR should be supposed to rank top athletes competing at international level. Now it ranks everyone who runs those WRE races, including tourists who just like to run a race while having a summer holiday. And because there is so few races, we end up with a list of 50% athletes and 50% tourists. And lots of tourists ahead of some true atheltes, because they have more races. Nice...

I dont think in tennis any US hobbyist can't get officially ranked top 50 of USA - better than far better players - just by having a nice holiday.

if it's so easy for our curious outsiders to understand the tennis points system, perhaps they could also deduce that the guy with two 1200 scores in orienteering is probably better than the guy with 4*700, despite his lower ranking.

In any case not enough outsiders will find the ranking page for it to be "terribly bad publicity".

Any "top" athlete who cares can get to 4 races in a year pretty easily (even NAmericans... two WOC races and two WC races this year, for ex).

the guy with two 1200 scores in orienteering is probably better than the guy with 4*700, despite his lower ranking

True, any fool can see that. But still they are listed like that....

Any "top" athlete who cares can get to 4 races in a year pretty easily

But nobody cares, of course, why shuold they because it means nothing.

But well, I give up, you are right. IOF ranking system is perfect and there is no way to improve it and we shouldn't try to figure out ways to fix it. And it is OK to keep these slightly fat overweighted middle aged hobbyists getting ranked so high ahead of national team runners even if we could easily change it. And I guess it OK everybody keeps thinking it as a one big joke and our national federations keep funding this joke - I guess we do that on way or an other.

But it would be nice to have a somehow realistic ranking of top O athletes instead of a joke.

BTW the penalty formula with 4% increments was copied form our ranking system, used about two decades for ascribing entries, start orders etc without any bigger whining. Why should anyone whine if one single run is enough to get your ranking about right.

"And I guess it OK everybody keeps thinking it as a one big joke and our national federations keep funding this joke - I guess we do that on way or an other."

Because organizers are stupid. I was thinking about this when we organized WRE event in 2006. We were thinking like this: WRE will bring more siriously runners and our home runners will have chance to get points as our elite depth esp in W class is small. But in W class didn't run nough good runners and nobody got any points. As organizers we did everything to make good race, even small money prizes. So I was wondering if we could ask IOF to get our money back?


"But it would be nice to have a somehow realistic ranking of top O athletes instead of a joke."

It is not a joke. I think that TOP athletes have WOC, EOC, WC ranking and they don't need WRE ranking to have info who is the best. But from IOF perspective I think they need coomon ranking to collect fees. Around 20.000 EUR/anualy.

BTW I will be glad if someone will start to run parralel ranking. Free for organizers / some fun for hobby runners but withouth runners with WC points.

And it is OK to keep these slightly fat overweighted middle aged hobbyists getting ranked so high ahead of national team runners even if we could easily change it.

Hey! I'm not fat!

:)

Actually I was imaging me buying some plane tickets and strolling four 600 point races I might be capable of with some luck. That might take me all the way to top 500 just ahead of some runners who will run for Finland at WOC. So I am the slightly fat overweighted (actually not only slightly these days) middle aged hobbyists I was talking about...

And yes, evereyone here laughs if someone talks about thinks IOF rankings and WR event status, it's not considered more than a joke. you can't expect much runners from here just because you got a WR status.

:( oh jagge, don't let a bit of satanic fruit (or veg?) discourage you.

Two things WREs have going for them:
- it's a pretty good indication for foreigners an event will be well-run with competent planners/controllers, which unfortunately is not always the case
- there tend to be more decent runners turning up, possibly due to the previous point, possibly because some people use the WRE event calendar as a way to find nice tourist destinations :)

Here WREs are almost never in a good terrain. Occasional WRE organizers does not have to use a good terrain for it, because there will be enough local runners anyway. And for some reason one WRE is always in Forssa, that's really not the right places for serching good terrains as some you may have seen.

Anyway, I can see troube coming if we start using WR for deciding who will get a place in a limited WRE class (like 40 is taken as usual) and who will not. Sometimes about 300 would like to run the WRE race. In Finland 500th best ranked runner is still easily capable of making 800-900 point races, but still we may take even worse runners if we use WR - those who had a nice holiday last winter. And end up putting some winner candidates to B class. But I can see it coming if we have to select between runners from different countries.

About

"Two things WREs have going for them"

I think we all agreed about those two things. But still this is good mostly for new/not so known event. If WR should mean something also for elite runners IOF must made some changes in the system otherwise I see useless and it is maybe batter to have unofficial event calendar with common agreed ranking for hobby orienteers around the world. Ranking which will be free to enter for everybody, maybe also for veteran categories as I don't understand why hobby orienteers want to compare with the best on the same ranking list. For me is just O.K. to run Elite to get real time comparison.

Regarding J-J's comment above, for all the complication of the IOF World Ranking formula, it uses a non-robust measure of central tendency (mean points) to calibrate the points for a race. This is the thing that could use the most improvement, in my opinion. Standard deviations or no standard deviations--one could argue this either way, but get rid of the average time and average points. Basing the ranking points on the time and using personal scoring values (ranking points multiplied by time), with a measure of central tendency that is less outlier-sensitive, would be better. Not necessarily the JJUNAR=>USOF system, but something similar.

True. Difficult to understad why they have to use times and points of those almost dead last hobby orienteers to calculate points for top runners. For example If I remember right, winner's points given by this formula can easily change ~100 points just by removing same poorly ranked people from the race. It shouldn't matter almost at all how some additional competitors far behind top runners are doing. Anyone can take some results from the web, take IOF formula and try what happens.

A bad race by a top runner can also sway the results, although I don't know whether it would have as much effect as a "hobby" orienteer running or not running (and doing poorly or well). A harmonic mean or percentile measure of all runners' performances would be less sensitive to outlier issues at both ends of the results: well-ranked runners doing unusually poorly or poorly-ranked runners doing unusually well (or at least better than their ranking would indicate).

"winner's points given by this formula can easily change ~100 points just by removing same poorly ranked people from the race". If this is true, does this mean poorly ranked people shouldn't do WREs as they hurt the rankings probably needed by the better orienteers?
I am seriously asking this as I trying to decide if should sign up for some upcoming WREs or run my age group. I fall into the the "poorly ranked".

As far as I understand it essentially the scenario is that everyone in the race throws all their average points into the pool and those points get shared out depending how everyone does. Therefore if you are a lower ranked runner you don't put so many points in; if you have a good run e.g. if the setter does a poor job and it turns into a running race then you get more points back - but these points are at the expense of the other runners - and it lowers their average for the next time.

ebone put it well, the probem is uses average times and average points. And possibly standard deviation and using hobby orienteers time to calclulate it. System works somehow only for those runners who ran average times, but at both ends points can be anything and two separate races are not comparable.

what this really says is that WREs are for 'hobby' runners to compare / contrast / aspire to and the 'big' events like WCs, WOC, EOC etc are for the 'elite' to do well at and get their counting ranking points at.

I wonder how many of the top ranked runners have counting points from WREs...

Where is the women's list?

tdgood: I am seriously asking this as I trying to decide if should sign up for some upcoming WREs or run my age group.

Do whatever you want to do. It's just a ranking system, and it shouldn't drive your decision to run or not. I (World Ranking ID USA11) am one of the better ranked U.S. runners, and I don't care enough about my ranking points to ask anyone to try to game the system on my behalf. As good or bad as the formula is, it is still possible to improve one's score by running fast times, and that is all I intend to concern myself with, when it comes to my own participation in WRE events.

At one point, I noticed that the top Canadian and American women had both beaten the then 100th ranked woman, an Estonian, when they had run against her in WOC, but were ranked well behind her, in part because she had dozens and dozens of ranked runs (and the North Americans only a half dozen). But North Americans also seem to get lower points when running WREs in NA than abroad (with the exception of Pam James' COC runs last year). Maybe the Good effect? But then, hard to get enough ranked women to keep the WREs ranked, so maybe need the faster middle aged to get and stay ranked?

At most WREs organizers made random starting list. Which maybe is not good. At least "R" runners should start first or last (8-10' gap to the hobby runners) or half of them first and other half at the end to minimize following problem; elite vs hobby runners.

Following is an issue when you have two runners of similar ability close together. Hobby runners can't keep up with elite runners.

North Americans also seem to get lower points when running WREs in NA than abroad

World Champs qualifiers are worth an extra 5%, so if the events abroad that you're considering are world champs qualifiers, divide by 1.05 first for a fair comparison (world champs finals are worth an extra 10%, world cup finals and world games finals an extra 5%. See IOF ranking specifications).

When I compare elite vs. hobby I was thinking about all 'R' runners just because they are important for inputs for formula at each race. As 'R' runners are not just Top elite runners but very large portion of other runners, I predict that also hobby runners can follow some of those runners and get better time results, more points and maybe R status. At WRE statistics I didn't find answers to these questions. Maybe a few analyzes should be done.

Is it true that in avg. Hobby runners (without 'R' status) which run among first get better results than last ones?
No data about how was changing the avg. mean or total number of 'R' runners over the years.

For now I am just assuming that avg. mean on all WREs without (WOC, WC,...) must be each year smaller than year before even if it is just for a few points. I don't mean that competitiveness or quality of entered runners is lower each year. It is probably just what happened when you have so called 'Law of big numbers'

To preserve elite maybe number of 'R' runners should be limited per one country in advance.

I predict that also hobby runners can follow some of those runners and get better time results...

Following happens. The ranking scheme isn't intended to do anything about that.

To preserve elite maybe number of 'R' runners should be limited

If a "hobby" runner is able to get enough points to be among the "elite", then maybe (s)he deserves to be there. What is this notion that the "elite" needs to be "preserved"?

For now I am just assuming that avg. mean on all WREs without (WOC, WC,...) must be each year smaller than year before even if it is just for a few points

The numerical value of the mean is entirely irrelevant for generating a ranking list. It could be normalized to 1, 1000000, or even a googolplex; why is it a problem if it varies by a few points per year?

My early post was about that we shouldn't have random start list at WREs. It is not very important improvement and also hobby runners really don't care at all. It is small error but still if we can minimize this effect why we don't do it.

If you look at FIN (around 150 "R"runners)
http://www.6prog.org/iof/wre_aths$.asp?mw=M&FED=FI...

I doubt that all of them are elite runners and also how many of them really start at other WREs out of Scandinavia? If I understand WRE Formula correctly; MP is made just out of avg. points of "R" runners. Because TOP Elite runners don't need to run those small WREs you get additional margin in avg. points for MP at normal WRE or at big international race (WOC, WC, some WREs..).

If we say that WREs are international races than differences between WREs and WOC and WC should be just in IP factor (1.10 for WOC Finals, 1.05 for WOC Qualification races). Also my stand point is that WE organizers pay to IOF to get the international rank for our event and not to get good TOP runners. I think that formula should calculate MP as avg. of country avg. points of all "R" country runners. So if you have runners in elite category from 10 countries you made avg. out of these 10 countries. I am not sure but MP will probably be smaller compared to other WREs and importance of WRE location will not be so crucial for runners. Suddenly all WREs will have meaning for international elite or local "R" runners as you will need only one SWE, FIN, NOR runner to get their avg. points for formula. If you get very good runner to your event than he must performed good also on your type of terrain. We could predict that all of them will not make it and than really start WR tic tacking on the long run. Who is the best on different terrains?

Example
One runner came from SWE to WRE race in Hungry and he ran very bad race. His country MP go slightly down and other SWE runners should do batter next time if they want to preserve elite in SWE. IF all of them made bad races abroad then their Eliteserien as WRE event will not be so important for ranking. It is very exceeding example but logic seems to me O.K. Now this is not so important as some SWE hobby runners who run elite in abroad they probbbly don't run or they are not allowed to run Elite in SWE.

I am not sure if all solutions are good but I think that this kind of system with other solutions could be batter than this one as WR will have meaning for runners and organizers. Now we count only on runners which is not good as they don't need our WREs. Check andrewd: post

difficult to understand. But it doesnt sound good to use worst runs instead of the best runs for calculating rank points. Or use hobby runners who like to travel for ranking nations.