Discussion: World Ranking calculation question
in: Orienteering; General
Jan 9, 2012 12:50 PM
#
I'm new to this site and have seen quite a bit of debate on the calculation methods etc but had a look at the most recent rankings and had a question I hadn't seen answered by previous posts.
What is the rationale for excluding 0 point runs in the calculation of "R" - ranked runner status? By excluding a 0 pt finish or "DSQ/MP" it appears benefitial for athletes who only attend few WREs to rather DSQ or get a 0 run then get a lower than 600 (or lower than their ave) score. Obviously this assumes the athlete cares about their "R" status.
Eg. If I look at the most recent uploaded WRE result (a Long event in South Africa where Olav Lundanes won and received 1257 points) there is an obvious example of this anomaly. The 4th (of the 4) "R" runners used in the calculation appears to have "R" status based on 2 runs at WOC last year, one with a points score of over 700 and the other a run where 0 points were scored (outside of the points run) but thus ignored for "R" ranking status.
It could be argued I guess that in such cases the runner should not have an "R" status as their actual average on 2 completed courses is <600. This would greatly change the points allocation from the above mentioned event (if my maths is correct, Lundanes would have received a considerably lower score of 829 and likewise the rest of the field). Quite a big anomaly, and thus my question. Any thoughts?
What is the rationale for excluding 0 point runs in the calculation of "R" - ranked runner status? By excluding a 0 pt finish or "DSQ/MP" it appears benefitial for athletes who only attend few WREs to rather DSQ or get a 0 run then get a lower than 600 (or lower than their ave) score. Obviously this assumes the athlete cares about their "R" status.
Eg. If I look at the most recent uploaded WRE result (a Long event in South Africa where Olav Lundanes won and received 1257 points) there is an obvious example of this anomaly. The 4th (of the 4) "R" runners used in the calculation appears to have "R" status based on 2 runs at WOC last year, one with a points score of over 700 and the other a run where 0 points were scored (outside of the points run) but thus ignored for "R" ranking status.
It could be argued I guess that in such cases the runner should not have an "R" status as their actual average on 2 completed courses is <600. This would greatly change the points allocation from the above mentioned event (if my maths is correct, Lundanes would have received a considerably lower score of 829 and likewise the rest of the field). Quite a big anomaly, and thus my question. Any thoughts?
Jan 9, 2012 1:35 PM
#
Agreed this has always seemed a little odd to me. A "0" which is from DNF or MP might be ignorable, as this is perhaps a one-off event that isn't indicative of that competitor's typical performance, but a "0" which is just such a long time they are out of the points getting ignored seems less explainable.
I'd guess the WRE folks are mostly focused on the middle to top of the field, and for those folks, even a "0" due to very high time is also likely a one-off event - e.g. the competitor twisted an ankle, or had food poisoning, but decided they'd go round the course anyway?
It does provide strange incentives though for folks near the edge of "R". While it doesn't hurt one's overall WRE score to get a 1 point run, vs. a 0 point run, it does hurt "R" status, and the presumption of that individual's strength vs. other runners, which is somewhat important in maintaining a functional WRE-rankings pool, and balanced scores in low elite population corners of the world (e.g. USA, South Africa...)
I'd guess the WRE folks are mostly focused on the middle to top of the field, and for those folks, even a "0" due to very high time is also likely a one-off event - e.g. the competitor twisted an ankle, or had food poisoning, but decided they'd go round the course anyway?
It does provide strange incentives though for folks near the edge of "R". While it doesn't hurt one's overall WRE score to get a 1 point run, vs. a 0 point run, it does hurt "R" status, and the presumption of that individual's strength vs. other runners, which is somewhat important in maintaining a functional WRE-rankings pool, and balanced scores in low elite population corners of the world (e.g. USA, South Africa...)
Jan 9, 2012 5:15 PM
#
I would say that your interpretation of the Rules is 100% right. I will not go in discussion of how good WRE formula is but rather how good definition of “R” runner is; rule 2.3
A ranked runner is defined as one who has scored World Ranking points....
a) in the 18 months before the event
b) whose average points score is greater than or equal to 600
c) who finishes within the winner's time plus 50%
He has scored points before the event (a) and based on this you predicted that also his 0 points run should be valid. Rule says so but not clear enough in my opinion. Why FOC don’t use this interpretation? I see two possible reasons for different interpretation of the Rules.
1) His 0 points run was achieved when runner has not had R status yet. So maybe FOC decided to ignore previous runs (a) and calculated his 18 months average from the date onwards when he becomes R runner. I agree it is not strictly by the rules so I think that here rule should be corrected because it is nonsense to use previous runs before he become R runner. In this way we prevent many runners to become ranked.
2) I am not sure but I think that maybe FOC uses all three conditions also for 18 months average score. I would say that rule is also not 100% clear here, although FOC and we all have understood that all runs are included. But based only on (a) you can't say that all valid runs should be included. Today rule doesn't say anything about which valid runs should be included into 18 months avg. score so this could be the only logical explanation why FOC excluded these bad runs from 18 months avg. score. If unranked or “R” runner gets 0 points or any other sum of points and his run is out of condition (c) then these points/runs are also not used for 18 months average score. I thought that this condition (c) is valid only for each single race but as it appears now this is another hidden/unclear WRE rule. I made a quick check and I found one more example which support my explanation and it clearly shows that 1) option was not used. Her 18 months avg. score doesn't include 0 run but includes all other runs in last 18 months and this includes also runs before she become R runner. With 0 run she would be without R status and without previous runs before she got R status she would have avg. score of 712,16 points. What is best for Rankings? The problem is that no rule says which runs should be and which shouldn’t be included into 18 months average score. I think FOC made its own interpretation outside the Rules and it is a good one. I would rather change the rule as today interpretation is not good for the World Rankings.
You are right, today rules are in favour of your explanation but not the previous examples and that is why I see FOC's decision better when we speak about which rules give better quality of World Ranking Sheme. I hope that they will work on this and make a change. In the last years I haven’t seen any notice in the FOC meeting minutes about this or any other problematic issues so I hope that they are aware of it. It just needs to be corrected with better wording of the rule.
A ranked runner is defined as one who has scored World Ranking points....
a) in the 18 months before the event
b) whose average points score is greater than or equal to 600
c) who finishes within the winner's time plus 50%
He has scored points before the event (a) and based on this you predicted that also his 0 points run should be valid. Rule says so but not clear enough in my opinion. Why FOC don’t use this interpretation? I see two possible reasons for different interpretation of the Rules.
1) His 0 points run was achieved when runner has not had R status yet. So maybe FOC decided to ignore previous runs (a) and calculated his 18 months average from the date onwards when he becomes R runner. I agree it is not strictly by the rules so I think that here rule should be corrected because it is nonsense to use previous runs before he become R runner. In this way we prevent many runners to become ranked.
2) I am not sure but I think that maybe FOC uses all three conditions also for 18 months average score. I would say that rule is also not 100% clear here, although FOC and we all have understood that all runs are included. But based only on (a) you can't say that all valid runs should be included. Today rule doesn't say anything about which valid runs should be included into 18 months avg. score so this could be the only logical explanation why FOC excluded these bad runs from 18 months avg. score. If unranked or “R” runner gets 0 points or any other sum of points and his run is out of condition (c) then these points/runs are also not used for 18 months average score. I thought that this condition (c) is valid only for each single race but as it appears now this is another hidden/unclear WRE rule. I made a quick check and I found one more example which support my explanation and it clearly shows that 1) option was not used. Her 18 months avg. score doesn't include 0 run but includes all other runs in last 18 months and this includes also runs before she become R runner. With 0 run she would be without R status and without previous runs before she got R status she would have avg. score of 712,16 points. What is best for Rankings? The problem is that no rule says which runs should be and which shouldn’t be included into 18 months average score. I think FOC made its own interpretation outside the Rules and it is a good one. I would rather change the rule as today interpretation is not good for the World Rankings.
You are right, today rules are in favour of your explanation but not the previous examples and that is why I see FOC's decision better when we speak about which rules give better quality of World Ranking Sheme. I hope that they will work on this and make a change. In the last years I haven’t seen any notice in the FOC meeting minutes about this or any other problematic issues so I hope that they are aware of it. It just needs to be corrected with better wording of the rule.
Jan 9, 2012 5:17 PM
#
Any statistics based on 4 athlete is going to be pretty rubbish, but here it seems to have worked out OK. 1257 is Lundanes lowest score this year, but not unreasonably so, so the event probably contributed some reasonable scores to SA runners. 829 would have been absurd: even I've scored more than that, when I was M40!
(in case you don't know, I've never troubled the international selectors, even as an M21, and I'm not R on account of an average score of (sob) 599)
(in case you don't know, I've never troubled the international selectors, even as an M21, and I'm not R on account of an average score of (sob) 599)
Jan 9, 2012 5:41 PM
#
Double checked and I come to a different conclusion. Sveng used times of top 3 R runners to calculate MT but he forgot to change denominator as he did in case of MP. He divided SUM with 4 not by 3 and that is why he got MT=3.486,25 and 829 points. So, without Michael Crone more points would be distributed than today: Lundanes - 1272 points, etc
Jan 10, 2012 8:34 AM
#
Ah, thanks kofols, there does appear to be an error of sort, formula I apply not updating.
I guess I still ask the question why 0 results are not included in R status calculation but perhaps the impact of the current method is less than as in my incorrect calculation. It could be argued that a 0 points run should be included as would a low 10 point run. It could also be argued that a MP 0 should also be counted as an incentive to check control number and/or be sure navigation are correct, but this can be argued both ways and not probably an issue for the top athletes.
I guess the SA results could been cleared for any number of unrelated reasons or even a relook at the rebasing, so I won't think much there.
I guess I still ask the question why 0 results are not included in R status calculation but perhaps the impact of the current method is less than as in my incorrect calculation. It could be argued that a 0 points run should be included as would a low 10 point run. It could also be argued that a MP 0 should also be counted as an incentive to check control number and/or be sure navigation are correct, but this can be argued both ways and not probably an issue for the top athletes.
I guess the SA results could been cleared for any number of unrelated reasons or even a relook at the rebasing, so I won't think much there.
Jan 10, 2012 10:22 AM
#
Just had a look at the ranking for some of our runners. It does seem a bit random, e.g. British Champion Doug Tullie got more points for coming next to last in the WOC final than for destroying the field at BOC, and JK champion (the major UK international event) Hector got about the same for failing in the qualifying race as for winning the JK by 2 mins ahead of higher ranked runners.
Including zeroes for people who usually score 600+ would make their points unrepresentative, which would mess up the statistics for everyone else, especially in small fields. The problem for the ranking list is trying to balance "rewards" with accurate statistics.
The UK system now has two numbers, one (meant to be) the best statistical rating, which is used to calculate points, and the published one which comprises you best 6 runs. The key difference is that the first ranking can go down if you have a bad run, while the second can't.
Including zeroes for people who usually score 600+ would make their points unrepresentative, which would mess up the statistics for everyone else, especially in small fields. The problem for the ranking list is trying to balance "rewards" with accurate statistics.
The UK system now has two numbers, one (meant to be) the best statistical rating, which is used to calculate points, and the published one which comprises you best 6 runs. The key difference is that the first ranking can go down if you have a bad run, while the second can't.
Jan 10, 2012 1:33 PM
#
Results are back. Some problems with times, still a very strange example to me. Runners got different time deductions from previous times (-9s, -25s, -10s)
@graeme
This was well known problem and the solution is in the best interest of World Rankings. From 1.1.12 weighting factors are history. But still this solve only part of the anomalies incorporated in the WRE Rules and Formula. I will try to use my formula to calculate your examples.
@graeme
This was well known problem and the solution is in the best interest of World Rankings. From 1.1.12 weighting factors are history. But still this solve only part of the anomalies incorporated in the WRE Rules and Formula. I will try to use my formula to calculate your examples.
Jan 10, 2012 2:40 PM
#
@kofols. Agreed, there are two problems with any stats based system.
1/ Having bad statistics
2/ Doing statistics badly
Not much can be done about the first, but it's nice to hear they're improving up the second.
1/ Having bad statistics
2/ Doing statistics badly
Not much can be done about the first, but it's nice to hear they're improving up the second.
Jan 10, 2012 9:28 PM
#
I guess I still ask the question why 0 results are not included in R status calculation...
From my point of view it is an arbitrary decision by FOC because none WRE rule says anything about this. It is not just a question to include or to exclude. With inclusion you get even worse WRE system and here I am with graeme.
Also exclusion is not the best solution but it is far better than inclusion based on today rules. To get better solution for this problem we need first to change rule 2.3 and make better definition of R runners. Changing this rule means changing WRE formula. So here I must disagree with graeme that we can’t do anything. With today rule we get skewed MP value when good runners make bad runs or make MP/DNF and their avg. points are not used for calculation. This is not good. I have made a few calculations based on different assumptions and come to very interesting conclusions.
For most of the people today system is fair enough just because they have been told that is fair. It is not bad but far from being fair enough. I think every small part of this puzzle; like 0 results should be resolved in the best way if we want to get fairer and better WRE system at the end.
From my point of view it is an arbitrary decision by FOC because none WRE rule says anything about this. It is not just a question to include or to exclude. With inclusion you get even worse WRE system and here I am with graeme.
Also exclusion is not the best solution but it is far better than inclusion based on today rules. To get better solution for this problem we need first to change rule 2.3 and make better definition of R runners. Changing this rule means changing WRE formula. So here I must disagree with graeme that we can’t do anything. With today rule we get skewed MP value when good runners make bad runs or make MP/DNF and their avg. points are not used for calculation. This is not good. I have made a few calculations based on different assumptions and come to very interesting conclusions.
For most of the people today system is fair enough just because they have been told that is fair. It is not bad but far from being fair enough. I think every small part of this puzzle; like 0 results should be resolved in the best way if we want to get fairer and better WRE system at the end.
Jan 10, 2012 11:31 PM
#
@sveng
ah,..also my 2nd reason is not the reason for FOC decision. I found example of a R runner Sarah-Jane Gaffney, GBR who finished over winner's time+50% (at JK Trophy) and her 428 points were used for 18 months avg. score.
Wyatt makes his point. I hoped that behind this is a theoretical reason or at least any kind of official explanation than just an ordinary lack of interest for the issue. So instead to find a solution FOC made an algorithm to exclude only results with 0 points. Maybe this was the easiest way to exclude 0 runs from R status calculation when you have "0 points = Insufficient Ranked Athletes to proceed" and why not used also for "0 points = valid results/bad runs" in the same time.
I really can't understand why FOC is not able to collect different views to find a suitable solution and make clearer rules. Now WRE rules really start to look like an Emmentaler.
ah,..also my 2nd reason is not the reason for FOC decision. I found example of a R runner Sarah-Jane Gaffney, GBR who finished over winner's time+50% (at JK Trophy) and her 428 points were used for 18 months avg. score.
Wyatt makes his point. I hoped that behind this is a theoretical reason or at least any kind of official explanation than just an ordinary lack of interest for the issue. So instead to find a solution FOC made an algorithm to exclude only results with 0 points. Maybe this was the easiest way to exclude 0 runs from R status calculation when you have "0 points = Insufficient Ranked Athletes to proceed" and why not used also for "0 points = valid results/bad runs" in the same time.
I really can't understand why FOC is not able to collect different views to find a suitable solution and make clearer rules. Now WRE rules really start to look like an Emmentaler.
Jan 11, 2012 7:54 AM
#
We were also surpised when Michael was given 'R'anked status after WOC. His two runs gave him 754 and 0 points (bad run), but the 0 was discounted and his average given as 754.
It seems like the WRE points calculations in respect to 'R'anking aren't being fully applied given the rules mentioned by kofols. I suspect that the +50% is being ignored, whilst all 0 points (despite their different reasons) are being lumped together and taken out of the 'R' equation. Maybe this is for simplicity sake or maybe VIProgammers are not fully aware of the smaller details.
Graeme, with regards to Doug Tullie's points, keep in mind that their is a multiplier (IP) in the WRE calculation that gives WOC Qualifiers a 1.05 higher points score, with a 1.10 in WOC Finals. Although this was intended to emphasize the importance and standing of WOC, in effect it means that other WREs will never feature highly in the total WRE points score of the world's best orienteers.
It seems like the WRE points calculations in respect to 'R'anking aren't being fully applied given the rules mentioned by kofols. I suspect that the +50% is being ignored, whilst all 0 points (despite their different reasons) are being lumped together and taken out of the 'R' equation. Maybe this is for simplicity sake or maybe VIProgammers are not fully aware of the smaller details.
Graeme, with regards to Doug Tullie's points, keep in mind that their is a multiplier (IP) in the WRE calculation that gives WOC Qualifiers a 1.05 higher points score, with a 1.10 in WOC Finals. Although this was intended to emphasize the importance and standing of WOC, in effect it means that other WREs will never feature highly in the total WRE points score of the world's best orienteers.
Jan 11, 2012 10:49 AM
#
Woo woo! I seem to be the worlds highest rated non-R athlete! How exciting is that?
@kofols Statistically/theoretically, (rather than by the letter of the rules) I would say it is reasonable to include Sarah-Jane's 428 points: it looks like a bad run (one big miss) and not completely unrepresentative (sorry SJ). It makes much more sense to exclude people whose score is well below their own average than on the basis of who turned up to win the race.
@kofols Statistically/theoretically, (rather than by the letter of the rules) I would say it is reasonable to include Sarah-Jane's 428 points: it looks like a bad run (one big miss) and not completely unrepresentative (sorry SJ). It makes much more sense to exclude people whose score is well below their own average than on the basis of who turned up to win the race.
Jan 11, 2012 12:14 PM
#
I mentioned her run just because I wanted to present that as my first possible reason also my second reason (+50%) which could have been in theory the FOC's reason to exclude 0 runs also doesn't hold the water. It is no difference between 428 and 0 points so this is anomaly in the Rules. Check Wyatt post. I still think that FOC's decision is good even if it is not consistent with the today rule 2.3. So should we be consistent in the future or should we change the rule 2.3?
It makes much more sense to exclude people whose score is well below their own average than on the basis of who turned up to win the race.
Yes and no. I agree with you when we speak about their times and reasons to exclude them from the MT/ST equation. But I disagree when it comes to MP/SP equation. I think that their avg. scores shouldn’t be excluded from MP/SP.
One thing about which I am sure that would work better (based on my calculations) is that MP should be calculated based on all R runners in the race. Weaker runners should be rewarded with more points in case good runners make bad runs or M.P. Now these bad runs (over +50%) have no effect on MP and Points calculations.
Going into details I think we should get one WRE formula at the end and not three as we have it today. Small Event calculation and weighted average of Large and Small Events calculations are based on arbitrary decision by FOC. Also 50% border is arbitrary decision by FOC. That are two more reasons why I think we should change rule 2.3.
It makes much more sense to exclude people whose score is well below their own average than on the basis of who turned up to win the race.
Yes and no. I agree with you when we speak about their times and reasons to exclude them from the MT/ST equation. But I disagree when it comes to MP/SP equation. I think that their avg. scores shouldn’t be excluded from MP/SP.
One thing about which I am sure that would work better (based on my calculations) is that MP should be calculated based on all R runners in the race. Weaker runners should be rewarded with more points in case good runners make bad runs or M.P. Now these bad runs (over +50%) have no effect on MP and Points calculations.
Going into details I think we should get one WRE formula at the end and not three as we have it today. Small Event calculation and weighted average of Large and Small Events calculations are based on arbitrary decision by FOC. Also 50% border is arbitrary decision by FOC. That are two more reasons why I think we should change rule 2.3.
Jan 11, 2012 5:46 PM
#
I think that [dnfs] avg. scores shouldn’t be excluded from MP/SP.
In which case, the next WRE I plan I'll give a free entry to all the top ranked athletes and dnf them when they don't show up. Loads more points for everyone else!
In which case, the next WRE I plan I'll give a free entry to all the top ranked athletes and dnf them when they don't show up. Loads more points for everyone else!
Jan 11, 2012 11:43 PM
#
I meant this only for runners +50% and dsq/rtd (DNF/MP) and not for runners who DNS.
And it is not necessarily that everyone else would get more points. Lower ranked runners make in general more mistakes than TOP runners so it could also be possible to have scenario that everyone else would get fewer points. Fair enough, each race - all R runners in equation.
In today system TOP runners are not punished if they make a mistake and then DNF. In every sport you are awarded for your achievements over your opponents no matter what is the reason for your win. As we have stats based system we decided differently. Take a look an example of Kauppi at WOC LongF; why ranking system should not punish her mistake? DNF and 0 results have the same analogy - both are excluded from equation.
And it is not necessarily that everyone else would get more points. Lower ranked runners make in general more mistakes than TOP runners so it could also be possible to have scenario that everyone else would get fewer points. Fair enough, each race - all R runners in equation.
In today system TOP runners are not punished if they make a mistake and then DNF. In every sport you are awarded for your achievements over your opponents no matter what is the reason for your win. As we have stats based system we decided differently. Take a look an example of Kauppi at WOC LongF; why ranking system should not punish her mistake? DNF and 0 results have the same analogy - both are excluded from equation.
Jan 12, 2012 9:03 AM
#
You might like the ranking system to punish someone for breaking a leg and likewise Kauppi for a mistake, but including her DNF=0 would make no difference to her. because she only counts her best six runs.
All you do is mess up the statistics. Why? Because "0" is a completely arbitrary number, you could equally well argue for +100 or -100 or half her average score.
All you do is mess up the statistics. Why? Because "0" is a completely arbitrary number, you could equally well argue for +100 or -100 or half her average score.
Jan 12, 2012 12:13 PM
#
You should not take this too personal. You also exaggerate about my intensions. It is not so much about her, it is more about others. It is true that her avg. score would not be hurt but other runners would have more chances to make better scoring at other races where TOP runners (including Kauppi) don't take part. Mathematically this rule would not change much. It was still better for her to do DNF than come to the finish with bad time which could spoil her avg. score and influence on Points Calculations because her time would be included also into MT/ST equation. We need to fix this anomaly if we think that this is anomaly in the system.
It may sound odd but without including her avg. score all other runners were punished with lower points. This difference in MP would be very small also in extreme cases (DNF from TOP or lower R runners) but it is important to acknowledge it for the sake of the fairness of the system. With this anomaly we give TOP runners in combination with ranking system a small chance to control the system. If you have 50 TOP R runners in a race and 48 of them would make a mistake then also other two may end up without points. Ranking system should be valid also in such an extreme situations.
What would be mess up? I don't get it. Only MP/SP for a single race would change. I didn't have in mind to use her DNF also for her 18 months avg. This 18 months avg. is also one of the unresolved problems in the Rules. Anomaly here in my opinion is conviction that all runners with avg. scores >600 gives us a very representative pool of R runners whose performances don’t shift too much. Now you could earn R status with already only one good race. These R runners with one, two,.. runs are less representative than runners with 10 or more runs but the problem is that they are not treated equally by the rules!
I don’t have a problem with this approach where runner with only one good run achieve R status but as I said in earlier post I think that 18 months avg. should be written in the Rules more clearly.
It may sound odd but without including her avg. score all other runners were punished with lower points. This difference in MP would be very small also in extreme cases (DNF from TOP or lower R runners) but it is important to acknowledge it for the sake of the fairness of the system. With this anomaly we give TOP runners in combination with ranking system a small chance to control the system. If you have 50 TOP R runners in a race and 48 of them would make a mistake then also other two may end up without points. Ranking system should be valid also in such an extreme situations.
What would be mess up? I don't get it. Only MP/SP for a single race would change. I didn't have in mind to use her DNF also for her 18 months avg. This 18 months avg. is also one of the unresolved problems in the Rules. Anomaly here in my opinion is conviction that all runners with avg. scores >600 gives us a very representative pool of R runners whose performances don’t shift too much. Now you could earn R status with already only one good race. These R runners with one, two,.. runs are less representative than runners with 10 or more runs but the problem is that they are not treated equally by the rules!
I don’t have a problem with this approach where runner with only one good run achieve R status but as I said in earlier post I think that 18 months avg. should be written in the Rules more clearly.
Jan 12, 2012 12:18 PM
#
To show you an example I will use the same SA event.
Garry Morrison, RSA got his R status based on this event with one good run. In contrary Dylan Hemer, RSA and Bradley Lund, RSA made one good and one bad run and so they didn’t get R status. Is now Garry more representative than Dylan and Bradley?
18 months avg. should include only runs from the date onwards when runner for the first time achieve more than 600 points (and change his status from unranked to Ranked) or maybe for the sake of representativeness also runs in 3-6 months before this date but not 18 months as says today rule. In this way all runners would have chance to have some influence on Points calculation no matter how bad they performed in the past. Using runner’s previous bad runs from 1 year or more ago is a very conservative approach. Why his previous bad fitness is more important in comparison with today performance?
I think this approach could easily be more flexible. It is also not good that we don't have fixed max. number of runs included into 18 months avg. score. Fixing min. number of runs for R status would be wrong direction but fixing max. number of runs would be good in my opinion. Now all runs in last 18 months are included and this cause some theoretical and evident problems.
I agree that is nonsense to use 0 run or DNF for calculating 18 months avg. for a runner who already proved that he is a TOP runner. His 8, 10 or 15 TOP runs in last 18 months should be enough to get his avg. score and to see how good he is and representative in general. I think that fixing 18 months avg. score calculation on the basis of the best 8-10 runs would be quite enough to get representative avg. score for each runner. TOP runners wouldn’t lose their high avg. scores with competing at small WRE events. It is an anomaly that Lundanes’s high avg. score went down with winning the race.
I am sure that these changes can give us better representative pool of R runners, better distribution of points for a single race and more interest from TOP runners to run small WRE events.
Garry Morrison, RSA got his R status based on this event with one good run. In contrary Dylan Hemer, RSA and Bradley Lund, RSA made one good and one bad run and so they didn’t get R status. Is now Garry more representative than Dylan and Bradley?
18 months avg. should include only runs from the date onwards when runner for the first time achieve more than 600 points (and change his status from unranked to Ranked) or maybe for the sake of representativeness also runs in 3-6 months before this date but not 18 months as says today rule. In this way all runners would have chance to have some influence on Points calculation no matter how bad they performed in the past. Using runner’s previous bad runs from 1 year or more ago is a very conservative approach. Why his previous bad fitness is more important in comparison with today performance?
I think this approach could easily be more flexible. It is also not good that we don't have fixed max. number of runs included into 18 months avg. score. Fixing min. number of runs for R status would be wrong direction but fixing max. number of runs would be good in my opinion. Now all runs in last 18 months are included and this cause some theoretical and evident problems.
I agree that is nonsense to use 0 run or DNF for calculating 18 months avg. for a runner who already proved that he is a TOP runner. His 8, 10 or 15 TOP runs in last 18 months should be enough to get his avg. score and to see how good he is and representative in general. I think that fixing 18 months avg. score calculation on the basis of the best 8-10 runs would be quite enough to get representative avg. score for each runner. TOP runners wouldn’t lose their high avg. scores with competing at small WRE events. It is an anomaly that Lundanes’s high avg. score went down with winning the race.
I am sure that these changes can give us better representative pool of R runners, better distribution of points for a single race and more interest from TOP runners to run small WRE events.
Jan 12, 2012 10:50 PM
#
@kofols Nothing personal, even if we're boring everyone else, its fun to discuss with you because you obviously understand the consequences of scheme in more detail than those who set it up. To me, most of what you say e.g. the absurd results arising from the weighting is so obviously right I haven't bothered to say I agree with you. But to be clear, I do agree with 90% of what you've said hare (and elsewhere).
So excuse me if I still poke you if you say something silly, like...
18 months avg. should include only runs from the date onwards when runner for the first time achieve more than 600 points
So given two guys, running 3 races a year, both of whom are improving, scoring
Joe 550 550 550 750 750 750
Moe 600 600 600 800 800 800
you reckon Joe should have R=750 and Moe R=700, even though Moe beat Joe every time?
So excuse me if I still poke you if you say something silly, like...
18 months avg. should include only runs from the date onwards when runner for the first time achieve more than 600 points
So given two guys, running 3 races a year, both of whom are improving, scoring
Joe 550 550 550 750 750 750
Moe 600 600 600 800 800 800
you reckon Joe should have R=750 and Moe R=700, even though Moe beat Joe every time?
Jan 13, 2012 9:21 AM
#
How about comparing Jeremy Green, RSA with Michael Crone, RSA. Mike is ranked but an injury at WOC 2010 results in Jeremy being unranked. But come February when WOC 2010 is no longer within the 18 months, Jeremy will become ranked above Mike. Without competing again?
Jan 13, 2012 9:52 AM
#
That's a very good comparison to illustrate the problem - how to know just from the results whether your very low score is due to limping round injured (i.e. should be discarded as unrepresentative) or making a single error costing 300+ points in France (i.e. should be included as representative).
It doesn't affect your personal ranking, (you are already ranked above Mike - 396th vs 511th) but does matter a great deal to the WRE points available to everyone competing in the South African WRE. In that particular case, it seems clear that you ability would be better represented if the very low scores were discarded.
It doesn't affect your personal ranking, (you are already ranked above Mike - 396th vs 511th) but does matter a great deal to the WRE points available to everyone competing in the South African WRE. In that particular case, it seems clear that you ability would be better represented if the very low scores were discarded.
Jan 13, 2012 11:33 AM
#
I know you understand scheme very well. My apologies but it was part of a discussion. I know that this discussion has gone into details and it is probably uninteresting for others. Sometimes it seems that we all forgot why we establish the scheme, including IOF. I see Rankings as a tool to promote elite orienteering, orienteers and their achievements and not just Rankings itself. It is a pity that scheme rules are not clear enough and many things are staying the same for a long time.
I think that idea behind the scheme is very good but in last few years scheme doesn’t evolve into one of the more important IOF tools. So I have doubts what is real IOF intension to do with the scheme. I have got the feeling (also from other comments elsewhere) that IOF don’t care much about the future of the scheme. About my suggestions, I just made an extensive analysis and try to see where could lie the problems, how to see them from other perspective and to figure out possible solutions which could work better than today rules. If we agree on that than I am satisfied that also some other people and particularly someone like you agree that these solutions could be serious proposals and acceptable for the future development of the scheme.
your example
I didn’t test this idea on all possible examples but I appreciate your case as this is still more or less draft idea. I also wrote this part of the same statement….or maybe for the sake of representativeness also runs in 3-6 months before this date but not 18 months as says today rule.
And I added that his avg. score should be calculated based on last 8-10 best runs in last 18 months. Because Joe has only 6 runs his avg. score would be 650 and not 750. And this would be true only in case he achieved all his three 550 results in 3-6 months before he has achieved 750 for the first time. In case he achieved all 550 results 1 year ago then you were right, his score would be 750. In this way we could give more credit to runners for today performances but in same time we would punish them only for bad runs <600 in previous 3-6 months period (and even then only in case of <10 runs in last 18 months) before they become Ranked and not for whole 18 months. I think this approach could be a little bit fairer for TOP and lower Ranked runners than what we have today.
So with this rule also 0 result or DNF (0 points; in case of MP and not in case of an injury) could be part of the avg. score for particular R runner. This would be a problem only for R runners who make 0 and DNF after or 6 months before they achieved their R status or >600 points (unranked runners) for the first time and would had less than 10 runs in last 18 months. I don’t have a problem with 0 results but for DNF I see two solutions. Same approach as for 0 results; runners with just a few races would need more time (races) to get rid of it and somehow this is understandable because runners with less races are less representative R runners. The other solution could be that all DNFs have expiration date, valid only for a fixed shorter time (3 months?). What could be other improvements of this idea?
I think that idea behind the scheme is very good but in last few years scheme doesn’t evolve into one of the more important IOF tools. So I have doubts what is real IOF intension to do with the scheme. I have got the feeling (also from other comments elsewhere) that IOF don’t care much about the future of the scheme. About my suggestions, I just made an extensive analysis and try to see where could lie the problems, how to see them from other perspective and to figure out possible solutions which could work better than today rules. If we agree on that than I am satisfied that also some other people and particularly someone like you agree that these solutions could be serious proposals and acceptable for the future development of the scheme.
your example
I didn’t test this idea on all possible examples but I appreciate your case as this is still more or less draft idea. I also wrote this part of the same statement….or maybe for the sake of representativeness also runs in 3-6 months before this date but not 18 months as says today rule.
And I added that his avg. score should be calculated based on last 8-10 best runs in last 18 months. Because Joe has only 6 runs his avg. score would be 650 and not 750. And this would be true only in case he achieved all his three 550 results in 3-6 months before he has achieved 750 for the first time. In case he achieved all 550 results 1 year ago then you were right, his score would be 750. In this way we could give more credit to runners for today performances but in same time we would punish them only for bad runs <600 in previous 3-6 months period (and even then only in case of <10 runs in last 18 months) before they become Ranked and not for whole 18 months. I think this approach could be a little bit fairer for TOP and lower Ranked runners than what we have today.
So with this rule also 0 result or DNF (0 points; in case of MP and not in case of an injury) could be part of the avg. score for particular R runner. This would be a problem only for R runners who make 0 and DNF after or 6 months before they achieved their R status or >600 points (unranked runners) for the first time and would had less than 10 runs in last 18 months. I don’t have a problem with 0 results but for DNF I see two solutions. Same approach as for 0 results; runners with just a few races would need more time (races) to get rid of it and somehow this is understandable because runners with less races are less representative R runners. The other solution could be that all DNFs have expiration date, valid only for a fixed shorter time (3 months?). What could be other improvements of this idea?
Jan 13, 2012 12:41 PM
#
@jgreen and @graeme
In that particular case, it seems clear that you ability would be better represented if the very low scores were discarded.
That is what I am saying. Nice case. With above rules you would have R status with =636,44 points (last four races). Before 16 Aug 2011 you were without R status (important) but achieved again more than >600 points. With including all races in 6 months period before 16 Aug 2011 only result from 14 Aug 2011 would be in equation. And with the same rules Mike would become unranked =544,83 points because of his 0 result.
It is a nonsense that WOC 2010 results are more important than your today performances. 18 months period is good to retain R status for some runners and regions to hold greater pool of R runners but we need also rules to have better representativeness of R runners within the pool. MP (mean points) of a single race should better represent today abilities of the runners.
I would say that Lundanes could afford to have DNF (miss punch) or 0 result (limping round injured) without risking to affect his personal avg. score. He has more than 10 races and he deserve to hold his high avg. score also after competing at small WRE or maybe with less successful performance. In your case I would say you can't afford it as you are less representative R runner. So where to put the line between representative/less representative R runner? For a start I would say 10 is quite reasonable.
I think this would be fairer, althought Mike should got this avg. score =544,83 already based on today rules.
In that particular case, it seems clear that you ability would be better represented if the very low scores were discarded.
That is what I am saying. Nice case. With above rules you would have R status with =636,44 points (last four races). Before 16 Aug 2011 you were without R status (important) but achieved again more than >600 points. With including all races in 6 months period before 16 Aug 2011 only result from 14 Aug 2011 would be in equation. And with the same rules Mike would become unranked =544,83 points because of his 0 result.
It is a nonsense that WOC 2010 results are more important than your today performances. 18 months period is good to retain R status for some runners and regions to hold greater pool of R runners but we need also rules to have better representativeness of R runners within the pool. MP (mean points) of a single race should better represent today abilities of the runners.
I would say that Lundanes could afford to have DNF (miss punch) or 0 result (limping round injured) without risking to affect his personal avg. score. He has more than 10 races and he deserve to hold his high avg. score also after competing at small WRE or maybe with less successful performance. In your case I would say you can't afford it as you are less representative R runner. So where to put the line between representative/less representative R runner? For a start I would say 10 is quite reasonable.
I think this would be fairer, althought Mike should got this avg. score =544,83 already based on today rules.
Jan 13, 2012 2:28 PM
#
@kofols That is what I am saying.
I don't think it is. I'm saying that the score should be discounted because Jeremy was injured, you're saying it should be discounted because it was long ago.
Obviously there must be some time limit on points. I think 18 months is about right because it includes this year and last year's main racing season, and that averaging over several races is better than just the most recent. But that's just an opinion.
Stepping way, way back, I'm trying to understand what the numbers mean. In particular, everyone's scores seem to be "about 1000", but that number doesn't appear in the specifications. http://iof.6prog.org/IOF_Documents/FootO/wrscheme.... The formula means that the average score will stay about the same, but where did it come from - is there a full explanation? I notice I wasn't always the world's best bad athlete , I got my 599 from some rescaling of Points
value in 2012.
n.b. in the UK ranking, which looks similar and uses the same formula, 1000 points is the "average runner", 1200 should be one standard deviation faster better etc. The top scorer averages 1350, quite a bit less than the WRE leader on 1516
I don't think it is. I'm saying that the score should be discounted because Jeremy was injured, you're saying it should be discounted because it was long ago.
Obviously there must be some time limit on points. I think 18 months is about right because it includes this year and last year's main racing season, and that averaging over several races is better than just the most recent. But that's just an opinion.
Stepping way, way back, I'm trying to understand what the numbers mean. In particular, everyone's scores seem to be "about 1000", but that number doesn't appear in the specifications. http://iof.6prog.org/IOF_Documents/FootO/wrscheme.... The formula means that the average score will stay about the same, but where did it come from - is there a full explanation? I notice I wasn't always the world's best bad athlete , I got my 599 from some rescaling of Points
value in 2012.
n.b. in the UK ranking, which looks similar and uses the same formula, 1000 points is the "average runner", 1200 should be one standard deviation faster better etc. The top scorer averages 1350, quite a bit less than the WRE leader on 1516
Jan 13, 2012 4:04 PM
#
I'm saying that the score should be discounted because Jeremy was injured, you're saying it should be discounted because it was long ago.
Yes but just because he would not be R runner in entire 18 months period. With the same case in 2011 his "injured - valid run" would be in my calculation. This "injured - valid run" is not a Ranking's problem. It is a runner's problem. Ranking system and rules should be clear and everyone should care about personal avg. score. In the past vast majority of runners didn't care about it because it was not so important. With elimination of weighting factors this will became more important. With these rules also avg. score, 0 runs, DNF and representative status could be resolved in much better ways. To understand which rules are fairer both rules should be tested on all possible cases but today we see that formulation of 18 months cause too much problems.
I think 18 months is about right because it includes this year and last year's main racing season....
For Lundanes is valid whole 18 months period. His avg. score should be based on ~10 best races out of all races in 18 months. I am saying that is not necessary that 18 months is valid for all runners equally. I think for the fairness of the system it is more important how we define R status and that we should extract other rules based on this rule.
..and that averaging over several races is better than just the most recent.
Yes, but as I said you also have runners like Garry Morrison, RSA who got R status based only on 1 race. How representative is he? So for some runners we use 1 race and for others all races. Take a look at previous posts.
In particular, everyone's scores seem to be "about 1000", but that number doesn't appear in the specifications.
You were looking into wrong link. It is here. Rule 2.12. All scores are rebased only once a year. Sometimes I also don't know which rules are still valid and which aren't. Many rules are out of date or they are not even written anywhere at all. FOC use also some hidden rules. Some of them have effect on Rankings so it is normal that we all see WRL more like a fun Rankings than true Elite Rankings with certain degree of respect.
The top scorer averages 1350, quite a bit less than the WRE leader on 1516
Yes, one of the problem with 1year rescaling. You were looking at points (which include also IPs). For comparison you should look at Highest Scoring Runs in 2011.
Yes but just because he would not be R runner in entire 18 months period. With the same case in 2011 his "injured - valid run" would be in my calculation. This "injured - valid run" is not a Ranking's problem. It is a runner's problem. Ranking system and rules should be clear and everyone should care about personal avg. score. In the past vast majority of runners didn't care about it because it was not so important. With elimination of weighting factors this will became more important. With these rules also avg. score, 0 runs, DNF and representative status could be resolved in much better ways. To understand which rules are fairer both rules should be tested on all possible cases but today we see that formulation of 18 months cause too much problems.
I think 18 months is about right because it includes this year and last year's main racing season....
For Lundanes is valid whole 18 months period. His avg. score should be based on ~10 best races out of all races in 18 months. I am saying that is not necessary that 18 months is valid for all runners equally. I think for the fairness of the system it is more important how we define R status and that we should extract other rules based on this rule.
..and that averaging over several races is better than just the most recent.
Yes, but as I said you also have runners like Garry Morrison, RSA who got R status based only on 1 race. How representative is he? So for some runners we use 1 race and for others all races. Take a look at previous posts.
In particular, everyone's scores seem to be "about 1000", but that number doesn't appear in the specifications.
You were looking into wrong link. It is here. Rule 2.12. All scores are rebased only once a year. Sometimes I also don't know which rules are still valid and which aren't. Many rules are out of date or they are not even written anywhere at all. FOC use also some hidden rules. Some of them have effect on Rankings so it is normal that we all see WRL more like a fun Rankings than true Elite Rankings with certain degree of respect.
The top scorer averages 1350, quite a bit less than the WRE leader on 1516
Yes, one of the problem with 1year rescaling. You were looking at points (which include also IPs). For comparison you should look at Highest Scoring Runs in 2011.
Jan 16, 2012 7:48 PM
#
What is also interesting about these small WREs is how accurate small event formula really is.>> RP = 2600 - RT x (2600 - MP)/MT
Results of SA event tell me that Lundanes & Taivainen underperformed and Haines and Crone outperformed to their Ranking. Did Lundanes really underperform relative to others and how much?
An example:
Assumption is that we have one TOP runner and two Ranked runners and both have a race time = winner’s time +50%.
3 R runners
Times (60-90-90 min); MT = 4800; SP = 1039,230
Avg. Points (1367,73-600-600);MP = 855,91; SP = 443,2491
So how many points would TOP runner score in this extreme example with large and small event formula? How good should be his run to retain his avg. score?
Large formula:
With Large Event formula: RP = (MP + SP x (MT- RT)/ST) his score would be exactly the same as his avg. score = 1367,73 points! This is also the Points limit because if he performs even better then both other runners would lose their R status because time difference would be over +50%. So even in extreme situations TOP runner can at least retain his avg. score based on large event formula.
Small formula:
RP = 2600 - 3600 x (2600 – 855,91)/x = 1367,73
x = MT = 5095,25
RP = 2600 - 3600 x (2600 – 855,91)/4800 = 1291,93
If runner wants to retain his avg. score, MT must be 5095,25. But this is impossible because MT can’t be higher than 4800. Higher MT would mean that not all runners retain R status (winner’s time +50%).
Unfortunately small event formula predicts that at small WREs we have more less representative R runners and more chances that someone performs differently to his ranking. So formula doesn’t allow him to retain his avg. score with just a normal relative run to others.
My prediction was that Lundanes should be able at least to retain his avg. score also when small event formula is used. In reality this would be possible only if he would slightly outperform compare to other runners. A little bit unfair.
Results of SA event tell me that Lundanes & Taivainen underperformed and Haines and Crone outperformed to their Ranking. Did Lundanes really underperform relative to others and how much?
An example:
Assumption is that we have one TOP runner and two Ranked runners and both have a race time = winner’s time +50%.
3 R runners
Times (60-90-90 min); MT = 4800; SP = 1039,230
Avg. Points (1367,73-600-600);MP = 855,91; SP = 443,2491
So how many points would TOP runner score in this extreme example with large and small event formula? How good should be his run to retain his avg. score?
Large formula:
With Large Event formula: RP = (MP + SP x (MT- RT)/ST) his score would be exactly the same as his avg. score = 1367,73 points! This is also the Points limit because if he performs even better then both other runners would lose their R status because time difference would be over +50%. So even in extreme situations TOP runner can at least retain his avg. score based on large event formula.
Small formula:
RP = 2600 - 3600 x (2600 – 855,91)/x = 1367,73
x = MT = 5095,25
RP = 2600 - 3600 x (2600 – 855,91)/4800 = 1291,93
If runner wants to retain his avg. score, MT must be 5095,25. But this is impossible because MT can’t be higher than 4800. Higher MT would mean that not all runners retain R status (winner’s time +50%).
Unfortunately small event formula predicts that at small WREs we have more less representative R runners and more chances that someone performs differently to his ranking. So formula doesn’t allow him to retain his avg. score with just a normal relative run to others.
My prediction was that Lundanes should be able at least to retain his avg. score also when small event formula is used. In reality this would be possible only if he would slightly outperform compare to other runners. A little bit unfair.
Jan 17, 2012 9:04 AM
#
Tweaking small event formula to give more points might open door to get excessive points from small WR event. I'd see it's better if you get the most points when all the big guns are around, not in small event with just one or two elite runners around.
I's see better fix would be something like changing the way average points are calculated. Like we could calculate four different scores: 0.85 x best race, 0.90 x average of two best, 0.95 x average of three best and 1 x average of four best races. And best of those would be your score. Like this running a smaller race would never do no harm, you would not need to travel to four races to gets somewhat reasonable overall ranking position, just one would do. And one bad race for being sick/injured or something would never make your ranking worse. Now you need to have four good result before you get that advantage.
I's see better fix would be something like changing the way average points are calculated. Like we could calculate four different scores: 0.85 x best race, 0.90 x average of two best, 0.95 x average of three best and 1 x average of four best races. And best of those would be your score. Like this running a smaller race would never do no harm, you would not need to travel to four races to gets somewhat reasonable overall ranking position, just one would do. And one bad race for being sick/injured or something would never make your ranking worse. Now you need to have four good result before you get that advantage.
Jan 19, 2012 4:00 PM
#
@Jagge
Interesting idea. 18 months avg. would be in each case a compromise between good/bad runs and all anomalies in stats based system. I would say that historical data should be used somehow to get these factors. Also using only 4 runs for avg. might be not sufficient enough to have "good" average. But using this system also for overall ranking is something different. I think we speak here about two very different things. I made my logic how I see possible changes into overall ranking and consequently to Federation League Table.
I realized that my idea of 10 best races is also not good enough. It works only in case when bad runners become good compare to their last season. I was thinking to slighty change it into 5 last races + 5 best of all other races in last 18 months.
One of the solution could be also using a some kind of "factors" and 3 separately averages based on 3 different WRE formulas to get valid average score.
The obviously problem is that many runners in Europe and particularly out of Europe have very little chances to run enough WRE races to achieve "representative" 18 months average no matter how we decide to calculate 18 months average. More WRE races per country is probably not the right solution but I see a chance that in addition we could use also all national champs (SML) from all IOF countries as valid races only for 18 months average. We must acknowledge to ourselves that elite compete at these events and we could use these races to get more accurate information about their current average value. It may help mainly less developed nations and regions to get more R runners in shorter time.
Interesting idea. 18 months avg. would be in each case a compromise between good/bad runs and all anomalies in stats based system. I would say that historical data should be used somehow to get these factors. Also using only 4 runs for avg. might be not sufficient enough to have "good" average. But using this system also for overall ranking is something different. I think we speak here about two very different things. I made my logic how I see possible changes into overall ranking and consequently to Federation League Table.
I realized that my idea of 10 best races is also not good enough. It works only in case when bad runners become good compare to their last season. I was thinking to slighty change it into 5 last races + 5 best of all other races in last 18 months.
One of the solution could be also using a some kind of "factors" and 3 separately averages based on 3 different WRE formulas to get valid average score.
The obviously problem is that many runners in Europe and particularly out of Europe have very little chances to run enough WRE races to achieve "representative" 18 months average no matter how we decide to calculate 18 months average. More WRE races per country is probably not the right solution but I see a chance that in addition we could use also all national champs (SML) from all IOF countries as valid races only for 18 months average. We must acknowledge to ourselves that elite compete at these events and we could use these races to get more accurate information about their current average value. It may help mainly less developed nations and regions to get more R runners in shorter time.
Jan 19, 2012 4:01 PM
#
More WRE races per country is probably not the right solution
Not here. We put them on, they don't come.
Not here. We put them on, they don't come.
Jan 31, 2012 9:12 AM
#
Jagge>> Take a look in old Ski-O WRE rules (2.11a and 2.11b). It seems that your idea has been in force for some time in SKI-O.
This discussion thread is closed.