The IOF Foot O' Rules (
384kb pdf) have a lot of rules for World Cups, WREs, WOC, and JWOC.
On page 37 it says:
A mixture of technical [control] difficulties
Small and medium scale route choice
[Type of Running] High speed, often in close proximity to other runners who may, or may not, have the same controls to visit.
[Terrain with] Some route choice possibilities and reasonably complex terrain.
[Map Scale] 1:10000 (or sometimes 1:15000)
Mass Start so the finish order is the order across the line.
Winning Times (for Senior Elite): 30-60 minutes per leg. Men Total 135 minutes. Women Total 120 minutes. [3 runners per team]
Summary: Relay orienteering is a competition for teams of three runners running on a virtually head-to-head basis with a first-past-the-post winner. Exciting for spectators and competitors.
It has more to say about relays starting on page 39.
I've set 2 relays. Each had about 160 runners in 40 teams, but the normal US rules for 4, 8, and 12 point teams. (So about 10 4-pt, 20 8-pt, and 10 12-pt). I wouldn't say I'm a good relay setter, but I have made a lot of mistakes setting relays.
Unless you have a *lot* of runners, really work to keep the runners together. If you have multiple categories, try to keep the slower, shorter legs matched to the longer, faster legs so the fast runners are overtaking the slower runners (who have run less distance). Have the threat of forking more than a big forking diagram.
While the main goal (for me) is to keep runners together, you also need to give the leaders chances to break away cleanly. If you are blessed with flat, open forests, it's easy for the followers to see the runner for a long distance ahead. So look for opportunities to limit visibility either by green or by terrain for the leader to escape. {Edit: When the conditions are right,} Avoid doglegs and "situations short of a dogleg" where the follower can see the leader leave the control.
If you have a winning time requirement, shoot for the low end of it (or just ignore the rule, set it shorter, and apologize when someone complains).
Don't fork early in the mass start or late in the final leg.
Have the runners pass the event center 3-10 minutes before they finish to act as notification to the next runner (rather than requiring a radio link and a PA and placing this responsibility on the organizers).
Since the first leg finishers can talk with their team but *not* the second leg runners, the second leg can be identical to the first. (This is actually more general to each N and N+1 leg.) Starting with the third leg, you should consider changing some aspect of the course. You should definitely save some surprises for the actual finish such as an additional finish loop (twice by the finish rather than once).
Do make up starts early and often (if your scoring software allows it). If you only have 1 make up start, and enough time, consider making the mass start 5 or 10 minutes after the 2nd or 3rd place team finishes. That way everyone can see who won and experience some of that head-to-head excitement.
With small fields, a relay gets spread out and is not that much different from a regular event with single runners finishing alone---So consider a billygoat instead. Look at the US Champs format---At any time before the make up start there are about 40 people on the course (exactly 1 person per team). After the first leg or so, these 40 people are spread over maybe 5km. That's a spacing of 125m. If you look at the 10 4-pt teams, that average spacing is 500m. Neither of those sounds exciting.