Discussion: Karttapullautin Tutorial

Is there a good step by step tutorial for using Karttapullautin? On the OUSA website I found the Lidar Mapping Guide and Resources document that seems to be pretty good. I've searched YouTube and found a couple KP videos with not really any explanation or details. I guess what I'm looking for is:
1) What are Lidar data files and which ones do I need (DSM, DTM, Metadata, Point Cloud, etc.)?
2) How do I get those lidar files ready for KP? How do tiles work etc.?
3) Using KP outputs for base maps and training purposes.

I have yet to find any comprehensive, excellent tutorial series for this, but Greg Wilson's writeup from 5 years ago is still the best I have found. https://medium.com/@somegreg/orienteering-mapping-...

Short answers to your specific questions...
1) .las or .laz (the same data as .las, except in a compressed format) files are the only file types that you want to get your LidAR in. These files contain the raw LiDAR point cloud that you can then use to process out whatever you want (like the Digital Surface Model (DSM), Digital Terrain Model (DTM), etc.). The Metadata file for your .las and .laz is also very helpful to have so you know the projection and units or your LiDAR - but this can get very messy very quickly.
*Note that some jurisdictions will only provide ground points only LiDAR (still as a .las or .laz), which will mean you won't get any vegetation information just contours - instead get the "full point cloud" version (still as a .las or .laz).

2) LiDAR files are really, really big. All that tiles are, is that instead of taking the entire LiDAR data collection (that could quite literally be terabytes in size), whoever is distributin the LiDAR breaks it up into "tiles" of more manageable sizes. Just get the tiles for the area that you need. You shouldn't* need to do any conversions of your .las or .laz files to make them work with Karttapullautin (KP).
*Although I think you are based somewhere in the states, so depending on where you get your LiDAR from, the units embedded into the LiDAR file might be in feet, not meters, which will wreck havoc on the scale when KP tries to process it.

3) Well ... OpenOrienteeringMapper and time are your friend. The map product strait out of KP will be totally usable for training (sort of), but if you import it into OOMapper, then the more time you spend improving the map (drawing trails, buildings, water features, etc. that KP can't directly map [other than sort of based on OpenStreetMap data, but it is strongly recommended to do this processing in mapper instead]) the better your map will be.

---

If this isn't enough to help get you started, or if you run into very specific problems a little bit down the line, I could probably do a Skype/Zoom/Teams call and help you out.

I remember finding this useful:

https://github.com/jgaffuri/OriMap/tree/master/doc...

It is written for use of Luxembourg free lidar but applicable more broadly

Thanks for the responses. Thanks robplow for that link. That looks like one of the most helpful resources that I've seen.

I do have a few follow on questions/comments.
1) I'm assuming the point cloud data takes longer to process? Would it be easier for KP to crunch data that is already processed to DSM and DTM? FYI, I'm located in WA state so have access to the WA State Lidar Portal.
2) If that lidar data is in feet, is there a method to scale it before crunching in KP? My question about tiles was more in reference to merging them (should have been more specific). I think the GitHub link above covers how to merge the tiles.
3) I've used OOM in the past and am ok at it. I've made a few basic small maps for training, but never tried starting a map with lidar. I reviewed the British Orienteering Mapping webinars Parts 1&2 on YouTube and found them pretty informative. My goal is to get some base maps started in my area and it looks like there is lidar available for many of the areas I'm interested in. If I can do all the right things with getting the base maps generated (georeferenced, magnetic declination, good templates, etc.) then that's a start.

Really appreciate the help and it's always good to learn new things. Thanks.

I seem to recall I got that link Spike a couple of years ago when I posted a similar request for help on AP.

Would it be easier for KP to crunch data that is already processed to DSM and DTM?

No. Jagge can explain way better than me but with just DTM (last returns - ie ground points only) and DSM (first returns - ie the top of the vegetation or buildings etc) all you can do is calculate the height of the vegetation. As I understand it KP uses returns in the 0-2m range (which have been discarded when you create DSM and DTM) to estimate the density of the vegetation (OCAD vegetation base map does something similar I guess). You need the full point cloud for that.

I usually process KP in batches of up to 30 km2 at a time. Depends on how big the las/laz files are but that usually takes 6-8 hours. The bigger files take longer obviously but they give better results - so see it as a positive - what you get is worth waiting for.

I tend to set KP up for batch processing and let it run overnight - otherwise it slows the PC down for everything else. Or I run KP on my newer PC and do other stuff on my old one.

But all of this depends on what you are trying to achieve. I use KP to get an idea of what the terrain is like - especially the vegetation. But when I want a base map for fieldwork I just use OCAD. For a forest map usually I create 1m contours (0.5 in flatter areas) and a vegetation height image. Usually that is enough to be able identify where I am most of the time. I don't need the auto-generated cliffs - if I have a small enough contour interval they are obvious from that.

If you are trying to create training maps without having to do fieldwork then KP is great, but for base maps for fieldwork I prefer the usual OCAD functions. Certainly for fieldwork I prefer the unsmoothed contours in OCAD vs the generalized contours that KP produces. I will do the generalizing myself.

I don't know what OOM is capable of with lidar.

I've created a lot of base maps over the last 8-9 years, including both JWOC2015 and WOC2019, today I am putting the finishing touches on the Norwegian Ultra champs 2024 area (about 50 sq km).

My usual process consists of first locating and extracting the relevant LAZ tiles (from https://hoydedata.no/, but I have already downloaded all of them for https://mapant.no/ so I can skip that part), then I run my own LAStools-based pipeline, which includes a custom vegetation recognizer which is quite a bit more sophisticated than either KP or OCAD. At this point I have raw 1m contours, slope and cliff images and vegetation objects.

Next I strip the lidar tile buffers and send them through OCAD's smooth contour generation process, before I run another custom program to convert Norwegian vector topo data into OCAD objects.
BTW, all the outputs I generate are in the form of DXF files which are compatible with both OOM and OCAD.

Right, as robplow wrote with DSM + DTM points needed for vegetation density is missing. If you need to use DSM + DTM as source for KP, you can do it by converting them to laz, DSM points as ground and DTM points as something else, like vegetation.

Note, KP makes any interval contours you want (as a side product dxf), I usually make 0.625 contours (or 1.25). Ini file parameter "basemapinterval". In batch mode it make seamless contours across tile edges. contours are raw & unsmoothed. By default that parameter value is set to zero (feature turned off).

Hey Alex, I've already spent a couple hours and worked out a simple pipeline and prepped basemaps for Chamna and WE Johnson - I can share that later this month. It's obviously not a good time for field checking here, so I haven't pushed further. But I expect KP should produce almost ready-to-use products for the Rattlesnake foothills (once you overlay roads from an alternative source)

Hi Jagge, thanks for pointing out the basemapinterval parameter. I had not previously noticed that option. Does this result in a second set of generated files? I haven't tried opening DXF files into OOM, will that generally work?

I would note, as Alex mentioned, that here in the US, we also need a preliminary step to massage the input LAS data from feet into meters. I found a method to do that using LASTools las2las. I'll have to share that later.

blegg about feet to meters I strongly suggest you look at whether your area is in the area covered by NOAA. Their program lets you select whether you want the download in State Pane or UTM or one other I do not know and whether you wat the horizontal and vertical distances in feet or meters. With that you can skip the LASTools steps. Believe me, you can.

Thanks blegg, wasn't sure if you had availability so I was thinking about giving this a go to learn the process. I'll be in touch.

blegg, it writes out one additional dxf file per tile to output folder. All other files and images are as usual, so it doesn't render them to png files or anything.

Thanks Jagge, that makes sense. I'll look forward to trying it. The next thing I will need to learn is how to optimize the vegetation classification for our unique area.

Gordhun - good to know about NOAA. The LAStools trick seems to be working quite well for me right now but I'm always glad to hear about other options.

FrankTheTank - I started working on the maps for some of these places years ago, so I'm glad that they might get some use now. It's a good time for this because the local LIDAR availability has dramatically increased in the last 5 years.

OCAD has a check box to convert feet to metres so no need to convert the data using lastools etc. But that doesn't help if you are using KP.

Isn't there also a problem in the US with two different standards for the foot - survey foot and normal foot?

My laptop w/ OCAD is currently dead, but I thought it converted just feet to meters just in height. Did a planar option get added?

The nice thing about the survey foot versus normal foot problem is, that if you run las2las with the wrong one since the metadata is garbage and doesn't clearly specify, when you try to import into say ocad with the open mapper roads/trails already in the right spot, it'll be grossly wrong, and then you know and use the other one. It's fun when your state has a mix, even in the same sort of county size area.

@gord - NOAA downloads do not always work properly for KP purposes. Just downloaded files for West IL, they worked just fine in OCAD, but do not work in KP, NOAA stripped something out in the conversion to UTM, meter and vertical meter. Downloaded the files from https://apps.nationalmap.gov/downloader, did the LAStools conversion, and they worked just fine in KP.

Alex, I generally do feet to meter conversion for elevation in KP.
In NH, lidar for different areas will have different units for grid and/or elevation, depending on which river drainage run (data collection)they were flown for. I convert and merge tiles with a state plane grid in feet or survey feet to UTM in meters using las2las. For some reason that I haven't spent the time to figure out, when I try to simultaneously also convert elevation from feet to meters, it sometimes work and sometimes doesn't.

Then in KP, I take the generic pullauta.ini file and edit that to have the Z coordinate do the transformation - this is about 2/3 of the way down the .ini text file.

# xyz factors, for feet to meter conversion (0.3048 for feet, 0.3048006096012 - survey feet) etc.
coordxfactor=1
coordyfactor=1
coordzfactor=0.3048006096012

( I keep a copy of this with the feet and survey feet factors in a handy place - although the difference is not material for any map that I've done. Just enter the right z factor (1 for the one lidar data collection in NH that is already in meters for elevation.)

I also adjust the angle for the north line in this file as well, a couple of dozen lines up. It's either -12 or -13 degrees in UTM zone 19 or -16 in UTM zone 18 for the areas I generally work in.

WIth this, my KP outputs align well with what I do in OCAD - can use either system as background files for the other.

@jima - you may want to run lasinfo on one of the downloaded tiles first and see what it says. I have run into having height in different dimension from x and y, even if not downloaded from NOAA.
The other odd thing is that if you download from NOAA, if you chose to download in UTM, then it has the download default for x and y in meter, but not for z, that default is feet - go figure - frustrating when one notices one forgot to change that only after the download is complete.

Now that I have a proper break from work, this is how I did the conversion.

My local datasets are all in feet, both horizontal and vertical, and referenced to NAD83. I had thought about just writing a python script to scale all the numbers, but of course that would have broken the georeferencing and neighboring tiles might stop lining up properly. What ended up working was using las2las to convert the datum from NAD83 to WGS84. (I figured out my NAD83 situation by the method Andreais described with lasinfo)

My reasoning was that NAD83 is a foot-based datum and WGS84 is a meter-based datum. (My local WGS84 Zone 11 is represented by epsg code 32611) To be extra safe, I used the -target_meter and -target_elevation_meter to be sure the conversion happened. So the line of looks like this:

las2las -cpu64 -i C:\InputFolder\*.laz -target_epsg 32611 -target_meter -target_elevation_meter -odir C:\OutputFolder\

I'm not sure if the conversion from NAD83 to WGS84 is strictly necessary, maybe I could have just converted to meters and stayed in NAD83. But regardless, it worked. All the tiles stayed properly co-registered when I imported to OOM, the output was KP compatible, and it runs fast, just a couple minutes.

Slightly off topic, but those who know how to handle/convert laz files and use KP can easily learn how to do 3D visualizations like the video one I posted to my log yesterday. Let me know if some of you likes to try it out.

Yes please. Nice visualization: https://youtu.be/drNuI5DvNbs

Frank the OP may, or may not, find useful the Karttapullautin stuff in the second post from the bottom on the first page of this thread;

http://forum.nopesport.com/viewtopic.php?f=1&t...