Register | Login
Attackpoint - performance and training tools for orienteering athletes

Discussion: Strange lidar problem

in: Orienteering; General

Oct 2, 2020 4:24 PM # 
Canadian:
I have a strange lidar problem with a new dataset that I can't make sense of and I'm wondering if anyone here can point me in the right direction.

The problem is that I can't get either OCAD nor Karttapullautin to generate contours. I've tried multiple different tiles and partial tiles and always get the same result...

OCAD runs just fine and it says all the points are there including several million ground points but the end result is 0 contour objects, a solid grey hill shade and white slope image. The veg height image is solid yellow. The intensity and classification raster images are what one would normally expect.

KP gives the following messages:

Preparing input file
.......... done.
Knoll detection part 1
.. done.
Knoll detection part 2
..Illegal modulus zero at script/pullautin.pl line 6163.

Contour generation part 1
.................... done.
Contour generation part 2
.Illegal division by zero at script/pullautin.pl line 4982.

Contour generation part 3
.................... done.
Contour generation part 4
.................... done.
Vegetation generation
....


In the hopes that someone is willing to take a look at the data I've uploaded two tiles to my google drive here: https://drive.google.com/drive/folders/1IiSB3d2nOo...

The entire dataset (100+ sq km - 134 tiles) is downloadable here: https://opendata.nfis.org/mapserver/PRF.html

Many thanks!
Advertisement  
Oct 2, 2020 5:17 PM # 
Jagge:
all ground points has elevation value 0.0. So it is entirely flat and even.

I think data is pre-processed for some kind of vegetation height analysis, elevation (z) for all points is relative to the ground, not sea level. So you can't use these files to calculate contours.

open the laz file with lasview and tilt it in 3D, it is entirely flat.
Oct 2, 2020 5:48 PM # 
Canadian:
Thanks Jagge.

That would obviously do it. They have a dtm available but it's been processed to 1 point /m2 so I was hoping I wouldn't have to use it directly.
Oct 2, 2020 8:42 PM # 
gordhun:
Jeff, I hope you get it worked out. I'm told that is an area with great potential as an orienteering venue.
I tried a couple of things with OCAD 2020 Dem Import Wizard . First: those tile areas are not side by side, are they?
The first time I tried the DEM wizard I got contour lines but they were just a jumble - I had chosen a 1 m contour interval and chsoen to clasify vegetation height.
The second time choosing just to get hill shading and contours with a 2.5m interval I got nothing but grey hill shading, same as you. The good thing is that it took virtually no time to process.
The third time besides ground contours I had asked for a vegetation base map. I got a jumble of contours that seem to represent the height of the vegetation, typically 7, 8 and 9 metres. Also almost a solid mass of spot elevations again probably for individual trees and small clusters of trees.
Seems to make sense that thy have processed the LiDAR relative to vegetation. It is after all a forestry research station. But can the data not be reprocessed for the true ground elevation?
Oct 3, 2020 7:58 AM # 
robplow:
It seems like a case where you want to try to get in touch with the owners of the data and ask if they have the original data, before it was processed into whatever it is now.

For some areas in MB I could get DTM's from the web, but to get the all points laz data I had to actually contact someone. They were quite happy to provide it.
Oct 3, 2020 10:26 AM # 
Canadian:
Thanks all, I've sent an email to a co tact listed on the site and will see what kind of a response I get.

Gord, those two tiles were chosen at random and are not side by side.
Oct 3, 2020 4:57 PM # 
robplow:
Using the OCAD DEM wizard settings below I was able to generate what looks like a good vegetation height image. The vegetation base map image was not so good. Click on the image to see the whole thing.



Oct 3, 2020 5:02 PM # 
robplow:
vegetation height
Oct 3, 2020 5:05 PM # 
robplow:
vegetation base map - seems to me it works in open areas but not under tree cover. Only first returns - no returns under the canopy to work out vegetation density

Oct 3, 2020 9:11 PM # 
Terje Mathisen:
This could even be a first return only, -replace_z, i.e. vegetation height file, with no intermediate returns. In either case you really need to get hold of the original scan.
Oct 4, 2020 1:10 AM # 
GuyO:
Is this near Ottawa?
Oct 4, 2020 11:45 AM # 
gordhun:
Northwest of Ottawa some 160 km / 100 miles between towns of Petawawa and Chalk River.
Oct 4, 2020 12:45 PM # 
Hammer:
The Petawawa Research Forest is over 100 years old and a world famous experimental forest for ecology, wildfire behaviour, and forestry.
Oct 4, 2020 12:58 PM # 
robplow:
Worst case - if you can't get the original data you could still make a pretty good base map with veg height from OCAD and contours from the DTM.
Oct 4, 2020 1:01 PM # 
robplow:
contours from the DTM using QGIS 0.5 and 2.5m

Oct 5, 2020 6:52 PM # 
Canadian:
Thanks all,

To be clear, I know I can use the DTM to generate contours. I thought I would take advantage of the huge amount of data to experiment with some new (to me) techniques for extracting maximum information from the lidar. Not having the full dataset in the laz files doesn't allow me to do that so unless I hear back from someone saying they can get me the full dataset I'm putting this project aside for now.

Hopefully some day in the future this area will be mapped for orienteering but, as far as I know, it's not on anyone's immediate to do list.
Oct 13, 2020 5:24 PM # 
cedarcreek:
I've had good luck with a kinda roundabout way to get the nice KP contour tweaking from a DEM (the ground DEM = DTM?), that is, I've used KP with the DEM as an input. I'd suggest it for lidar-derived DEMs, but not low-res DEMs, although you can certainly try it.

I use this command on the DEM (here a .tif) to create a uniform grid LAZ format file of ground points:

demzip -i *.tif -o test.laz -set_classification 2

Then I run that LAZ file through KP.

I'd suggest trying both a normal contour generating process as well as KP. But on the two times I've done this, the KP contours seemed like a better starting point for the mapper. But test it yourself.

I use my normal CRT file, which is a text file with the lines below. In Windows Notepad, I change the "save as TXT" to "All files", or you might get a .CRT.TXT file extension. I save this as "KP_out2_formlines_dot_knolls_dxfs.crt", and the filename reminds me to select the "out2.dxf", "formlines.dxf" (if you asked for formlines), and the "dot_knolls.dxf". Here are the lines for that CRT file:

115.000 udepression
206.000 uglydotknoll
112.000 dotknoll
313.000 uglyudepression
103.000 formline
101.000 contour
705.000 depression_intermed
0.0 contour_intermed
102.000 contour_index
707.000 depression_index
704.000 depression

{Edit: If you tiled the KP input and merged the tiled output, the out2.dxf file becomes "contours.dxf" or "merged_contours.dxf".}

Verify your symbol set has numbers that match mine, or update the symbol numbers to match your map symbols.

You can consider not importing the dot_knolls.dxf, which contains depressions, dot knolls, and ugly depressions and ugly dot knolls (which means the contour probably needs to be cut). I import the ugly ones with black boulders and blue U-depressions, and it's often not very good. Especially for a 1m or 2m grid or whatever grid dimension your DEM has, it's probably okay to not import them.

Oh---After you import, ideally in to a clean, new map, hide all the symbols and delete everything that wasn't converted. In settings in OCAD, you can change the color of unconverted symbols to something like 100% magenta, so you can see them. The old default was gray, and it was very easy to miss them. I click "entire map", zoom out one click, and select almost the whole screen when I delete the unconverted symbols.

If you set a smaller basemap interval, like 1.25m, those all import as a single symbol from basemap.dxf. I often use a different color than brown for these, and open them as a template (background map) so the symbols don't mix with the final map.

I'm not sure if this import process works identically in OOMapper. I use OCAD for importing the KP vector contours. I think it can be done in OOMapper.

Please login to add a message.