Discussion: LiDAR data for download?

Any hints or links where one can download free Lidar data for testing purposes? It doesn't matter where is it from. Plan is to test drive automatic O map generation application.

Something like this site I already know: http://giswww.westchestergov.com/gismap/

As an example, here one of the test maps I have made using Westchester data:
https://picasaweb.google.com/lh/photo/bt1tw9k5SPBy...

I have been doing most of the development with Finnish data, auto generated maps look like this (app can use Finnish terrain database data too):
https://picasaweb.google.com/lh/photo/4iC8b9uNwuWt... (Jukola 2012 area)

http://lidar.cr.usgs.gov/LIDAR_Viewer/

This tells you where free and non-free data is available, but you'll have to go down the instruction tree to actually get the data.

Jagge, there's a nice set here covering the area recently used for the NA rogaine champs. Its very high sample density (8/m^2) and the forest is mixed from heavy conifer cover to open. Its part of the Puget Sound lidar collection, there are lots of other datasets covering that region of the US. Some of these are available through the CLICK link above, but not all.

Request a username and password here to access the Wenas data at the link above.

Thanks. That opentopo.sdsc... site is perfect. Here is couple of clips I my laptop did last night:
https://picasaweb.google.com/lh/photo/4DpuvmUjTbFu...
https://picasaweb.google.com/lh/photo/AgV_MPLNL6Xk...

Not quite like Helsinki parks. These two gives me already plenty of issues to think about.

Jagge, I really like the maps you generated. I've been waiting for Lidar data to be complete in my area- but I have no skills in turning Lidar data into a usable map. How "automatic" is this application that you are developing, and is your intent to make it generally available? Very impressive.

Damn, we should've had that first map a year ago..

Jagge's automatic map generator (I guess it doesn't have an english name yet?) Karttapullautin is already available for everyone:
http://www.routegadget.net/karttapullautin/

And as a person who knows maybe average amount about computers, I can say it's rather easy to use. Depending on what sized and in what format your lidar-data is. In best case, it's drag and drop. And in worst case, some tweaking with lastools to get the area cropped to as little as possible (too big and you're out of memory) and to correct format.

You drag and drop your las/laz file on the tool and just wait.
http://www.youtube.com/watch?v=V2NjyAqRndc

How large a tile is a reasonable tile? Our data tends to come in 1/16th of a quad, i.e. 112.5 × 112.5 arc-seconds (~= 2.8 × 3.5 km). With something like 1 meter post spacing.

The largest clip I have done was 3 x 9 km = 27 km². There was 0.6 points per m2. It took several hours, four or something. Old versions failed sometimes with 3x3 km tiles as Hirppa wrote.

Ok, bear with me here, I'm trying to learn this stuff.

One of my first projects is to map this, probably at 1:2000, which is close to my in-laws' lake house. The area has LIDAR from this project.

Data:
I think the data I need is in the two following: 47120F2123.laz and 47120F2124.laz. However, I don't seem to be able to download it. Broken link?

Software:
What's the difference between the standalone version and the OL-Laser version? I downloaded the standalone one. I think I've got the las2txt.exe file (from a separate download) into the right spot, but since I can't get the right data, I can't finish the job.

The software output is a bitmap file? What about vectoc/OCAD?

Pullautin outputs bitmap with contours, knolls, depressions, clifss, yellow and green. It also generates temp folder, from where you find dxf-files: out2, c1g, c2g, c3g, there is also a bitmap called vegetation.png.

Importing those dxf-files to ocad (and converting each layer to appropriate symbol) and opening the vegetation as background map gives you in Ocad a view that is similar to pullautus.png.

Hope i didn't forget any dxf-files from the list above.

I have made a .crt conversion file for ocad that includes also the pullautin layers, but I haven't yet uploaded the latest version online yet (my current online crt file only includes finnish land survey database bindings to isom).

I had an interesting learning experience tonight, and I'm not sure I've done the right thing, but this is what happened:

This command, specifying the input plane as OH_N and the output as UTM 17N does convert to meters:

las2las -i *.las -sp83 OH_N -feet -elevation_feet -target_utm 17N (I think the default utm units are meters.)

However, the original tiles were square, and the lidar points are square in the state plane, but in UTM, they're skewed, so it will be hard to work with in OCAD. So, I decided to keep the data in the state plane, but convert it to meters.

This conversion apparently just changes the header from feet to meters, but doesn't convert the points---everything is still in feet:

las2las -i *.las -feet -elevation_feet -target_meter -target_elevation_meter

The only way I could get it to convert feet to meters (and not just change the header) was to type this, converting *from* the state plane *into* the state plane:

las2las -i *.las -sp83 OH_N -feet -elevation_feet -target_sp83 OH_N -target_meter -target_elevation_meter

I'm sure this is probably a dumb thing to do, but I've got OCAD8 and I figure this will work.

I tried Pullautin with one of the UTM tiles, but there are wedge-shaped areas along each edge with no data, and I think that caused it to crash. I'll try it again with one of the state plane metric tiles and see if it works.

I'm currently waiting on the OL-Laser batch processing to finish. I'm doing 12 tiles in Toledo and (fingers crossed) it seems to be working really well. This is my first time figuring out the batch processing, and it was completely painless (knock on wood).

My attempt with some PSLC data didn't work so well. It was thinking all day, but it didn't spit out an image file when it was done, unfortunately.

I've had trouble with OL-Laser when using feet rather than meters, particularly with the cliff detection. I'd recommend to first make sure your input file is metric, both in the xy plane as well as for elevation. Just a guess.

{edit: I will go out-on-a-limb and guess that Karttapullautin will also not like feet.}

Make sure las3txt pullauta.exe and your las/las file are all in same folder. I haven't yet finds any files making it crash other than being too big. But it will msot likely crash if point cloud is unclassified (includes no points classified as ground).

The ol laser version is older one, it just polishes OL Laser's output. Not automatic, there is about 20 manual steps in the work flow.

Karttapullautin doesnt know is numbes feet or meters. It reads xyz file and there is only numbers there. If inut is las/laz it uses las2txt to xyz first and continues form there.

And note, Karttapullauin is not designed to be used for making base map for mapping. It is designed to make something usable directly for training purposes. For mapping you may want to have 1m contrours and stuff (something you dont want to train with). The trick is Pullautin doesn't draw mathematical controus, instead contour elevations drift up and down to hit pre-detected land forms. It also smooths and generalizes quite heavily flat areas it didi not detect anything to be mapped. That's why it get lot more distinct stuff mapped with 2.5m contours than you usually get with 2.5m contours and there is lot less jagged noise and lot less mapped indistinct land forms. Making it quite good to run with out of the box, but maybe not best possible data as base map.

Ok Jagge, we give up. Which side of Tahoe did your second map come from? We can't place it anywhere.

http://www.gmap-pedometer.com/?r=5692626

So it's actually not even in California...

Californian side of Tahoe:
https://picasaweb.google.com/lh/photo/qemHLKZATtS-...

Jagge wrote: Karttapullautin doesnt know is numbes feet or meters. It reads xyz file and there is only numbers there.

I think the problem is that the post size is constant. In the case of feet, the data appear to be sparse, while in the case of meters, the program has been tested there. I'm guessing, but if you try to make a 1 or 2 ft grid from data that is just over 1m post (i.e., over 3ft post) size, the grid is too sparse and the gridding operation explodes.

That's correct. Max area end up only about one tenth of the usual max.

For Ohio, I use this site: http://gis4.oit.ohio.gov/osiptiledownloads/default...

The tiles are 5000 feet on a side, so each tile is about 0.9 square mile, or 2.3 square km.

Occasionally I get lucky and can get a whole park by processing one tile, but that is not what usually happens. I like to use lastools to combine the 2, 3, or 4 tiles into one, and then clip it so I have one single file, but if that single file is too big, I have to use the tiles anyway. For small parks, the clipping operation saves a lot of time even though it is cumbersome because you end up with 1 set of images rather than 2, 3, or 4 times that.

I have been converting the tiles to metric. It gives actual metric contour interval, plus it allows the cliff detection to work. If I use a feet tile in OL-Laser, I get *zero* cliffs, everytime.

Here is a command that works for me on Michigan state plane tiles, you also have to be careful which feet you are using international ones or survey ones.

las2las -target_utm 16T -sp83 MI_S -feet -elevation_feet -i tile.las -o tile_utm.las

I also started processing on a one square km tile basis, it makes it really easy to manage data, while still keeping OL-laser happy.

This is from east coast, near Harriman (no idea is this already mapped or not):
https://picasaweb.google.com/lh/photo/1VK7S0iBpZOD...

Lake Tahoe is so cool area I could not resist making two more:
https://picasaweb.google.com/lh/photo/c6Z4Tiz1lwfY...
https://picasaweb.google.com/lh/photo/I6NbaRZGaRdG...

If someone likes to have better resolution ones or vectors, just let me know or simply use same lidar source and Karttapullautin to do the same.

All the cool kids are making maps from Laramie, so I did one too
https://picasaweb.google.com/lh/photo/B8N_Pd8QeB5M...

I really should stop doing this...
https://picasaweb.google.com/lh/photo/xXpTEy7YbQvQ...

Hawaii:
https://picasaweb.google.com/lh/photo/Hwizu8Ph-s_6...

Pink, I think it run out of memory, same happened to me with the Hawaii one. It ran out of memory if point density was highh enough even if are wasnt really that big. Latest verson should behave slightly better.

The NY-NJ area is Sterling Forest State Park; it's not mapped as far as I know. (In fact it was only founded as a state park in 1998, so it post-dates the big expansion of O maps in HVO's territory.) I doubt it will ever be mapped; HVO can barely maintain the existing Harriman maps let alone spend vast sums on making new ones.

I think Neil started mapping a little bit in Sterling Forest but don't think he got far.

I didn't. [I may have asked about the area at one point because it looked cool!] HVO has money to spend if maps need to be made. Since the club is not generating huge sums of money, it has to be careful where the investment is made. Personally, I think more local maps would be good, plus an update or two of the good maps, but it has to be based on where the active, motivated members are, and right now...

Anyway, I no longer have responsibility :)

I am really anxious to try this when I get my hands on some lidar. Sounds exactly want I want---training maps.

But, I have to ask----Would it be possible for the program to put boulders and houses on the map too? I know that's asking a lot and I am way over my head here but as I understand, typical lidar processing removes the houses and boulders to arrive at the bare earth contours. So, if the processing can recognize a boulder or house in order to remove it, couldn't it instead, put the boulder or house directly on the map?

Just take a look at Jagge's links. Some are replete with boulders and cliffs.
Supplementary question. Will the availability of this great tool for the creation of quality low cost maps destroy the sport of rogaining. ;-)

I may have to explore this area tomorrow morning (Wednesday). Now if only I had a map! http://opentopo.sdsc.edu/gridsphere/gridsphere?gs_...

Note, if you think Pullautin is cool, or may soon become cool, you can vote it. I entered it into an app competition and there is public voting. How to vote, details here.

I find it difficlut to believe poor Pullautin will do well in public voting for it is a command line tool while most other apps are sleeky touch UI tablet/smart phone apps. But decent amount of public votes might make the actual jury take it more seriously. Because O is such a main stream sport here(?!), maybe it will do well just because it is O app, who knows.


Video of Karttapulaltuin in action, fist processing a clip of lidar and then adding there some vectors from natinal topo database:
http://www.youtube.com/watch?v=VXoyQ4bDJVA
(area between Helsinki Olympic stadion - Linnanmäki amusement park).

Jagge, been following this thread and some of the output, very nice.

I finally got a chance to download the program and try playing with it, but I'm trying to do so under Wine on Ubuntu 12.04 - so the instructions to drag & drop the xyz file onto pullauta won't work. Is there a command line structure I can use instead? I tried "wine pullauta.exe test.xyz", but it just crashes on that. Might not work under wine, but figured I'd check if I'm missing something important if trying to run from command line. Thx

try "wine pullauta.exe < test.xyz"

(I'm guessing.)

thanks, i'll give that a try tomorrow, perhaps also using a -i directive. Was too tired to try random stuff tonight...

Played with a data set from Mobile area.

Here is the result

I would say program worked as described

I thought you would say the mapper did work of fair quality.

cmorse in windows you just command pullauta file.xyz, but I can't recall how are cl parametres given in wine. I'll try it this evening on my own box with ubu 12.04. Currently I use windows for ocad and pullautin only. :)

I made a rogaine base for Alan Young (HVO) last year which covers at least part of Sterling Forest. 220 km^2, just contours and vector data.

is it what they call sleepless in Seattle?

I did again same part of Sterling forest as before, now with 5 m contour interval (as Eddie's base). And now only with bigger cliffs. Should here be same resolutin and rotation as in Eddie's gif.
https://picasaweb.google.com/lh/photo/DjaqH41JdXxj...

I'm currently using OCAD 8. Is there a better way to manage processed lidar tile images than the template method in OCAD 8 where each tile is aligned individually?

Ideally, I'd like to be able to open a layer of images (such as slope or shaded-relief) and have all the tiles open using the georeferencing information (such as jgw files).

Do I need a newer version of OCAD, or maybe GIS software?

That works on OCAD 9. It lines up the geo-referenced image outputs from OL-Laser automatically.

Do you have scale and coordinate system set up in Map options to use real world coordinates? I mean if there is such a setting in OCAD 8, I am not sure.

cedarcreek if you convert the pullautin png:s to jpg, you have to rename each pgw to jgw (e.g.pullautus.pgw file to pullautus.jgw). Then ocad finds the georeferencing information.

or if you convert them to tif, just rename "word file" to .tfw
Works in ocad 8.

Apparently OCAD 8 only uses the geo-tiff automatic alignment in the Professional edition. I've got the standard version:

Professional version only:
- geo-referenced TIFF files can be opened and are automatically adjusted to the real-world grid

I'll have to go through the OCAD website to figure out if this is a pro-only feature of OCAD 10, or whatever version they're up to now.

I've been meaning to spend time going through the readme files for the lastools. Are any of those tools especially useful?

in current ocad (11) at least standard version can align geotiffs (pngs, jpgs and bmps,) automaticly.

I've found geo-jpgs align themselves in OCAD 10 Std.

cedarcreek I'm not a Lidar expert but anyhow I have been through the same process as you lately with similar issues processing USGS Lidar files in feet. The solution I came to was to use the following parameters: las2las -i inputfilename.las -o outputfilename.las -target_utm 18N -olas SP83 NJ -feet -elevation_feet -vertical_navd88 (the -merged function was also useful to merge Lidar tiles)
As a final test i use the following website www.apricotvision.com to verify the converted files. If the tiles end up in the right location it seems to work.

I have alos been using OCAD 11 which does a great job reading lidar files, creating contours and generating very nice images all positioned in the right spot. With OCAD 11 I was facing the same issues that there is a need to convert the Lidar files to metric files before starting. OCAD will assume that the Lidar files are in metric format.

Jagge, your application is absolutely great. I am trying to create practice maps using Karttapullautin for some of our juniors here, and tried to change the contour lines, especially as the area is rather flat. I must say I was unsuccessful. I keep seeing maps created by you with 2.5 m intervals. Can I do that and how?

You can always use coordzfactor to tweak z values. But with smaller than 2.5m interval you end up mapping lots of details that doesn't exist. it is not designed to be used like that. It doesn't draw mathematical 2.5 m contrours, it already tweaks controur elevations and maps all ~40cm knolls and so you easily end up trying to map 20 cm knolls and your lidar accuracy may be only +/- 15 cm, so it may not end up well. If it's flat, it may be better map it as such.

I made a different error, I think - the assumption you have 4 regular contours between index contours. It took me a while, but the application is already having the intervals at 2.5, and the index contours at 25 m, is that correct?

index contour interval is set in the ini file. I guess you have 25 there.

@Jagge I think I have figured out the difference between c1, c2, c3.dxf files, but I have not really figured out the difference between c1 and c1g, between c2 and c2g.

Andeais: Pullautin does (or at least used to do) some generalization to the cliffs it detects, so the c1 c2 and c3 are those intermediate cliffs and the c1g and so on are the final ones. And the difference from 1 to 3 is the height of the cliff (you can change the steps in pullauta.ini).