Discussion: I'm pathetic...and need help

So I've been toying with these images for a while now trying to stitch them together into a base map and can't seem to do it.

Its for a small park just outside of our high school... and I'm hoping to make it into a training ground and eventually perhaps host a sprint on it.

I've conned the good folks at TNRIS (Texas natural resources) into donating their LIDAR data for this purpose and then ran it through Kartapullautin. I now have the four subsequent tiles that I just can't seem to mesh, I never was any good at photoshop, irfanview, and the like...

Of course all tiles are more than I need, but there is a small piece of relevant data on each one. If you need google maps for a reference, the location is in Mesquite, TX and is Samuel Farm off of US-80.

http://s8.postimg.org/fraaycrk5/image.png
http://s28.postimg.org/xi84p2i25/image.png
http://s7.postimg.org/3tyrxr4ej/image.png
http://s28.postimg.org/7039lknhp/image.png

So if one of you awesome folks were able to help me with this, I would be forever grateful.

I would need more info to do it right, like the scale of the images. I tried bringing the images into Open Orienteering Mapper as templates, and it looks like the first image is upper left, second upper right, third lower left and fourth lower right. I just dragged them into place manually until they seemed to match up. Can you provide info on scale and such? What mapping program are you using? I'd suggest something like Open Orienteering Mapper instead of PhotoShop. Let me know and I'll try to help a bit more.

Those images are huge and don't overlap much, which would make it really hard to properly stitch them together.

Especially considering you only want a small portion of the region, you should try using lasmerge from the LAStools suite to merge your four LiDAR tiles into a single file and filter down to the region you want. Coordinates are in UTM so you'll also have to convert your bounding GPS coordinates using something like this. Having a smaller las file specific to the area you're trying to map is super helpful, because it makes generating basemaps from the data with kartapullautin, ol-laser, and ocad significantly easier and faster.

If you want specific help with this process let me know, I'd be happy to help.

Here are the four pieces georeferenced as a TIFF
http://maphiker.info/SamuelPark.tif
You should be able to open this as a template in OCAD and have a basemap with a UTM grid.

Awesome, thank you all.

This is only the second map I've made... with the first being of the high school campus (no useful contour). I used OOM for that, and have access to OCAD8.

In the future I'll try the lasmerge (if haywoodkb's solution doesn't work)... it looks like the provided data may have a small gap in it. But small enough to where I should be able to draw in the differences.

I appreciate the fast and detailed replies. I believe those are 1:10000... I'll have to go back and look...

From the size of the building lots and houses I'd put my money on 1:10,000 images as well.
Haywoodkb is a miracle worker with creating base maps., LiDAR images, Kaartapaullatin and all.

But for school and small municipal parks there is another source of base information that many of us have been overlooking. To build a park, a school, a shopping center, a college campus, anything pretty well anywhere in North America the property owner- school boards and municipalities included - must get a document called a site plan approved. The site plans have to be certified as correct before the development can proceed and the development must be built according to the site plan.
And these are documents meant to be available to the public.
Check with the town for a copy of the park's approved site plan.
I'm finding these are great ways to start an orientering map

Thanks Gord, great tip.

I downloaded the openstreetmap.org data as a topo reference then went looking at the TNRIS site for lidar around Mesquite but I see that (a) all lidar is by phone order only, and (b) I could not find Mesquite there. :-(

BTW, I have visited Mesquite a number of times back in the nineties, when my friends John Cash and Mike Abrash worked with John Carmack at id Software, writing Quake. Quake was of course the first ever 3D game/first person shooter, you can still locate my name in the manual for the first version. :-)

Ooooh, old-skool gamer cred. :D

Quake, wow, I remember people waxing lyrical about that too.

OK, this thread is hijacked.
I remember playing Quake (and of course the forbidden Wolfenstein) about 20 years ago, thinking that someone should write a 3D orienteering simulator. Friends told me this would never happen.

Haha some googling brought me to the fast inverse square root method used in Quake 3. I've read about it before but never knew that an orienteer was involved in this crazy code. :) (I'm not really a programmer.)

Wolfenstein was awesome... but ... DOOM > Quake...

Anwyway, back to the thread. I'll keep you updated with my (rather slow...progress)

I'll keep in mind the public site plans, however I don't think there is much promise for this park as it is A) actually in unincorporated land between mesquite and sunnyvale
B) operated by Dallas city parks as part of a land gift
C) almost 100 years old and not officially 'developed'...

I forgot to mention... that yes lidar is available for purchase, but as I initially stated, the good folks at TNRIS donated it to me. If you'd like the original files I can see if the link they gave me still works.

If it doesn't ... I can probably find a place to upload them.

More DOOM memories please! (Apologies to the serious thread, I had the Aliens version which was deadly).

Almost totally irrelevant but I was watching this clip the other night and you just reminded me of it - 0:37 into it might make you chuckle (or might not, you have to watch it all up to that point for the surprise).

OK, here's more Quake background:

I worked with John Cash at Novell in Utah in 91-92, at that time John and friends played DOOM deathmatch at night and discovered that the network code was really horrible (every single packet was sent out globally, across all connected networks) and he found out that Carmack seemed to have something to do with the game:

He wrote him an email and complained, then a few days later he got back a response: "Here's the source code, fix it!"

So of course he had to spend a few days doing so, rewriting that part from scratch, and a few years later he got hired as the network programmer for Quake.

(In the meantime Mike Abrash had asked me if I was interested but I told him that I wanted to keep on living in Oslo and stay married to my engineer wife, and that John Cash was a better choice.)

BTW, both Carmack and Abrash are now at Oculus working on AR/VR which of course is the obvious endpoint for a virtual O simulation.

Cash moved to Blizzard just as they were starting up a big multiplayer game you might have heard about, he's been the lead engineer/tech director for WoW since the start.

BTW, I'm back working on stuff like the invsqrt() code now, I'm writing sw emulation for all hw fp operations for a new cpu architecture, the Mill. Take a look at MillComputing.com. (This is a hobby, pro bone work like most of my base map generation work.

@skarabrae: Just dump the lidar on dropbox/google drive or similar and send me a link, I'll see what I can make out of the data.

IDKFA

link sent via google drive

For the best map, you should consider merging the lidar tiles with lasmerge (part of the lastools you already have for Kartapullautin), then crop/clip that big file to a file that only has the data for your school area. Like:

MERGE

lasmerge -i *.las -merged -o BIGMERGE.las

METERS
change to Meters and UTM zone (if needed ... you'll need to find your UTM zone and replace the 18N with it)

las2las -i BIGMERGE.las -o BIGMERGE-meters.las -olas -target_utm 18N -feet -elevation_feet -vertical_navd88

INFO
get info about the new file

lasinfo -i BIGMERGE-meters.las -o BIGMERGE-meters_info.txt

Take a look at BIGMERGE-meters_info.txt. The most useful lines are "min x y z:" and "max x y z:". These are the borders of the 3D space represented by the LAS file.

VIEW
open in lasview and estimate clip area ... there are probably good tools for this, but I don't know them. :)

lasview -i BIGMERGE-meters.las

CLIP
clip the area you want to keep

las2las -i BIGMERGE-meters.las -o INPUT.las -keep_xy 465000 4496000 465500 4496500

Drag-n-drop to Kartapullautin!

I have a bit more detail if you're interested and this seems useful.

And another option is to just do a 'pullauta pngmerge 1' to merge your PNG output image files. See the Kartapullautin readme.txt file for details!

What hughmac⁴ said except use .laz instead of .las. The laz files are about 1/7th the size of las.

For determining UTM coordinates for clipping, I use Acme Mapper with preferences set to UTM. The + in the middle of the map image is the point the coordinates refer to.

I have txt files I use to keep track of the process---mostly to help with the lastools command line stuff. I have one for each different state projection I tend to use:

UTM Projection is 16 Northern Hemisphere (32616).

From NOAA program: WMMGUI
Grid-Magnetic, for Karttapullautin: - 7° 12' = -7.2

clipping boundary is: -keep_xy 698300 4392100 702500 4393900
Commands used:

lasmerge -i *.las -olaz -o Park_Name_merged.laz

las2las -i Park_Name_merged.laz -o Park_Name_UTM.laz -sp83 OH_S -survey_feet -elevation_surveyfeet -target_utm 16N

Entire Park: clipped:
las2las Park_Name_UTM.laz -o Park_Name.laz -keep_xy 698300 4392100 702500 4393900

lastile -i Park_Name.laz -olaz -tile_size 1000 (only if the park file is too big to process)

las2dem -i Park_Name.laz -oasc -keep_class 2 -utm 16S (might not work if too large)

So:
merged: state projection, not clipped (full tiles)
UTM: not clipped (full tiles, but reprojected)
Park_Name.laz: reprojected, clipped to "rectangle of interest".

lastile strips out some data above a threshold point number, so I try to not tile unless I need to. Sometimes, I use the -keep_xy las2las switch and create tiles that way. I recently had a park with such dense tiles I had to create 500m tiles to get it to run.

Also, note that -keep_xy is not the same as -keep_XY.

I use OL-Laser and Karttapullautin. I've started trying to figure out Terje's python script method. I've done a few lidar things with OCAD 11, but it seems to be more automatic and I don't like the loss of control. If it can create useful vector layers, I'll be more inclined to use it. For large parks that require tiling to get small grid sizes, I also process contours at the smallest grid I can and still get the whole park. For example, I find a 3m grid gives a good starting point for fieldchecking---it's not too noisy, but it shows fairly small details. But for a big park, I'll use a grid of 6m (or even 10m) if I have to just to get vector contours of the whole park. For the tiles, I typically run contours at both 1m and 3m grids, but if I can do the whole park file at 3m grid, I won't run the tiles at 3m.

I process state-plane aerials in QGIS. I merge, reproject (warp), and clip. Merging requires certain image types: GeoTIFF works and jpg doesn't, so I tend to work in TIFF until I clip, and then I "extract projection" to create world files (.wld), then "Convert" tiffs to jpgs. Finally I rename the .wld to .jgw---as long as the pixel sizes are the same, the world files will work. The newer versions of QGIS might not need the jgw---the "Convert" process creates an xml that seems to do the job. I think I've tiled automatically, but I can't remember. I've definitely tiled using manual coordinates typed into the clipper. Also, when I have used tiles, I have started using QGIS to merge the various lidar products into a whole park image as long as the file sizes aren't too big.

I keep seeing non-orienteering references to processing lidar with GRASS-GIS, but I haven't figured that out yet, either. I use two SAGA-GIS features, but in the QGIS Processing toolbox. I create a DEM (last command line, above), and run the Wetness Index algorithm, then I save (as rendered image, not data) two images: Catchment Area and Wetness Index. This seems to help find ditches and streams, especially on slopes. (You need to end up with the georeferencing world files, and you can use QGIS to convert to jpg if desired.) I'm trying to create vectorized streams and ditches, but I haven't had any luck. TauDEM and LSDTopoToolbox are supposed to be able to vectorize streams and extract channels, but I haven't tried them yet.

Also, one trick I've discovered (thanks to Mike Minium) is that for OCAD 8, be sure to create only TIFF files.

Also be sure to open the contours and a raster to see if they align, especially if you're trying to start at a scale other than 1:15000. I have to do a map scale conversion in OCAD 8 or assign the projection and "change map scale" in later versions. The georeference functions aren't perfect. Sometimes things don't align if you use an angle rotation to account for grid-to-magnetic. Open Orienteering Mapper, I am told, handles that better. If if doesn't work (in OCAD or OOMapper), try opening first the contours then the raster, and then the opposite: first the raster and then the contours.

I've made a q&d base map of the Mesquite park, I put it on my personal server:

http://tmsw.no/o/mesquite.zip

The georeference is UTM 14N, I saved it in OCAD 11 so OpenOrienteeringMapper can open it!

This version uses the default ground classifications as I got them, but with my custom vegetation classes to make it more suitable for orienteering maps.

It is quite obvious that there are significant areas under dense vegetation which have too few ground points, but when I tried lasground_new from LAStools I got a huge amount of brush vegetation that was classified as small dot knolls. :-(

@cedarcreek: yup, LAZ, good call.

Text file save = scripting! Hooray! I actually don't work on Windows with Kartapullautin much, but when I do I do the same thing (save my steps as a script/historical document).

I ported Kartapullautin to Linux at some point, and wrote some wrapper scripts so that I can feed a list of LA[SZ] tile URLs and some options to a script, and it will fire up a job for each tile (or even split the tiles ... 4:1 is a good compromise option with PA tiles between speed and 'the uglies') as last-priority jobs on a cluster at work, and merge stuff at the end if requested. I've tested with up to 128 cores ... that gets some mapping done!

Nice stuff, @Terje!

All, you have been most generous. I am very greatful. This is my first real map... so its going to take me a bit to process all you have told me. Hopefully, I'll have reasonable feedback for you all as I progress. Good news is I should have a good bit of time on my hands next week to try and process it all.

I just realized the Wetness Index isn't in GRASS, it's from SAGA-GIS, but I still run it from the QGIS Processing toolbox. I'll edit the above.

If you'll indulge me... I'm using OOM6, I've loaded the basemap provided by Terje as a template, and I've also matched an image to it with the same scale as a 2nd template.

1. I know how to draw over the image...
2. Is there a way to bring all that contour and vegetation into the actual map, as opposed to using it as a template and drawing over it?
-I have tried opening as the map in order to draw on it and just edit it... but that brings up all sorts of crazy grey area that looks like slope data
3. How do I interpret the contour interval?

skarabrae---regarding question 2: The contours are fairly easy to import, but in most cases you'll want to visit the park and fieldcheck them. My favorite "camera ready" contours are those from Karttapullautin. I don't know why they seem so much better and smoother than other lidar outputs, but if you're doing a quick and dirty map, those are available in vector form in the temp directory. It helps if you can run it as one lidar file, though, because otherwise it's more difficult---you have to import multiple files.

All---If anyone cares, this is a link (70MB) to a recent park basemap I did, showing basically what I try to do with OL-Laser, KP, and QGIS. It's a zip, so unzip it and just look at the images. There are several ocad files. The one labeled "best starting point" was KP contours from dxf plus OSM imports like roads and (interestingly) buildings. The lidar had buildings classified (seen in the pullauta image as black), but the OSM buildings were either imports from a GIS system or drawn by hand in OSM. If you've got any recommendations on how I can improve it, please let me know. This basemap has some recent innovations, such as OSM vector import, which when shown over the KP output looks remarkable for a map purely from data with zero fieldchecking.

@cedarcreek

1. Thank you, but I'm brand new (a newb if you will...) to this so
2. Can you give me the steps to import said contour? I can't seem to figure it out. I will say OOM is more friendly than OCAD to the non-pragrammer rookie
-I have Terj's basemap... which looks very similar to the KP output
-I have Karttapullautin images and files from the lidar
-do you suggest I go back and make it one large tile first as suggested by miclaraia above
-I planned on making field check adjustments after I had done contour, veg, buildings, etc. so I can *hopefully* visit each section of the park once. Is that a poor plan?

skarabrae:

I'll post how to import the KP output as vector. It doesn't do the vegetation, just contours, broken formlines, dot knolls, and U-depressions.

I always try to do a park as one lidar file if I can. It depends on how much RAM you have. My computer is pretty middle-of-the-road, so I can process up to maybe 4 sq. km (I can do one 5000ft/1524m tile (about 1 sq. mile) almost all the time, and sometimes two (5 sq km) if the lidar post spacing is large. Just try it with KP and see if it runs. Maybe start it at bedtime if you need your computer for other stuff.

Hopefully some experienced mappers can recommend strategies for fieldchecking and drafting. I noticed they typically prefer to not have imported features, especially for parks with few roads and buildings, because most of the available vector data is not good enough (too noisy or otherwise wrong). They prefer importing the data to make a basemap, and then basically redraw everything to make the final map.

I find it really helps with KP to have some imported layers, like roads. But OSM in rural areas sucks. The best I've seen is called "road centerlines", which are made for emergency response, and have pretty good accuracy (the spec is like 0.5m or something). But, it's usually not too big a task to draw the roads from aerials or from the lidar layers that show the road cuts. I've even driven roads with a GPS and gotten data good enough for the basemap. (A lot of OSM in the US is unedited "TIGER" data, and it's often grossly wrong. Some OSM editors let you highlight unedited TIGER data features.)

Ok currently running LASmerge... then I'll rerun it through KP and see what pops up tomorrow

From there, hopefully I'll be able to read your directions! =)

I'm comfortable hand drawing all of the veg... probably better that way anyhow for overall generalization of specific areas.

TIGER data is fine for mapping a city or county, but at orienteering scales, it has very serious flaws. I don't image it would be worth the trouble.