add forward option to retain las file scale and offset

[originlake]

When creating the georeferenced laz file, the scale is set to 0.001m(or value of spacing if it's smaller), offset is set to origin of the dataset. But the following classification steps do not respect the scale and offset, using the default value of pdal instead, which restricts the precision of point cloud data to 0.01m.

Fixed the smrf filter that used pdal directly. Still working on fixing OpenPointClass

[originlake]

I have one question, how should I prepare the fix for OpenPointClass? It's not a fork and the fix would be a special patch just for ODM I suppose.

[originlake]

https://github.com/originlake/OpenPointClass/tree/las-scale fix for openpointclass is here

[smathermather]

Thanks for this!

As OpenPointClass (AFAIK) only operates on PLYs, there shouldn't be any floating point issues requiring setting las scale. But, double check my assumptions on that. I haven't looked under the hood of OpenPointClass in any serious way.

Take a look at this pull request to appropriately set the scale instead of fixed:
#1788

[originlake]

The classification starts with a smrf filter then followed with a openpointclass step. OPC does use laz file and overwrites the old laz file, leading to this issue. So both need to be fixed.

Code for classification steps:

ODM/opendm/point_cloud.py

Lines 280 to 300 in 5d4862e

	if args.pc_classify:
	pc_classify_marker = os.path.join(tree.odm_georeferencing, 'pc_classify_done.txt')
	if not io.file_exists(pc_classify_marker) or rerun:
	log.ODM_INFO("Classifying {} using Simple Morphological Filter (1/2)".format(tree.odm_georeferencing_model_laz))
	commands.classify(tree.odm_georeferencing_model_laz,
	args.smrf_scalar,
	args.smrf_slope,
	args.smrf_threshold,
	args.smrf_window
	)
	log.ODM_INFO("Classifying {} using OpenPointClass (2/2)".format(tree.odm_georeferencing_model_laz))
	classify(tree.odm_georeferencing_model_laz, args.max_concurrency)
	with open(pc_classify_marker, 'w') as f:
	f.write('Classify: smrf\n')
	f.write('Scalar: {}\n'.format(args.smrf_scalar))
	f.write('Slope: {}\n'.format(args.smrf_slope))
	f.write('Threshold: {}\n'.format(args.smrf_threshold))
	f.write('Window: {}\n'.format(args.smrf_window))

Code where the opc is used:

ODM/opendm/opc.py

Lines 7 to 30 in 5d4862e

	def classify(point_cloud, max_threads=8):
	tmp_output = io.related_file_path(point_cloud, postfix=".classified")
	if os.path.isfile(tmp_output):
	os.remove(tmp_output)
	try:
	model = get_model("openpointclass",
	"https://github.com/uav4geo/OpenPointClass/releases/download/v1.1.3/vehicles-vegetation-buildings.zip",
	"v1.0.0",
	name="model.bin")
	if model is not None:
	run('pcclassify "%s" "%s" "%s" -u -s 2,64' % (point_cloud, tmp_output, model), env_vars={'OMP_NUM_THREADS': max_threads})
	if os.path.isfile(tmp_output):
	os.remove(point_cloud)
	os.rename(tmp_output, point_cloud)
	else:
	log.ODM_WARNING("Cannot classify using OpenPointClass (no output generated)")
	else:
	log.ODM_WARNING("Cannot download/access model from %s" % (model_url))
	except Exception as e:
	log.ODM_WARNING("Cannot classify using OpenPointClass: %s" % str(e))

The scale and offset are computed dynamically in an early stage, my fix proposed here follows what is being set in the original laz file, which should be expected as the classification step should not alter the point cloud itself.

Code where the offset and scale are set in the original laz file:

ODM/stages/odm_georeferencing.py

Lines 131 to 160 in 5d4862e

	las_scale = 0.001
	filtered_point_cloud_stats = tree.path("odm_filterpoints", "point_cloud_stats.json")
	# Function that rounds to the nearest 10
	# and then chooses the one below so our
	# las scale is sensible
	def powerr(r):
	return pow(10,round(math.log10(r))) / 10
	if os.path.isfile(filtered_point_cloud_stats):
	try:
	with open(filtered_point_cloud_stats, 'r') as stats:
	las_stats = json.load(stats)
	spacing = powerr(las_stats['spacing'])
	log.ODM_INFO("las scale calculated as the minimum of 1/10 estimated spacing or %s, which ever is less." % las_scale)
	las_scale = min(spacing, 0.001)
	except Exception as e:
	log.ODM_WARNING("Cannot find file point_cloud_stats.json. Using default las scale: %s" % las_scale)
	else:
	log.ODM_INFO("No point_cloud_stats.json found. Using default las scale: %s" % las_scale)
	params += [
	f'--filters.transformation.matrix="1 0 0 {utmoffset[0]} 0 1 0 {utmoffset[1]} 0 0 1 0 0 0 0 1"',
	f'--writers.las.offset_x={reconstruction.georef.utm_east_offset}' ,
	f'--writers.las.offset_y={reconstruction.georef.utm_north_offset}',
	f'--writers.las.scale_x={las_scale}',
	f'--writers.las.scale_y={las_scale}',
	f'--writers.las.scale_z={las_scale}',
	'--writers.las.offset_z=0',
	f'--writers.las.a_srs="{reconstruction.georef.proj4()}"' # HOBU this should maybe be WKT
	]

[smathermather]

Got it. Pull request against OpenPointClass and explain your concerns. If rejected and we need to maintain a fork with this functionality under an OpenDroneMap banner, happy to do so.

[originlake]

Thanks, created the PR, will see how it goes. uav4geo/OpenPointClass#33

[smathermather]

Nice work! Just update SuperBuild/cmake/External-OpenPointClass.cmake to point to the hash of the commit (dd6a560a1d43cb709f7b220b19a436e25a889e3e), since we don't know when the next release will be cut, and this is similar to how we're handling other upstream libraries now, and we should be good to merge.

[smathermather]

Awesome. Thanks! I'll update SuperBuild/cmake/External-OpenPointClass.cmake to point to the appropriate commit.