Add mbmesh: generate 3D point clouds (projected XYZ + ECEF XYZ + GLB + optional HTML viewer) directly from swath datalists

[Lyn-Solarn]

Addresses: #1559

Summary

This pull request introduces a new MB-System program, mbmesh, that reads swath sonar datalists (MB-System datalist framework) and extracts valid multibeam bathymetry soundings to generate 3D point cloud outputs. The current implementation focuses on producing point clouds and lightweight visualization artifacts that can be used immediately in browser-based and desktop 3D tooling, and serves as the foundation for future mesh generation and OGC 3D Tiles output.

What’s included

• New executable: mbmesh (src/mbmesh/mbmesh.cpp)
• GLB (glTF binary) point cloud export:
  • src/mbmesh/pointcloud_glb_writer.cpp
  • src/mbmesh/pointcloud_glb_writer.h
• Documentation:
  • src/mbmesh/README.md (usage, outputs, build notes)
  • src/mbmesh/mbmesh.1 (man page)
• Build integration:
  • src/mbmesh/CMakeLists.txt
  • src/mbmesh/Makefile.am

How to use mbmesh

Synopsis

mbmesh -I<datalist> [-O<outputdir>] [-Rwest/east/south/north] [-html] [-V [-V]] [-H]

Inputs

• -I<datalist>: MB-System datalist file that references one or more swath files (and optionally processed versions).
  Internally, mbmesh iterates entries using mb_datalist_open() / mb_datalist_read3(), then reads each swath file using mb_read_init() and mb_read().

Options

• -I<datalist>: Input datalist file (default: datalist.mb-1)
• -O<outputdir>: Output directory (default: ./tileset). Created if it does not exist.
• -Rwest/east/south/north: Geographic bounds filter (degrees).
• -html: Also generate adjustedPointcloud.html and attempt to launch a local web server + open the viewer.
• -V / -V -V: Increase verbosity (progress reporting, stats, sample soundings).
• -H / -h: Print help and exit.

Examples

Basic run (writes to ./tileset):

mbmesh -Idatalist.mb-1

Custom output directory:

mbmesh -Idatalist.mb-1 -O./my_output

Geographic filtering:

mbmesh -Idatalist.mb-1 -R-122.5/-121.8/36.5/37.2

Generate an HTML viewer and attempt to launch it:

mbmesh -Idatalist.mb-1 -O./my_output -html -V

Output files

mbmesh writes outputs into the directory specified by -O (default: ./tileset):

1. adjustedPointcloud.xyz
   Local projected XYZ (meters) designed for numerical stability in 3D viewers:

   • X,Y: meters relative to the data centroid (derived from mean lon/lat)
   • Z: depth centered around mean depth
     The header records the reference lon/lat/depth used for centering.

2. ecefPointcloud.xyz
   WGS84 ECEF point cloud (meters) written with a GeoOrigin (centroid) offset applied:

   • Header includes reference lon/lat/depth and the ECEF offset (x0,y0,z0)
   • Points are written as (x-x0, y-y0, z-z0)

3. adjustedPointcloud.glb
   Binary glTF (GLB) point cloud generated from adjustedPointcloud.xyz:

   • Uses TINYGLTF_MODE_POINTS
   • Includes a basic PBR material (double-sided)

4. adjustedPointcloud.html (only when -html is provided)
   Minimal X3DOM-based viewer that inlines adjustedPointcloud.glb.
   When -html is enabled, mbmesh attempts to:

   • Start a local server: python3 -m http.server 8000 --bind 127.0.0.1 --directory <outputdir>
   • Open: http://127.0.0.1:8000/adjustedPointcloud.html

Notes / known limitations (current implementation)

• mbmesh stores all extracted soundings in memory prior to writing outputs; very large datalists may require substantial RAM.
• The -R option is documented as a geographic (degrees) bounding box. Reviewers should confirm that the coordinates being stored/filtered correspond to lon/lat as intended in the current extraction path.
• The ECEF output currently uses a GeoOrigin offset; code comments note potential precision considerations and suggest making GeoOrigin configurable or supporting absolute (non-offset) ECEF output.

Future work (not part of this PR)

• Generate triangle meshes from the point cloud while preserving true 3D structure (cliffs/overhangs/caves).
• Add OGC 3D Tiles output (tileset.json + tile payloads) and LOD streaming.
• Evaluate py3dtiles and alternatives for 3D Tiles generation.
• Implement Draco compression to reduce GLB payload size.
• Improve/standardize GeoOrigin handling and ECEF precision strategy.
• Make X3DOM visualization the default path.

[MBARIMike]

On first scan this appears to be a significant contribution. Thanks!
I will need some time to review it all, but I expect that this can be merged without too many change requests.

[MBARIMike]

Testing this PR in my working directory... Here's how I did it:

1. I have a git alias named pr in my .gitconfig:

cat ~/.gitconfig
# This is Git's per-user configuration file.
[user]
# Please adapt and uncomment the following lines:
name = Mike McCann
email = MBARIMike@gmail.com
[core]
	editor = vim
[pager]
	diff = false
[alias]
    pr = !sh -c \"git fetch upstream pull/${1}/head:pr/${1} && git checkout pr/${1}\"

2. First made sure that the master branch in my working directory is synchonized with upstream and origin:

git pull upstream master
git push origin master

3. See that I can build and install without error:

cd build
cmake ..
make
sudo make install

4. Now, use git pr to pull the Pull Request into a new branch:

➜ git pr 1584
remote: Enumerating objects: 194, done.
remote: Counting objects: 100% (188/188), done.
remote: Compressing objects: 100% (90/90), done.
remote: Total 168 (delta 109), reused 131 (delta 77), pack-reused 0 (from 0)
Receiving objects: 100% (168/168), 428.04 KiB | 2.58 MiB/s, done.
Resolving deltas: 100% (109/109), completed with 17 local objects.
From github.com:dwcaress/MB-System
 * [new ref]             refs/pull/1584/head -> pr/1584
Switched to branch 'pr/1584'

5. Build and try it:

➜  build git:(pr/1584) cmake ..
➜  build git:(pr/1584) make
➜  build git:(pr/1584) sudo make install

No errors, and can display the mbmesh help message:

➜  MB-System git:(pr/1584) mbmesh -h

Program mbmesh
MB-system Version 5.8.3beta12

mbmesh generates 3D point clouds from swath sonar bathymetry data.
This tool reads swath data files and produces multiple output formats
suitable for 3D visualization and analysis of seafloor bathymetry.

usage: mbmesh -Idatalist [-Rwest/east/south/north] [-Ooutdir] [-html]
       [-V -H]

Required:
  -I<datalist>       Input datalist file [datalist.mb-1]

Optional:
  -O<outputdir>      Output directory [./tileset]
  -R<w/e/s/n>        Geographic bounds (degrees)
  -html              Also write adjustedPointcloud.html viewer file
  -V                 Increase verbosity (can repeat: -V -V)
  -H                 Print this help message

Example:
  mbmesh -Idatalist.mb-1 -R-122.5/-121.8/36.5/37.2 -V


Program <mbmesh> Terminated

[MBARIMike]

Test execution with sample data: OceanImaging2018/20180923d1/multibeam/:

➜  multibeam git:(pr/1584) mbmesh -I datalist.mb-1 -html -V

Program mbmesh
MB-system Version 5.8.3beta12

mbmesh settings:
  Input datalist: datalist.mb-1
  Output directory: ./tileset
  Geographic bounds: [unbounded]
  HTML output: enabled
  Verbose level: 1

=== Reading Swath Data ===
Datalist opened: datalist.mb-1

Processing file 1: 20180925_144609.mb89
  Opening file (format 89)...
  File opened: 2048 beams, 2048 amp, 2048 ss
  [DEBUG] About to start mb_read loop, mbio_ptr=0x128008000
  [DEBUG] mb_read loop exited: status=0
  File complete: 0 pings processed
...

Processing file 28: 20180925_180007.mb89
  Opening file (format 89)...
  File opened: 2048 beams, 2048 amp, 2048 ss
  [DEBUG] About to start mb_read loop, mbio_ptr=0x138008000
  [DEBUG] mb_read loop exited: status=0
  File complete: 0 pings processed

Swath data reading complete

========================================
    Swath Data Reading Statistics
========================================
Files in datalist:       28
Files successfully read: 28
Pings processed:         58481
Beams total:             29882368
Beams good:              22634536
Beams flagged:           7247832
Soundings stored:        22634536
Acceptance rate:         75.7%
========================================

Data bounds:
  Longitude: -122.304680 to -122.303812
  Latitude:    36.392409 to   36.393160
  Depth:          833.64 to      850.68 meters


Writing projected XYZ point cloud: ./tileset/adjustedPointcloud.xyz
  Reference point: lon=-122.304274 lat=36.392804 depth=843.2
  Scale: 1 deg lon = 89457.7 m, 1 deg lat = 111132.0 m
  Points: 22634536
  Projected XYZ file written successfully

Writing ECEF XYZ point cloud: ./tileset/ecefPointcloud.xyz
  Points: 22634536
  ECEF XYZ file written successfully
  Location: ./tileset/ecefPointcloud.xyz

Processing complete
Soundings collected: 22634536
Adjusted XYZ file written: ./tileset/adjustedPointcloud.xyz
Point cloud GLB written successfully: ./tileset/adjustedPointcloud.glb
GLB point cloud file written: ./tileset/adjustedPointcloud.glb
HTML viewer file written: ./tileset/adjustedPointcloud.html
Started Python web server at http://127.0.0.1:8000/adjustedPointcloud.html
Server logs: /tmp/mbmesh_http.log

Program <mbmesh> completed successfully

I had another program using port 8000 so the automatic display did not work. However, I fired up a new web server and am able to visualize the point cloud:

➜  multibeam git:(pr/1584) python3 -m http.server 8001
Serving HTTP on :: port 8001 (http://[::]:8001/) ...
::ffff:127.0.0.1 - - [11/May/2026 13:29:59] "GET /tileset/adjustedPointcloud.html HTTP/1.1" 200 -
::ffff:127.0.0.1 - - [11/May/2026 13:30:01] "GET /tileset/adjustedPointcloud.glb HTTP/1.1" 200 -

[MBARIMike]

I'm seeing a few nits, mostly my preferences on how things should work, but mbmesh works well.

Now, to try out mbm_tmap...

[MBARIMike]

Working in the tileset directory where the mbmesh generated .xyz files are:

Install py3dtiles in a virtual environment:

➜  tileset git:(pr/1584) python -m venv venv
➜  tileset git:(pr/1584) source venv/bin/activate
(venv) ➜  tileset git:(pr/1584) pip install py3dtiles
(venv) ➜  tileset git:(pr/1584) mbm_tmap --input=adjustedPointcloud.xyz --port=8001

View frustum based tile loading and unloading are seen in this view and as the viewpoint moves about the scene.

Nice work!

[MBARIMike]

One final nit...

After doing a sudo make install the man mbmesh command did not bring up the man page. I think the src/mbmesh/mbmesh.1 should be moved to src/man/man1/ and mbmesh.1 should be added to src/man/man1/CMakeLists.txt to fix this.

[MBARIMike]

Did you move src/mbmesh/mbmesh.1 to src/man/man1/?

I get this error when doing a sudo make install:

CMake Error at src/man/man1/cmake_install.cmake:41 (file):
  file INSTALL cannot find
  "/Users/mccann/GitHub/MB-System/src/man/man1/mbmesh.1": No such file or
  directory.
Call Stack (most recent call first):
  src/man/cmake_install.cmake:42 (include)
  cmake_install.cmake:177 (include)

This is a small nit, but the entries in src/man/man1/CMakeLists.txt should be alphametized. mbmesh.1 needs to be placed after mbmakeplatform.1.

[MBARIMike]

I'm getting a clean build and sudo make install and man mbmesh now works:

➜  MB-System git:(pr/1584) man mbmesh
mbmesh(1)                                          General Commands Manual                                         mbmesh(1)

NAME
       mbmesh - Process swath bathymetry data and generate 3D point clouds and GLB models

SYNOPSIS
       mbmesh -I datalist [-O outputdir] [-R west/east/south/north] [-html] [-V [-V]] [-H]

DESCRIPTION
       mbmesh reads swath sonar data files and processes bathymetric soundings to generate 3D point cloud data in multiple
       formats. The primary use is to extract multibeam echosounder (MBES) data and convert it to point clouds suitable for
       3D visualization and analysis.

       mbmesh reads a datalist file containing references to swath data files in MB-System supported formats. It extracts
       all valid bathymetry soundings, applies optional geographic filtering, and outputs the data in multiple formats:

              adjustedPointcloud.xyz
                     Point cloud in local projected coordinates (meters) relative to data centroid

              ecefPointcloud.xyz
                     Point cloud in WGS84 Earth-Centered Earth-Fixed (ECEF) coordinates

              adjustedPointcloud.glb
                     Binary GLB (glTF) format suitable for 3D viewer applications

              adjustedPointcloud.html
                     Optional HTML viewer for interactive 3D visualization


OPTIONS
       -I datalist
              Input datalist file containing list of swath data files. This is a required option. Each line in the datalist
              file should contain: path format [processed_path] following MB-System datalist format conventions.  The
              default datalist file is datalist.mb-1 if not specified.

       -O outputdir
              Output directory for all generated files. The directory will be created if it does not exist. Default is
              ./tileset.

       -R west/east/south/north
              Specify geographic bounds for filtering soundings. Only soundings within the specified rectangle (in degrees
              longitude/latitude) will be included in the output. The format is west/east/south/north where west and south
              are typically negative. If not specified, all soundings are included.

       -html
              Also generate an HTML viewer file (adjustedPointcloud.html) for interactive 3D visualization of the point
              cloud. When this option is used, mbmesh will attempt to launch an HTTP server and open the viewer in the
              default web browser.

       -V
              Increase verbosity level. By default, only essential information is printed.  Can be specified multiple times
              (-V -V) to increase debugging output.  Higher verbosity levels provide detailed progress information, data
              statistics, and sample soundings from the dataset.

       -H
              Display help message and exit.

       -h
              Display help message and exit (alternative to -H).


EXAMPLES
       Process all files in a datalist and output to default directory:
              mbmesh -Idatalist.mb-1

       Process files in a specific geographic region:
              mbmesh -Idatalist.mb-1 -R-122.5/-121.8/36.5/37.2 -V

       Process files with custom output directory and HTML viewer:
              mbmesh -Idatalist.mb-1 -O./my_output -html -V

       Process files with increased verbosity for detailed debugging:
              mbmesh -Idatalist.mb-1 -V -V


OUTPUT FILES
       mbmesh generates the following output files in the specified output directory:


       adjustedPointcloud.xyz
              Standard XYZ point cloud in local projected coordinates where longitude/latitude are converted to meters
              relative to the data centroid and depth is centered around the mean depth. This format is suitable for
              visualization in standard 3D point cloud viewers. Format: X(meters) Y(meters) Z(meters)

       ecefPointcloud.xyz
              Point cloud in WGS84 ECEF coordinates with GeoOrigin offset applied. This format preserves accurate Earth-
              centered coordinates suitable for analysis requiring geodetic precision. Format: X(meters) Y(meters) Z(meters)

       adjustedPointcloud.glb
              Binary GLB (GL Transmission Format) file containing the point cloud as a 3D model.  This format is widely
              supported by web-based 3D viewers and applications.  Can be directly embedded in web pages or used with tools
              like Cesium.js or Three.js for interactive visualization.

       adjustedPointcloud.html
              Optional HTML file generated when -html option is used. This file provides an interactive web-based viewer for
              the GLB model. When generated, mbmesh attempts to launch a local HTTP server and open the viewer in the
              default web browser.


DATA STATISTICS
       Upon completion, mbmesh prints detailed statistics including:

              Number of datalist files processed and successfully read

              Total pings processed from all files

              Total beams encountered vs. valid beams (acceptance rate)

              Geographic bounds of the processed data

              Sample soundings (first 10) with coordinates and quality flags


PROCESSING NOTES
       Beam Quality Filtering
       mbmesh automatically filters out beams flagged as bad or suspect using MB-System's standard beam quality flags. Only
       beams marked as good are included in the output point clouds. The percentage of valid beams vs. total beams is
       reported in the statistics.

       Geographic Filtering
       When -R is specified, soundings outside the geographic bounding box are excluded from output. This is useful for
       processing large datasets or focusing on areas of interest.

       Coordinate Transformations
       The adjusted point cloud uses a local projected coordinate system relative to the data centroid for better numerical
       stability in 3D applications. The ECEF point cloud preserves absolute WGS84 coordinates. Both formats include
       reference information in file headers for reproducibility.

       Memory Usage
       mbmesh stores all soundings in memory during processing. For very large datasets, this may require significant RAM.
       The program reports total soundings collected upon completion.


MB-SYSTEM INTEGRATION
       mbmesh integrates with the MB-System datalist processing framework, allowing it to process multiple files in any MB-
       System supported format.  File format is determined automatically or can be specified via the datalist file.
       Processed files (if available) are preferred over raw files unless the datalist indicates otherwise.


FUTURE DEVELOPMENT
       Planned enhancements to mbmesh include:

              Triangle mesh generation from point cloud with seafloor feature preservation

              Py3dtiles integration and evaluation of alternative 3D Tiles libraries

              Draco compression support for efficient GLB file transmission

              Improved X3DOM visualization and viewer enhancements

              Refinement of GeoOrigin handling and direct ECEF coordinate output


AUTHOR
       David W. Caress and Dale N. Chayes
       Monterey Bay Aquarium Research Institute
       Original implementation: CSUMB Capstone - Spring 2026


SEE ALSO
       mb-system(1), mbgrid(1), mbgrd2gltf(1), mbgrdviz(1)

                                                        6 March 2026                                               mbmesh(1)

[MBARIMike]

@dwcaress This PR is ready for merging into the master branch.