Issue with Extrinsics pipeline

[machinethinking]

Hello, thank you for making this as it greatly helps some experiments I'm working on.

I'm having an issue with the extrinsics pipeline. Intrinsics has calculated just fine. But when I drop two files of the same name into the extrinsics directory it's prompting me for a timestamps.csv file. I do not have hardware sync on my videos, but I did do a clap and synced them in post and exported them with exactly the same length. Do I still need a timestamps.csv file? Is there a work around? Thank you

[mprib]

@machinethinking,

Thank you for raising this. As a quick workaround you could try mocking a timestamps.csv, but certainly if it requires that then it is a bug. I need to clean up the safety checks and hopefully can get a fix pushed out by tomorrow afternoon.

And just to clarify:

• do your files have the exact same number of frames?
• when you say "two files of the same name", do you mean "both following the same naming convention"?

Regards,

Mac

[mprib]

@machinethinking,

I just pushed a patch to main and PyPI (0.7.3). This latest version will create an inferred_timestamps.csv file based on the frame counts and the assumption that that all videos start and stop at the exact same time. Drift can cause the frame counts to get unaligned, and this approach attempts to smooth that out. I tested it on some local files (deleted my timestamps.csv) and it appears to be working.

I would greatly appreciate your feedback to see if this did the trick.

Thank you again for raising this issue!

Mac

[machinethinking]

It sort of works. If I reopen my project, it takes what feels like a few minutes to load. I thought it was Unresponsive until the third time when I went to get a drink and discovered it did eventually load. I am able to click 'Start Processing' on the Multi-Camera tab and it says "Starting" but nothing appears to happen for a couple of minutes. Then the status bar updates to "Processing complete" (it had not moved before) and in the box it says Shared Point Observations No structural issues detected. Then I head over to the Capture Volume tab and click Calibrate and it fails. I've included the logs below. Happy to give more debugging info if needed.

2026-03-08 18:42:42 | INFO | caliscope.gui.views.multi_camera_processing_widget | 413 | Starting processing with subsample=5
2026-03-08 18:42:42 | INFO | caliscope.gui.presenters.multi_camera_processing_presenter | 280 | Starting multi-camera processing: C:\Users\wfran\Desktop\MT-sync\o\Stereo2\calibration\extrinsic
2026-03-08 18:42:42 | INFO | caliscope.recording.synchronized_timestamps | 285 | No timestamps.csv found in C:\Users\wfran\Desktop\MT-sync\o\Stereo2\calibration\extrinsic, inferring from videos
2026-03-08 18:42:42 | INFO | caliscope.recording.video_utils | 39 | Reading video properties from: C:\Users\wfran\Desktop\MT-sync\o\Stereo2\calibration\extrinsic\cam_0.mp4
2026-03-08 18:42:42 | INFO | caliscope.recording.video_utils | 39 | Reading video properties from: C:\Users\wfran\Desktop\MT-sync\o\Stereo2\calibration\extrinsic\cam_1.mp4
2026-03-08 18:42:43 | INFO | caliscope.recording.synchronized_timestamps | 266 | Inferred timestamps for 2 cameras, -2 total frames, avg duration -0.017s
2026-03-08 18:42:43 | INFO | caliscope.task_manager.task_manager | 89 | Starting task 'Multi-camera processing' (id=416cef13-96c2-4391-aab5-7d63ade941c9)
2026-03-08 18:42:43 | INFO | caliscope.core.process_synchronized_recording | 68 | Processing 0 sync indices (subsample=5, total available=0)
2026-03-08 18:42:43 | INFO | caliscope.workspace_coordinator | 146 | Directory changed: C:\Users\wfran\Desktop\MT-sync\o\Stereo2\calibration\extrinsic
2026-03-08 18:42:43 | INFO | caliscope.workspace_coordinator | 271 | All extrinsics calculated: False
2026-03-08 18:42:43 | INFO | caliscope.workspace_coordinator | 275 | Point estimates available: False
2026-03-08 18:42:43 | INFO | caliscope.workspace_coordinator | 278 | All underlying data available: True
2026-03-08 18:42:43 | INFO | caliscope.workspace_coordinator | 146 | Directory changed: C:\Users\wfran\Desktop\MT-sync\o\Stereo2\calibration\extrinsic
2026-03-08 18:42:43 | INFO | caliscope.workspace_coordinator | 271 | All extrinsics calculated: False
2026-03-08 18:42:43 | INFO | caliscope.workspace_coordinator | 275 | Point estimates available: False
2026-03-08 18:42:43 | INFO | caliscope.workspace_coordinator | 278 | All underlying data available: True
2026-03-08 18:44:42 | INFO | caliscope.gui.presenters.multi_camera_processing_presenter | 407 | Multi-camera processing complete: 0 points
2026-03-08 18:44:42 | INFO | caliscope.workspace_coordinator | 597 | Persisted extrinsic image points: 0 observations to C:\Users\wfran\Desktop\MT-sync\o\Stereo2\calibration\extrinsic\CHARUCO\image_points.csv
2026-03-08 18:44:42 | INFO | caliscope.workspace_coordinator | 271 | All extrinsics calculated: False
2026-03-08 18:44:42 | INFO | caliscope.workspace_coordinator | 275 | Point estimates available: False
2026-03-08 18:44:42 | INFO | caliscope.workspace_coordinator | 278 | All underlying data available: True
2026-03-08 18:46:19 | INFO | caliscope.task_manager.task_manager | 89 | Starting task 'Extrinsic calibration' (id=03c7ec25-2a46-45fe-af04-2a9823791757)
2026-03-08 18:46:19 | INFO | C:\Users\wfran\o\caliscope-0.7.2.venv\Lib\site-packages\caliscope\core\bootstrap_pose\pose_network_builder.py | 95 | ==================== Step 1a: Camera-to-Object Poses ====================
2026-03-08 18:46:19 | INFO | C:\Users\wfran\o\caliscope-0.7.2.venv\Lib\site-packages\caliscope\core\bootstrap_pose\pose_network_builder.py | 227 | Computing per-frame camera poses with PnP (min_points=4)...
2026-03-08 18:46:19 | INFO | C:\Users\wfran\o\caliscope-0.7.2.venv\Lib\site-packages\caliscope\core\bootstrap_pose\pose_network_builder.py | 230 | Pre-undistorting points...
2026-03-08 18:46:19 | ERROR | C:\Users\wfran\o\caliscope-0.7.2.venv\Lib\site-packages\caliscope\core\bootstrap_pose\pose_network_builder.py | 114 | Failed to estimate camera-to-object poses: No valid camera data found for PnP
2026-03-08 18:46:19 | ERROR | caliscope.task_manager.task_handle | 90 | Task 'Extrinsic calibration' failed: ValueError: No valid camera data found for PnP
2026-03-08 18:46:19 | ERROR | caliscope.gui.presenters.extrinsic_calibration_presenter | 514 | Calibration failed: ValueError: No valid camera data found for PnP

[machinethinking]

Ok, turns out this was (mostly) on me. When I made the synced extrinsic files I exported them with the format of mp4, but the codec of APV (Davinci Resolve default). I switched codec to h.265 and the Multi-camera tab ran and seemed to give some useful output. However, when I switch to the Capture Volume tab and run Calibrate there, I get this error:

2026-03-08 20:14:02 | INFO | caliscope.task_manager.task_manager | 89 | Starting task 'Extrinsic calibration' (id=94bc0cd2-883c-4887-bbad-0e5ebe378ab4)
2026-03-08 20:14:02 | ERROR | caliscope.task_manager.task_handle | 90 | Task 'Extrinsic calibration' failed: SchemaError: Error while coercing 'sync_index' to type int64: Could not coerce <class 'pandas.Series'> data_container into type int64:
index failure_case
0 9987 NaN
2026-03-08 20:14:02 | ERROR | caliscope.gui.presenters.extrinsic_calibration_presenter | 514 | Calibration failed: SchemaError: Error while coercing 'sync_index' to type int64: Could not coerce <class 'pandas.Series'> data_container into type int64:
index failure_case
0 9987 NaN

[mprib]

@machinethinking,

Thank you for this valuable feedback. I'm gathering a few things from the information you are sharing and want to lay out my plan:

1. Video File Read Error: I appreciate you catching the file format issue. I will update the video reader to detect unsupported codecs and give a clear error message.
2. Long load times: Sorry that it's taking something measured in minutes. I've noted slow loads on projects locally, but for me that's been more like 5 seconds so I deferred looking into it in greater detail. I will dig into this to look for opportunities to streamline the launch.
3. Schema error on image_points: Could you share the image_points.csv and inferred_timestamps.csv files? The image_points.csv should be in calibration/extrinsic/CHARUCO/ (or ARUCO) and the inferred_timestamps.csv should be in calibration/extrinsic/. If you ended up creating a timestamps.csv yourself, please share that as well. The input validation is rejecting the image_points data which is what feeds into the calibration. It's not clear to me how things might have gotten out of whack there, and I'd really need to see the data to sort it out.

I can get started on 1 and 2 now and will wait for those csv files to get started on 3.

Thank you,

Mac

[machinethinking]

@mprib,
Getting back to your question in your first reply, yes, "two files of the same name" meant following the same convention.

Now that I have the video files in the right format, the load time has dropped a lot when I load a project. Maybe 30 seconds now. However, I also have another slow load time, maybe also 30 seconds, when I type 'caliscope' at the command line before the GUI comes up.

Here are the two files you requested. Let me know what else I can do to help.

image_points.csv

inferred_timestamps.csv

[mprib]

@machinethinking,

Thank you for sharing those files. The timestamps look good. The image_points.csv looks good as a csv but has a weird entry in one field that may be an artifact of a corrupted save. The current version (0.7.5) changes how things are saved so that nothing get saved mid-process, which will hopefully protect from corruption if there is a crash or forced close.

I've also pushed some performance improvements that should help with load times by running more reading from disk in parallel. I was able to trim my load times from about 3.5 seconds to 2.5 seconds, but this was on linux with an NVME so its hard for me to know how this will translate to windows. Hopefully this allows for faster processing on your end, and no mid-process closure. Fingers crossed....

If you delete calibration/extrinsic/CHARUCO/image_points.csv and re-run "Start Processing" on the Multi-Camera tab it should regenerate cleanly.

PyPI is updated now with the latest version (0.7.5) which reflects the current main.

Thank you for your patience while we sort this out!

Mac