Advanced COLMAP Tips: Improving SfM and MVS Results

Advanced COLMAP Tips: Improving SfM and MVS Results

Structure-from-Motion (SfM) and Multi-View Stereo (MVS) with COLMAP can produce excellent 3D reconstructions, but achieving clean, accurate models requires careful capture, preprocessing, and parameter tuning. Below are practical, advanced tips to improve both sparse (SfM) and dense (MVS) stages, organized by capture, preprocessing, SfM, MVS, and post-processing.

Capture: set the foundation

• Overlap: Aim for 70–85% overlap between adjacent images for smooth feature tracking. For difficult surfaces (low texture, repeating patterns), increase to 85–95%.
• Baseline: Keep a mix of short and medium baselines. Short baselines help dense matching; longer baselines improve scale and robustness.
• Angles & coverage: Capture from multiple viewpoints and heights—top, oblique, and close-up—so features are visible from diverse directions.
• Consistent exposure: Use manual exposure where possible to avoid large brightness changes; shoot in RAW to preserve detail.
• Scale references: Photograph a few scale bars or rulers if absolute scale is needed; otherwise ensure some objects of known size are present.

Preprocessing: clean and organize images

• Lens correction: If using wide or fisheye lenses, undistort images first (COLMAP can estimate distortion, but pre-undistortion with known calibration reduces errors).
• Remove duplicates/blurry frames: Use automated scripts or quick visual checks to delete near-duplicates and motion-blurred images.
• Masking: Create masks for moving objects or irrelevant sky/vehicle areas — masking reduces false matches and improves reconstruction of static geometry.
• Downsampling strategy: For initial SfM, downsample large images (e.g., to 2–6 MP) to speed up feature detection; keep originals for final dense reconstruction.

SfM (sparse reconstruction) tips

• Feature detector/descriptor: Use SIFT (default) for robustness. For very large datasets, consider RootSIFT variants or GPU-accelerated options if available.
• Matching strategy: Use exhaustive matching for small-to-medium datasets. For large datasets, use vocabulary/tree-based or sequential matching to limit memory and time.
• Feature filtering: Increase the ratio test strictness or enable cross-checking to remove ambiguous matches. In COLMAP, adjust the “max-reference” and “max-forward” matches to control graph connectivity.
• Keypoint tuning: Lower the keypoint detection threshold slightly for low-texture scenes; raise it for very noisy images to avoid spurious features.
• Camera model choice: For standard lenses use the pinhole-radtan model; for fisheye use the fisheye model. Correct camera model reduces reprojection errors.
• Robust initialization: Start with well-overlapped image pairs for incremental reconstruction. Use registered GPS/EXIF hints to seed ordering if capture was sequential.
• Bundle adjustment settings: Use robust loss functions (e.g., Huber) and set reasonable convergence thresholds. For large projects, run BA on subsets first, then global BA.
• Outlier removal: After initial reconstruction, remove cameras with large reprojection errors or very few registered points. Re-run BA to refine.

MVS (dense reconstruction) tips

• Image resolution: Use the highest practical resolution for MVS; downsample only to speed up processing when necessary. Preserve details where depth precision matters.
• Stereo pair selection: Adjust the maximum image pair overlap and the two-view consistency to include more reliable neighbor pairs for depth estimation.
• PatchMatch parameters: Increase the number of PatchMatch iterations and propagation steps for difficult surfaces. Increase the depth candidate sampling density when fine detail is required.
• Photometric consistency: Tighten photometric thresholds for consistency checks to reduce false depths on reflective or textureless surfaces. Use multi-scale consistency to handle varying detail.
• Filtering: Post-filter depth maps aggressively where noise appears—use geometric consistency, median filtering, and photometric filters provided by COLMAP.
• Fusion settings: Use conservative fusion settings (higher min-consistent-views) to avoid spurious triangles. For complete surfaces, lower the threshold slightly but follow with mesh smoothing and cleaning.
• Normals & visibility: Recompute normals carefully from fused depth maps and ensure visibility tests are strict to prevent occluded triangles being kept.

Post-processing: cleanup and enhancement

• Mesh simplification: Reduce triangle count while preserving shape using quadric edge collapse or similar tools; keep a high-resolution mesh for detailed areas.
• Hole filling: Use local Poisson or screened Poisson with masked guidance to fill small holes; avoid aggressive filling that invents geometry.
• Laplace/HC smoothing: Apply mild smoothing to remove noise but preserve edges. Use HC (Humphrey’s Classes) smoothing for better feature preservation.
• Texture baking: Generate high-resolution textures from original images, blend exposures, and correct seams manually if necessary.
• Scale & align: Use known scale markers or ICP alignment to align and scale reconstructions to real-world coordinates for measurement tasks.

Troubleshooting common problems

• Sparse model fragmented: Increase feature overlap, add intermediary images, relax matching thresholds slightly, or use sequential matching if capture was along a path.
• Many outlier points: Tighten ratio test, enable symmetric matching, and remove images with few matches before BA.
• Dense noise on reflective surfaces: Mask reflections if possible; use stronger photometric consistency and geometric filtering.
• Thin structures missing: Increase image resolution, ensure sufficient viewpoint diversity, and lower fusion view-count threshold for those areas (then clean up false positives).

Example parameter suggestions (starting points)

• SfM: SIFT peak threshold 0.01–0.02; matching ratio 0.7; exhaustive matching for N<2000.
• BA: robust loss (Huber), max 50 iterations for incremental, global BA until convergence.
• MVS: PatchMatch iterations 6–10; min-consistent-views for fusion 3–5; photometric consistency threshold 0.9–0.95 (tune per dataset).

Final checks before delivery

• Inspect camera trajectory for jumps or isolated cameras.
• Visualize reprojection errors and remove outliers.
• Compare reconstructed dimensions against known measures.
• Run a quick textured-render pass to verify appearance and seam quality.

Use these tips as a baseline; tune per dataset and iterate—small changes in capture or parameters often yield the largest improvements.