PointCal-C — selective zero-shot 3D recognition under corruption

A post-hoc calibration and abstention head on top of a frozen OpenCLIP ViT-B/32, for zero-shot 3D point-cloud recognition under corruption, plus the complete evaluation artifacts backing it.

Code: https://github.com/crimsonKn1ght/pointcal-c

What the model is

The backbone is never trained. laion/CLIP-ViT-B-32-laion2B-s34B-b79K is loaded frozen, in eval mode, with requires_grad_(False).

The fitted model is four scalars, in runs/full/results/calibration.json, fit on clean calibration objects only — corrupted labels, corruption identity and severity never touch a fitted parameter:

parameter	value	role
temperature	1.2309	divides logits before softmax
bias	-1.5723	combined-score intercept
weight_confidence	+1.7071	on logit(p_calibrated)
weight_disagreement	+4.9158	on cross-view disagreement d

Combined score: sigmoid(bias + w_conf * logit(p_cal) + w_dis * d), where d is 1 - mean pairwise Jensen-Shannon divergence across the six views.

Converged, unclamped, not degenerate. Fit in 0.94 s on CPU.

How inference works

point cloud (N,1024,3)
  -> centroid-center, unit-radius normalize
  -> 6 orthographic cameras  [front right back left top bottom]
  -> 64x64 splat raster + nearest-depth z-buffer -> upsample to 224x224
  -> frozen OpenCLIP ViT-B/32 image encoder
  -> per-view logits vs a fixed 8-template prompt ensemble   <-- cached here
  -> mean over views  ->  the prediction
  -> four confidence scores over that one prediction

All four confidence methods (msp, temperature, disagreement, combined) rank the same predictions. A confidence method that changed a prediction would be a different classifier, and assert_predictions_unchanged raises if one ever does.

Results

All 75 corrupted conditions pooled against clean, 1975 evaluation objects, 95% bootstrap intervals resampled over base object IDs:

	accuracy	ECE (MSP)	ECE (combined)	AURC (MSP)	AURC (combined)
clean	0.2896	0.1160	0.0245	0.4339	0.4147
corrupted	0.2354	0.1416	0.0229	0.5217	0.5008

Accuracy degrades under corruption and MSP calibration degrades with it. The clean-fit combined score holds ECE near 0.023 under shift and lowers AURC by 0.021.

The ECE improvement is large and consistent. The AURC gain is small and should be read against the intervals in results.csv, not as a headline. This is a reliability audit and a bounded abstention baseline, not a state-of-the-art claim.

Accuracy by severity: 0.2896 (clean), 0.2593, 0.2515, 0.2388, 0.2252, 0.2023 — monotonically non-increasing.

Repository contents

The expensive artifact is runs/*/logits/ — per-view logits for every condition under both prompt modes. Every metric, ablation, bootstrap interval and figure here was computed on CPU from those caches, so any re-analysis is free: a different aggregation, disagreement statistic, coverage level or bootstrap depth costs no GPU time, because CLIP never runs again.

runs/<tier>/
  logits/<condition>.npz     per-view logits, both prompt modes (~1.4 MB each)
  results/calibration.json   the fitted scalars above
  results/results.csv|.json  metrics table with intervals
  results/ablations.csv      view-count, prompt-mode, disagreement ablations
  results/predictions.npz    per-sample confidences and correctness
  results/results_summary.md auto-generated tables + hypothesis checks
  figures/fig1..fig4         accuracy, ECE, risk-coverage, cost
  provenance/                data manifest, run manifest, split audit
  ledger_inference.json      measured GPU-hours, dollars, throughput, memory
artifacts/split.json         the frozen calibration/evaluation object split
env/requirements.lock.txt    exact resolved environment (171 packages)

Tiers: xs (5 conditions, 100 objects), s (13 conditions), full (all 76 = clean + 15 corruptions x 5 severities).

Training / evaluation data

Evaluated on ModelNet40-C (2468 objects, 40 classes, 15 corruption types, 5 severities). The calibration scalars are fit on the clean split of 493 calibration objects; 1975 evaluation objects are held out.

The split is over base object IDs, not samples, and every corruption array is verified row-aligned with the clean array, so holding an object out of calibration holds it out under all 76 conditions. Audited: overlap: 0, 0 row-misaligned conditions, all 40 classes present on both sides.

Compute

tier	conditions	GPU-hours	USD	views/s	peak VRAM
xs	5	0.001	0.0003	1091	2.5 GB
s	13	0.015	0.004	2816	12.1 GB
full	76	0.089	0.025	2808	12.1 GB

0.105 GPU-hours and about $0.03 of inference in total, on a single Runpod RTX 4000 Ada at $0.28/hr (2026-08-16).

Limitations

• full uses 200 bootstrap replicates, not the preregistered 1000. The CPU bootstrap dominated runtime (~2 h at 1000 vs ~5 min of GPU work). Deeper intervals recompute from the cached logits at no GPU cost.
• NLL and Brier require a distribution over all 40 classes, so they are reported for msp and temperature only. For the two ranking scores they are null, not a lookalike substitute.
• Cross-view disagreement is prior art, and calibration degrading under 3D corruption is a confirmation rather than a discovery.
• Absolute accuracy is low (~0.29 clean). This is zero-shot transfer of a 2D model to depth renders, with no 3D training whatsoever.
• Severity is reported 1-indexed, matching the on-disk release.

Provenance

• Frozen spec hash: ca487597c978ab17217bd640ab8bf95c8a21db0eaacd51ae5ba4b00618dcc820
• Backbone checkpoint SHA-256: 1bd3c7172de5b207ceac554f5ab5266166f3b9baccc9af5989bc801016d080ad
• Split fingerprint: cc5ed4b3ce7108154039f497d474f3ac6bdbfe7eb7dbfe4d0f90785e1829bb31

Licence and attribution

Cached logits and metrics are derived from ModelNet40-C and redistributed under CC BY 4.0, as the source record declares.

• ModelNet40-C — Zenodo 6017834, DOI 10.5281/zenodo.6017834, CC BY 4.0
• Generation code — https://github.com/jiachens/ModelNet40-C, BSD-3-Clause
• Underlying ModelNet40 carries its own original terms
• OpenCLIP — MIT · PointCal-C code — MIT

The multi-view depth-projection approach follows the PointCLIP paper (CVPR 2022); the projector is an independent reimplementation from the paper text, with no PointCLIP repository code used.