Upload via Autoresearch export_data
Browse files
README.md
CHANGED
|
@@ -1,3 +1,47 @@
|
|
| 1 |
-
|
| 2 |
-
|
| 3 |
-
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Polymorphic
|
| 2 |
+
|
| 3 |
+
**Compute-efficient video-to-3D research by Modular Computing.**
|
| 4 |
+
|
| 5 |
+
This repository contains the Polymorphic video-to-3D experiments and releases. The current public experiment combines **AnySplat** for scene/camera initialization with **Fourier Splatting** for per-scene optimization and novel-view synthesis.
|
| 6 |
+
|
| 7 |
+
> **Important:** This is an experimental reconstruction pipeline, not a feed-forward model that instantly converts any video into a finished asset. Each new scene requires optimization. The released representation is a renderable splat scene; mesh/GLB conversion is not included as a validated workflow.
|
| 8 |
+
|
| 9 |
+
## Featured experiment: Fourier Splatting
|
| 10 |
+
|
| 11 |
+
[**Open the full experiment model card and run instructions →**](experiments/fourier-splatting/README.md)
|
| 12 |
+
|
| 13 |
+
The experiment directory contains:
|
| 14 |
+
- Runnable preparation and evaluation scripts
|
| 15 |
+
- Apartment and kitchen scene checkpoints
|
| 16 |
+
- PLY exports at Fourier frequency levels K=1–6
|
| 17 |
+
- Per-scene metrics, experiment report, and third-party license notices
|
| 18 |
+
|
| 19 |
+
### Held-out novel-view results
|
| 20 |
+
|
| 21 |
+
Metrics are from two indoor scenes, using the same VGG LPIPS implementation for both methods. Higher PSNR/SSIM is better; lower LPIPS is better.
|
| 22 |
+
|
| 23 |
+
| Scene | Method | PSNR ↑ | SSIM ↑ | VGG LPIPS ↓ | Primitives |
|
| 24 |
+
|---|---|---:|---:|---:|---:|
|
| 25 |
+
| Apartment | AnySplat baseline | 19.1235 | 0.6612 | 0.3154 | 1,858,896 |
|
| 26 |
+
| Apartment | Fourier Splatting K=6 | 20.3469 | 0.7041 | 0.3173 | 99,483 |
|
| 27 |
+
| Kitchen | AnySplat baseline | 21.6155 | 0.6660 | 0.2678 | 1,983,229 |
|
| 28 |
+
| Kitchen | Fourier Splatting K=6 | 25.3078 | 0.8212 | 0.2586 | 120,000 |
|
| 29 |
+
|
| 30 |
+
Across these two scenes, Fourier Splatting used approximately **16.5–18.7× fewer primitives** than the AnySplat baselines. Apartment LPIPS was slightly worse, so the results do not indicate uniform improvement. This is a small exploratory evaluation, not a generalization or state-of-the-art claim.
|
| 31 |
+
|
| 32 |
+
## Quick links
|
| 33 |
+
|
| 34 |
+
- [Fourier Splatting model card and instructions](experiments/fourier-splatting/README.md)
|
| 35 |
+
- [Experiment report](experiments/fourier-splatting/FOURIER_SPLATTING_REPORT.md)
|
| 36 |
+
- [Metrics JSON](experiments/fourier-splatting/fourier_results.json)
|
| 37 |
+
- [Generic video-frame preparation script](experiments/fourier-splatting/scripts/make_fourier_scene_generic.py)
|
| 38 |
+
- [Apartment PLY, K=6](experiments/fourier-splatting/models/apartment/lod/fourier_k6.ply)
|
| 39 |
+
- [Kitchen PLY, K=6](experiments/fourier-splatting/models/kitchen/lod/fourier_k6.ply)
|
| 40 |
+
|
| 41 |
+
## Reproducing the experiment
|
| 42 |
+
|
| 43 |
+
Start with the [experiment README](experiments/fourier-splatting/README.md), which documents dependencies, frame extraction, scene preparation, optimization, rendering, results, and known limitations. The research run was conducted on an NVIDIA H100 80 GB with PyTorch/CUDA 12.9 at 448×448 resolution. The complete workflow has **not yet been validated from a clean environment**, and dependency compatibility may require adjustments.
|
| 44 |
+
|
| 45 |
+
## Licensing
|
| 46 |
+
|
| 47 |
+
This is a research release that incorporates upstream AnySplat and Fourier Splatting code. Their respective license terms apply; see the license notices in the experiment directory. In particular, review Fourier Splatting's non-commercial research/evaluation terms before any commercial use.
|