Model Card for ALICE

ALICE (Agglomerative Learning via Integrated Computational pathology Embedding) is a unified general-purpose pathology foundation model trained through multi-stage agglomerative distillation, which sequentially distills eight vision-only, vision-language, and slide-level teacher models into dedicated modules of a single backbone. It provides a unified representation for ROI tissue analysis, vision-language multimodal understanding, and whole-slide clinical assessment, achieving the best average rank among task-matched pathology foundation models across 21 task scenarios, 96 downstream tasks, and 48 data sources.

Model Description

• Developed by: Tsinghua Shenzhen International Graduate School, Tsinghua University
• Model type: Unified vision, vision-language, whole-slide image encoders
• Pretrained datasets: 24,985,184 tile-level pathology images + 155,604 high-resolution pathology images
• License: CC-BY-NC-ND-4.0

Requirements

pip install torch torchvision timm transformers

Model Usage

from huggingface_hub import login
from transformers import AutoModel

login()

alice = AutoModel.from_pretrained("JWonderLand/ALICE", trust_remote_code=True).eval()

Image Preprocessing

The image preprocessing is equivalent to alice.image_transform:

from PIL import Image
from torchvision import transforms
from torchvision.transforms import InterpolationMode

image_preprocess = transforms.Compose([
    transforms.Resize(224, interpolation=InterpolationMode.BICUBIC, antialias=True),
    transforms.CenterCrop(224),
    transforms.ToTensor(),
    transforms.Normalize(
        mean=(0.48145466, 0.4578275, 0.40821073),
        std=(0.26862954, 0.26130258, 0.27577711),
    ),
])

example_image_path = './example.jpg'
image = Image.open(example_image_path).convert("RGB")
input_img_tensor = image_preprocess(image).unsqueeze(0)  # [1, 3, 224, 224]

Direct Use

ALICE contains three architectures: vision-only, vision-language, and slide-level.

import torch
from PIL import Image

example_image_path = './example.jpg'

with torch.no_grad():
    # Vision-only: image -> raw vision feature
    # [B, 3, H, W] -> [B, 3840]
    image = Image.open(example_image_path).convert("RGB")
    image_tensor = alice.image_transform(image).unsqueeze(0)  # [1, 3, 224, 224]
    vision_features = alice.vision_stage(image_tensor)
    # vision_features: [B, 3840]

    # Vision-language: vision_stage output -> dict of teacher-head features
    # [B, 3840] -> dict[str, Tensor]
    vl_features = alice.vl_stage(vision_features)
    # vl_features["keep"]:     KEEP head,     [B, 768]
    # vl_features["conch_v1"]: CONCH v1 head, [B, 512]
    # vl_features["musk"]:     MUSK head,     [B, 1024]

    # Slide-level: patch_features + coords -> slide feature
    # [N, 3840] + [N, 2] -> [B, 2048]
    patch_features = torch.randn(100, 3840)
    coords = torch.randint(0, 10000, (100, 2))
    slide_features = alice.slide_stage(patch_features, coords=coords, patch_size_lv0=512)
    # slide_features: [B, 2048]