Cloud vs Local — MF preference model

Simple Matrix Factorization model that predicts which model to use for a prompt: cloud or local. Trained on preference data of the form (user, prompt, chosen) where chosen ∈ {cloud, local}.

🚨 Use real preference data for better results

This model is trained on synthetic demo data (12,000 generated ratings) — it exists to validate the pipeline and the tooling, not to be the final product. For a model that actually routes your traffic, retrain on your real preference logs:

Export a CSV with at least three columns: user_id, prompt, chosen (values cloud or local).
Open mf_cloud_vs_local.ipynb in Jupyter and run the last code cell: train_on(load_preference_data("your_real_data.csv")).
The whole pipeline — EDA, stratified train/val/test split, MF training, evaluation, bundle export — runs unchanged on your file.

Real data gives you: trustworthy metrics, per-topic routing insights from your own prompts, and a cleaner cold-start story (see Limitations).

Model

r̂(u,i) = μ + b_u + b_i + ⟨p_u, q_i⟩ — user bias + prompt bias + dot product of latent factor vectors (k = 8), trained with binary cross-entropy + L2 regularization via minibatch SGD (NumPy), early-stopped on validation AUC.

Decision rule: p(cloud) = σ(r̂) ≥ 0.5 → cloud, else local.

Metrics (held-out test, n = 1,200 — synthetic data)

metric	value	baseline (always cloud)
accuracy	0.677	0.483
AUC	0.734	0.500

The model recovers the latent topic structure: privacy-sensitive and latency-sensitive prompts are routed to local; complex-reasoning and long-context prompts to cloud.

Files in this repo

README.md — this model card
mf_cloud_vs_local.ipynb — full source notebook (data gen, EDA, training, eval, bundle, diagram, inference demo)
mf_inference.py — sample inference script (CLI, loads the bundle with SimpleMF.from_pretrained)
mf_model_diagram.png — training + inference pipeline diagram
config.json — hyperparameters + metrics
mf_params.npz — mu, bu, bi, P, Q, user_ids, prompt_ids (weights + vocab)
test_predictions.csv — per-pair predictions on the held-out test set

Usage

pip install numpy datasets huggingface_hub

# score specific (user, prompt) pairs
python mf_inference.py --model_dir mf_bundle --user_id U0007 \
    --prompt_ids P0001,P0400,P0572 --csv preference_data_synthetic.csv

# rank all known prompts for a user
python mf_inference.py --model_dir mf_bundle --user_id U0007 --top_k 5 \
    --csv preference_data_synthetic.csv

Clone this repo and run mf_inference.py locally, or open mf_cloud_vs_local.ipynb to retrain on your own preference data.

Limitations

Cold start: prompts not seen in training have no q_i embedding — the model cannot score brand-new prompt text. Extend with a text encoder (e.g. embed prompt text into the latent space) for production cold start.
Synthetic data: metrics above are on synthetic ratings; expect different numbers on real preference logs — see "Use real preference data" above.
No leakage checks were needed for synthetic data; run a near-duplicate check between train/val/test when real data arrives.

Downloads last month: -