Article
AX-Ray, Finding Causal-Leakage Defects in Two General-Purpose Public Models
FINAL-Bench
β’ β’ 13
gen
ginigini
AI & ML interests
None yet
Recent Activity
upvoted an article about 8 hours ago
AX-Ray, Finding Causal-Leakage Defects in Two General-Purpose Public Models liked a dataset about 8 hours ago
FINAL-Bench/AX-RAY liked a Space about 8 hours ago
FINAL-Bench/AX-RAYOrganizations
None yet
upvoted an article about 8 hours ago
liked a
dataset about 8 hours ago
liked a
Space about 8 hours ago
reacted to SeaWolf-AI's post with π about 8 hours ago
Post
540
AX-Ray: Safety Diagnostics for AI/AX Models
AI models can no longer be evaluated only by capability scores. As models move into public services, enterprise workflows, scientific research, and administrative decision support, we need a second layer of evaluation: whether the model behaves safely, structurally, and consistently under real deployment conditions.
VIDRAFT AX-Ray is a public AI/AX safety diagnostic initiative powered by FINAL-Bench Diagnostics. AX-Ray evaluates models across a structured guideline framework, including model-level safety, AX deployment readiness, and agent/service operation risks. The public diagnostic catalog contains 117 diagnostic items, mapped to legal, regulatory, ethical, and religious-law governance contexts so that safety review can be discussed in a form closer to real institutional responsibility.
A central finding of AX-Ray is causal leakage: a structural defect where information that should not influence an earlier reasoning state appears to affect model behavior. AX-Ray presents a public case of diagnosing, reproducing, and demonstrating causal leakage in two general-purpose public models. This matters because such defects are not exposed by ordinary benchmark scores. A model can appear capable while still carrying hidden safety or integrity risks.
Explore the live leaderboard, diagnostic reports, and public dataset here:
- AX-Ray Space: FINAL-Bench/AX-RAY
- AX-Ray Dataset: FINAL-Bench/AX-RAY
- Technical Article: https://huggingface.co/blog/FINAL-Bench/ax-ray
AX-Ray is intended as a practical guideline for moving AI evaluation beyond βhow smart is the model?β toward βcan this model be trusted, governed, and deployed safely?β
AI models can no longer be evaluated only by capability scores. As models move into public services, enterprise workflows, scientific research, and administrative decision support, we need a second layer of evaluation: whether the model behaves safely, structurally, and consistently under real deployment conditions.
VIDRAFT AX-Ray is a public AI/AX safety diagnostic initiative powered by FINAL-Bench Diagnostics. AX-Ray evaluates models across a structured guideline framework, including model-level safety, AX deployment readiness, and agent/service operation risks. The public diagnostic catalog contains 117 diagnostic items, mapped to legal, regulatory, ethical, and religious-law governance contexts so that safety review can be discussed in a form closer to real institutional responsibility.
A central finding of AX-Ray is causal leakage: a structural defect where information that should not influence an earlier reasoning state appears to affect model behavior. AX-Ray presents a public case of diagnosing, reproducing, and demonstrating causal leakage in two general-purpose public models. This matters because such defects are not exposed by ordinary benchmark scores. A model can appear capable while still carrying hidden safety or integrity risks.
Explore the live leaderboard, diagnostic reports, and public dataset here:
- AX-Ray Space: FINAL-Bench/AX-RAY
- AX-Ray Dataset: FINAL-Bench/AX-RAY
- Technical Article: https://huggingface.co/blog/FINAL-Bench/ax-ray
AX-Ray is intended as a practical guideline for moving AI evaluation beyond βhow smart is the model?β toward βcan this model be trusted, governed, and deployed safely?β
upvoted an article 6 days ago
Article
Model Genome: Fingerprinting Whether an LLM Was Trained From Scratch or Derived
mayafree
β’ β’ 16 liked a
Space 6 days ago
reacted to mayafree's post with πππ₯ 6 days ago
Post
2369
𧬠Architecture lineage of Korea's sovereign-AI foundation models β checked with public data
In late July 2026, as Korea released self-developed foundation models competing with DeepSeek and Qwen (e.g. LG K-EXAONE 2.0, 750B), interest grew β including a Zhihu thread with 2.7M+ views (β https://www.zhihu.com/question/2067512422555029717 ) β over whether these models are trained from scratch or built on foreign open-weights.
Sharing a tool that answers this with public data rather than opinion.
π Model Genome Korea β mayafree/Model-Genome-Korea
It classifies the public models of 9 Korean organizations that released "self-developed, from-scratch foundation models" on HuggingFace β 3 large enterprises (LG, NAVER, Kakao), 2 telcos (SKT, KT), 2 mid-size firms (NCSOFT, Upstage), 2 startups (Motif, VIDRAFT) β on two axes measured from public config.json + model weights:
β’ Architecture fingerprint β does model_type + (hiddenΒ·intermediateΒ·layers) match a foreign open-weight model
β’ Weight fingerprint β embedding similarity (from-scratch vs continued-pretraining)
Genotypes: π’ Native Β· π΅ Adapted Β· π‘ Mixed Β· π΄ Ported
The results are not uniform. Some models match foreign architectures (Qwen, Llama, β¦) exactly; others use self-built architectures and weights with no foreign match. Which company/model falls where is shown per model in the Space, along with attention originality, license, and reproducible open-source status.
This is a neutral transparency tool, not an accusation β building foundation models on open-weight bases is a legitimate, industry-standard practice. The exact same yardstick is applied to every model, without exception.
Features a 3D lineage graph, search, EN / δΈζ / νκ΅μ΄, and dark mode. Corrections are welcome via the Community tab.
Articles: https://huggingface.co/blog/mayafree/model-dna
#KoreanAI #LLM #ModelLineage #OpenSource #SovereignAI
In late July 2026, as Korea released self-developed foundation models competing with DeepSeek and Qwen (e.g. LG K-EXAONE 2.0, 750B), interest grew β including a Zhihu thread with 2.7M+ views (β https://www.zhihu.com/question/2067512422555029717 ) β over whether these models are trained from scratch or built on foreign open-weights.
Sharing a tool that answers this with public data rather than opinion.
π Model Genome Korea β mayafree/Model-Genome-Korea
It classifies the public models of 9 Korean organizations that released "self-developed, from-scratch foundation models" on HuggingFace β 3 large enterprises (LG, NAVER, Kakao), 2 telcos (SKT, KT), 2 mid-size firms (NCSOFT, Upstage), 2 startups (Motif, VIDRAFT) β on two axes measured from public config.json + model weights:
β’ Architecture fingerprint β does model_type + (hiddenΒ·intermediateΒ·layers) match a foreign open-weight model
β’ Weight fingerprint β embedding similarity (from-scratch vs continued-pretraining)
Genotypes: π’ Native Β· π΅ Adapted Β· π‘ Mixed Β· π΄ Ported
The results are not uniform. Some models match foreign architectures (Qwen, Llama, β¦) exactly; others use self-built architectures and weights with no foreign match. Which company/model falls where is shown per model in the Space, along with attention originality, license, and reproducible open-source status.
This is a neutral transparency tool, not an accusation β building foundation models on open-weight bases is a legitimate, industry-standard practice. The exact same yardstick is applied to every model, without exception.
Features a 3D lineage graph, search, EN / δΈζ / νκ΅μ΄, and dark mode. Corrections are welcome via the Community tab.
Articles: https://huggingface.co/blog/mayafree/model-dna
#KoreanAI #LLM #ModelLineage #OpenSource #SovereignAI
- 3 replies
upvoted an article 11 days ago
Article
The Fast Gemma Challenge: our verified-SOTA recipe, in full
FINAL-Bench
β’ β’ 24 reacted to SeaWolf-AI's post with π§ π 11 days ago
Post
3427
We wrote up our run in The Fast Gemma Challenge β as vidraft-darwin β and wanted to share the recipe. π
gemma-challenge/gemma-dashboard
Verified result: 510.58 TPS at PPL 2.3930 on a single A10G (fw188-ctk49-n64-patchbridge, re-run & VERIFIED). Honest note: on raw TPS there are faster runs (535+), but those went over the PPL bar and didn't verify β what we're proud of is the fastest result that keeps quality.
The recipe is already open, so we explained each piece: sliding-window W188, CTK49 kernel tuning, noprecache (honest, verifiable measurement), and an N64 synthetic warmup bridge that shrinks the publicβprivate gap (~15 TPS), plus INT4 + MTP K=7 + CUDA-graph capture. One rule: only stack quality-neutral speedups.
Huge thanks to @firfir-cast , @gemma-slayer , @chiku-inu , @kenyan-duma , @dixie-flatline and everyone who shared their experiments. Full write-up
π
https://huggingface.co/blog/FINAL-Bench/fast-gemma
gemma-challenge/gemma-dashboard
Verified result: 510.58 TPS at PPL 2.3930 on a single A10G (fw188-ctk49-n64-patchbridge, re-run & VERIFIED). Honest note: on raw TPS there are faster runs (535+), but those went over the PPL bar and didn't verify β what we're proud of is the fastest result that keeps quality.
The recipe is already open, so we explained each piece: sliding-window W188, CTK49 kernel tuning, noprecache (honest, verifiable measurement), and an N64 synthetic warmup bridge that shrinks the publicβprivate gap (~15 TPS), plus INT4 + MTP K=7 + CUDA-graph capture. One rule: only stack quality-neutral speedups.
Huge thanks to @firfir-cast , @gemma-slayer , @chiku-inu , @kenyan-duma , @dixie-flatline and everyone who shared their experiments. Full write-up
π
https://huggingface.co/blog/FINAL-Bench/fast-gemma
- 8 replies
liked a
model 12 days ago
liked a
Space 19 days ago
reacted to SeaWolf-AI's post with π₯ 20 days ago
Post
3130
POCKET now speaks Gemma 4 β a 26B model that loads in every app, and runs on your PC with no GPU
We're adding a Gemma-4 sibling to POCKET: POCKET-26B, built from Google's Gemma-4-26B-A4B (Apache-2.0). Our flagship POCKET-35B is a Qwen-family MoE and needs a recent llama.cpp; POCKET-26B trades a little size for the thing people kept asking for β it just loads, everywhere, today: Ollama, LM Studio, PocketPal, MLX, any stock llama.cpp. No fork, no bleeding-edge runtime, no CUDA, no cloud.
It's a sparse Mixture-of-Experts (25.2B total, ~4B active per token), so the work per token stays small β a real 26B that generates on a CPU with no graphics card.
Two things make it stand out:
1) Universal compatibility. Gemma 4 is a standard, widely-supported architecture, so POCKET-26B runs on the tools you already have β no waiting for your app to add a new model type.
2) Quality that survives compression. Measured GPQA-Diamond (198 q, greedy):
β’ Full base: 67.7%
β’ POCKET-26B Q4_K_M (17 GB): 67.7% β lossless
β’ POCKET-26B Q2_K (11 GB): 67.2% β near-lossless, at 11 GB
Live, on a CPU-only box (our demo Space β POCKET-26B vs Bonsai-27B, same machine, same stock llama.cpp): POCKET-26B β 19 tok/s vs Bonsai β 6 tok/s β about 3Γ faster generation, no GPU. (Honest notes: shared CPU box, sequential race; a dedicated machine is faster.)
Where it fits in the family:
β’ POCKET-35B (Qwen MoE) β bigger, top-tier, needs a recent llama.cpp.
β’ POCKET-26B (Gemma 4) β loads in any app, quality-robust when compressed. The demo runs the Q4_K_M build; Q2_K (11 GB) is the smallest footprint. For a true β€8 GB phone, the 5 GB POCKET-KR (Qwen) is still the pick.
Try it and grab it:
π₯οΈ Live demo (Gemma4-based, answering on a CPU, no GPU): FINAL-Bench/POCKET-26B-CPU
π¦ POCKET-26B-GGUF (Q4_K_M 17 GB Β· Q2_K 11 GB): FINAL-Bench/POCKET-26B-GGUF
π POCKET collection: https://huggingface.co/collections/FINAL-Bench/pocket-models
We're adding a Gemma-4 sibling to POCKET: POCKET-26B, built from Google's Gemma-4-26B-A4B (Apache-2.0). Our flagship POCKET-35B is a Qwen-family MoE and needs a recent llama.cpp; POCKET-26B trades a little size for the thing people kept asking for β it just loads, everywhere, today: Ollama, LM Studio, PocketPal, MLX, any stock llama.cpp. No fork, no bleeding-edge runtime, no CUDA, no cloud.
It's a sparse Mixture-of-Experts (25.2B total, ~4B active per token), so the work per token stays small β a real 26B that generates on a CPU with no graphics card.
Two things make it stand out:
1) Universal compatibility. Gemma 4 is a standard, widely-supported architecture, so POCKET-26B runs on the tools you already have β no waiting for your app to add a new model type.
2) Quality that survives compression. Measured GPQA-Diamond (198 q, greedy):
β’ Full base: 67.7%
β’ POCKET-26B Q4_K_M (17 GB): 67.7% β lossless
β’ POCKET-26B Q2_K (11 GB): 67.2% β near-lossless, at 11 GB
Live, on a CPU-only box (our demo Space β POCKET-26B vs Bonsai-27B, same machine, same stock llama.cpp): POCKET-26B β 19 tok/s vs Bonsai β 6 tok/s β about 3Γ faster generation, no GPU. (Honest notes: shared CPU box, sequential race; a dedicated machine is faster.)
Where it fits in the family:
β’ POCKET-35B (Qwen MoE) β bigger, top-tier, needs a recent llama.cpp.
β’ POCKET-26B (Gemma 4) β loads in any app, quality-robust when compressed. The demo runs the Q4_K_M build; Q2_K (11 GB) is the smallest footprint. For a true β€8 GB phone, the 5 GB POCKET-KR (Qwen) is still the pick.
Try it and grab it:
π₯οΈ Live demo (Gemma4-based, answering on a CPU, no GPU): FINAL-Bench/POCKET-26B-CPU
π¦ POCKET-26B-GGUF (Q4_K_M 17 GB Β· Q2_K 11 GB): FINAL-Bench/POCKET-26B-GGUF
π POCKET collection: https://huggingface.co/collections/FINAL-Bench/pocket-models
- 3 replies
liked a
Space 20 days ago
liked a
model 20 days ago
upvoted 2
papers 22 days ago
upvoted an article 22 days ago
Article
POCKET: a 35-billion-parameter model that runs on your iPhone β and on your PC with no GPU
FINAL-Bench
β’ β’ 12