TheHFStack

HFStack is an open-source organization focused on building reproducible ML infrastructure around the Hugging Face stack.

---

Focus Areas

Datasets & Storage

Building reproducible workflows around Hugging Face Datasets and HF Buckets.

Trackio & Observability

Experiment tracking, artifact lineage, and reproducible evaluation pipelines using Trackio.

Benchmarking & Runtime Systems

Inference benchmarking, optimization workflows, and runtime evaluation tooling.

Orchestration & Integrations

Composable integrations with tools like Dagster and ecosystem-native ML workflows.

---

Philosophy

HFStack focuses on the systems surrounding modern ML:

• reproducibility
• interoperability
• observability
• infrastructure simplicity

The goal is to make Hugging Face workflows easier to build and operationalize.

Projects & Integrations

• dagster-hf-datasets: Dagster-HF-Datasets integrates Hugging Face datasets with Dagster for building reproducible, observable data pipelines. Load datasets directly as Dagster assets, apply transformations, and publish results back to the Hub.

• Open-Source AI Cookbook with Transformers and Optuna: Contributed a recipe which showcases how best hyperparameters to fine-tune a lightweight BERT model for text classification on a subset of the IMDB dataset.

---

Contributions and ecosystem collaborations are welcome.