Skip to content

Welcome to Pulp Hugging Face

The pulp_hugging_face plugin extends pulpcore to support hosting and caching Hugging Face Hub content. This plugin is a part of the Pulp Project, and assumes some familiarity with the pulpcore documentation.

If you are just getting started, we recommend:

Features

  • Pull-through Caching: Automatically fetch and cache models, datasets, and spaces from Hugging Face Hub on first access
  • Authentication Support: Use Hugging Face tokens to access private repositories
  • Full Compatibility: Works seamlessly with huggingface-cli, transformers library, and other HF tools
  • All Content Types: Support for models, datasets, and spaces
  • On-demand Downloads: Reduce storage by only downloading content when requested
  • Versioned Repositories: Every operation creates a restorable snapshot

Documentation Sections

For Users

  • User Guide - Getting started and feature documentation
  • Tutorials - Step-by-step guides
  • Guides - Detailed workflow documentation

For Administrators

For Developers

Quick Start

  1. Create a remote pointing to Hugging Face Hub:
curl -X POST https://pulp.example.com/pulp/api/v3/remotes/hugging_face/hugging-face/ \
  -H "Content-Type: application/json" \
  -u admin:password \
  -d '{"name": "hf-remote", "url": "https://huggingface.co", "policy": "on_demand"}'
  1. Create a distribution for pull-through caching:
curl -X POST https://pulp.example.com/pulp/api/v3/distributions/hugging_face/hugging-face/ \
  -H "Content-Type: application/json" \
  -u admin:password \
  -d '{"name": "hf-cache", "base_path": "huggingface", "remote": "<remote_href>"}'
  1. Access Hugging Face content through Pulp:
# Use with huggingface-cli
export HF_ENDPOINT="https://pulp.example.com/pulp/content/huggingface"
huggingface-cli download bert-base-uncased

Site Map