Replacing a Domain storage implementation¶
Domain records retain historical storage class paths so older Pulp processes can read them during
a rolling upgrade. Pulp resolves those paths to stable backend IDs (filesystem, s3, azure, and
gcs), then selects the installed class through DOMAIN_STORAGE_BACKENDS. The API still advertises
the historical class paths and returns them on reads. It also accepts stable IDs in raw requests,
but generated clients may reject those IDs because the schema still lists the historical paths.
Existing S3Boto3Storage paths are preserved rather than rewritten to the newer S3 path.
This boundary replaces the code for an existing storage type; it does not register a new type. Only filesystem, S3, and Azure are currently offered for new Domains. GCS is recognized for existing configurations, and SFTP is not supported.
Select an implementation¶
Set the class path for each non-default Domain storage type you are replacing:
DOMAIN_STORAGE_BACKENDS = {
"filesystem": "pulpcore.app.models.storage.FileSystem",
"s3": "my_storage.backends.s3.S3Storage",
"azure": "storages.backends.azure_storage.AzureStorage",
"gcs": "storages.backends.gcloud.GoogleCloudStorage",
}
The default Domain instead uses Django's STORAGES["default"]. When replacing that class, set
both its BACKEND and the corresponding DOMAIN_STORAGE_BACKENDS entry. The mapping lets Pulp
identify its storage type without requiring the replacement's Python path in a Domain record.
An unrecognized default class can serve redirects only if it implements pulp_url.
For direct content redirects, a replacement may implement
pulp_url(name, headers=None, method=None) to sign URLs with Pulp's response-header overrides.
Without that hook, Pulp calls the django-storages-style url() method for recognized object storage
types. For ranged content reads, a replacement may expose a synchronous
open_stream(name, start=0, length=None) context manager. Pulp falls back to Django's open() and
seek() if the hook is absent; that fallback bounds the returned bytes but may buffer a whole
remote object. The stream must release its resources when the context exits.
Use storage from a plugin¶
Plugin code should use pulpcore.plugin.util.get_local_storage_path(file, domain=None) when it
needs an actual filesystem path. It returns None when the backend does not provide one; other
storage errors are propagated. Use get_direct_storage_url(file, headers=None, http_method=None,
domain=None) when a consumer needs a URL directly to object storage. It returns None when the
Domain uses filesystem storage or has redirects disabled, and raises NotImplementedError when
redirects are requested from an unsupported backend. Both functions accept a Pulp-managed file
such as Artifact.file or PulpTemporaryFile.file. If the file belongs to another Domain, pass
that Domain explicitly; otherwise they use the current Domain.
The URL function accepts generic Content-Type and Content-Disposition response headers and
translates them into the selected provider's signed URL parameters. Plugins should not inspect
storage_class or pass provider-specific parameters to storage.url(). The existing
get_artifact_url(artifact, ...) differs: it always returns a URL, falling back to the content
app when a direct URL is unavailable.
Roll out a replacement¶
Keep the Domain's storage type, bucket or container, prefix, object names, credentials, and URL behavior compatible. Install the replacement and its dependencies on every API, content, and worker process, and test reads, writes, range requests, and signed redirects against existing objects. Then change the deployment settings. A code-only change needs no Domain data migration. Moving objects to a different location still requires a storage migration.
During a rolling upgrade, new processes continue to store and return historical class paths so
older processes can read the same Domain rows. Keep the old implementation installed until those
processes and the rollback window are gone. Do not switch persisted storage_class values to IDs
while any process still requires the historical paths.