Skip to content

Pulp Installer 3.5 new roles and new deployment scenarios

There have been some big changes to Pulp Installer in 3.5.0, in terms of the Ansible role structure, including breaking changes. Merely reading the release notes for individual changes does not paint a complete or clear picture, hence why we are writing this blog post.

In short, we refactored the Pulp Installer roles to acheive an SOA (Service Oriented Architecture).

We now have 1 role for each service, and each can be installed to a server of its own.

This does mean that users who wrote custom playbooks need to update them for 3.5.0 with a new roles: list.

So we created 2 new meta roles as well, to prevent users from having to change their roles: list again in the future: * pulp_all_services * pulp_services

The refactoring benefits every deployment scenario that users may have, 4 of which are explained in greater detail with example playbooks below. * Users that want a single "all-in-one" Pulp server (including database, redis & webserver). * Users that want to install Pulp itself on a server, but want to use existing infrastructure (servers) for database, redis & webserver. * Users that want to install Pulp itself on a server, and want to install database, redis & webserver on separate servers. * User that wants to install each and every Pulp service on a separate server.

"All-In-One" Pulp Server:

This is a single server, containing all Pulp services, including database, redis & webserver.

This is the new example playbook, almost identical to the one found in pulp_installer/playbooks/example-use/playbook.yml and in the quickstart guide in the Pulp Installer docs.

The benefits / differences from 3.4.1 are: * The old roles: list (with 5 roles) will no longer work. * The new roles: list is now 1 meta role: pulp_all_services * The new roles: list is very unlikely to change again in the future.

    ---
    - hosts: pulp1
      vars:
        pulp_default_admin_password: << YOUR PASSWORD FOR THE PULP APPLICATION HERE >>
        pulp_settings:
          secret_key: << YOUR SECRET HERE >>
          content_origin: "http://{{ ansible_fqdn }}"
        pulp_install_plugins:
          pulp-container: {}
          pulp-rpm: {}
      roles:
        - pulp_all_services
      environment:
        DJANGO_SETTINGS_MODULE: pulpcore.app.settings

Pulp Server with Existing Infrastructure:

This is a single server, that relies on existing database, redis & webservers.

This example playbook is almost identical to the one found in the pulp_services documentation.

The benefits / differences from 3.4.1 are: * The old roles: list (with 4 roles) will no longer work. * The new roles: list is now 1 meta role: pulp_services. * The new roles: list is unlikely to change again in the future. * pulp_redis will no longer be applied on this server as a dependency of pulp_workers or pulp_resource_manager. This avoids an additional & unused redis service being pointlessly installed and running. * Telling the installer to not install a postgres database server is now a function of the roles: list. The following variable which used to control that is now ignored: pulp_install_db. * Therefore, you no longer need to set pulp_install_db=false in your vars:.

    ---
    - hosts: pulp1
      vars:
        pulp_default_admin_password: << YOUR PASSWORD FOR THE PULP APPLICATION HERE >>
        pulp_content_bind: pulp1:24816
        pulp_api_bind: pulp1:24817
        pulp_settings:
          secret_key: << YOUR SECRET HERE >>
          content_origin: "http://webserver1.fqdn"
          redis_host: redis1
          redis_port: 6379
          redis_password: << YOUR REDIS PASSWORD HERE >>
          databases:
            default:
              HOST: postgres1
              ENGINE: django.db.backends.postgresql_psycopg2
              NAME: pulp
              USER: pulp
              PASSWORD: << YOUR DATABASE PASSWORD HERE >>
        pulp_install_plugins:
          pulp-container: {}
          pulp-rpm: {}
      roles:
        - pulp_services
      environment:
        DJANGO_SETTINGS_MODULE: pulpcore.app.settings

Note the following hostnames: * pulp1: The Pulp server * webserver1.fqdn : An existing webserver, specified as FQDN, and with http:// or https:// * redis1: An existing redis server * postgres1: An existing PostgreSQL database server

Pulp Server + Separate Servers

Here the installer installs Pulp itself on a server, the database on a 2nd server, redis on a 3rd server, and the webserver on a 4th server.

Note that the code for hosts: pulp1 is identical to the above example.

We consider this to be a new deployment scenario for Pulp Installer, because although it may have worked previously, odd settings/workarounds were involved (we listed the ones we noticed below), and it was never tested or documented.

The benefits / differences from 3.4.1 are: * The new roles: list for the main Pulp server is now 1 meta role: pulp_services. * The new roles: list for the main Pulp server is also unlikely to change in the future. * pulp_redis (and thus a redis server) will no longer be installed on the main Pulp server as a dependency of pulp_workers or pulp_resource_manager. This avoids a pointless service being installed and running on it. * When you apply pulp_database to install the Posgres database server, it no longer installs the Pulp application on it (it no longer depends on pulp_common, formerly pulp.) This is because the functionality to configure the database itself is no longer in pulp_database. It is now in pulp_database_config, which is a dependency of pulp_services. * You no longer need to set pulp_install_api_service=false and pulp_install_db=false on postgres1 and webserver1. * Because of the above change, the following variables are now ignored: pulp_install_api_service and pulp_install_db.

Note that the following non-SOA behavior still exists: * pulp_webserver still depends on pulp_common (formerly pulp), so as to provide the per-plugin webserver snippets. So webserver1 will have pulpcore & all plugins installed on it. Now however, pulp-api is no longer running & listening by default (no need to set pulp_install_api_service=false.)

    ---
    - hosts: postgres1
      vars:
        pulp_settings:
          databases:
            default:
              HOST: postgres1
              NAME: pulp
              USER: pulp
              PASSWORD: << YOUR DATABASE PASSWORD HERE >>
      roles:
        - pulp_database

    - hosts: redis1
      vars:
        pulp_redis_bind: 'redis1:6379'
      roles:
        - pulp_redis

    - hosts: pulp1
      vars:
        pulp_default_admin_password: << YOUR PASSWORD FOR THE PULP APPLICATION HERE >>
        pulp_content_bind: pulp1:24816
        pulp_api_bind: pulp1:24817
        pulp_settings:
          secret_key: << YOUR SECRET HERE >>
          content_origin: "http://webserver1.fqdn"
          redis_host: redis1
          redis_port: 6379
          databases:
            default:
              HOST: postgres1
              ENGINE: django.db.backends.postgresql_psycopg2
              NAME: pulp
              USER: pulp
              PASSWORD: << YOUR DATABASE PASSWORD HERE >>
        pulp_install_plugins:
          pulp-container: {}
          pulp-rpm: {}
      roles:
        - pulp_services
      environment:
        DJANGO_SETTINGS_MODULE: pulpcore.app.settings

    - hosts: webserver1
      vars:
        pulp_content_bind: pulp1:24816
        pulp_api_bind: pulp1:24817
      roles:
        - pulp_webserver

Note the following hostnames: * pulp1: The Pulp server * webserver1.fqdn : The new webserver, specified as FQDN, and with http:// or https:// * webserver1: The same new webserver * redis1: The new redis server * postgres1: The new PostgreSQL database server

Separate Servers for Each & Every service

Here every single service is on its own server.

And in the example playbook, there are actually 2 worker servers.

We consider this to be a new deployment scenario for Pulp Installer, because although it may have worked previously, odd settings/workarounds were involved (we listed the ones we noticed below), and it was never tested or documented.

pulp_services is expanded from the previous example 5 roles: pulp_api, pulp_content, pulp_workers, pulp_resource_manager and pulp_database_config. pulp_database_config is actually not a service, but it does depend on pulp_common (which installs the Pulp application) and it only needs to be run once. It can be run on any server (ideally one where pulp_common is being run for other roles anyway), so we simply picked the API server.

The benefits / differences from 3.4.1 are: * The new roles: list is 1 role per service and per server, plus pulp_database_config. * pulp-api is now its own role. It is no longer part of pulp (now called pulp_common). * pulp_common (formerly pulp) is not meant to be called directly anymore, either as an undeclared dependency or for the pulp-api service. * pulp_redis (and thus a redis server) will no longer be installed on the pulp_workers servers or pulp-res-man1 server as a dependency of pulp_workers or pulp_resource_manager. This avoids a pointless service being installed and running on them. * When you apply pulp_database to install the Posgres database server, it no longer installs the Pulp application on it (it no longer depends on pulp_common, formerly pulp.) This is because the functionality to configure the database itself is no longer in pulp_database. It is now in pulp_database_config, which is a dependency of pulp_services. * You no longer need to set pulp_install_api_service=false on all the servers other than pulp-api1. * You no longer need to set pulp_install_db=false on postgres1 * Because of the above changes, the following variables are now ignored: pulp_install_api_service and pulp_install_db.

Note that the following non-SOA behavior still exists: * pulp_webserver still depends on pulp_common (formerly pulp), so as to provide the per-plugin webserver snippets. So webserver1 will have pulpcore & all plugins installed on it. Now however, pulp-api is no longer running & listening by default (no need to set pulp_install_api_service=false.)

    ---
    - hosts: postgres1
      vars:
        pulp_settings:
          databases:
            default:
              HOST: postgres1
              NAME: pulp
              USER: pulp
              PASSWORD: << YOUR DATABASE PASSWORD HERE >>
      roles:
        - pulp_database

    - hosts: redis1
      vars:
        pulp_redis_bind: 'redis1:6379'
      roles:
        - pulp_redis

    - hosts: pulp-api1
      vars:
        pulp_default_admin_password: << YOUR PASSWORD FOR THE PULP APPLICATION HERE >>
        pulp_api_bind: pulp-api1:24817
        pulp_settings:
          secret_key: << YOUR SECRET HERE >>
          content_origin: "http://webserver1.fqdn"
          redis_host: redis1
          redis_port: 6379
          databases:
            default:
              HOST: postgres1
              ENGINE: django.db.backends.postgresql_psycopg2
              NAME: pulp
              USER: pulp
              PASSWORD: << YOUR DATABASE PASSWORD HERE >>
        pulp_install_plugins:
          pulp-container: {}
          pulp-rpm: {}
      roles:
        - pulp_api
        - pulp_database_config
      environment:
        DJANGO_SETTINGS_MODULE: pulpcore.app.settings

    - hosts: pulp-content1
      vars:
        pulp_content_bind: pulp-content1:24816
        pulp_settings:
          secret_key: << YOUR SECRET HERE >>
          content_origin: "http://webserver1.fqdn"
          redis_host: redis1
          redis_port: 6379
          databases:
            default:
              HOST: postgres1
              ENGINE: django.db.backends.postgresql_psycopg2
              NAME: pulp
              USER: pulp
              PASSWORD: << YOUR DATABASE PASSWORD HERE >>
        pulp_install_plugins:
          pulp-container: {}
          pulp-rpm: {}
      roles:
        - pulp-content
      environment:
        DJANGO_SETTINGS_MODULE: pulpcore.app.settings

    - hosts:
        - pulp-workers1
        - pulp-workers2
      vars:
        pulp_settings:
          secret_key: << YOUR SECRET HERE >>
          content_origin: "http://webserver1.fqdn"
          redis_host: redis1
          redis_port: 6379
          databases:
            default:
              HOST: postgres1
              ENGINE: django.db.backends.postgresql_psycopg2
              NAME: pulp
              USER: pulp
              PASSWORD: << YOUR DATABASE PASSWORD HERE >>
        pulp_install_plugins:
          pulp-container: {}
          pulp-rpm: {}
      roles:
        - pulp_workers
      environment:
        DJANGO_SETTINGS_MODULE: pulpcore.app.settings

    - hosts: pulp-res-man1
      vars:
        pulp_settings:
          secret_key: << YOUR SECRET HERE >>
          content_origin: "http://webserver1.fqdn"
          redis_host: redis1
          redis_port: 6379
          databases:
            default:
              HOST: postgres1
              ENGINE: django.db.backends.postgresql_psycopg2
              NAME: pulp
              USER: pulp
              PASSWORD: << YOUR DATABASE PASSWORD HERE >>
        pulp_install_plugins:
          pulp-container: {}
          pulp-rpm: {}
      roles:
        - pulp_resource_manager
      environment:
        DJANGO_SETTINGS_MODULE: pulpcore.app.settings

    - hosts: webserver1
      vars:
        pulp_content_bind: pulp1:24816
        pulp_api_bind: pulp1:24817
      roles:
        - pulp_webserver

Note the following hostnames: * pulp-api1: The pulp-api server. We also chose this to run pulp_database_config. * pulp-content1: The pulp-content server * pulp-workers1, pulp-workers2: The pulp-workers servers * pulp-res-man1: The pulp-resource-manager server * webserver1.fqdn : The new webserver, specified as FQDN, and with http:// or https:// * webserver1: The same new webserver * redis1: The new redis server * postgres1: The new PostgreSQL database server