My Profile Photo

Thiago Vinhas


Thoughts on DevOps, cloud architecture, coding and random geekiness



Creating a local Chef repository with Pulp

It's very common these days to leave part, or even your whole farm, on an air gapped environment. It will definitely make it harder for malicious visitors to gain unauthorized access, but you'll have some other things to worry about regarding the management of that infrastructure.

Chef is one of the most popular orchestration tools around, and if you need to bootstrap servers on a airgapped environment, it's important that you keep a local repository at least with the products your company use.

Unfortunately, Chef's rpm packages are made using omnibus, and that means every single dependency of that software will be declared in the package, making the repository's yum metadata HUGE (currently it has about 2.1 GB).

Here is a simple way to create a local copy of the packages, and then creating a local repository using Pulp, the best tool around for creating and managing repositories:

  • Create a local user, let's say yum with all the defaults and add it to your /etc/sudoers file and login using that user

  • And then:

$ sudo yum install wget createrepo
$ sudo setenforce 0 # I won't cover SELinux permissions here
$ chmod o+x . # You'll need execute permission in your home directory to serve http

You'll have to create the directories where your repo will be located, and run the wonderful wget tool for mirroring the packages:

$ mkdir -p repos/chef ; cd repos/chef
$ wget -r -N -np -nH --cut-dirs=3  https://packages.chef.io/repos/yum/stable/el/7/x86_64/

Brace yourself because there's a LOT of packages to download and it will take a while. The good thing here is that this command will only download new/changed files, so it's safe to add it to your crontab and let it run every night.
Be aware that Chef imposes a transfer limit per IP, so if you abuse you'll be temporarily blacklisted. Use wget to fetch the packages to a single central location, and redistribute from there internally as needed.

Once done, you'll need to run:

$ createrepo /home/$USER/repos/chef/el/7/x86_64

Now follow Pulp's documentation to successfully install it, and then change the verify_ssl parameter to false on /etc/pulp/server.conf.
Run the following commands so Pulp creates your repository:

$ pulp-admin login -u admin -p admin
$ pulp-admin rpm repo create --repo-id=chef --serve-http=true --serve-https=true \
 --relative-url=chef --description 'Chef Software Repo' \
 --display-name 'Chef Software Repo' --feed file:///home/$USER/repos/chef/el/7/x86_64/
$ pulp-admin rpm repo sync run --repo-id=chef # This will probably take a while
$ pulp-admin rpm repo sync schedules create -s 2017-04-12T12:00:00Z/P1DT  --repo-id chef

Now you can access your own local Chef repository at:

http://ip-address/pulp/repos/

That's it! Hope this helps you.