Or: How I spent an afternoon doing a deep dive into the RPM spec and solving a problem for myself
tl;dr – Nginx Mainline packages are being built for Fedora & CentOS at copr.fedoraproject.org/coprs/kyl191/nginx-mainline/
My webserver’s running nginx 1.4.7, a version that hasn’t gotten non-bugfix attention since March 2013, according to the changelog. Oddly enough for a Fedora package, the version in koji is the stable branch – something that makes sense for CentOS/EPEL since that’s a long term support release, but not for Fedora. Doubly annoying was the fact that the packages for Fedora 20 (because Fedora 21 isn’t supported on OpenVZ yet) was one step behind the official stable – 1.4 instead of 1.6.
If I wanted mainline on Fedora, I was going to have to build it from source.
Over-complicating things
I like Nginx, having gotten fed up with fiddling with Apache (aka httpd) to make it run under memory limits in the past. And I’m trying to define the configuration of everything as much as possible, which means I need a way to:
- Reliably reproduce and automate configurations, so I don’t have to manually touch each server
- Distribute it automatically
Building RPMs and dumping them in a yum repo sounded like the perfect thing. I immediately jumped to hosting it myself – I had bandwidth to spare, which meant that I could open it up to the public. And I could do Jenkins slaves! And rsync! And DNS round robin load distribution! And play with repo configurations! And, and, and! So much fun!
First things first
The first thing to do was to get the new source actually building. I looked into how Fedora builds RPMs. Turns out that koji has a bunch of VMs that build packages using mock as necessary. It seems to pull from the Fedora git repo – So I grabbed a copy of the nginx.spec file and installed mock on a VPS.
Getting it building was pretty much iterating between attempting to build, fixing, and building again. So here are my notes from the process:
- Mock requires that you not be running as root, and the user you’re running as be in the group mock
- usermod -a -G mock user should get you working
- Mock command to rebuild from the source RPM is mock -r <platform> –rebuild <src rpm>
- Platform can be any of the config files in /etc/mock.
- Platform format is (epel|fedora)-release-arch
- Installed zlib-devel, pcre-devel, gcc, openssl-devel as build pre-reqs
- To extract files from an RPM, create a directory, change to that directory, and run rpm2cpio <RPM>|cpio -idmv.
- Create a directory because cpio will indiscriminately dump files in your current directory
- Build a source RPM in mock with /usr/bin/mock -r fedora-20-i386 –spec=nginx/nginx.spec –sources=nginx/ –buildsrpm –resultdir=.
- /usr/bin/mock because there was also /usr/sbin/mock, and PATH was preserved su-ing from root, so /usr/sbin/mock complained about not being root
- RPM has fiddly little bits, and is annoyingly ill documented.
- Draft documentation was really helpful in documenting what the %setup macro did in the .spec file
- You must follow syntax for Requires/Obsoletes sections.
- Under all the macros and RPM build processing, the spec file is actually a shell script. You could go wild… but that would be silly.
- Other resources used:
Well, this changes things
Hidden under “Useful Tools” in one of the documents was a single sentence pointing out the existence of Copr. A public build service, it allows you to submit source RPMs, and Copr will build architecture specific RPMs for you and setup a repo and automatically update it when you start a new build. Suddenly, my plans of creating a distributed Jenkins workforce + downloading infrastructure were replaced with “Now, how can I get this working with my source RPM?”
Long story short is: Trial and erroring my way through worked! Copr has a new nginx-mainline repo.
Future things
- Possibly add it to the list of Fedora People repos – not too sure if I want to do this because it’s extra work (setting up + syncing the repos) for not much benefit (you still have to download the repo files anyway)
- Automate builds – Jenkins to the rescue! There is still a bunch of manual work
- Download the Nginx source file & gpg file
- Run the source RPM build
- Move it to Web-accessible directory
- Trigger Copr build with the new source RPM
- Add a build integrating ngx_pagespeed as an additional source file
Miscellaneous things from my notes
If I want to configure nginx (read: add extra modules), I need to extract, edit and recompile the package…
Not really, Fedora requires the sources in any case – can’t rebuild it from a src rpm and get it into their repo
Koji in a box/Kojak: github.com/sbadakhc/kojak/blob/master/scripts/install/install
Creating a repo: wiki.centos.org/HowTos/CreateLocalRepos
nginx -V to see the configure line
Jenkins only needs to be installed on the master node, because slave nodes are managed by the master, no install required. Script the config somehow?
Slaves:
- Add Jenkins user
- Install mock
- Add jenkins user to mock group
- Launch Jenkins slave from systemd?
- Would allow me to get around requirement to script config (I think) and muck around with SSH keys
- wiki.jenkins-ci.org/display/JENKINS/Distributed+builds#Distributedbuilds-Launchslaveagentheadlessly
- wiki.jenkins-ci.org/display/JENKINS/Distributed+builds#Distributedbuilds-Transitionfrommasteronlytomaster%2Fslave
As an aside: I do want to look at Apache 2.4, because that has PHP-FPM and a new event MPM instead of worker. Considering that webapps seem to be built pretty much exclusively for Apache with all their custom .htaccess logic.