The Wizard of Yum: Upgrade from FC3 to CentOS 5.2

If you have a production server, you would like to stay on the yellow brick road of upgrades for the operating system and installed software. However, it might happen you forget to support a server... and get 'scared' of performing updates: "If it ain't broken, don't fix it"! This mentality is not always correct, since errors you don't have at the moment, might still be a problem later on. The further away you get from the updates, the more difficult it will become to perform an upgrade to a later version of a distribution.

Why to write about it
The motivation to describe my experience of upgrading is because I hear a lot of people still complain about RPM and the dependency issues. Most of them have never used it and call the apt system the best solution. In my opinion, the key to success for apt is the reliance on a single repository. Debian people seldomly use dpkg -i to install package, since they use apt-get install to resolve dependencies. YUM solves the issue for RPM systems in a similar fashion. Instead of using rpm -i you can install packages using yum install, which resolves necessary dependencies from the configured repositories.

Note: With Ubuntu and Maemo I have noticed that more repositories get introduced which cause the same problems for apt-get. Dependency resolving might not work, since packages and depencies get fragmented and sometimes even unavailable or conflicting. The Red Hat community recently merged several repositories (RPM Fusion and RPM Forge) which does solve a lot of issues.

Setup
I was faced with the upgrade problem with my own personal (web)server for quite some time. I have no idea what I originally installed on it, but it has always been a Red Hat Linux system. It has been upgraded many times until it ran Fedora Core 3. At least the oldest file on the server was from January 2000. Since the server ran FC3, there was a problem. Fedora Core 3 was released in November 2004 it therefore only had updates available until about 2006.

Last year the machine was taken apart and virtualized to run on a newer VMware environment. I did update the system quite often... but never upgrade the distribution again. I had strayed from the yellow brick road and had no clear upgrade path any more. To solve this issue I could do to things:

1. Install a new server and migrate the files
   
2. Upgrade to a new distribution and hope this would start

No matter what option I would choose, I wanted to have similar functionality. Currently it runs:

• Webserver:
  Apache HTTPd (with various modules) → Zope
  
• Mail infrastructure:
  Postfix → Amavisd-new → Cyrus-imapd
• Database:
  PostgreSQL

I wanted to upgrade (option 2) as I had always done. Because of this, my options for upgrade were kinda limited: Fedora Core 4? Fedora Core had no support either... and the support for Fedora 8 would also expire soon (as it did this week). On the internet I had seen people mention it would be impossible to perform an upgrade from FC3 to a later version, like FC9 or they strongly advice others not to do so...

Hmmm, so instead I copied my 'sites', 'imap', 'zope' and database dumps to an NFS. Did I have to start a new installation?!??? All other servers I installed allready ran CentOS, I would also make this a CentOS installation. Well, considering CentOS is derived from RHEL (RedHat Enterprise Linux) it is in fact a Fedora installation. If you look at the release details:

• RHEL-4 is based on FC3
• RHEL-5 is based on FC6

You can upgrade from RHEL-4 to RHEL-5, this would mean like you jump from FC3 to FC6. CentOS 5.2 is a RedHat derivative from RHEL-5.2... So I wanted to upgrade my FC3 to CentOS 5.2. I know this would be difficult, but in my opinion possible. The key to success would be YUM, the chosen applications and the self-contained installs. The structure I had laid out was simple

• Amavisd-new ran its own Perl installation from /opt/activestate/perl/
• Zope
  ran from a Python installation from /opt/activestate/python/
  datafiles are in /home/zope/
  
• Cyrus was built from source
  installed in /opt/cyrus/
  datafiles are in /home/cyrus/, totalling 7GB.
  
• Each website has its own vhost-[domainname].conf for Apache and a separate directory in /home/sites/[domainname]/[hostname]/web/ which contains the files. Several websites, subdomains and databases, totalling 4GB of data.
  

Description
The following description are the actions I took, but it is NOT a howto for guaranteed success. It might work for you, although you could also end up with a non-booting system. Only perform this installation if you have enough knowledge of a Linux system and have backups available if it goes wrong.

On VMware I made a snapshot of the VM and opened the console, logged in on the 2nd and 3rd console as root. The main installation I would perform from two SSH connections (one as a backup). The tool I would use for the upgrade is YUM. YUM is a tool which resolves dependencies from repositories you have specified.

As root I created a 'frankenstein' directory with the following files from a CentOS mirror.

centos-release-5-2.el5.centos.i386.rpm

beecrypt-4.1.2-10.1.1.i386.rpm
centos-release-notes-5.2-2.i386.rpm
db4-4.3.29-9.fc6.i386.rpm
db4-devel-4.3.29-9.fc6.i386.rpm
db4-utils-4.3.29-9.fc6.i386.rpm
elfutils-0.125-3.el5.i386.rpm
elfutils-libelf-0.125-3.el5.i386.rpm
elfutils-libelf-devel-0.125-3.el5.i386.rpm
elfutils-libelf-devel-static-0.125-3.el5.i386.rpm
elfutils-libs-0.125-3.el5.i386.rpm
glib2-2.12.3-2.fc6.i386.rpm
glib2-devel-2.12.3-2.fc6.i386.rpm
glibc-2.5-24.i386.rpm
glibc-common-2.5-24.i386.rpm
glibc-devel-2.5-24.i386.rpm
glibc-headers-2.5-24.i386.rpm
keyutils-1.2-1.el5.i386.rpm
keyutils-libs-1.2-1.el5.i386.rpm
krb5-devel-1.6.1-25.el5.i386.rpm
krb5-libs-1.6.1-25.el5.i386.rpm
krb5-workstation-1.6.1-25.el5.i386.rpm
libselinux-1.33.4-5.el5.i386.rpm
libselinux-devel-1.33.4-5.el5.i386.rpm
libsepol-1.15.2-1.el5.i386.rpm
libsepol-devel-1.15.2-1.el5.i386.rpm
libxml2-python-2.6.26-2.1.2.1.i386.rpm
m2crypto-0.16-6.el5.2.i386.rpm
mcstrans-0.2.7-1.el5.i386.rpm
openssl-0.9.8b-10.el5.i386.rpm
popt-1.10.2-48.el5.i386.rpm
python-2.4.3-21.el5.i386.rpm
python-devel-2.4.3-21.el5.i386.rpm
python-elementtree-1.2.6-5.i386.rpm
python-ldap-2.2.0-2.1.i386.rpm
python-sqlite-1.1.7-1.2.1.i386.rpm
python-urlgrabber-3.1.0-2.noarch.rpm
readline-5.1-1.1.i386.rpm
readline-devel-5.1-1.1.i386.rpm
rpm-4.4.2-48.el5.i386.rpm
rpm-build-4.4.2-48.el5.i386.rpm
rpm-devel-4.4.2-48.el5.i386.rpm
rpm-libs-4.4.2-48.el5.i386.rpm
rpm-python-4.4.2-48.el5.i386.rpm
sqlite-3.3.6-2.i386.rpm
sqlite-devel-3.3.6-2.i386.rpm
yum-3.2.8-9.el5.centos.1.noarch.rpm
yum-metadata-parser-1.1.2-2.el5.i386.rpm

The centos-release file tell your YUM to which repository it should refer for an upgrade. Your version number may differ, but this would generally give you a working YUM installation from a CentOS system. Install all the packages at once.

$ rpm -Uvh *.rpm --nodeps.

This is what my console showed after the installation. Since the berkely database version has been upgraded, you need to remove the previous RPM database and rebuild it.

$ rm -f /var/lib/rpm/__db.*
$ rpm --rebuilddb

From this moment on, you could upgrade the system. The command

$ yum upgrade

will show you what issues you need to satisfy for the dependencies. These you can install using:

$ yum install [packagename]

Before I upgraded, I disabled several repositories in my /etc/yum.repos.d/ directory. You can do this by adding enabled=0 to a repo definition. Before continuing the upgrade I stopped unnecessary services and disabled SELinux for now. Depending on the use of your installation you may need to also remove some packages. I had to do so.

$ rpm -qa

shows you all the installed packages. For my installation I removed some Novell branded packages for Mono and most of the X11 packages. After this remove the installed kernels. You might to iterate these steps (rpm -qa and rpm -e or yum remove) several times before you can perform the upgrade.

$ yum upgrade

starts the actual upgrade... be careful during the installation, since some services might not function as expected. Do not abort it. This would render your system not usable.

After the yum upgrade has been performed, you need to check if you have a kernel configured in /boot/grub/grub.conf (/etc/grub.conf). I had to install the kernel myself due to architecture conflicts?!

$ rpm -ivh --ignorearch kernel-2.6.18-92.el5.1686.rpm

Even after this installation you would need to check the grub configuration. Reboot the system and when it starts, you would have a CentOS system.

After this you can cleanup your system.

$ rpm -qa > /tmp/rpmlist
$ cat /tmp/rpmlist | grep fc3
$ cat /tmp/rpmlist | grep FC3

Remove any remaining package which refer to your previous installation. Note: it is normal to find references to fc6 packages for your CentOS installation.

After the reboot, I noticed that most of the applications still worked as they should. Most of the time went into getting Apache and Zope running well together. Now a week later, I removed the VMware snapshot, even consolidated two harddisc files into one. Cyrus-impad is now makes use of the official release package (using my own config file) and upgraded amavisd-new to the RPMforge release. Only zope is running the old installation...

Conclusion
Always plan your system for future use. A sever has a lifespan which may exceed your initial thoughts. Try to compose the server as seperate responsibilities (mail, web, etc) as this would make upgrade easier. Try to stay close to the release schedule of patches and updates for your system... try to avoid building from source or compiling your own kernels as this would make management more difficult later on. And when a distribution gets the end-of-life status, try to move away from it immediately.

As you can see, it would still be possible to upgrade to CentOS 5.2 from Fedora Core 3... but this is NOT advised or a common practice. For me it shows how well thought the components and layout were, since this is also the setup I had used for SpotlightMedia and my previous employer.

In my opinion it shows how well the YUM dependency resolving and upgrading works. The wonderful wizard of Yum gave me a working CentOS system!