Interview: Fedora developers Seth Vidal and Will Woods

by Jonathan Roberts

This is the second in our series co-publishing with Fedora Interviews.

In Fedora, the supported method for upgrading from one release to the next is by downloading the media and rebooting from the media and using Anaconda. Why was this the case, and what’s wrong with a yum upgrade?

Seth Vidal: Anaconda has the advantage of running outside of the system it is upgrading. This means it can do certain tricks in the event of big changes. The lvm->lvm2 migration, for example. Yum can’t do this. Changing the partition or fs type on a running partition is a bad idea :)

Will Woods: Another good example: In Fedora 9 we have (experimental) ext4 support, and Anaconda is capable of migrating your ext3 filesystems to ext4 automatically. But yum upgrades can’t handle this.

Even if it was possible–would you really want your F8 kernel (with old, broken, or non-existent ext4 drivers) messing around with your filesystems? No, you should probably be running the new F9 kernel to do that.

Seth Vidal: However, in a huge number of the cases, a yum upgrade worked fine for updating your distro. I’ve been updating my laptop for quite some time that way; lots of people have.

Will Woods: Live upgrades are kind of crazy. It’s a frequently-requested feature, especially from people coming from Ubuntu or Debian: “Why doesn’t Fedora have ‘apt-get dist-upgrade’! It’s so simple and easy!”

Well, it’s actually not that simple or easy. Have you actually read the 20 pages of directions for preparing for an upgrade? Or the 11-page-long list of potential problems with performing such an upgrade?

Yum upgrades use your (now old and out-of-date) yum, rpm libraries, kernel, etc. to perform the upgrade. And they do it on a running system! All sorts of weird things can happen when you upgrade the entire system while it’s running.

Bisection divides users and developers

The last couple of years have seen a renewed push within the kernel community to avoid regressions. When a patch is found to have broken something that used to work, a fix must be merged or the offending patch will be removed from the kernel. It's a straightforward and logical idea, but there's one little problem: when a kernel series includes over 12,000 changesets (as 2.6.25 does), how does one find the patch which caused the problem? Sometimes it will be obvious, but, for other problems, there are literally thousands of patches which could be the source of the regression. Digging through all of those patches in search of a bug can be a needle-in-the-haystack sort of proposition.

One of the many nice tools offered by the git source code management system is called "bisect." The bisect feature helps the user perform a binary search through a range of patches until the one containing the bug is found. All that is needed is to specify the most recent kernel which is known to work (2.6.24, say), and the oldest kernel which is broken (2.6.25-rc9, perhaps), and the bisect feature will check out a version of the kernel at the midpoint between those two. Finding that midpoint is non-trivial, since, in git, the stream of patches is not a simple line. But that's the sort of task we keep computers around for. Once the midpoint kernel has been generated, the person chasing the bug can build and test it, then tell git whether it exhibits the bug or not. A kernel at the new midpoint will be produced, and the process continues. With bisect, the problematic patch can be found in a maximum of a dozen or so compile-boot-test cycles.

Bisect is not a perfect tool. If patch submitters are not careful, bisect can create a broken kernel when it splits a patch series. The patch which causes a bug to manifest itself may not be the one which introduced the bug. In the worst case, a developer may merge a long series of patches, finishing with one brief change which enables all the code added previously; in this case, bisect will find the final patch, which will only be marginally useful. If the person reporting the bug is running a distributor's kernel, it may be hard to get that kernel in a form which is amenable to the bisection process. Bisection might require unacceptable downtime on the only (production) system which is affected by the bug. And, of course, the process of checking out, building, booting, and testing a dozen kernels is not something which one fits into a coffee break. It requires a certain determination on the part of the tester and quite a bit of time.

Bugs And Bureaucracy

A thread on the Linux Kernel mailing list discussed the process in place for reporting, bisecting and fixing bugs. In response to a suggestion that some of the issues could be solved by introducing new procedures, Al Viro retorted, "we've got ourselves a developing beaurocracy. As in 'more and more ways of generating activity without doing anything even remotely useful'. Complete with tendency to operate in the ways that make sense only to bureaucracy in question and an ever-growing set of bylaws..." Later in the thread, David Miller agreed and noted that ,"the resulting 'bureaucracy' or whatever you want to call it is perceived to undercut the very thing that makes the Linux kernel fun to work on. It's still largely free form, loose, and flexible. And that's a notable accomplishment considering how much things have changed. That feeling is why I got involved in the first place, and I know it's what gets other new people in and addicted too."

Andrew Morton tried to return the discussion to its original topic, "the problem we're discussing here is the apparently-large number of bugs which are in the kernel, the apparently-large number of new bugs which we're adding to the kernel, and our apparent tardiness in addressing them." Al noted that some of the problem is that git is so efficient at merging code, "the patches going in during a merge (especially for a tree that collects from secondaries) are not visible enough. And it's too late at that point, since one has to do something monumentally ugly to get Linus revert a large merge. On the scale of Great IDE Mess in 2.5..." Another suggestion was made to replace bugzilla with something better, to which Andrew replied, "swapping out bugzilla for something else wouldn't help. We'd end up with lots of people ignoring a good bug tracking system just like they were ignoring a bad one."

The iPhone SDK and free software: not a match

The SDK itself is a free download, with which you can write programs and run them on a software simulator. But in order to actually release software you've written, you must enroll in the iPhone Developer Program -- a step separate from downloading the SDK, and one that requires Apple's approval.

Since its release, many in the free software and open source community have debated whether the terms of the iPhone Developer Program are compatible with common licenses such as the GPL. In a search for a definitive answer, we asked the principal parties themselves. Apple did not reply to our inquiries, but Free Software Foundation (FSF) Licensing Compliance Officer Brett Smith was happy to discuss the licensing issues in depth.

First, let's look at the SDK and the developer program that accompanies it.

To download the SDK, you must first sign up for a free Apple ID -- an existing Apple Developer Connection, .Mac, or iTunes Store account will do -- and use it to register with Apple as an iPhone Developer. The SDK by itself won't let you create applications that run on actual iPhone devices, though. To do that, you must enroll in Apple's iPhone Developer Program, for a fee starting at $99.

For the time being, Apple is not accepting all applicants. Currently only US residents age 18 and up are eligible, and Apple is selecting a limited number of applicants. Who gets approved and speculation as to why are popular discussion topics on Apple-centric Web sites.