I’m Sam. I’m a programmer, hacker and scientific researcher.
I have been a free software user and developer for a long time, and a Debian developer since the year 2000.
Around the year 2004 I started to have a feeling that Debian was no longer the fun it used to be, though I kept finding it fulfilling to contribute to the project. Now I want to make it fun again, including for newcomers.
And for those wondering, I am as serious as you think I am.
I can be reached by email at email@example.com.
Hi! I’m Sam, and I’m serious.
I have been a Debian developer since the year 2000. Prior to that, I was a happy user since around 1996. Having 150 highly qualified developers working together on the biggest Linux distribution out there sure was impressive, and I have always admired both the project for its noble goals and the distribution for its technical excellence. It was an honour and a challenge to become part of that project and it helped me become better at programming, communicating, working in teams and even speaking English.
At about the same time, in 1998, I started helping on a small project called VideoLAN. Within that project I quickly evolved from the shy but enthusiast newcomer to the talented contributor. As I gathered knowledge about the codebase and respect from other developers, I became omnipresent and started controlling what others were doing, by mentoring them or by making enlightened project-wide decisions. I was naturally overwhelmed with work and had to manage my priorities, dismissing less urgent stuff and locking tasks because I did not trust others to complete them in the perfect way I expected. Several contributors left in frustration and today I believe that despite VideoLAN’s quality and popularity it would be even greater today had I not prevented developers from working freely.
This is a personal story but certainly not an isolated one. History of other projects must not be ignored: egcs (though we now call it gcc) overtook the FSF control freaks’ suffocating gcc, X.org quickly replaced XFree86 after everyone got fed up with the core team’s control freaks. And I foresee the same fate for the glibc if Ulrich Drepper remains in charge of everything.
I believe a similar thing is happening to Debian. I see the enthusiasm and I see the conservatism and I see the overwhelmed control freaks and I see the frustration. I understand both points of view but I also think the confrontation is hurting the project badly. And I am running for DPL mostly because I love Debian and I want to change it before it’s too late and something else rises from Debian’s ashes (and some people say it has already happened).
Debian is afraid to change too fast. We have a history of being good, innovative and popular, but it’s a good idea to scratch our itches from time to time. We are not alone and just saying “Debian will always be the best” does not make it true.
I don’t think so. I believe we can have the three of them, both in terms of people and of products. The one we’re currently slowly dropping is dynamism, and I want to bring that back without impacting on the other factors. It is challenging, it is hardly a one-man task, and it would be dishonest to pledge that others will do what I suggest. But even if I can’t make you do it, I sure hope I can make you want to do it.
Admit it, we have become a bunch of old farts who have a lot less fun than before. I find it hard to have as much fun working on Debian when there is so much frustration.
There are many ways to reduce the frustration:
Team work is good. It makes you communicate. The “Uploaders:” field is one of the best things that happened to Debian’s organisation. It is truly fucking brilliant. Of course it allows several people to work on the same package, but it also helps reduce the notion that packages are “owned” and it makes us create teams.
I want to encourage co-maintenance by making it the norm, not the exception, by having NM candidates join an existing team instead of adding new packages to the archive, by trying to have at least one backup uploader for each “Priority: standard” package, by having a policy of light highjacking (forced co-maintenance) of neglected (eg. unanswered old bugs, unapplied patches) packages, including for NM candidates. No more “I’ve been busy with real life for the last six months but don’t you dare touch my package, I promise I will take care of its bugs within the next month”.
I believe having bigger teams is amongst the solutions to the high latency problem some of our core teams have. I want to be able to appoint additional, voluntary assistants to those of our core teams (DSA, FTP-master, buildd admins, DAM) that appear to need it. I know some of them already have, but for so long a time that you may wonder why they are still assistants.
After a reasonable period (say, 3 months) I want these assistants to be either approved by the team as full members or rejected with proper justification (eg. inactivity, incompetence, important human relationship issues).
Ideally, these assistants should be co-opted. There is however, for historical reasons, an important concentration of powers amongst a very small number of individuals. I have every reason to believe that it makes some of the work more efficient, but it also has drawbacks:
All this adds to the frustration. I would like, in the long term, these teams to have as few common members as possible, with inactive members (eg. buildd admins that do not run a build daemon) being demoted to assistants.
I know it is hard to handle newcomers. They need training and help, and all the training and help is time and energy that could have been used doing real work instead. But many people can learn by just watching, and since some current members of the teams appear to be effectively just watching, there should be no problem adding volunteers to the team.
I like small projects. Small projects are fun. Small projects take less time, and by the time we finish a big 6-month project we can run a dozen 2-week projects, and if someone gets bored by a small project, they can just start another one and not waste 6 months of work.
We have trouble managing big projects. We struggle to release because releasing is a huge project and we don’t know how to manage a huge project (or just don’t apply what we know). We eventually succeed, but I would not exactly call what we do “project management”. Being volunteers, not all rules of project management apply, but it would be foolish to ignore all of them.
Proper reporting is part of these rules. It not only helps the current project by making communication better, but it also helps future projects learn from our schedule, missed deadlines and the reasons for them. Sadly, I don’t know how to get proper reporting without making it a mandatory condition for staying at the appointed position. I have seen many people playing dead then suddenly reacting to public blame. But I’m willing to consider more constructive alternatives.
We currently have issues that we do not talk about publicly. We announce new developers, yet we don’t announce people who leave or take long breaks, even when they are key people. I do not think there is a valid reason for that as long as we do not disclose their reasons.
We are also afraid of saying we are late on schedule or missing manpower. I’d prefer we admit “at the current rate, we’re going to miss the deadline by 2 months” 4 months before the deadline so as to kick our asses into working more or reconsider priorities, rather than just say “nothing too dramatic” and watch the deadline pass.
An ill-effect of our high quality requirements for new maintainers is that we end up with people who are highly skilled in package maintenance, whose energy would thus be wasted not actually working on packages, while there is equally important work to be done. I want translators, documentation writers, graphists etc. to be Debian developers, too. We want excellence, but our recruiting process is only about package maintenance excellence and Debian is not only about that.
To achieve this I want to lower the threshold for becoming a Debian developer, while not giving immediate upload rights and keeping the present requirements for obtaining them. The FTP-masters have recently proved it to be trivial to add per-developer upload restrictions to dak, so this should be really easy to implement.
I would also like the mentorees’ work to be more visible to our users and I am open to ideas on how to achieve that. For instance, just like we have an experimental section for experimental software, we might have a “mentors” or “untrusted” section to which NM applicants would get upload rights after a while and some trust requirements (such as a minimal number of advocates). This section would not be autobuilt or signed, and packages would transition to unstable once a sponsor has approved and signed them.
Debian has money. Last time someone wanted to use that money for something, we disagreed, and he found money somewhere else. So we still have that money, and I would like to use it at least to fix our broken hardware. I cannot believe this is in a DPL platform, but escher has been down for ages, developers do not have access to an alpha machine, and we have not even tried to fix that problem with money.
Speaking of money, one thing that requires lots of it is meetings. IRC meetings are not enough for some tasks, and isolating people in a remote place to work together on nothing else than their Debian project has proven to work very well. Though I am getting scared by the escalating luxury of the DebConf accomodations, I believe we can and should afford even more meetings to take place. There are local structures in many countries (Extremadura in Spain, Cetril in France) that can take care of the logistics.
There was a recent announce that I find quite perfect. Should the decision to carry on with this initiative after a few months be mine, I’d gladly agree to it. I would prefer a full report rather than a “brief summary of what went on”, but maybe that’s what’s already expected and the wording is just inaccurate.
Admit it, Debian could be sexier. Currently, Debian rhymes with epicurean, octogenarian and caveman. Even the FreeBSD website is sexier than ours, how can we expect to attract users? Have you seen how the website renders on Internet Explorer?
We lost countless users to Ubuntu. Admittedly, Ubuntu drew a lot of its users from other distributions and even from Windows. But there is no reason for Debian to be simply “the distribution upon which Debian-based distributions are based”, we can also get new users by being more appealing.
There are plenty of talented web designers out there who definitely would want to help design a website that looks like it was updated at least once in the 21st century. At least a few of them consider fame a sufficient reward, hence the only thing to do is to say we’re interested in a new design. We also need a more dynamic website, that does not for instance have a pathetically short list of 10 “news” items a year. There are other newsworthy things to tell our users on the front page, we just need web maintainers to add them. With my plan of having more developers who do not focus on packaging, we will have more than enough people to do the work.
Our bug tracking system is ugly and hardly usable. If you don’t think so, you probably have never been to the WNPP pseudo package. I am sick of hearing that our BTS is better than Bugzilla because Bugzilla does not have an e-mail interface as an excuse to hide the fact that our BTS is uglier than Bugzilla and does not let you perform simple tasks using the web interface.
Though I’d really like a web interface to our BTS, very minor additions can be done very rapidly, for instance by having artists work on meaningful icons for bug tags (mockup here).
There are very useful services around, such as http://www.buildd.net/, http://bts.turmzimmer.net/ or http://bjorn.haxx.se/debian/. I cannot understand why they are not Debian services. One may see them as simple external services, I call them NMUs on our infrastructure that should be acknowledged and merged back. And if we lack the computing power, I have explained that we have money to fix this. There is also http://www.backports.org/ that is an impressive service but that would be tricky to integrate into Debian, especially because of security updates, but we can at least try to make transitions from it to testing and unstable smoother.
Package descriptions are what the user sees before even installing the software. Translating them (by distributing Packages-xx.gz files) would also benefit to our users.
The size of our distribution is both a pride and a curse. Etch is about to be released and we managed to reduce the time between releases with a stricter process, but we’re still not performing very well. As free software gets more users, more developers and more funding, projects are released more often and instead of following the trend we’re releasing less often.
It is high time we implemented one of the more aggressive strategies being discussed to get more releases with more up-to-date software, starting with Lenny. I have my personal favourites, but I do not think it is up to the DPL to decide which way to go. I will just make sure the people who want to make it possible get the tools and the power to do it.
Debian is great, and there is nothing like Debian. But we’re often too proud to admit we can do better or that others are doing better than us.
There is information that can be automatically fetched from other distributions, even from non-Linux ones such as the BSD projects. Of course the dedicated Debian maintainer is well aware of it, and already communicates with upstream and maintainers from other distributions, but a way to monitor which patches are applied upstream and which patches are shared across distributions would be nice. Upstream and other distributions could benefit from a tool that monitors:
Ubuntu is “not evil” only in a Google kind of way, we won’t get patches from them out of pure philanthropy. But we can learn from their processes as much as we can from their patches, so let’s just get them. Another excellent distribution we should keep a close eye on is Gentoo. Despite all the criticism that can be done, there is also much to learn. They have been faster than us at adding new architectures and kernels to their core infrastructure, for instance.
I would love to see alternate kernel projects benefit from the rest of our infrastructure (especially buildd.d.o) far sooner in the porting process. We’re right in having requirements for these new architectures to be included, but ignored portability bugs are common, as are regressions, in which case the maintainer is not necessarily to blame if build logs are not available. The inclusion of amd64 suffered from pathetic delays that I do not want to happen again.
Another great addition to the distribution would be multiarch support. It is underway, and if everyone works together we can have it in Lenny.
I am also interested in pushing the dpkg-cross approach in order to cross-build the distribution for targets that cannot run a build infrastructure (such as very specific CPUs on embedded platforms).
As far as organisation is concerned, I think at least one porter per architecture should have access to wanna-build.
We could use some improvements to the archive management software. For instance, avoid unnecessary NEW waits (such as simple soname bumps) by allowing new binary packages for a source package that was already in the archive. NEW processing latency should not affect the everyday maintainer, and multiple-binary abuse can be dealt with with the BTS.
Lintian could also be integrated with dak to automatically reject uploads with serious errors (binary-in-etc, missing-copyright, shared-lib-without-dependency-information...).
A debugging infrastructure is nothing new, many developers have talked about it and worked on it. There is still much work to do and I would like to make it a priority: transparently generate .ddeb debug packages (for instance during the dh_strip or equivalent step), integrate them with dak, integrate them with apt, integrate them with higher level infrastructures such as the GNOME and KDE SIGSEGV handlers.
I would also like to make build logs mandatory. There is currently no way to know how exactly the packages we upload were built, and we could easily hook to dpkg-buildpackage to log the build and attach that log to the .changes, then later make that mandatory. dh_buildinfo is another underexploited initiative.
Our transverse teams (especially the translators) are already NMUing packages during release periods. Let’s allow them to do this constantly, with reasonable rules. Under these rules I’d like LowThresholdNmu to eventually become opt-out rather than opt-in (currently around 150 people subscribed, who probably are amongst the most active maintainers).
The QA team looks like they’re the garbagemen of the archive, doing QA uploads of orphaned packages. I want it to become completely natural to see QA uploads by anyone for any packages that have unaddressed old bugs with patches, for instance. To avoid abuse, one could make the DELAYED queue mandatory for QA NMUs.
Ubuntu manages transitions far better than us because no one owns packages. By having some sort of transition strike teams, or more generic squads blessed by the release managers, I think we can achieve the same result. These teams (ideally all maintainers, of course, with anti-abuse protection) would also have restricted wanna-build access for fast requeueing.
That was it. Thank you for reading.