Zimbra Source Code distribution should be improved

Introduction

As you might know source code is very similar to a cooking recipe. I’m going to borrow (and adapt) some lines from Free as in Freedom by Sam Williams for describing the analogy:

Both program and a cooking recipe provide useful step-by-step instructions on how to complete a desired task and can be easily modified if a user has special desires or circumstances. «You don’t have to follow a recipe exactly. You can leave out some ingredients. Add some mushrooms, ‘cause you like mushrooms. Put in less salt because your doctor said you should cut down on salt-whatever.»

As software programs and recipes are both easy to share. That usually turns out into that the person giving the information gains two things: increased friendship and the ability to borrow interesting recipes in return.

In order to distribute source code (I.e. the last changes the recipe has (E.g. Salt proportion increased by 25%) ) the most used distribution mechanism is called GIT branches.

That technical gibberish can be understood as the once famous rss feeds for tracking news on line which are now being of replaced by twitter timelines.

What I mean is that following someone on twitter and getting his latest tweets is very similar on checking out (technical way of saying getting) a GIT branch which contains the latest changes performed in source code.

The same way as you don’t usually eat your uncooked vegetables and meat without cooking the code needs to be build. The build process takes source code as an input and outputs a binary program. This binary program is the equivalent of a cooked dish ready to eat. As it happens with expired ingredients which unables us to cook them, sometimes source code is unable to be build. That means that we cannot get a binary program out of it.

Building a program each time a new change has been made can be haunting. Another thing it’s usually done when building a program is to test it. One of the most well known technologies for such as task are the unit tests. A unit test validates that an expected functionality still works as expected. The unit test might be as stupid as the one you would expect for a sum program.

Number 1 to sum: 2 Number 2 to sum: 2 Expected result: 4

If the program source code is changed, the source code will be rebuilt so that we get a new binary. If we test the new binary again this unit test it will try to automatically input to it : 2 as the first number to sum and 2 as second number to sum. If the result is something else than 4 that means that the binary does not satisfy all the binary tests and it should probably be fixed.

Current layout for Zimbra Git branches

Zimbra has recently switched to use Git branches for distributing its source code. Its branches are available only for their Open Source Edition product. There are three different types of branches:

• Last unstable code for a Zimbra release : This is the Zimbra release name and its ‘foss’ suffix. You can track the latest applied fixes for a given Zimbra released there.
  • helix-foss (Helix release which was mapped to 7.x.x versions)
  • ironmaiden-foss (Ironmaiden release which was mapped to 8.0.x versions)
  • judaspriest-foss (JudasPriest release which was mapped to 8.5.x versions)
• Last stable code for a Zimbra release specific version. This is the Zimbra release name, its associated version and ‘foss’ suffix. You can check the exact associated source code (if you are lucky enough that Zimbra has not released it with two different build numbers) to a given Zimbra release version. This is compulsory for Zimbra to comply to its license.
  • helix-700-foss
  • helix-712-foss
  • ironmaiden-800-foss
  • ironmaiden-806-foss (Ironmaiden release mapped exactly to 8.0.6 version)
  • judaspriest-851-foss
  • judaspriest-860-foss
• Last unstable code for  the next Zimbra version. For this type of branch there is only one branch named main. As always it has a foss suffix.
  • main-foss

You can learn more on how these branches are organized in the Building Zimbra using Git wiki article.

 Community agreement on public repository branches

In the open source there are many projects that use branches in their repositories (A repository is a collection of branches of the same software). The development branch sometimes is named main and the stable branch is named stable. In other projects main is always the development branch while the stable releases get their name from its version or its release name. Whatever the name is given to the unstable branch there’s a common criteria on handling pushes (updating source code changes to one branch) to the repository. This criteria is:

If it does not build do not push it to our public repository.

This criteria ensures that everyone who downloads the source code and tries to build the source code will be able to build it if the required conditions for it to build are met. Yes, sometimes, a commit (branch update) breaks the project (source code set) build but that’s an exception not a rule. Check what Info Molnar said in Linux Kernel mailing list:

In the future please do not push broken code upstream or put it into drivers/staging/scsi/ if it has outstanding problems. I’m using the hack below for the time being.

Community agreement on build branches for quality assurance (QA) based on unit tests

Another agreement that it’s usually found on big project such as openoffice, firefox and other projects is to use temporary branches based on last source code contributions in order to build them and automatically verify its unit tests. You can see ReactOS buildbot status for a live example. Not only you can verify if the code can build but also if the unit tests have been succesful or not.

These kind of process usually copies (or merges in git terminology) its verified for build and unit tests changes into another branch for testing consumption. You can read an actual example in Factory wiki page which finishes into a rolling released named Tumbleweed which it’s under the openSUSE GNU/Linux distribution.

The Factory project is the rolling development codebase for both openSUSE Tumbleweed and the next stable openSUSE release. There is a constant flow of packages going into Factory. There is no freeze; therefore, the Factory repository is not guaranteed to be fully stable and is not intended to be used by humans. The core system packages receive automated testing via openQA. When automated testing is completed and the repo is in a consistent state, the repo is synced to the download mirrors and published as openSUSE Tumbleweed. That usually happens once or twice a week.

Finally I’m going to quote metux comment about how main branch is handled and what’s his opinnion (probably better technically than mine) on how Zimbra should proceed (this is an offtopic in this article):

#1: keep mainline as stable as possible #2: when doing a release, tag the current state and fork off a maintenance branche #3: run all the final tests (those that cannot run automatically by the CI) #4: do final fixes in the maintenance branch and thag the final release there #5: forkoff from the maintenance branch, rebase it onto mainline (which might be proceeded meanwhile) and merge in there #6: do further bugfixes in the maintenance branch and do further bugfix releases from there, reintegrate them like #5

 Main-foss branch build problem

The problem about main-foss branch in Zimbra distributed source code is that the branch is not garanteed to build at all. We can read the explanation for the former mentioned wiki page:

main is the development branch. The ‘main’ branch, (often referred to as mainline) contains the latest code – if your interested in seeing pre-release software check it out. It is constantly in flux (for instance we might stagger spec files to avoid the need to always rebuild ThirdParty), if you’re hitting a error in the Zimbra build side the first thing to do is re-sync, as we typically resolve these fast.

That means that main is the development branch. However it is not guaranteed to build at all! Current versions of Zimbra (as per writing this article) are: 8.5.1 and 8.0.9. 8.6 is on the way and Zimbra people are supposed to be working on Zimbra 9 in the main branch. However if you want to test Zimbra 9 you cannot. Yes, you can, if the build instructions are updated and if the source code builds. The current problem about Zimbra declaring its main branch as maybe hitting an error implies that you cannot completely discard downloaded source code as being the reason for the faulty build. Specially if it’s the first time that you try to build it. You can see metux comment on this problem (which, by the way, inspired me on writing this article):

 Anyways, the Zimbra source tree is horribly broken – at least the mainline. It doesn’t even compile, and it’s getting worse day by day. I had fixed that, but now they introduced even worse regressions.

Next branch proposal

How can Zimbra improve its source code distribution so that it meets community minimum expectations? A new branch named: next-foss should be created. This branch should ensure two things:

• Its internal documentation on how to build it is updated
• It builds when you use suggested Operating Systems and suggested build instructions (internal documentation)

Optionally if there’s a quality assurance (QA) flow already in place for internal Zimbra development:

• QA should be adapted so that it works using main-foss as an input and saves or merges only its validated changes into next-foss branch. This last piece is very important because as QA being usually an automatic process it avoids to commit a person into doing this validation work.

  

 

 Why a next branch is so important?

With a next branch you can:

• Work on latest versions without worrying about broken code
• Test if your developed zimlet still works with Zimbra latest version
• Test your if your custom hacks (modifications) are compatible with Zimbra latest version
• Show a new feature for a client that it’s doubting between Zimbra and other email and collaboration suites.

This is very important for the zimlet development because it will make easier for the zimlets to be tested with the new version. Sometimes using the latest version in production is not about that latest version being stable but about ensuring that its associated programs (its environment) work as expected. Ensuring a healthy development environment for contributors will help Zimbra to sell more licenses when a new release is offered to the public and thus decrease its amortization time. The reason is that not only the developers community will be able to report bugs more easily which will in turn improve Zimbra stability and reputation but the environment programs will probably be ready from day 1 of the release. Another side effect of the next branch is that community would be able to offer a daily build of Zimbra next for the most adventorous users and developers.

Further improvements needed

If you have read metux suggestions you will probably understand that some other improvements will be needed:

• Use tags to identify exactly which commit (change) corresponds to a given release. This is not only technically desired but compulsory for legally respecting open source licenses.
• Build each feature in a separate branch if you already don’t do it (I have no information on this subject).
• (Not sure if it’s technically possible)Totally migrating from perforce source code system to a similar system which uses Git under the hood.
• Open a foss development mailing list. If you are told in the Developers subforum that a bug has to be submitted so that developers see it that’s a not a valid Development forum (as per place where to talk). Don’t get me wrong. If you happen to find a bug you have to submit it to bugzilla. If you ever have a doubt on how to modify or compile your Zimbra build developers forum seems not to be visited by Zimbra (as in Zimbra Inc) developers. At least, as not frequently, as one would hope.

I will probably write more articles for explaining each one of these issues based on this initial article. Was I not to write them that might mean that Zimbra is improving on source code building and distribution or that I just don’t have time for writing them.