User-friendly continuous integration

Hello everyone,

It has been some time, since the last post. Rest assured, that easyCIS is still alive and under development. In fact, there has been so many of new functions and under the hood technology changes that I've come to decide not to release any more changes into 1.0 branch and wait with the release for a brand new 2.0 version. It is still some time untill the release, but most of the features are already implemented.

So what those changes will be?

Plugin architecture

easyCIS began using Ninject as DI framework and started utilizing its capabilities for loading modules to build a plugin architecture. It will be possible to write C# plugins with build tasks or source control access and just drop it into installation to start using it. No configuration needed. 

Current easyCIS features are build as a plugin as well

New web

Web application has been moved to MVC 3 with Razor view engine. This is in no way only a cosmetic change, because a fresh good design allows much more flexibility in implementing features and greatly improves implementation speed. In regards to plugin architecture, web application is dynamically loading and displaying build tasks configuration. This allows for plugin developlent in separate assembly or several assemblies.

easyCIS will feature user groups and rights management. Feature much needed in larger companies with many teams and many projects. This was somewhat neglected before, but 2.0 will be fully prepared even for not so open environments :)

Users and groups management

Internaly, data access layer has moved from linq to sql to entities framework 4.2 with code first POCO entities, no code tracking proxies and T4 autogenerated CRUD repositories. This is just FYI, that easyCIS is using good and robust design with shiny new technology :)

With the improved data access and better web platform I've been able to implement some better editors and dialog windows. For example in 1.0 order to select a project reference (dependent project) you had to click on a listbox with all projects, then select project from popup listbox and again (after form postback) enter a directory to put reference integration output into. In 2.0 there is a modal editor for selecting project references. After it is confirmed, ajax call will update reference directories textboxes. Overall it is much more intuitive now.

Major feature that will improve usability with say 100+ projects is project dashboard that will replace the simple table with all projects. But more of that later, because this is still a feature on a TODO list.

Being the easyCIS developer, this is going to be a very subjective point of view of course. But let me tell you what I like the most about easyCIS and what I don't about other continuous integration tools.

High level build tasks and no scripting

First of all easyCIS is designed with a strong emphasis on simplicity and user-friendliness. OK, other tools market themselves they are too, one might say. But as far as I can tell what you can expect in other tools is simplification of previously used schema of running of build scripts. First of approaches others went is the way of creating those scripts by means of web interface. This means not actually writing a complex script, but selecting and configuring a ton of tasks that you could possibly want to use. I find that confusing. Second approach I've seen is a nice web interface with a configuration of ridiculously named (probably consensus) features like jobs or agents and than you're left with something like execute ant. What the hell is ant? Some hacker attack? (: If you don't know already that ant is the build script you surely must feel like in the wonderland. easyCIS on the other hand focuses on tasks that developer (at least .NET developer at time of writing this post) does and needs most when building software. Everything else is supposed to be subsidiary. With that in mind easyCIS offers just a few high level build tasks. For example to build a project you just have to use one build task. Provide the solution name, the project name and you're done.

Figure 1 - high level build task

That's what makes me excited the most. You don't have to know anything about CI and it's projects life-cycles or anything about MSBuild or what languages build scripts use to be able to use easyCIS. For basic setup which should be enough for about 95% of all projects, I'm perfectly happy with the tools selection and there is still the possibility to write the build script for the rest 5%.

It's needed to say that the learning curve of easyCIS is much smoother, because you're done with setup and had integrated at least once already before you find out that just your project falls into those 5% and you start learning a build script syntax.

Branches and subprojects - logical way to relationship management

When I used to struggle with CI tools, they either didn't offer any relationship management at all or offered something not very convincing. Relationships could be formed widely by means of "triggering" builds between projects. You could never be sure what depends on what. Relationships was not obvious and they were especially hard to form. That's actually the main reason to develop easyCIS - to make order in relationships and provide features that would make working with them fun.

First of those relationships is a branch tree. Projects can be branched and then branched again and again. Those branches form a tree in easyCIS that can be displayed on several places (project tree, project details on web, project tree in tray app) and you allways know what projects are trunk projects and what are derivates.

Second relationship is a dependency between project and its subproject. This is an essential relationship. If you need to build libraries separately from your application you form a dependency between those libraries and the application. See my previous post Keeping build time reasonably low. If you build a complex project with many dependencies, easyCIS can draw a nice dependency diagram of the entire tree so you never end up with so complex setups to be lost within them.

Figure 2 - dependency diagram

Branching

Next thing is branching. I remember the pain when I was assigned to branch a complex project consisting of 5 projects in one CI tool which uses only a text file configuration. Task took me 4 hours to just edit the file. Next couple of days I was fixing all the bugs in config file introduced by this branching operation. What a pain! Branching in easyCIS is a matter of this simple form (Figure 3). All you have to do is to select the project, click create branch and then select which of it's subprojects to branch as well. The whole project tree is branched in one step. The only manual task to be done is to check the settings of branched projects, if there is something needed to be changed (version control url perhaps). Now thats easy.

Figure 3 - easy branching

The last thing is price. I know, it's not free. But I suppose 399 EUR is low enough for anyone to afford for a tool that saves so much in configuration time. Just consider. There is a online demo and a trial version available.

Keeping the build fast is a necessity for CI to provide rapid feedback. Developers need to know quickly that the build has been broken in order to fix it and allow other team mates to work with a functional version. But what to do when building your project takes a long time? Say two hours? Or if you have extensive testing that involves use of another servers and it takes about a day to complete? Should you reject the continuous integration idea at all or is there a solution for you?

The answer to these questions is staged build. Basically it is about dividing one large build into smaller pieces, called stages and forming a relationships between them.

Say your compile takes two hours. Ok, maintaining rapid feedback from the whole compilation is impossible. But you can stage your compile into module layers e.g. data access, business layer, myapplication. Add or modify these to your liking and compile only those modules/libraries on each stage that falls into appropriate application layer. Next, setup your build server in a way that it will form a dependency tree according to your build server abilities. It will take a form of one of following pictures.

Figure 1 - Triggered build

Figure 2 - Build dependencies

What is different with this setup?

• Each time there is a change somewhere in data access, whole chain of dependencies is recompiled, but you will see the status of data access compilation much earlier than before.
• Each time there is a change somewhere in myapplication, only myapplication is recompiled and yet again you will see its status faster than before.

Same technique applies to tests. Huge automatic tests which needs a long time to complete can be split into several stages and usually they can even run in parallel. Consider this dependencies schema.

Figure 3 - Tests schema

The idea is obvious. The tests make a dependency tree. If any of these shall fails, the whole branch will fail as well  (or not continue with integrating depending on your setup). On the figure both feature tests from groups 1 and 2 wait for db tests to complete, then run in parallel, possibly on different machines. Only when they both finish the last project myapplication integrated is run. This actually does't do anything (maybe deployment), just signals the final health status of integration status of all stages and provide functional, well tested build artifact.

When to use staged build

Multi stage build has more advantages than just keeping build time low. It provides additional clarity to the continuous integration process. There is a much better feedback about where exactly (in what module/what phase/what stage) the build failed if it happens. If it is properly setup, it is possible to avoid error promotion of low priority modules into big applications. The whole idea of integration makes more sense with this setup. It is especially useful for bigger teams or companies consisting of several teams working on large projects.

It does not make much sense for a one .NET solution that compiles under 5 minutes where there are no more than 3 people working on it.

How many stages do your project need? There is no merit to that. Use your own judgment and follow two basic practices:

1. Keep the build time fast.
2. Keep the project structure as complex as necessary, but no more.

That's the ultimate question. Why should we be using this kind of tools at all? For most of us it is a common tool, but there are developers and software companies that are still untouched by this "phenomena".

I remember what were the reasons when the thought for a build server first came up in my former employment. There was a hardcore rogue developer there (actually not just one of those) who over the years gain too much control over key company software. He never used any version control software, he never cooperated with anyone else, he never followed any best practice. Nobody but him was able to read and understand his code. But still he got the job done, so management was happy until the work overgrown him and the workflow problems became visible. I suppose this is pretty common story in companies that grow from a small poorly managed teams into something bigger and then they find out how hard it is to change, what was supposed to be done right from the start.

The reason for a build server is obvious then. Both management and other developers (not counting the rogue of course, he protested :) agreed it was necessary to extract as much competences from this developer as possible without actually hurting the build quality. Reasoning is this: If it builds on his machine, it must build on another machine, right?

This is called build neutrality and is an essential feature of build automation.

By the time we've been considering implementing a build server, we've already been using a subversion for many of our projects, so it was not such a big move in thinking as if we haven't been.

The thought of automation somewhat crystallized by itself. If the build is to be done on the server, if it shall be done from the subversion, why not run it by itself on a regular basis? Now we are talking about something that just takes a new version of everything in svn, meaning all company projects with all of their dependencies that no one knows anymore, placing them to specific location on C drive, because that's what the build tool expects them to be and building them the precise way the rogue developer did. Difference here is that it can happen in regular, automated manner and it can include other projects of other people. Not just the rogue's.

Even on this primitive stage you can see first signs of continuous integration, meaning taking people's inputs and integrating them together on regular basis. It's there even if nobody realizes that yet.

The important thing that comes along (or better that precedes) implementing a build server is the change in business processes according to software releases. More on that some other time, but basically whoever works with the release doesn't go for it to the developer guy, but takes it from the build server. Build servers often offers features to support this processes. Sometimes don't. But the main requirement here is always supported. That is build availability. When the rogue is sick and support guy needs a release to write to the cd, then there is a problem, because no one can find it in the mess of his files. Build server sorts releases into understandable folder tree that everyone can easily find.

Good story end then?

It seems there are only positives, right? Actually there are setbacks as well. The biggest one I mentioned is the need to change business processes. The change in a medium-small company is always painful when it involves learning, change in thinking and most importantly replacing what's somewhat working as it is by something that may introduce problems. Even if many understood build automation was a necessary step, some saw it only as a complication and it involved emotion. Enthusiasm on one hand and rejection on the other hand.

After some talks and research, we've settled on night build done by cruisecontrol.net. We've modified and reconfigured it many times after.

 Hi,

for some time now I've been considering opening up a blog on easyCIS site. So as of this day, I've moved the site to mojoPortal which contains this nice blog module.

As the blog name suggests, it will be devoted to build automation and continuous integration. I'll be posting here some of my knowledge on this area I've gained in my career in software development and when developing easyCIS. You can also expect some insight on coming features of easyCIS and perhaps its comparison with other tools.

That's pretty much it. Hope you enjoy my blog!