The main idea behind Continuous Integration (CI) is to be able to integrate all the source code for any given component into a shared build system, which has as a purpose to check that every project builds successfully after changes are made to it, and most importantly, that all the unit and integration tests run correctly. These verifications can be set to run numerous times per day, in order to aid early detection of bugs that may have been introduced.
Why do we need CI?
Let’s picture a scenario without CI, where everything is manually tested by developers when they need to get ready for deployment. Normally, developers would do all their coding within the IDE, where they can add all the required new features, browse through the code when debugging is necessary, do all the needed refactoring to keep the source code clean and manageable, and even sometimes, run the unit and integration tests of the component they’re working on right from the IDE as well. Although this helps to speed up development, as every developer can code, debug and test quickly, if this process is not performed on dedicated build servers, we will easily find ourselves in the “It works on my machine” scenario. This, in turn, leads to a manual and error-prone build and delivery process, where sporadic compilation and integration issues may appear, as well as failed deployments. In order to be able to detect compilation and deployment problems early on, we need an automated CI system capable of executing the entire compilation process on a clean machine, and also running the entire set of unit and integration tests. This way, we’ll be able to detect issues that appear while we’re still in the development cycle, and not when deployment begins.
How do we achieve CI?
To allow C/C++ projects to jump on board the CI cycle, two main tools were selected: Make and Gradle. On the one hand, Make is a well known tool to anyone working with C/C++ projects. It’s responsible for compiling the entire source code of a project, and it allows for great customization and extensibility by allowing each user to write their own compilation rules. On the other, Gradle, is a build management tool that comes from the Java world, yet aims to be language-agnostic. This tool allows for specifying all the required tasks and dependencies each project requires for building and executing, and then fetches such dependencies and makes them available for the project to use. When it comes to Java projects, everything works out of the box, yet when dealing with C/C++ projects, a little extra effort is required.
Make
To keep the migration of existing projects to the CI cycle for C/C++ simple, a Make framework was created, which hides all the complexity required and exposes a common set of targets that allow for compiling and testing all the associated modules. This way, the only thing that is required when migrating to the new Make structure, is to define a small subset of variables that indicate which files need to be built and their additional flags. All the logic required to actually carry out the compilation process is hidden behind the Make framework.
Gradle
As the transitive resolution of compilation dependencies was not possible from the Make framework alone, a special plug-in for Gradle was written. This plug-in searches and fetches all the required dependencies, in a transitive fashion, and compiles each of them in logical order. Once all the dependencies are compiled and available through the CFLAGS, CXXFLAGS and LDFLAGS variables, it proceeds to compile the project. This is why every project needs to execute its compilation process though Gradle rather than Make directly.
Considerations
One keen reader could wonder why each of the dependencies needs to be compiled, and is not directly used as an already compiled version of that static/shared library. The reason why all the required dependencies’ source code is fetched rather than their binary is to ensure that everything works perfectly and that it has been compiled natively, thus assuring that no problems may arise from different system libraries or dependencies among the different OS that developers have.
Jenkins
The final part for achieving true CI with C/C++ is to add all the desired projects to Jenkins, the cross-platform, continuous integration and continuous delivery server. It’s used for building and testing software projects continuously. Now that projects have made their changes in order to use the Make framework and have added the required Gradle components, it becomes much simpler to add CI jobs, as all the projects will have the same targets available, and will behave exactly the same, thus special or custom logic for each project is no longer required.
What’s next
Now that C/C++ projects can be added to the CI cycle, we need to move forward to mastering the Continuous Deployment (CD) phase. To do so, additional tools and changes may be required, in order to create a single deployment preparation stage which works for every system on which we may want to deploy.
Continuous Integration Image: http://blog.jelastic.com/wp-content/uploads/2015/07/devopsimage.png
