What is continuous integration?
Continuous integration is a software development strategy that increases the speed of development while ensuring the quality of the code. Developers continually commit small increments of code (at least daily, or even several times a day), which is then automatically built and tested before it is merged with the shared repository.
In a nutshell:
- Every developer commits daily, or even more often, to a shared mainline
- Every commit triggers an automated build and test
- If build and test fails, it’s repaired quickly -- within minutes
Continuous integration works hand-in-hand with Agile methodologies. Team members work on incremental “stories” and the code for these software changes is merged incrementally into the shared software repository multiple times a day.
Continuous integration can be used for many types of software projects.
For example, we use continuous integration for updating the CircleCI website:
- Developer or content publisher creates a new branch of code in GitHub, updates the HTML on the page, and then commits the changes.
- When the developer has completed all the changes on the particular branch, he/she then creates a Pull Request in GitHub.
- GitHub is integrated with the CircleCI. CircleCI runs the build processes on the code and then runs the automated test scripts.
- If the CircleCI encounter any errors (build status red), the developer is notified via email.
- Otherwise, the user is notified that the build was successful (green), and the code has been pushed to the staging server that allows us to preview the web page.
- Once verified, the developer can then merge the new branch of code into production.
Read this blog to view the technical details behind our CircleCI website.
Benefits of continuous integration
The fundamentals of continuous integration is quite simple: commit and integrate your code often — daily at a minimum. Such a small change in your software development process can yield big results.
Using this development strategy, you can:
- Improve team productivity and efficiency
- Accelerate time to market
- Build the right product
- Release higher quality, more stable products
- Increase customer satisfaction
- Keep developers happy and shipping code
How do these benefits accrue from such a simple change in your workflow?
When you commit more often, you can identify and resolve merge conflicts earlier, or avoid them altogether. So, instead of writing a thousand lines of code and finding an error, you’ve only written a hundred. Finding the bugs in your code becomes an errand of a few minutes instead of a few hours. And, you can spend that time instead working with your fellow developers to determine how you can work more efficiently. This leads to improved team productivity and your developers shipping working code more quickly.
Since you’re shipping new features faster, you’re getting to market more quickly. This enables two things:
- Your customers are getting access to new features faster, leading to higher customer satisfaction and
- Your company is getting a faster return on investment from the new features that have been developed. You no longer have to wait until the next milestone to release new code, which might be months in the future. Instead, you’re delivering value as soon as the feature is ready.
The continuous integration toolchain
The software development process consists of three main centers of work:
- Creation of the code: This is where the software development team is the most creative. They are turning user stories into elegant code.
- Orchestration of new code as software: This is where the heart of continuous integration lives. Once the developer commits code, the continuous integration build server automates the build, test, and deploy process.
- Logistics: This is the final step of shipping the new software to production. Tools in this part of the process monitor and manage the software in production.
The most important tool for the creation process is the repository used to hold the code. CircleCI integrates with GitHub and Bitbucket. Our server version also integrates with GitHub Enterprise.
CircleCI lives in the orchestration center of work. It does the heavy lifting of turning the code into working software. Users configure how the code is processed via the YAML file. The YAML file allows configuration via code and is almost infinitely configurable, allowing teams to build and test their way.
The logistics part of the software development process is getting the tested and approved code into production. This includes the set-up, configuration, and management of the production servers. To gain the full benefit of continuous integration, continuous deployment is necessary. This is achieved through platforms such as Heroku.
Continuous integration best practices
Developers who practice continuous integration commit early and often. This allows them to detect conflicts early. And, having a smaller commit makes it easier to troubleshoot the code if there are any issues. Committing software daily, or even more often, is necessary for continuous integration, but not sufficient.
To use continuous integration successfully, the team must:
- Make testing an integral part of the development process. Tests should be written as code is created.
- Ensure that the testing environment mirrors production.
- Use coding best practices, such as pair programming.
- Automate the deploy workflow.
The most important element a company needs to have for successful continuous integration is a rigorous testing culture. In order to confidently integrate new code into the mainline, the team needs the confidence that the code is sound. This is achieved by testing, which should be happening regularly. Engineers should write tests as each feature is being developed. At CircleCI, we run tests on any new code that is committed and our system alerts the developer if they have a successful “green” build, or if they need to fix issues because the build was “red.” Without tests, of course, green builds are meaningless. Ideally, the team is using automated testing, though that is not a requirement. QA services, such as Rainforest QA, can be integrated into the continuous integration process.
To support your rigorous testing culture, it’s important that the testing environment mirrors the production environment. Otherwise, you have no guarantee that what you’re testing will work in production. This means the testing environment should use the same version of the database, web server configuration, artifacts, etc.
Another best practice for software development is pairing during coding. For more complex pieces of functionality, teams discuss the architecture approach before a single line of code is written. Before any code is merged into production, another developer always reviews the code. This helps ensure that coding best practices have been used, the code does not conflict with existing code or code that another developer is working on, and the new functionality is scalable.
Finally, to ensure that the entire software development pipeline is fast and efficient, builds need to be fast and the deploy workflow should be automated. Every minute that code is building is a minute wasted. And by automating the deploy workflow, the team gets their finished code to production more quickly. Because, after all, what’s the point of developing software quickly if it’s not getting to the customer?
For all the reasons above, continuous integration is a worthy goal of any software development team. But, how do you know if you’ve achieved your goal? This is where key performance indicators help. You need to measure the processes that make up the software development lifecycle in order to make improvements to it. This is how you will know if the changes to your process are making a difference.
Here are 4 key performance indicators (KPIs) to measure your software development:
- Commit-to-Deploy Time: This is the time it takes for code to go from commit to deploy. The goal of measuring CDT is to understand how long it’s taking code to get from one end of the pipeline to the other and what roadblocks you’re encountering, if any.
- Build Time: Build time encompasses both the running of tests, as well as any preparation needed to make those tests run.
- Queue Time: The amount of time engineers have to wait before their build executes.
- How Often Master Is Red: Every time master breaks, start a cumulative timer. Divide that number by the rest of the time in the year so far. That’s the percentage of time master spends “red”.
It’s helpful to have a regular cadence of reporting. Automated daily reports help ensure that if there is an error, the team catches it fast. And, analyzing the weekly reports allows teams to dive much deeper into the processes and find improvement areas.
How to get started with continuous integration
Continuous integration is not just about implementing a new tool. It is about changing how the development team works. It’s a new mindset. And, making culture changes in an organization is not easy. The best way to get started with continuous integration is to start small.
Use your next small project as a proof of concept and get the following fundamentals into place:
- A rigorous testing practice, using automated tests ideally.
- Consistent software environments, from testing to production.
- Training on continuous integration practices.
- Reports to measure key metrics.
Once your team starts committing regularly and in small increments, they’ll see how much easier it is to be responsive to bugs. And, as they resolve bugs faster, they will deliver features even faster. This will give your team the momentum to scale continuous integration across the organization.