Development | 14th December 2018

What is a CI pipeline, and why should I care?

Whenever we begin the coding stage of a new project, one of the first things we set up is an automated process called a pipeline. In development terms, a continuous integration (CI) pipeline is a group of tests and jobs that we may want to run against the code we are writing. These can include tests to help prevent bugs and ensure code is well written, and automated processes to release software. Broadly speaking, we will split the pipeline into a number of sections for each of these jobs, which we will delve a little deeper into below.

Syntax Checks

We really care about how the code we write is formatted. Really really care. It matters a lot to us if we've accidentally used a capital letter somewhere it should be lowercase, or put a tab somewhere there should be a space. We aim to have our code consistent enough that it's impossible to tell at a glance which developer has written it, because our style matches. Why do we care so much? It's because it helps keep the code maintainable as we grow and take on new developers. If one developer chooses to indent code with tabs and another chooses spaces, we end up with a mish-mash of styles. The same goes for labels and file names, if one prefers lowercase, and another uses capitals, it quickly gets messy, and it's slower to go back and check if the one you need has capitals in the name or not.

However, we are all human, and with the best will in the world, we sometimes make mistakes. That's where pipelines come into play. Whenever we have finished working on some code and save it to our shared server, an automated pipeline runs checking that it meets our agreed formatting rules, highlighting problems and rejecting it if it doesn't. We sometimes run an additional automated process to fix any minor issues like this, saving valuable developer time hunting out those stray tabs!

Unit Tests

Unit tests are an important part of Test Driven Development (TDD). The idea behind these is that before we write any code to perform an action, we write a test to check the outcome. So for example, let's say we want to write a function that adds two numbers together. Before we start, we would write a test with some different pairs of numbers and the expected result. When we write these tests, it's important to try and account for unexpected input as well, so we'd include not just 1 + 1 = 2, but also 0 + 0 = 0 and 2 + -3 = -1.

Over time, we build up a library of these tests that run automatically each time we make a change to the project. This helps ensure that not only does the code work at the point at which it is written, it also works in a year's time when a different developer makes a change to it, because the tests are run against it again.

Integration Tests

Once the unit tests have run and passed, we use a system called Cypress that allows us to set up a temporary version of the application and simulate a real user interacting with it. We can test that it works, and even when a user tries to do something they shouldn't be able to, that the correct errors are shown. If these tests highlight any problems, we can also obtain screenshots and video of the automated tests taking place to help us understand what didn't work and why.

This kind of test is different from a unit test, because a unit test is only concerned with making sure a small part of the system works as a standalone unit. For example, if we had asked two craftspeople to make a window frame and a glass pane to go in it, but had given neither any further direction, we would hopefully end up with two parts that pass tests in their own right (is it a frame, and is it a glass pane?). However, when we try to fit them together we may discover other issues, they may not fit together, or one may even break when we try to put them together. This is the purpose of integration testing - we know the individual parts work, but do they work together as a whole?

pankaj-patel-729895-unsplash.jpg

Build

For most of our projects we need to compile some of the code to make it run. This process converts the code we have written into code the computer can quickly understand and run, as well as making it smaller by stripping out unnecessary code. Smaller files load and run faster, especially over the internet. It also optimises the code to ensure it runs as well as possible on every device, including specific parts of code required by different devices to function well.

Deployment

When we've finished all of the above steps, we're ready to deploy! There are three stages of deployment we generally use. Whilst a feature is being developed, it will have it's own temporary deployment that can be used by our Quality Assurance testers. Once development has finished and it has been approved by the test team, we will merge it into the main code for the product, but it's not quite ready to be released and go live yet. At this point, it is automatically deployed to a second environment, that we call Staging. This houses a cutting-edge copy of the software, with all the latest pre-release features included. This allows our testing team, and depending on the project, sometimes our client, hands-on access to review and test functionality and stability before we release. Finally, we deploy a live version of the system. We call this deploying to Production, and it is the version of the software that will be used for day-to-day use.

We prefer automated deployment to other alternatives such as copying files manually as it significantly reduces the risk of human error; sometimes a single forgotten file can bring down an entire application! By automating this process we can eliminate that risk and help ensure stability for your system.

So that's a short overview of what pipelines are, and why and how we use them. Of course, we view every project we undertake as unique, and look to find the best bespoke solutions for each of our clients, so it's possible the pipeline for your project will look a little different. We're always happy to explain what we're doing behind-the-scenes to deliver a great product for you and your stakeholders, so please ask. If we're not yet developing something to solve problems for your business, you may be interested to learn about our design sprint process, where we take an idea to development in just 4-days.

Written by
Christopher Johnson
Christopher is a Full Stack PHP Developer and DevOps Engineer at New Socks Media. He cares deeply about making sure our servers and code stay in tip-top condition, and in his spare time he volunteers to help improve the prospects of young people in Bedford and Malawi.