In this series of posts, we define a simple and open source CI/CD pipeline for Linux embedded device development on GitLab that takes advantage of the power of Pantacor Hub and Pantavisor Linux.
| Table of Contents (4 part series) |
|---|
| Part 1: Automating CICD Pipelines for Embedded Linux Projects |
| Part 2 – Generate Flashable Images with CI/CD Pipelines |
| Part 3 – Templated CICD pipelines for embedded Linux Projects |
| Part 4 – Customizing CICD pipelines for Embedded Linux Projects |
In the previous post, we added the needed configuration to generate and store flashable images with the stable version of your project to the pipeline.
In this Part 3, we will look at the internals of our CI script to understand how it works.
Background
If you completed Part 1 and Part 2 of this series, you may wonder why we didn’t set up templates. The answer to that is we started this CI process for our initial devices. The pipeline keeps a number of initial devices that helps anybody to get started with Pantavisor development by keeping the code base up to date on a daily basis. In this way, it is easier to integrate the different apps (containers) and the bsp that forms those devices into the master branch. It’s also convenient to automatically detect at the end of the day if any of the landed merge requests break the baseline boot for different devices.
Automation also helps us track and revert failing commits even after running a modest set of tests. Furthermore, we can automatically publish the initial images built by the CI for anyone who wants to flash their devices and start their projects from there.
At the beginning, everything was kept in the .gitlab-ci.yml inside of our project, but as other projects came up, we saw the opportunity to reuse the code. Therefor, we moved the common script to a template that can be included in any GitLab project. The motivation to start a new CI project to use this template is always the same: keep groups of related Pantavisor-enabled devices up to date, which means keep their bsp and any apps (containers) up to date.
In the end, the proposed CI from these blog post is just a relatively simple script run by GitLab CI in a GitLab runner. The strength of it comes from the use of Pantacor Hub and Pantavisor. The GitLab runner does the actual update and build, and the results are published to Pantacor Hub, and remotely updated to a number of Pantavisor-enabled devices.
It is time now to explain the template internals so you can discover the simplicity of it first hand (if you haven’t already).
Explaining the CI template
The example CI template used in this tutorial is hosted here. In this next section, we will discuss its architecture, the state machine and tools.
Architecture
This diagram below is an overview of the CI architecture and shows how the scripts run in a GitLab runner sit in relation to the device and the cloud.
To the left are number of devices that we want to keep up-to-date. At the center, the .gitlab-ci.yml with a list of devices (pairs of latest and stable devices, generated with the mkdevice-ci script) in the repo and GitLab CI variables. At the right, Pantacor Hub receives the updates and is in charge of storing and updating each Pantavisor device state in turn.
The logic of the CI is therefore contained inside the code repository (included from the templates) and can be overridden following the rules of GitLab CI. This logic is then executed in a GitLab runner, which can be either the GitLab shared ones or one of your own.
The storage of the binaries is managed by Pantacor Hub and run in a Pantavisor-enabled device, while the storage of the images is done in an AWS S3 bucket, although this last part optional.
Stages
There are five stages and they can be divided in two groups and run sequentially: scheduled stages and pushed stages.
#1. Scheduled stages
The scheduled jobs are executed either by the GitLab scheduler or by the user via GitLab user interface. They are two: post and push.
- Post: checks for bsp and apps updates in the list of latest devices. If positive, it updates them by posting to Pantacor Hub, which in turn updates the Pantavisor devices.
- Push: gathers all update information and advances the state of the device in the repository. This is done by committing to the repository the device revision that was updated in the last stage into the recipes folder. This is important because each commit in the repo represents a state of the devices.
#2. Pushed jobs
The pushed jobs are executed when either a commit or a tag is pushed to the repository. There are three jobs: build, validate and deploy.
- Build: Only triggered when pushing a new tag. It clones the device from Pantahub using the reference in the recipes folder and posts it to the stable device. Using that reference, it builds the flashable image and uploads it to AWS.
- Validate: Checks the device status is DONE and fails otherwise. It tests either latest device or stable device depending on how was the job triggered, so it will check the most recently updated one.
- Deploy: Only triggered when pushing a new tag. Uploads the tag metadata collected from the previous stages to the stable.json file at AWS.
Tools
The main script used in Part 1 to set up the device-ci project was the mkdevice-ci script. As of today, it supports two commands: init and install.
With the init command, you generate the default update-ci.yml configuration file. This file contains the list of device pairs.
The install command takes the information from update-ci.yml and automatically generates a hidden file named .gitlab-ci.yml. This one is important because it’s what GitLab CI takes to start the jobs. This is the file that you have to edit if you want to perform an override on the template, as we are going to see in the next post.
What’s next?
The original idea was to show how to override the pipeline to make is customizable, but we are going to do that in Part 4 since this post is getting a bit long.
In the not planned next part of the series, we are going to use the knowledge gained in this post to show how to customize our template to fit your needs.