Sweet CI – Homemade Jenkins server using the Raspberry Pi 3 (Raspbian)

In my previous posts I’ve gone into detail about writing Spring Boot apps, as well as how to dockerise them, adding a dynamic frontend to them with Thymeleaf etc. Now what I’d like to do is move away from the coding a little bit and focus on how we can implement a simple CI (Continuous Integration) environment to automate the compilation and testing of our code. We’ll tackle this using a popular (and free) CI tool called Jenkins, and a Raspberry Pi 3 (RPi3).

What is Continuous Integration?

Continuous Integrations (CI) is a principle in software engineering that ensures all code is regularly tested, this means that any changes made to a code base are not only tested independently, but all tests in the entire codebase are run as well, to ensure that any new changes integrate well with the existing functionality. Basically you’re continuously checking that your entire codebase is still happy after a change is made.

As you can imaging, running CI manually all the time could become laborious and tedious. Enter, Jenkins.

What is Jenkins?

Jenkins is a fantastic, free, piece of software that you can deploy incredibly easily to manage your CI requirements. This automates the process of running CI by being easily configurable to monitor a branch on GitHub for any new changes and automatically testing the entire changed codebase. This is exactly what we will do in this post.

What are we going to do?

  1. Configure Raspbian to access the Internet
  2. Update Java
  3. Install Jenkins
  4. Configure Jenkins
  5. Create a build job to automatically check our code

Stuff you’ll need & Prerequisite tasks

An RPi3

Amazon (at the time of writing) are selling all the parts you need (RPi3 board, power supply, case and 16GB SD card) for £47 in one bundle, so I would recommend getting that to save a couple of quid.

N.B. I initially used the RPi3 in this post as it is the quickest and most powerful RPi going at the time of writing. Since then I have also successfully got Jenkins running on an old RPi1 B+ (one of the oldest Raspberry Pis going) using the same steps outlined in this post. It worked absolutely fine, it was just quite a bit slower!

Raspbian

Once you’ve received your fantastic new Raspberry Pi, you now need to flash the SD card with a copy of the official Raspbian OS. You can download Raspbian here (download the one with the desktop included rather than the ‘lite’ version to help with configuration). Extract the contents of the zip once downloaded and you will presented with a .iso file, this is the Raspbian OS ‘image’ which you will burn onto the SD card.

Etcher

Etcher is a great free tool for burning .iso (image) files onto SD cards and can be downloaded for use on either Windows, Mac or Linux machines, so download Etcher here ready to burn Raspbian onto your SD card!

Insert your SD card using the adapter provided into the SD card slot of your computer or USB card reader and follow Etcher’s straightforward user interface (see below) to flash your SD card with the Raspbian .iso you extracted a moment ago.

Screen Shot 2018-01-22 at 17.30.09.png

Now insert the SD card into the RPi, connect a monitor, keyboard, mouse and power cable to go through the straightforward setup of your RPi.

A simple application

In my example below I will continue to use the Spring Boot app I wrote back in an earlier post. I’ve added one very basic unit test so you can see how the CI works (you can make the test fail, add more etc.), as well as adding a maven wrapper file (mvnw) and a maven directory (jar and config file inside).

The mvnw file and maven directory are just there to simplify our use of Maven. It means that we will not need to install Maven on the server (Jenkins) in order to see CI in action, the codebase will instead completely take care of the use of Maven in building and testing our project.

Feel free to use a project of your own creation, as long as it is built using Maven it will work fine. Just make sure to copy over the mvnw file and maven directory from my project to follow along completely with the below.

My example project can be found here on GitHub, please feel free to fork it, upload it to your own GitHub etc as long as it is on GitHub and you can push changes up to it, that’s all you need.

Let’s get to it!

Once you’ve got your RPi set up and an example project ready in your own GitHub repository, you’re good to go!

1. Configure Raspbian to access the Internet

First things’ first, let’s get the Pi connected to the internet via WiFi. Boot up the Pi if you haven’t already by attaching the power cable, once the desktop appears you should see the network symbol in the top right (icon with arrows pointing up and down). Choose your WiFi network and enter your WiFi password, your Pi should now be connected to the internet, you can test this by opening the Chromium web browser and entering a URL/searching for something. The network icon should now look similar to the one shown in the image below.

Screen Shot 2018-01-22 at 18.41.30.png

2. Update Java

This might seem an odd step in the context of getting Jenkins to work on your Pi, however for reasons unbeknown to me, the stock release of Raspbian comes with a slightly older version of Java (and accompanying certificates) than Jenkins will tolerate so we must update this to make sure Jenkins will work.

To do this, first you will need to manually download Java 8 from the Oracle website to your Pi. Using Chromium on your Pi, go to Google and search for ‘Oracle Java 8’, you should be presented with some results that will navigate to a webpage similar to this one.

The first grey boxed area for the Java SE Development Kit will look similar to the screenshot below. Click the button next to Accept License Agreement‘ and then click the download link for the Linux ARM 32 Hard Float ABI version, as this is the only version that will work on your Pi.

Screen Shot 2018-01-22 at 19.39.04.png

Go find the file in the Downloads folder on your Pi and double-click it to unpack it.

N.B. Some of you out there might be asking why I didn’t recommend downloadign the  file to the Pi using ‘wget <url>’ and then just use some command like ‘tar xvf <filename.tar.gz>’ to unpack it. The simple reason is, I battled that for about an hour and I couldn’t get it to unpack the file without error, something weird was going on with wget there. Anyways if anyone can get it working, please comment on this post and I will amend this section!

Now that we have the unpacked file, enter the below commands one-by-one in a terminal window to complete the Java upgrade.

  • General package update
    • sudo apt-get update
  • Navigate into same directory as extracted file
    • cd /Downloads
  • Move the extracted file to a root directory (/usr/local)
    • sudo mv jdk1.8.0_<version_number> /usr/local
  • Update system properties on version of Java to use
    • sudo update-alternatives –install “/usr/bin/java” “java” “/usr/local/jdk1.8.0_161/bin/java” 1000

    • Update system properties on version of Java compileer to use

      • sudo update-alternatives –install “/usr/bin/javac” “javac” “/usr/local/jdk1.8.0_161/bin/javac” 1000
  • Check the version has now been updated to the version number you installed
    • java -version

3. Install Jenkins

The following steps have been taken from a fantastic post on duskborn.com, which details how to get Jenkins onto an older version of the Raspberry Pi. The steps are the same though so I’ve summarised them below.

  • Get the required key to install Jenkins from apt-get
  • Add the key so as apt-get is happy to use the Jenkins binary
  • Update Raspbian packages in apt-get
    • sudo apt-get update
  • Install the new Jenkins package
    • sudo apt-get install jenkins -y
  • Check that Jenkins is running (should see active (exited) in the output of this command
    • systemctl status jenkins.service

If you now open Chromium on your Pi and navigate to http://localhost:8080, you should see something similar to the below screen prompting you to unlock Jenkins.

jenkins1.png

In a terminal window on the Pi, enter the following command to view your admin password (actual command will differ according to the red line in the image below, compare with your actual output).

cat /var/lib/jenkins/secrets/initialAdminPassword

The above command will return the contents of the file in a single line below the command itself, copy and paste this long string of characters and/or numbers into the Administrator password field.

Now you’ll be given the option to install default plugins (accept this option, it will take some time though) and then you’ll be asked to create your first admin user, do this as well.

4. Configure Jenkins

Now you should be up and running with Jenkins, from now on you’ll be working in the web browser (Chromium) on your Pi. The goal of this section and the next is to get your Jenkins installation to look similar to mine shown below. You’ll notice that you currently have a different colour/formatting scheme and no build jobs listed. That’s ok we’ll sort out the formatting now and the build jobs in the next section!

Screen Shot 2018-01-22 at 20.20.35.png

In all honesty this section is entirely optional, however it makes for a much more user-friendly interface compared to Jenkins’ default interface.

Screen Shot 2018-01-22 at 20.25.48.png

Click Manage Jenkins on the left hand side menu, then click Manage Plugins on the next screen (see screenshot on the right).

On the next screen click the Available tab and enter the following into the search bar ‘Simple theme plugin’.

Tick the box next to the appropriate plugin in the response and click the Download now and install after restart option a the bottom of the screen. Tick the box on the next screen to restart Jenkins automatically.

Now that you have the plugin installed, navigate back to the Manage Jenkins screen and select Configure System (the first option shown in the screenshot above). Scroll down to the Theme section and enter the required CSS URL in the format https://cdn.rawgit.com/afonsof/jenkins-material-theme/gh-pages/dist/material-<colour_name>.css as shown in the screenshot below. The available colours can be found on the official plugin site here.

Screen Shot 2018-01-22 at 20.35.16.png

5. Create a build job to automatically check our code

Now that we have Jenkins looking the way we want, we can add a build job to check out some code from GitHub and make sure it’s all working as expected. To start this off click the New Item option from the left hand menu on the main Jenkins page (http://localhost:8080) .

Now enter a name for your new build job and click on the Freestyle project option, then click OK.

N.B. I’m using a ‘Freestyle project’ in this example, take some time to play around with Jenkins and find the best types of build jobs for given scenarios.

Screen Shot 2018-01-22 at 20.41.38.png

You’ll now be redirected to a page where you can specify the configuration of you build job. Scroll down to the Source Code Management section and click Git then enter the url of the repository which you would like to run CI on. In my example below, I’m checking my simple spring boot application created in an earlier blog post.

Screen Shot 2018-01-22 at 20.45.15.png

We can now scroll down to the Build Triggers section. This is where we tell Jenkins how ofter to check the repo we specified above for new commits (any new code changes). This will then trigger some actions which we will define in a little while. For now tick the Poll SCM option to make Jenkins poll our repo for changes, and in the Schedule box just enter 5 stars in a row, separated by a space. This tells Jenkins to check GitHub every single minute. Feel free to play around with this, the “?” icon to the right will give you some guidance on how to manipulate these values.

Screen Shot 2018-01-22 at 20.48.22.png

If we now go down to the Build Environment section, we just need to tick a box to tell Jenkins to cleanup all old files in its workspace before checking the codebase. This is good for two main reasons. Firstly, it prevents a potential bloating of files in the workspace. Secondly (and most importantly), it prevents any risk of files from one build contaminating/affecting the result of any other builds as the slate is wiped clean every time.

Screen Shot 2018-01-22 at 20.52.29.png

Finally, we need to add something for Jenkins to do once it has been triggered, we add this into the Build section. Click the Add Build Step option to see a dropdown of options and select Execute shell now enter the command shown in the below screenshot. Here we are using our Maven wrapper command to clean, compile and then run all (unit) tests found in our project. Make sure you have the required mvnw file and maven directory in your project as I mentioned earlier otherwise this will fail. You can alternatively install Maven on your Jenkins machine, then replace ./mvnw below, simply with mvn and you’re all good, but this post doesn’t cover that.

Screen Shot 2018-01-22 at 20.55.53.png

That’s it, click Apply and then Save to confirm your configuration. You should now see your brand new build job in Jenkins as shown below. Click the Build Now button to kick it off and make sure it works.

Screen Shot 2018-01-22 at 21.02.44.png

Your build job is now set up and ready to automatically perform integration testing on your repo, try making a change on the master branch of the project you used and pushing it up, this should trigger a build within a minute automatically. Also, try playing around with the unit tests in the project, break the tests (make them fail) and push the change to master on GitHub, you should notice the job shows your code failed integration tests!

That’s all for now

Now you have Jenkins running on your Raspberry Pi watching your every move you can now rest assured that Jenkins will flag up and potential issues with your changes during development!

As an extension of this post I will come back and show you how to use the fantastic node build monitor to provide a really nice and simple dashboard to highlight build jobs to prevent you from needing to log in to Jenkins all the time.

As always, if you have any questions, suggestions or corrections please feel free to comment below, any and all feedback is gratefully received! 🙂

Thanks for reading,

Mark

Advertisements

Published by

mvee1987

I joined WordPress during my final year studies in Computer Science at Brunel University, back in 2015. When I realised it took me 100 Google searches to just about figure out something technical, I figured that it'd probably be best that I write it down! The hope is that by doing this someone else can benefit from instead of potentially doing another 100 searches! Nowadays my general motto is, if it took me more than 2 hours to figure out, I'll see if I can write something that can be understood in 30 minutes or less (by anyone with an interest in programming) to solve the same issue. Please feel free to comment or message me if you have any questions, queries or general feedback to help me provide the clearest and most accessible information possible. Thanks, Mark

2 thoughts on “Sweet CI – Homemade Jenkins server using the Raspberry Pi 3 (Raspbian)”

  1. I am so glad I found your article, It really helped me… I also have a few things, that would be nice to point out in it, here they are:

    I was getting:
    sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
    at sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:146)
    at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:131)
    at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:280)
    at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:382)

    And it was all because of version of Java. I firstly used (not like you mentioned in article) jdk1.8.0_151, because it was in official raspbian repositories. I had to install jdk1.8.0_161 and everything is ok now. If you also already installed older version then you can remove it by command found here https://askubuntu.com/questions/84483/how-to-completely-uninstall-java and reboot.

    The “Can’t download JDK by wget thing” is because of the licence of JDK. You have to agree by checking the radio button on Oracle download page, otherwise you will download unextractable binary nonsense file. There is a little workaround by putting cookie into the wget request and it is described here https://stackoverflow.com/questions/10268583/downloading-java-jdk-on-linux-via-wget-is-shown-license-page-instead then everything works as it should.

    Also be aware of the WordPress quotes representation, which neet to be replaces by ” while copy-pasting codes

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s