Getting Started with Spring

Note: I’ve now (Sep 2017) added an extra step ‘3. b)’ to cover running Spring apps using IntelliJ Community Edition, so it is now possible to follow along without needing IntelliJ Ultimate Edition 🙂

In the last couple of weeks I’ve moved into a new team at work – the people are great, the office is amazing, life is wonderful. However, joining a new team means learning how they work and learning the technology they use, in this case I’m facing down the Spring Framework.

In the past I’ve worked with the Play! Framework to create and deploy a webapp, choosing it more or less entirely due to its simplicity and close knit relationship with plain old Java. This isn’t to say that Spring isn’t closely knit with Java, just that learning a billion annotations and figuring out how Java Beans worked seemed quite a daunting task at the time, as opposed to being able to to get straight into Java code in a fully-featured (perhaps a little bloated, in retrospect) framework. In essence, I was being lazy and doing my best to dodge having to learn a ton of new stuff before developing anything meaningful. Lesson learnt!

Anyways, karma has come right back around on me and here I am, now attempting to get up to speed with Spring as soon as my little brain can manage!

The purpose of this post is to share a brief ‘getting started’ guide. This isn’t here to explain all of the intricate steps in detail, this will come when I know these steps intimately myself, this guide is here to get anyone from a blank Maven project in IntelliJ to a very basic web application hosted on a local Tomcat server. Any code I’ve used is available on my GitHub repo for anyone to clone/download and play around with.

FYI – this guide is based on me using a Windows 10 PC, so some steps will be slightly different (read: ‘easier’, ‘less painful’ etc.) if using a Mac.

What is Spring?

As a one-liner – ‘Spring is a framework for developing applications or services that can be accessed/used through the web (via http)’. Basically, it lets you write applications (in Java) that can be deployed and then accessed by users through their web browser of choice (Chrome, Firefox, Safari etc.) by navigating to a URL e.g. http://marksamazingsite.com.

The main benefit of Spring is that it is designed to be modular. You start with the core functionality (Spring Boot) and from there you can choose to extend that by installing additional Spring modules. For instance, if you wanted to hook your application up to a database, you can use the Spring Data module, or if you want to account for security concerns you could apply the Spring Security module, or what if you wanted to develop your app towards Android users, there’s the Spring For Android module! There are loads of different modules to fit most needs, these are all described on the Spring projects website.

The inherent modularity of Spring means that you don’t need to start with a monolithic project and then strip away what you don’t need, it means that you can start with a simple, understandable base and build up gradually according to your needs. The obvious downside to this if you are a beginner (like me), is that implementing additional modules may require a bit of a learning curve, but I’ll find out as I use it more in future posts!

What is Tomcat and how do we use it with Spring?

Tomcat is a ‘web server’ technology, this means that installing and running it on a machine (be it your laptop, or a server in the cloud etc.) will enable you to ‘serve’ content or applications to other machines (clients) in your network, or over the internet if you host your Tomcat machine publicly. If you’re new to the idea of client-server architecture, have a look at this quick (~2 min) video to get an idea of how a server (machine with Tomcat installed, for example) can provide content, data or services to clients (other machines out there able to connect to the server) through the use of requests and responses.

Just to give a quick example of client-server interaction (request-response); by typing http://www.google.co.uk into your browser you are issuing a http request to the UK Google URL, Google will issue a response in the form of a webpage (HTML/CSS/JavaScript content) hosted from its server(s). If you typed in http://google.co.uk/somejunk instead, you’ll notice that it causes a 404 error, this is still a valid response from Google, all it’s saying is that it doesn’t understand your request (it can understand and provide content for google.co.uk, but does not have anything it can serve you for google.co.uk/somejunk aside from an error code). But what if you mistyped the url of Google? Say you typed in http://gogogogogogogg.co.uk instead, you’ll notice that nothing is served back to you, not even a custom error message like the 404 we saw a minute ago, from Google or anyone, because there isn’t (at the time of writing) a web server in the world that is setup to handle requests and provide responses for that URL. Simple!

In our case, we are going to install Tomcat on our PC and rather than sending requests to our Tomcat server externally, we can use a special ‘localhost’ URL which will allow us to write a Spring web application, host it (serve it from) our local Tomcat server installed on our machine, then access it using a URL of http://localhost:8080. This just means “send a http request to a local server on port 8080 (Tomcat’s default port)”.

What are we going to do?

  1. Create a new Maven project
  2. Download Apache Tomcat 8 to deploy our application to later on
  3. Set up a run configuration in IntelliJ to run our project on a local Tomcat server
  4. Write a basic SpringMVC (Spring 4) application
  5. Deploy and run your webapp!

Stuff you’ll need

IntelliJ Ultimate or Community Edition

If you haven’t already got hold of a free copy of IntelliJ Community Edition, I would highly recommend it. For general coding, in my opinion, it is THE BEST integrated development environment (IDE) for Java, Scala or pretty much anything else it supports!

In this guide we can use either the Ultimate Edition or the Community (free) Edition, if you’re a student you’ll likely be able to get the Ultimate edition for free using your institution’s details here. Step 3 (Setting up Run Configurations) has now been split into part a, and b, to set up a run configuration in either Ultimate or Community Editions’ respectfully. If you’re using Ultimate Edition you can use either of the two methods, however Community Edition will only allow you to use the latter of the two steps.

Apache Tomcat 8

It’s all good and well writing a Spring application, but we need some way of ‘hosting’ it from our machine so we that it can be accessed using a web browser (Chrome, Firefox, Safari etc.) to see it working as a live application! We’ll install and set it up for sue with IntelliJ after we run it manually to test it. We’ll set this up a step in the guide so that you can see the process of going from nothing at all on your machine, to deploying a Spring webapp to your very own local Tomcat server as a matter of easy steps.

Java 1.8

In this guide we’ll be using Tomcat 8 which relies on Java 7 or upwards being installed on your machine. I’d recommend using Java 8 straight away anyway though just to stay a little bit more up to date and allow you to take advantage of features such as Java Streams later on in development if you take your Spring project further.

Head to the Oracle website to download Java for your machine.

Let’s get to it!

A lot of the steps in this guide have been taken from a fantastic YouTube video by LynAs Sazzad, have a look at his video to get up and running quickly! As he moves quite fast through the steps in his video (a lot of pausing and going back was required!) and as he already had Tomcat set up to run his application, I’ve decided to write out some of the required steps but for the most part I’ll point you at his video to set up the code you need.

1. Create a new Maven project

Open IntelliJ and choose File -> New -> Project… then select the option for Maven from the list on the left hand side (it should be about halfway down).

Leave the Create from archetype option un-ticked and click next.

Give it a GroupId e.g. com.myTestProject. This option isn’t too important, it’s mainly important only for enterprise-level project where you’d need to define the domain or team your app will fall under such as com.marksCompany.financeWebApp.

ArtifactID is the name you want to assign to your app type in something like springMVCTest and click next. In the following screen the Project Name will likely autopopulate with the ArtifactID you gave your project so just check it is set and that you’re happy with the location of where your project will be stored and then click Finish.

2. Download Apache Tomcat 8 to deploy our application to later on

On my hunt to find a nice easy way of understanding how to install Tomcat on Windows 10, I found this awesome guide. It is a lengthy read but if you want to get to grips with webapps and understanding Tomcat, this covers everything you’ll likely need to know, conveniently enough it’s also got a nice section of downloading and running Tomcat. Below I’ve summarised the steps to downloading Tomcat and testing it by running it on your machine manually.

To download Tomcat for Windows, the easiest way I found was to download a zip file from Tomcat’s official website. At the time of writing, under the 8.5.14 heading there is a section called ‘Binary Distributions’ and in there is an option to download the ‘Core’.zip file (see red box below).

downlaodingtomcat.JPG

When it had finished downloading, I moved the downloaded .zip file into a new folder called H://Tomcat/Tomcat8/ on the hard drive of my computer and unzipped it there (right click .zip file -> Extract Here). This is just to give it a permanent home on my machine so any of my run configurations that I might put together later on have a definitive location in which to find and use Tomcat from.

As mentioned at the start. we need at least Java 1.7 (Java 7) installed on the machine running Tomcat 8 in order for it to work. Additionally we need to set up an environment variable called JAVA_HOME to point anything on our machine to its location. Hopefully, you’ve already downloaded Java through the Oracle website as explained at the beginning, if so follow the very last few instructions under the heading ‘to install the JDK Software and set JAVA_HOME on a Windows System’ on this site.

Now we are able to test that our downloaded version of Tomcat works, to do this click into the newly unzipped Tomcat file and go into the /bin directory. In here hold down shift and right click a blank are of the window to view the option to ‘Open command window here’, click this to enter command prompt. The only thing you need to do now is type ‘startup’ (without quotes) and press enter, this should open up another little command window, the whole thing should look similar to the output shown below.

Note: the equivalent command in Linux/Mac is ‘./catalinasa.sh run’

manualtomcatrun.JPG

Great, so in this new little window it’s telling you that Tomcat is running and that it’s listening on port 8080, so how do we access this? As we mentioned earlier on, we need to send it a request, as Tomcat is a web server, this therefore needs to be via http, so let’s open a web browser such as Chrome and navigate to http://localhost:8080. The result should look similar to the screenshot shown below.

tomcattest.JPG

This is Tomcat’s default page to serve to any requests, all this means is that we’ve not told Tomcat to serve anything other than this in response to any requests it receives, simple!

I don’t want to go too far into Tomcat stuff in this guide but I do want to quickly show you how to host your own content from this server manually before moving on. To do this I have created a very simple HTML file called index.html and stored this within a folder called marksApp in Tomcat’s webapps folder (see below).

simplehtmlfile.JPG

You should now be able to go to localhost:8080/marksApp (in my example) and take in the awesome beauty of my website.

marskapp.JPGNow that we’re happy that we have a working Tomcat installation, we just need to set up IntelliJ to use it and write the webapp to be served from it!

3. a) Set up a Tomcat run configuration in IntelliJ Ultimate Edition

In this step we’re simply telling IntelliJ how we would like it to run a program, in this case we’ll be writing a bunch of code that we’d like to run on a Tomcat server locally, so we’ll tell it just that by defining a ‘run configuration’ – a way of setting a particular way to run something.

To do this, from the top menu bar in IntelliJ choose Run -> Edit Configurations. In the

runconf1.png

Click the little green ‘+’ in the top left of the window that opens (see screenshot on the right).

You might get an option to choose Tomcat Server as shown in the image to the right, if not, click the option at the bottom titled ‘x items more (irrelevant)…’ and scroll down to find the Tomcat Server option. Once you’v found this click it to choose the Local or Remote options, we want to serve our app from a local Tomcat server so click ‘Local’ as shown below.runconf2.png

A new run configuration has now been created, you just needintellijtomcatlocalserversetup.JPG to edit it to your needs.

Firstly, set the Name field to something meaningful like SpringTomcat, now click the ‘Configure…’ button in the Application server section to bring up a small window (see screenshot to the right).

Clicking the ‘…’ therunconf2.png the right of the Tomcat Home box will allow you to tell IntelliJ where your Tomcat installation lives. Simply put, navigate to the unzipped version of Tomcat that you downloaded earlier, and click OK as shown on the left. You should now be presented with the original small window without the error icons and with some details of your Tomcat installation, click OK.

explodedstuff.PNG

The final thing we need to do is address the warning at the bottom of the run configuration, click the Fix button and choose the option suffixed with ‘exploded’ then click Apply and OK to finalise the run configuration.

3. b) Set up a Maven run configuration in IntelliJ Community Edition

In the more likely event that you’re following along using IntelliJ Community Edition, here is how you can easily set up a Maven run configuration to run your Spring applications. In this section I will be using the Tomcat7 plugin for Maven (rather than Tomcat8 as used previously in this guide). For the purposes of this guide the version of Tomcat being used isn’t important though.

Add the following plugin to your pom.xml.

Screen Shot 2017-09-16 at 16.23.24.png

From the Run menu, select ‘Edit Configurations…’

step1.png

The Run Configuration window will now pop up – click the grey ‘+’ icon in the top left of this window and choose the Maven option.

step2.png

Enter a name for your new run configuration and add ‘tomcat7:run’ in the Command Line box. Now click Apply and then OK to exit the window. This may require you to download and install Tomcat

step3.png

All you need to do now is choose your new run configuration from the Run menu!

step4.png

If your application fails to start, make sure that no other applications are running on the same port as the current application you’re trying to run. Most tutorials use the default port of 8080 so if this is the case, just stop any other apps you might be working on and try again. Other common issues relate to either the contents of your WEB-INF folder, or the dependencies in your pom.xml, so be sure to check them and Google any specific errors that come up.

4. Write a basic SpringMVC application

For this section I will point you directly at LynAs Sazzad‘s fantastic video for instructions on setting up all of the Spring code and dependencies in the pom.xml file. If you get stuck at any point or are having difficulty getting it working, my code is up on GitHub so feel free to use it for comparison or as a starting point for your own app!

The reason I’ve followed this tutorial specifically is that it uses Spring version 4, the latest release. Older versions of Spring use different annotations and also rely on .xml files for configuration of Beans etc. So this is all in aid of using the most up-to-date (at time of writing) version of Spring which uses Java instead of xml to work with Beans and in my opinion uses better structured annotations.

5. Deploy and run your webapp!

Once you have completed all of the above steps you can now use your run configuration to deploy your Spring webapp to your local Tomcat server and access it through the browser!runapp.PNG

At the top of the window in IntelliJ you’ll notice a Run Configuration dropdown with a green ‘play/run’ button next to it, make sure your new run configuration is selected and click the green run button.

The run console at the bottom of the screen will start churning out all sorts of messages, a lot of them will be in red text but are only ‘info’ messages rather than errors! When it’s finished deploying your app to the local Tomcat server it will show you the ‘deployed successfully’ message (see white lines of text in screenshot below) and will open up your application automatically in a web browser window, if it doesn’t just navigate to http://localhost:8080/ in your browser to view it.
runsuccess.PNG

runningapp.PNG

That’s all for now

That’s it! You’ve successfully set up a local Tomcat server, written a Spring application and deployed it to your server to be accessible by clients on your network (localhost)! In addition to this guide, I intend to write a sequel showing you how to deploy your application to a remote Tomcat server so it can be accessed by anyone on the internet. But for now that’s pretty good going 🙂

Also, as I start to learn more than just the absolute basics I’ll figure out what I can write about coherently and add that up on there as well!

If you have any questions or comments please as always feel free to message me or put them in the comments section below 🙂

Thanks for reading,

Mark.

Advertisements

Programming the Parrot AR 2.0 Drone – Beginners Guide 2

In the previous guide we used NodeJS and some basic JavaScript to control the drone, however this is all a bit static – what if we want to control the drone in real-time during flight, rather than having to pre-define a route for it to fly? Or what if we want to view a live feed from the drone? This is where WebFlight comes in. WebFlight is a superb open-source project created by Laurent Eschenauer that can be downloaded/cloned for free from Laurent’s GitHub repository.

What are we going to do?

  1. Go over thenewboston tutorial on installing Git bash
  2. Download (Clone) WebFlight and install required components
  3. Configure WebFlight
  4. Download and install a program to render the feed from the drone’s camera as a live stream in our web browser (FFMPEG)
  5. Start flying our drone in real time using our laptop/PC keyboard!

Stuff you’ll need

Same as last time (mostly)

In this guide you’ll need exactly the same bits and pieces as in the previous guide  (Win10 laptop/PC with Wi-Fi, drone, Notepad++). You’ll also need to have completed the previous guide to ensure you have NodeJS installed as well.

Aside from that we also need an application called Git installed. We will need to use ‘Git Bash’ in order to get all the bits and pieces we need set up.

Git

Git is a piece of software that falls under the bracket of Version Control Software (VCS). Git’s job is to make sure files are up to date as well as keeping a history of the different versions of files up to the current point, just in case you need to revert to an older version.

Going into detail about Git and its uses is a little out of the scope of this guide, but you absolutely need Git installed on your laptop/PC to get WebFlight installed correctly. Bucky Roberts over at thenewboston.com has put together a great series of easy to understand video tutorials on getting up and running with Git so I would recommend you go through at least the first tutorial on installing Git before continuing. The tutorials after the first one are great if you have a GitHub account and want to understand a bit more about Git and its uses. For the sake of this guide, we only need Git to do some very basic stuff so it’s not totally necessary to know loads about it.

Also, have a look around the tutorials on thenewboston.com, there are absolutely loads of really well put together and clearly explained tutorials on programming. It’s an absolute goldmine of programming knowledge!

Let’s get to it!

1. Go over thenewboston.com’s tutorial on installing Git Bash

Make sure you’re familiar with Git (Git Bash) before going any further; this will ensure that you can get everything set up ready to use WebFlight! Head over to thenewboston’s Git tutorials here .

2. Download (Clone) WebFlight and install required components

Scrolling down a bit on the WebFlight GitHub page you’ll see the below instructions for installation. The yellow highlighted line has been added by me as I found that bower wasn’t recognised by the computer without this (bower install will fail, unknown command).

If you still have your ‘Drone’ folder from the previous guide and you have Git Bash installed, open the Drone folder and right click an empty area in the folder, now choose ‘Git Bash Here’ to open the Git Bash command window. Now run the below commands in the Git Bash window one at a time.

installCommands.PNG

What we are doing by entering these commands is explained below:

  1. “git clone https://github.com/xyz…”  “Grab me my very own copy of some software stored on GitHub”
  2. “cd ardrone-webflight” – change from the current directory into the (newly cloned) ardrone-webflight directory (cd = change directory, cd )
  3. “npm install” – get the Node Package Manager to auto-detect any dependencies within the current directory and install them
  4. “npm install bower -g” – , Bower is another package manager (handles any required packages that NPM doesn’t look after) so needs to be installed seperately and given global (-g) access so you can run it from anywhere in the system
  5. “bower install” – now Bower is installed we can get it to auto-detect any dependencies within the current directory (ardrone-webflight) and install them.

Above I mentioned ‘dependencies’ a couple of times. As I described in the previous guide dependecies can be though of as tools to complete a job. In the last guide we used the command ‘require(‘ar-drone’); this is saying that in order to complete some or all of the commands in the program, it requires (or depends on) the ar-drone library being installed. As programs get bigger and bigger, they may likely use many dependencies, hence the need for dependency managers and the ability for them to automatically detect any required dependencies and get them installed ready for you. Imagine if you had a program with 50 dependencies, it would take a loooooong time to type ‘npm install abc’ 50 times manually!

The three screenshots below demonstrate the process of installing everything, and installing bower using NPM, using the –g tag to set a global path variable to it as well (this just means that you can run the command ‘bower’ anywhere and it’ll be recognised, rather than having to run it from the directory it lives in).

Clone the WebFlight repo from GitHub, then using npm install:

cloneinstallwebflight.PNG

At first your computer will not understand what bower ‘is’

bowerinstall.PNG

Use NPMto install bower and use the -g tag to set it to the computers’ global path, bower is now recognised and you can run bower install

installbower.PNG

Random error I ran into

When testing the installation of WebFlight a second time for this guide, I ran into an error when running the bower install command (last line). Unfortunately I closed the window before I thought about screenshotting it so I apologise for that. However, part of it included “exit code #128 fatal” so if you see that at all, it’s the same error.

While I was googling the error and trying to figure out how to fix it, the gremlin in the system causing this issue magically disappeared and I was able to run the bower install command without any errors at all. I literally did nothing to my PC, nadda, it just started working all of a sudden!

3. Configure WebFlight

Now that everything is installed we need to make sure that Webflight is configured as we like it. To get a basic level of operation running we won’t need to do much so we’ll do the bare minimum for now to get up and running.

If you navigate to the ardrone-webflight folder that you recently cloned from GitHub, you’ll find a file called config.js.EXAMPLE – simply put, this is the configuration file that tells Webflight which plugins to use etc. For now, rename the file to config.js (remove the ‘.EXAMPLE’ bit at the end), this will make sure that this file is recognised as the correct file for Webflight to use.

Aside from config.js, there is one other file in the same directory that is very important, that being app.js. App.js is the main, central JavaScript program for controlling the drone. This file includes links to the config.js file (hence why renaming this a minute ago was important!) as well as bindings from the keyboard to the basic drone controls etc. There’s no need to edit this right now, that’s the basic configuration done!

If you now open a command prompt window in the ardrone-webflight directory and type ‘node app.js’ you will initialise the WebFlight application! At the moment you will see a couple of errors pop up under the ‘node app.js’ command to do with something called ‘ffmpeg’ not being installed, for now don’t worry about that, simply open up your web browser and type  localhost:3000 in the URL bar at the top – press enter and you should see the following.

stockimagewf.png

You’ll notice that your drone is (probably) not pointed at a nice view of clouds, this is a stock image. The reason it displays this stock image is due to FFMPEG not being installed, this is the program used to render any images and videos from the drone to a format our browser can understand and output, so until we get that set up we’re stuck staring at the stock image!

Saying that though, it’s not all bad! We’ve got some nice HUD (Heads Up Display) elements including a compass along the bottom of the screen and some subtle instruments in the centre of the screen so we must be doing something right!

4. Download and configure FFMPEG

This section is a bit fiddly so take your time and go through it carefully!

In the previous guide I took you through how to find out if your PC/laptop is 32 or 64-bit, the same rules apply to downloading FFMPEG. If you get stuck, go back to section 2 of the previous guide for a reminder. You can download FFMPEG here.

When the download has finished, navigate to your Downloads folder and copy your new FFMPEG ZIP file (notice it has ‘.zip’ after the file name) from the Downloads folder, into the ardrone-webflight folder, inside the Drone folder (Drone\ardrone-webflight\). As the folder is a zip file, we need to ‘extract’ it to use the contents of it. You can do this by right-clicking the zip file and then selecting ‘extract here’. Rename the extracted folder to ‘ffmpeg’ for easier referencing later on.

I found FFMPEG to be a bit of a pain to install overall, however after Googling around a bit I found this guide which helped me eventually. If you run into issues installing FFMPEG, I would recommend having a look at the guide linked above, or Googling any specific errors you run into.

Within the ffmpeg folder you’ll find another folder called ‘bin’, enter this folder and copy the location of this. To do this, click on the address bar (the bit that says ‘Drone -> ardrone-webflight > ffmpeg > bin’ in the image below) and copy the whole path that shows up. See the two images below, first image is before the address bar is clicked on my computer, second image shows it when it is clicked. In my example below, I copied all of the path highlighted in blue, you need to copy the whole path shown on yours as well.

ffmpeginstall.png

getpath.png

Now we need to edit the environment variables for your computer so you can use ffmpeg from anywhere. It would make sense that you could use the –g tag like we did earlier for bower, but I haven’t figured out why that doesn’t work in this case so I’m just putting it down to FFMPEG being awkward and doing it the good old fashioned wcortana.pngay.

In Windows 10 all you need to do is start typing ‘env’ in the ‘Ask me anything’ search bar in the bottom left of your screen (Cortana I think it’s called?), you should see an option to edit the system environment variables as shown in the screenshot to the right, click this.

The next bit is a little more fiddly so I’ve included three windows in one image to try to explain the process as clearly as possible!

You should now be presented with the top left window shown in the screenshot below, click the Environment Variables… button to be taken to open the top right window from the screenshot below. Click on the Path item (PATH, path – whichever way it’s shown on your computer), then click on the Edit button to reach the window shown at the bottom of the screenshot below. Click New and then paste in the path you copied earlier from the ffmpeg/bin folder.

envvar.png

Now if we close any command prompt windows and reopen them, typing ‘ffmpeg –version’ you should see an output similar to below, if so, FFMPEG is installed and ready to be used with WebFlight!

ffmpeginstalled.png

If you now navigate back to your ardrone-webflight folder and run ‘node app.js’ in command line, you should notice that the FFMPEG errors have gone and if you open ‘localhost:3000’ in your browser you are receiving a feed live from your drone! Below is my feed while writing this guide, the drone is currently looking out of the window enjoying a view of perfect English summer weather.

notstockimagewf.png

5. Controlling the drone using the keyboard

Seeing the live feed from your drone’s camera is all good and well but what about flying it? As long as your drone has at least 20% battery life (see top left of WebFlight display in your browser), you can press ‘T’ on your keyboard to take off and ‘L’ to land!

However you might notice that other commands seem to be missing. The up and down arrows seem to make drone go up and down, but what about forwards and backwards? By default the config.js file is set to use a different keyboard layout from QWERTY (UK/US layout) so we just need to change one line in there first.

Open the config.js file, you’ll notice on line 13 there is a keyboard variable set to ‘azerty’, simply change this to ‘qwerty’ to use WebFlight with a standard UK/US keyboard layout. Save and exit the file and restart WebFlight for the changes to take effect (close browser window/tab running WebFlight and use ctrl+c to stop it in the command prompt window – then re-run node app.js in command prompt and reopen localhost:3000 in your browser).

editconfig.png

Once you’ve reconfigured Webflight in the config.js file, start Webflight up again with ‘node app.js’ (make sure you’ve closed down all other windows running Webflight first). You’ll notice that you can run the following commands in the browser window:

  • W – Forward (pitch)
  • A – Left (roll)
  • S – Backward (pitch)
  • D – Right (roll)
  • Up – Increase height
  • Down – Decrease height
  • Left – Rotate left (yaw)
  • Right – Rotate right (yaw)
  • F – Flip (make sure you’ve got lots of room around the drone!)

That’s all for now

With any luck you now have a drone flying around according to your controls, as well as a feed from the drones’ camera in the web browser (localhost:3000)! Check out Laurent Eschenauer’s WebFlight repository on GitHub for more controls, plugins etc. As always, if you ran into any issues getting up and running with WebFlight please feel free to comment below and we’ll look into it together, alternatively head over to the WebFlight repository linked above and click the ‘issues’ tab to search any issues/fixes other users ran into!

In the next guide we’ll look into installing and some additional plugins for Webflight to extend its functionality even further!

Thanks for reading,

Mark

Programming the Parrot AR 2.0 Drone – Beginners Guide 1

Welcome to my first beginners guide on programming and flying the Parrot AR 2.0 drone! The guides I’ll be writing are based on my own experience of learning and working with the AR 2.0 drone. The aim is to write these in a way that I would’ve found useful and easily understandable when I was getting started.

Although there are other great resources out there such as this Instructables guide and this video, I wanted to write a guide for absolute beginners – explaining as much as possible as I go along. This way there is absolutely no expectation to have any prior understanding of programming or working with technology. I hope to assume as little knowledge as possible so please feel free to comment on this post if you have any questions or issues while following along.

What are we going to do?

In this guide we are going to achieve the following:

  1. Create a folder on the Desktop to work in
  2. Install an application that will allow us to send and receive information to and from our drone (NodeJS)
  3. Connect to the drone using Wi-Fi
  4. Use NodeJS to tell the drone to take off and land from our laptop/PC
  5. Write some simple JavaScript programs to save us typing lots of commands each time we want to fly the drone
  6. Add some more functionality into our JavaScript programs and understanding the use of these to control the movement of the drone

In writing this guide, I’ve used my experience of going through a fantastic guide from the Instructables website. This contains many of the commands that we’ll use, however I’ve tailored this guide to fill in any gaps in understanding that I came across, as well as explaining a little bit about the technologies we’ll need to use along the way.

Stuff you’ll need

Before we get into anything, these are the things you’ll need to get going with this guide.

1. Windows 10 PC/laptop with Wi-Fi

For these tutorials, I am using a Windows 10 PC with a USB Wi-Fi adapter to connect to the drone, if you don’t have Wi-Fi capability on your PC, something like this USB Wi-Fi adapter should solve that for you.

If you have a Windows 10 laptop, you should be all set (built-in Wi-Fi) 🙂

Windows: Other versions of Windows should be fine, I originally got this all working on a Windows 7 laptop so that should be fine, Windows 8 (and 8.1) are notoriously awkward operating systems however so there may be a need to Google some bits and pieces if you run into issues. Feel free to comment on this post with any issues as well though and we’ll try to work through them.

Linux/Mac: At the moment I’m looking into getting a MacBook of my own so one day I should be able to try this all out on Mac and write a guide on that if needed. Might as well wait to do Linux until that point as well!

2. A drone!

You’ll also need a Parrot AR 2.0 drone for this guide. You can find these on Amazon for under £200 brand new (at the time of writing), but check the used/new options from other sellers as I managed to get it even cheaper through Amazon Warehouse Deals, just with a slightly damaged box! Click this link to see the current cost of these drones on Amazon (UK).

3. Spare battery (optional, but recommended)

I would definitely recommend getting hold of an additional battery for the drone as the flight time on the standard battery (1000mAh Li-ion polymer) is around 10-12 minutes, give or take. I got hold of a 1500mAh battery to use while the other one is charging which worked out quite nicely. The 1500mAh I bought can be found here and will allow for about 18 minutes of flight time. This will cost around £40 which seems like a bit of a sting to the wallet but it with save you twiddling your thumbs for around an hour while you’re waiting for a battery to charge!

Third-party batteries: If you’re going to get a spare battery, I would absolutely recommend getting an official Parrot one rather than from another manufacturer. Although these batteries may well be a bit cheaper and probably absolutely fine for use, I remember reading some horror stories online when I originally started looking for a spare battery.

4. Notepad++

Aside from the above, I would also recommend downloading Notepad++ as we’ll be fiddling with some code in JavaScript. Notepad++ is a nice, basic text editor that understands code formatting so will highlight elements of code for us, which will really help anyone get to grips with basic coding.

Let’s get to it!

1. Create a folder to work from

Just before we get into installing NodeJS, the first thing I did was create a folder on the Desktop called ‘Drone’ which contained an empty folder called ‘nodejs’. When we install NodeJS below we’ll change the default install location from C:\Program Files\nodejs\ to C:\Users\Mark\Desktop\Drone\nodejs\ (just change ‘Mark’ to the user name on your PC!)

folderondesktop.png

2. Download and Install NodeJS

Now we’ve got a place for NodeJS and our other files to live, click this link to view the available NodeJS downloads.

The version you download will ultimately depend on the operating system you are running, in Windows you can check this in the following ways:

  • Windows 7 – right click the ‘Computer’ option in the start menu and click the ‘Properties’ option
  • Windows 8 and 10 – hold down the windows key (key in-between CTRL and ALT on the bottom left of keyboard) and press E to open the Windows File Explorer, then right click ‘This PC’ on the left hand side of the file explorer window and click ‘Properties’.

The resulting window should contain the ‘System’ section as shown below.

system64bit.png

According to the result given above for my PC, I’m running a 64-bit operating system (‘System type’ field) so I’ll need to download the 64-bit version of NodeJS (use the .msi installer) as shown by the red box in the next screenshot.

nodewebsite.png

Clicking the link in the red box above will start to download NodeJS, once the file has downloaded double-click it in your Downloads folder to start the installation process. Accept the license agreement and click Next on each screen if you’re happy with the defaults, apart from the screen that includes the below (install location of NodeJS).

installingnode.png

Click the ‘change’ button and in the new window that opens, type  ‘C:\Users\<yoursystemusername>\Desktop\Drone\nodejs\’ as the location to install NodeJS, click OK to confirm and close the new window and then click Next on the original window for all further steps of the installation.

We now have NodeJS installed on our machine! There’s just one last step to complete which will make sure that NodeJS knows what a drone is and how to work with it. To do this we need to use the windows command prompt within the nodejs folder – do this by holding down the left ‘shift’ key on your keyboard and right click an empty area inside the nodejs folder, then choose ‘Open command window here’ as shown below (ignore any additional files you can see in the background, you’ll create these soon!).

openingcmdfromlocation.png

At this point you should have a command window open much like the one below.

cmdwindowopen.png

Type in npm install ar-drone and press enter.

The result should look something like the output shown below, ignore the yellow warning (WARN) messages.

installingardrone.png

What this has done is tell the Node Package Manager that we need a new set of ‘tools’ (or ‘package’ of tools), the tool in question is one created by Felixge which enables the NodeJS application to work with our drone and perform commands such as take-off, land etc.

If you head to the npm website, you can search for available npm packages for a range of uses, below I’ve entered ‘ar-drone’ to see the packages available for our drone.

npmsite.png

3. Connect to the Drone

First things’ first, if you’re using a desktop PC like me – make sure you have a wireless network adapter (see recommended USB Wi-Fi device mentioned earlier in this guide), and be sure to test it works by connecting to your Wi-Fi at home and accessing the internet through your web browser.

Attach a charged battery to the drone and set it on the side for now, the lights should come on and the propellers should perform a quick test (small jolt of each propeller, don’t worry it won’t take off until you tell it!).

Assuming your Wi-Fi is working as expected on your PC, a new connection should appear if you click the Network icon in the system tray (bottom right of your screen), ‘ardrone2_062272’ in my case. Click on your ardrone2_xxxxxx network, and then click the ‘Connect’ button to establish the connection between your laptop/PC and the drone.

connectingbywifi.png

Note: I’m not under police surveillance (promise), my Wi-Fi network was named by my housemate!

When you’ve connected to the drones’ Wi-Fi it will say ‘No internet, open’ regarding the connection to your drone, don’t worry about this at all, it’s only showing this as your PC would normally expect a connection to a router/hub that provides internet access to your machine, our drone isn’t that clever unfortunately.

4. Running NodeJS and controlling the drone (takeoff and land)

Once you’re on the Wi-Fi of your drone, open the node program that we installed into ‘…Desktop\Drone\nodejs\’ earlier, double-click node.exe (green hexagon-shaped icon with a white ‘n’) to open the NodeJS terminal window.

NodeJS will be your means of sending commands to the drone and receiving data back (such as receiving a live stream from the drone to your PC, which we’ll get into in the next guide). Now that we have the NodeJS terminal window open, we can start typing in some commands to explicitly establish our connection between the NodeJS application and the drone to start sending it commands.

At this point I’m going to borrow some bits from the Instructables guide as they do an amazing job of going over the commands. To save you having to jump between different guides, I thought it would be better to copy and paste a couple of images from their guide (all credit for the below images of code to Instructables!)

In case you’d like to use the guide though, the Instructables guide can be found here.

If you’re still with me, we’re going to start entering some stuff into the NodeJS command window we recently opened (node.exe). The image below shows four lines that each needs to be entered into the NodeJS command window, one-by-one. Before we enter these commands, let’s go over each one briefly:

  • var arDrone = require(‘ar-drone’); this command lets NodeJS know that it needs to use the library of tools called ar-drone that we downloaded earlier to complete some or all of the commands that follow.
  • var client = arDrone.createClient(); we’re now getting a bit into networking, NodeJS is what’s called a ‘server-side’ language so its job is to provide stuff (commands, data, files etc.) to ‘clients’. In this case we’re telling Node that our drone is the client and we’re about to provide it some instructions, simple as that!
  • client.takeoff(); so we’ve told Node that our ‘client’ is the drone, this line is telling the drone to go ahead and run a command from the ar-drone library (see first command) called takeoff(). No prizes for guessing what happens when this command is executed!
  • client.land(); same idea as client.takeoff(); – it makes the drone land with the command called land() in the ar-drone library.

instructablescodescreenshot.png

So now that we’ve gone over each command hopefully you have a better idea of how everything we’ve done up to this point fits together. We need NodeJS to send data to and from our drone, but to do that we need the package of tools called ‘ar-drone’ to ensure NodeJS understands what a drone is and how to communicate with it.

Once you’re happy with the above ideas, go ahead and enter each command. I’ve written them out below so you can copy and paste them into the NodeJS window a line at a time if you like. Word of warning, when the drone takes off it will hover about a meter off the ground and stay roughly there until you tell it to land. Just in case you need to stop/land it quickly for whatever reason, I would make sure you copy the command to land the drone (highlight command with cursor, right click, copy) before pressing enter on the take-off command. This will make sure that you can paste the land command into the NodeJS command window (right click NodeJS window and choose Paste, ctrl+v doesn’t work!) and press enter to stop the drone whenever you need.

var arDrone = require(‘ar-drone’);

var client = arDrone.createClient();

client.takeoff();

client.land();

If you run into any errors, make sure each line ends with a semicolon (;) and you’ve run each line separately by pressing enter after each semicolon at the end of the line. If it presents you with an error about not knowing what ar-drone is, make sure you’ve used the npm install ar-drone command as shown earlier in the guide. When all is said and done your NodeJS window should look more or less exactly like mine below:

runningnodecommands.png

5. Running commands as a simple, repeatable JavaScript program instead

To write our JavaScript files I installed Notepad++ as this is a fantastic, free and simple text editor that comes equipped with code assistance tools which will help us more easily visualise the structure of our code. In case you skipped the link above, you can download Notepad++ here. You can just use Notepad it you like, but Notepad++ does make the code a little more understandable.

First thing’s first, let’s make a new Notepad++ file for our program. Open Notepad++ from wherever you installed it and enter the code shown below.

firstjsprogram.png

Now save the file as firstflight.js in the same folder as your node.exe application (e.g. …\Drone\nodejs\firstflight.js). To do this click File -> Save As…, enter firstflight.js as the ‘File name‘ and change the ‘Save as type’ option to JavaScript file (*.js) as shown below.

savingfirstjsprogram.png

If you don’t fancy writing this file for yourself, firstflight.js and some other JavaScript files (which we’ll be going over soon) can be found in my GitHub repository, feel free to download/clone this repository and mess around with the code! Alternatively, if you’re not familiar with working with GitHub/Git, typing out the files will achieve the same result.

GitHub: In case you’ve not bumped into these terms before, I’ll give you a quick explanation. GitHub is a website which software developers use to store and maintain code on. If they choose to store their code in public repositories then me or you can find it on the GitHub site, download it, play with it etc. The main major reason people use Git(Hub) as well is that it offers ‘Version Control’. This means that whenever updates are made to code,the previous versions of that code are still kept in case the creator needs to roll back to a previous version for any reason (say, the new code has a massive bug in it, but the old version didn’t).

Git: This is the application you install on your laptop or PC to manage the use of code repositories, enabling you to clone (make a copy of) someone’s code from the GitHub site (remote repository) onto your own machine (local repository) to mess around with. If you choose to set up a GitHub account, you too can create repositories of code that you can share with others and utilise Version Control.

To complete this guide, you don’t need to use GitHub at all if you don’t want to, nor do you need to install Git so don’t worry about these for now. In the next guide we will need to though so I’ll point you at some resources later on that will get you up to speed on what you need to know.

To run our program we’ll need to open the Windows Command Prompt (not the NodeJS command window) in the location of your firstflight.js file and enter the following command. Remember, to open the command window in a location in your file explorer you can hold left-shift and right click a blank area within the nodejs folder, then choose ‘Open command window here’.

node firstflight.js

Your drone should now take-off, and about 5 seconds after starting to take off it will start to land. So it’ll take-off, hover for a couple of seconds, then land.

Note: if your drone just sits on the ground with its lights flashing, it is likely you’ve got a NodeJS window open somewhere, make sure all NodeJS windows are closed! The above command calls node and tells it to run a particular program – node firstflight.js essentially means “computer, I want you to use a program called node [node.exe as we know it] to run a JavaScript program called firstflight.js, simple as that!

Another note: I asked you to open command prompt in the directory of the firstflight.js program you wrote, this isn’t mandatory, it just saves you having to type out the whole path to your program like node C:\Users\Mark\Desktop\Drone\nodejs\firstflight.js – that would be a pain to enter every time compared to node firstflight.js, much more typing involved each time!

6. Adding more commands in JavaScript

We now have a simple program that makes the drone take-off, float about a bit, then land. Altogether not too exciting, but we can add some more commands into the program to make it do a bit more for us.

When making this guide, I created a JavaScript file called secondflight.js to demonstrate a few more simple commands, these commands are described nicely in the Instructables guide so I’ve used their examples along with my own description of what each one does below.

Any reference to (speed) is a value between 0 (no speed at all) to 1 (max speed), you can replace the word ‘speed’ inside the brackets with any value from 0.1 to 1 to set the speed of the given action. (E.g. front(0.1) will make the drone go forward very slowly but front(0.9) will make it go forward at 90% of its maximum speed).

  • up(speed) – drone will gain altitude (go up)
  • down(speed) – drone reduce altitude (go down)
  • clockwise(speed) – drone spins clockwise at a given speed
  • counterClockwise(speed) – drone spins counter-clockwise
  • front(speed)/back(speed) – changes the pitch causing horizontal movement
  • left(speed)/right(speed) – changes the roll causing horizontal movement
  • stop() – keeps the drone hovering in place

The file secondflight.js in the screenshot below demonstrates us adding a bit more functionality to our drone program. In this file we’re telling the drone to take off, go forward a bit (front) at 20% (0.2) of maximum speed, then stop after 2 seconds/2000 milliseconds of going forward (lines 10 and 11, .after(2000, function() {this.stop()}. The same is then repeated but instead of going forward again (front), we’re telling it to go backwards (back), before stopping and then landing.

secondjsprogram.png

That was pretty straightforward, so just to concrete the idea in our minds about how these commands are built and work with the drone, thirdflight.js (below) demonstrates the concept of ‘yaw’ using the clockwise and counterClockwise commands.

thirdjsprogram.png

Yaw is a movement where the drone will stay at the same height/altitude, but rotate mid-air without going forward or backwards – like putting a pen flat on its side on a table and rotate it, this is yaw! The image below shows the concept of yaw in terms of an aeroplane, linear rotation on the perpendicular axis of the object.

pitchrollyawimage.gif

Above image source – http://machinedesign.com/site-files/machinedesign.com/files/uploads/2014/06/PRY.gif

Another example of using navigational controls in NodeJS is making the drone perform a simple route around an object. This is exactly what I did when I was learning about the use of NodeJS with the drone as shown in this video..

Feel free to copy the code I wrote to perform the square route in the video linked above, this too can be found in my GitHub repository.

That’s all for now

With any luck you now have a drone flying about breaking every lampshade in its vicinity! If you ran into any issues that you couldn’t figure out please comment below and we’ll have a go at fixing it together. Similarly, if you ran into issues using this guide and were able to overcome them, please comment as well so I can update this guide with your input!

In the next guide, I’ll be taking you through how to use an amazing open-source (free) application called WebFlight (all credit to Laurent Eschenauer) to enable us to control the drone in real time using the keyboard as well as allowing us to view a live feed from the drones’ camera through our web browser.

Thanks for reading,

Mark

Psychedelic Drone Images and Other Animals

Just like any real student I have once again taken an age (2 months) to get around to a deadline, or in this case, my previous comment dedicating me to providing more regular posts!

Apologies for the delay, the reason for this is two-fold:

  1. Believe it or not a lot of progress has been made, however as a lot of it wasn’t directly related to the drone, more related to some Java stuff I wanted to keep the bulk of that for later on.
  2. I am apparently terrible at writing blog posts regularly.

Provided you’ve forgiven me, I’ll move on, back into the world of drones as well as some work I’ve been doing away from drones on image comparison programming in Java, I’ll make a separate post (or posts) regarding this in detail, but will stick to the drone stuff and high-level image work for now.

Now, due to this all being a part of my Final Year Project (FYP, major dissertation-based project) at Brunel, I am very cautious of posting code up here until the submission deadline has passed and I have my degree in my hand. This is just to cover me from all angles as this is a BIG part of my degree! Boring bit aside, let’s get into it…

In the last post (Brave New World…) I’d managed to get the fantastic, open-source WebFlight application working on the laptop, therefore enabling the drone to fly around based on keyboard inputs while outputting a live feed to a browser window (Google Chrome in my case), however I needed to get my hands on those images for analysis.

Storing images to the laptop

This project is about collecting images from the drone and messing around with them on the laptop, so to do this I fiddled around with the WebFlight code and managed to get images saved to a local directory on my laptop during flight. Again, I’ll provide code and guidance to completing this in the coming months, for now you’ll have to take my word on it!

The images were stored to a local folder every few seconds while the drone was in flight. The result is shown below:

droneimages.png
Images from drone stored on laptop

Each image was stored in numbered order so the user can make sense of them as a series and easily choose which ones they would like to compare. In the example of the above, these are images of me wandering towards the drone, making a few faces and then moseying off.

Comparing images with Java

Moving on from this point, the next goal was to see about comparing the images for differences using Java. In the scope of this post (and because it’s quite code-heavy) I won’t go into detail about how this was achieved for now. What I have done however is provided a few output images based on two of the images in the folder shown in the above screenshot.

The first image-based output used a single level of tolerance (and therefore a single colour) to show the differences between two images, as shown below in yellow:

 

finaldifference.png
Original Output Style

Moving on from this, by creating multiple “levels” of change tolerance values, it was possible to assign various levels of change in a single output that gave a surprisingly pleasing and psychedelic (ish) output. Green indicates a very minor change, yellow slightly less minor, orange a bit more of a change and red picks out changes that are severe.

finaldifference_me.png
“Heat Map” (1970’s) Output Style
finaldifference_compleximage.png
Using another image pairing, one with a backpack in frame, then one without

The second of the two heat-map style outputs involved taking a picture from the drone, then getting up, moving the backpack out of frame and then sitting down again.Gripping stuff, I know. But the output looks fantastic. While taking all of the above images the drone was completely stationary (landed) on a solid surface, not in flight. The reason for this is it’s simply too unstable even hovering in an empty room to get two images from the same angle for comparison.

The point of all of this fancy stuff is to deploy it in the real world to see if damage to an object or structure has changed over a period of time (before and after spot the difference), and in doing so, doing it programatically, with greater precision than the human eye. The idea was to provide an output that enabled a user to very quickly and accurately identify differences between two images.

How does it work?

On a very high-level, here’s what the program I created does to achieve this:

  1. Takes in two images
  2. Checks they’re both the same size (stops here if they’re not)
  3. Goes through the first image pixel-by-pixel and compares the Red, Green and Blue (RGB) values of each pixel to the same pixel in the second image
  4. A new blank image is constructed of the same dimensions of the original images
  5. Wherever there is a difference between the two images under comparison (according to each tolerance value), the pixel in the new image is changed to either bright green. orange. yellow or red, otherwise the “alpha” (transparency) value is used to make the pixel transparent on the new image.
  6. New image is saved in a specific folder locally, this image is now treated as a “mask”
  7. The mask is then overlaid on top of the second of the original images up for comparison and saved as yet another new image
  8. This final new image is what you see in the outputs above. All changes are highlighted on the original image for easy viewing.

So there we go, we’ve taken images from the drone, compared them pixel by pixel and outputted something that can be used to quickly identify areas of major change.

Current/future work

When it comes to this being deployed in the real world, it faces a couple of issues at the moment, a few of which I’m working on fixing:

  • If the images aren’t taken from exactly the same position and angle the output is unbelievably screwed at the moment. Below I’ve simply shifted the image to the right by 3px for comparison, as you can see there is a LOT of additional noise

finaldifference.png

This could be mitigated for by going all “Inception” on it. By this I mean searching for an image, within an image, according to a point of interest, a certain pattern of pixels for example then using this to represent the pixel co-ordinate, copy out a certain sized area of the image using the co-ordinate gained as a corner.

Another way of mitigating against this is to ensure the drone checks the pixels in view through its camera as it flies around and automatically correcting its pitch, roll and yaw to get to the exact position that a previous image was taken at. However this option would take a lot of live processing resources to achieve and based on playing with the Parrot AR2.0, it’s a little unstable to fine-tune it’s location to the level of precision required.

  • As also noticeable in the above outputs, even the slightest change in lighting is highlighted as a change, albeit green – therefore slight, this adds a lot of noise to the image, distracting the user from the main points of interest.

This is currently being worked on, in that I’ve implemented varying level of tolerances to highlight the level that something has changed on the output (final) image.

  • Hard-coded tolerance values

As these values have been built in throughout the development process, they’re not dynamic to the users’ possible needs at the minute. The user interface could be improved to include options for the user to play with the tolerance values, maybe even showing changes of a certain tolerance only (singular, modifiable tolerance value). As my dissertation is based on providing a technical solution rather than a fancy user interface, for now this has been set aside. For the future this is a necessary consideration to take the work forward.

Overall…

So aside from failing to post my progress up on here for 2 months, things are going fairly well. It’s getting a bit more into the wordy side of things though as I’ve been typing my dissertation up on all the work done to date relentlessly, so forgive me again but I’ve tried to add some more media in where I can 🙂

In the scope of my dissertation this is more or less all that I’ll be doing in terms of building the application itself, unfortunately I’ve run out of time to play with LiDAR and the like but I may well revisit this idea one day. What I will be doing from here on regarding this inaugural project however, is writing posts when I get some time on how exactly it was all done (coded) so anyone out there can replicate and play with the ideas I’ve enjoyed working with. Throughout the development process I’ve maintained a private Git repository which I will aim to make public as soon as I’m hoofed back into the real world after graduating this summer, so that should also help.

As always, please let me know if you have any questions or if you have any feedback, writing these posts has been an enjoyable side-project so far on top of the dissertation so please keep all the input coming!

 

Brave new world – first manual flight!

As promised in my previous post (Hard Reset“), these articles will be coming in much more regularly now that I have a place to properly fly and test the drone in. Progress has already started to build 🙂

Over the last 24 hours the project has gone from sending the drone off on a pre-programmed flight, to enabling manual control through a laptop. Thanks to some helpful signposting of a fellow student at Brunel (Ben Evans – thanks for the advice!), I was able to implement Webflight (all credit to Laurent Eschenauer (eschnou) for building and providing this application). Below is a screenshot of me and two of my coursemates getting this working today, taken from the front camera of my drone.

drones

This program uses NodeJS to connect with the drone, display a live video stream straight to a web browser (“localhost:3000”) while also enabling the user to manually navigate the drone around using the keyboard. If you’d like to use Webflight yourself, below are some tips that helped me get up and running with it that should hopefully help you get set up and flying in no time at all:

  1. Make sure you’ve done the tutorial over on the Instructables site as described in my earlier post, takeOff();This guide provides a great foundation to help you understand NodeJS and the npm library usages in a practical manner and will greatly help you in getting to grips with further drone/NodeJS/npm related work.
    • Completing the Instructables tutorial will ensure that you have a separate “Drone” folder, as well as NodeJS and Ffmpeg installed so is a good head start!
  2. Visit the Webflight Github page and spend some time reading the README.md file, the author has spent considerable time and thought to provide a comprehensive guide to the application.This guide is immensely useful if you follow it word-for-word
    • If (like me) you are too keen to get the drone airborne and think you know best, you might end up spending quite a bit of time rectifying your mistakes later on!
  3. To be 100% clear, install git, NodeJS and ffmpeg before starting to work on the Webflight installation.
    • Also, make sure that your PATH (environment variable) includes a reference to “c:\…\Ffmpeg\bin\”, “c:\…\nodejs\”, “c:\…npm\ and “c:\…\Git\bin” once all of these have been installed on your machine through the hyperlinks in “3.”.
  4. Using Git Bash, clone the Webflight project into your Drone folder (see Instructables tutorial) on your laptop as shown in the first step of the Install instructions on the Github page.
  5. Open up windows command prompt to complete the second and third lines of the Install guide on the Github page, ensuring you have used “cd ” to navigate into the Webflight folder
  6. The fourth/final line about installing Bower caused me some issues as it couldn’t find Git on the laptop I was using so here’s what happened:
    • Firstly, make sure you run “npm install –global bower” instead and make sure you’re running the command in the Webflight folder that you cloned (see step two in the Install guide)
    • Checked environment variables, Git was in there
    • Ran “npm install –global bower”  through Command Prompt as administrator, no change on the output
    • Ran the same command in Git Bash (while inside the Webflight folder) and success! I have no idea why this worked, it just did!
  7. The initial control layout is not in QWERTY format, to change this edit the config.js file and save it

If you follow the guide on the Github page and use the above for reference, you should be fine. The Bower install issue was by far the biggest pain as this resulted in only seeing a blank webpage with the Webflight banner at the top when the app was running. Once all the above it set up, it’s as simple as connecting to the drone’s WiFi, executing the program in Command Prompt (cd to Webflight folder and type “node app.js”) and entering “localhost:3000” in a new web browser window or tab.

The above has been more of a how-to style post as this caused me a little trouble while setting it up so below I’ll talk about why I chose this existing application as part of my project.

When determining the best means of approaching drone programming there were several options out there in various different languages (see “Hard Reset”). Considering all of the available options, it became obvious to me that NodeJS is the most established and well documented means of creating drone applications for beginners. As I am a beginner and this project isn’t about me reinventing the wheel in terms of drone control, this offered the best platform for me to get on with programming a meaningful solution towards the objectives of my dissertation.

Webflight in itself was recommended to me as the best starting point to understand drone programming and to work with, this also gives a great insight into client-server interactions and JavaScript. For the sake of my work towards this, I’m aiming to work on a plugin for the existing application that will enable the drone to perform the specific navigational and surveying tasks that I require of it. In doing this, I’m looking to attempt the following:

  • Semi-autonomous navigation – upon request of user (JS)
  • Autonomous detection of objects (JS)
  • Image capture (JS)
  • Image comparison (Java)

Bundling all of this together in a Java project will enable a single point of usage, alleviating text-based input requirements of the user and allowing an image comparison function to be implemented in the same project (comparing a past and present version of a structure for differences). All in good time though…

So there you go, I’ll keep posting as this project progresses and hopefully provide some content to help other beginners get into drone programming 🙂

Thanks for reading,

Mark.

 

 

Hard Reset

Anyone following along with my little experiment to blog my FYP/dissertation progress will have noticed that it’s been a while since any update has been posted up here. Apologies, I’ll be posting more from now on (for reasons that’ll be explained shortly), I hope that these posts will continue to be useful to at least someone out in the world who’s interested in flying drones using code. I’m very glad to have received some feedback that people are enjoying the content and getting something out of it, thank you for that! Please let me know if I can add more to these posts to help out further. 🙂

So what’s been going on throughout this period of silence? In a nutshell, I have been working hard to confirm a better spot to fly the drone on campus here at university. In doing this, I’ve been working with my supervisor to make sure everything is above board and good to go, I’m very relieved to report that yesterday we were able to confirm that it is now A-OK for the practical coding and flight of the drone to go ahead!

Small victories aside, the title of this post is intended to give an idea of my current position – resetting my project, reviewing the relevant literature and working from the ground up. Before I get into the in’s and out’s of the work behind the scenes up to now, here’s a recap of my project’s focus:

Exploring the implementation of consumer-level (off-the-shelf) drone technology in the process of inspecting and gathering data on physical structures for the purposes of performing structural surveys.”

In other words…

“Let’s see if I can get a drone to fly around with some level of autonomy and enable a user to perform survey-related tasks while they’re at it. Preferably without crashing.”

Although some work had previously been done in getting the drone airborne, (see .takeOff();), I decided it best to take a step back, do more research and start afresh once I was able to fly the drone again. During the time I was able to really get into some background research which is now helping me steer development a little better. Through my research, I came to the following conclusions:

  1. Drones are currently being used by some companies in performing building surveys, however this a field in its infancy as far as I can tell (Kestrel CamFlying Eye).
    • The drones used in these (up-close) surveys are manually operated. Let’s see about getting some autonomy in there!
    • Singular hi-res images and videos are taken, so there’s an opportunity to implement Structure From Motion (SFM) to develop on this foundation without the need for additional hardware.
    • Recommended by supervisor – what about taking two photos of the exact same structure at two different points in time and comparing them for any differences using programming?
  2. There are APIs (Application Programming Interfaces) and SDKs (Software Development Kits) out there for the drone in C#, Java (Android), JavaScript etc.
    • Unfortunately these are (in the main) pretty poorly documented. At least, for my tiny brain to comprehend anyway.
  3. LiDAR (Light Detection and Ranging) payloads are being used on drones for the purposes of larger-scale surveying and 3D modelling.
    • Re-purposing this technology for up-close structural surveys could be a more accurate and useful means of collecting and presenting data of structures in hard to reach or unsafe environments or parts of a building?
    • However, these payloads are too heavy for my off-the-shelf drone and are over my budget (not so off-the-shelf or consumer-level!).
  4. The Xbox Kinect might be a better option for this project! The Kinect uses depth-imaging in a similar manner as LiDAR and has a dedicated SDK provided my Microsoft for Developers.

There’s quite a lot to take in as you can probably tell, my job at the moment is to pick a specific direction and go from there. Discussing this with my supervisor it was decided that I will start by working on semi-autonomous navigation of the drone as well as building a function to compare two images of the same structure (angle, lighting etc all the same), and check for any differences. This “spot-the-difference” function should allow a user to compare images taken at different points in time and identify changes or developments of the structure (or “subject”) in question, automatically.

Once these initial steps have been worked on, the more advanced aspects such as LiDAR or similar will be looked into. As the project moves on it will increment in technicality and the amount of tasks it can address.

That’s all for now from this post, I’ll be sure to post as this project progresses. Now that full development of the software has been given the green light, there will no doubt be a lot to say about mistakes made, lessons learned and tips that I recommend. Anything that I think could be of any use will be posted!

As ever, I hope this helps and please get in touch if you have any feedback at all 🙂

Thanks for reading.

Mark.

 

 

 

.takeOff();

Welcome to the first proper blog post about my Final Year Project (FYP)!

Since my first post on here (‘I have a blog.’) I’ve been looking into the legalities and applications of drones currently in the big wide world (pretty dry stuff). What I’ve really wanted to do instead is get the laptop out and start writing code. Up to this point however, I haven’t had the luxury of a ‘clean room’; a windless, relatively uncluttered, large, indoor space.

Today, I am glad to announce, I was finally able to sit down and work on programming my Parrot AR 2.0 drone (Dave, or dUAVe as he is known) in a spacious, indoor environment on campus at Brunel.

The goal of the day was to write a simple program on my laptop that would tell the drone what to do. The logic being that if I can get that element to work, I can write a whole PC-based program to communicate with the drone.

After a steady start I was able to follow along with an amazing tutorial I found on the Instructables website. This tutorial goes over connecting the drone to your laptop and sending basic commands to it through node.js, as well as  how to program a basic set of instructions for your drone to play out at the click of a button.

After 3 hours of fiddling with the code and witnessing dUAVe survive two fairly serious encounters with the wall, the end result can be seen on my YouTube channel. In the video you can see me hit enter on the keyboard and let the drone carry out the basic set of instructions sent to it.

Tips on the guide I used (Instructables):

  1. Firstly, it is very well written but if like me you have the attention span of a gnat, you’ll miss out on the handy bits of info at the end of each step so make sure to read each step all the way through before starting!
  2. In step 5 you are recommended to use Sublime as a text editor. This isnota good text editor. It is a one way ticket to launching your laptop through the window/drone. Try Notepad++ instead!
  3. When trying to run your automated program for the first time in step 6, make sure you haven’t stored jsin the folder titled nodejs. I did this, it will not work. Instead make sure to create a separate folder for it (called Drone or similar), in the same directory as the nodejs folder and ensure that you run ‘npm install ar-drone’ in the command line while in the new Drone folder you’ve created. By this I mean re-do step 3 in the Instructables guide for the Drone folder. Google the use of ‘cd’ to navigate through directories in command line if you’re unsure of how to get around in there.
  4. As ever, pay attention to any errors thrown back to you within the command prompt when running your program to find any rogue semicolons/brackets!

To help out, the code used in the video is in the image below  (Click to enlarge).

12038916_10156081370915722_1200142344215504560_o1

So there we are, a solid achievement for the first day of programming. Now to figure out how to get dUAVe to use his camera while we’re flying…

Please let me know if you have any questions or comments about this post or any other, I’ll do my best to answer any questions and all feedback is welcome!

Mark.

Note: In the above examples I am using a Windows 7 laptop so I will have very limited knowledge of Linux/OSX instances of this project!

Originally osted on September 26, 2015 Categories: Brunel FYP, Drone Programming