Wednesday, September 12, 2012

Continuous Deployment for Mobile Apps with Jenkins: Back-end Services and Integration Testing

In an earlier post, I described how to configure Jenkins (running in the cloud using the CloudBees PaaS service) to manage Xcode builds for iOS applications using MacOS slaves.  As part of our demo at the Silicon Valley Cloud Computing Group Meetup,  Kohsuke and I also showed how easy it is to do full, end-to-end continuous integration that includes integration testing with back-end services as well.  For this demo, I built a simple servlet-based chess server with REST web services to record games and moves, using JAX-RS, JAX-B and a MongoDB backend.  You can find the source code on GitHub at https://github.com/mqprichard/mongo-chess.

The MongoChess Server Project

Just to give you a quick sense for the application, here's a view of the Eclipse project:


The com.cloudbees.service package contains two classes (GameServlet and MoveServlet) which provide REST web service interfaces to create games, record moves and get the history for a particular game.  There is also a third class (MongoDAO) which abstracts away access to the MongoDB database, where the actual results are stored: it's a straightforward implementation, with a separate connection used for each REST service call.  I'm using the free MongoDB service provided by MongoHQ, which you can sign up for via the CloudBees Partner Ecosystem page on GrandCentral.  

There are classes (in com.cloudbees.model) to represent Game and Move objects and I use the standard JAX-B @XmlRootElement annotation to handle the mapping to the JSON format used to pass data via the REST interface and also for inserts and queries against the MongoDB database.  Incidentally, this was one of the reasons why I chose to use MongoDB for this example - this is where a NoSQL database that works naturally with JSON-formatted data is much simpler to work with than a more traditional relational database (by way of comparison, I used MySQL for my first cut at this application and you can see that version at https://github.com/mqprichard/chess).  

The Chess Server REST Interface

I'll describe in a minute how I use a Jenkins build pipeline to test the REST service interface, but first here's a quick description of what the interface looks like using curl to make POST and GET requests (notice that the service is configured to accept and return data using MIME type application/json, hence the need to set the Accept and Content-type headers):

Create a new game:
curl -s -H "Accept: application/json" -H "Content-Type: application/json" -X POST -d '{"white":"Player1","black":"Player2","description":"Championship"}' http://localhost:8080/chess/game/new
{"id":"5047b5e703640e8d8faf6b2a"}

Get information about a game:
curl -H "Accept: application/json" http://localhost:8080/chess/game/5047b5e703640e8d8faf6b2a
{ "white" : "Player1" , "black" : "Player2" , "description" : "Championship" , "result" : "" , "next" : "W" , "move" : 1}

Submit moves:
curl -H "Accept: application/json" -H "Content-Type: application/json" http://localhost:8080/chess/moves/new -X POST -d '{"game":"5047b5e703640e8d8faf6b2a","white":"e2-e4","move":"1"}'
{"game":"5047b5e703640e8d8faf6b2a","move":1,"white":"e2-e4"}
curl -H "Accept: application/json" -H "Content-Type: application/json" http://localhost:8080/chess/moves/new -X POST -d '{"game":"5047b5e703640e8d8faf6b2a","black":"e7-e5","move":"1"}'
{"game":"5047b5e703640e8d8faf6b2a","move":1,"black":"e7-e5"}
curl -H "Accept: application/json" -H "Content-Type: application/json" http://localhost:8080/chess/moves/new -X POST -d '{"game":"5047b5e703640e8d8faf6b2a","white":"g1-f3","move":"2"}'
{"game":"5047b5e703640e8d8faf6b2a","move":2,"white":"g1-f3"}
curl -H "Accept: application/json" -H "Content-Type: application/json" http://localhost:8080/chess/moves/new -X POST -d '{"game":"5047b5e703640e8d8faf6b2a","black":"b8-c6","move":"2"}'
{"game":"5047b5e703640e8d8faf6b2a","move":2,"black":"b8-c6"}
curl -H "Accept: application/json" -H "Content-Type: application/json" http://localhost:8080/chess/moves/new -X POST -d '{"game":"5047b5e703640e8d8faf6b2a","white":"f1-b5","move":"3"}'
{"game":"5047b5e703640e8d8faf6b2a","move":3,"white":"f1-b5"}

Get move history for a game:
curl -H "Accept: application/json" http://localhost:8080/chess/moves/5047b5e703640e8d8faf6b2a ; echo
[{ "white" : "e2-e4"}, { "black" : "e7-e5"}, { "white" : "g1-f3"}, { "black" : "b8-c6"}, { "white" : "f1-b5"}]

There are also a set of JUnit tests, which I use to test all the MongoDAO methods as well as the GameServlet/MoveServlet interfaces.  Of course, I need to run these every time I do a build but I also want to invoke the REST services over the wire to be certain that the JAX-RS interfaces are working as expected.  There are several ways to do this and the folks at SOASTA have a very nice capability to automate test like these: however, as an example of how to do this using the Jenkins community plug-ins, I put together some scripts that can be run as part of a Jenkins build pipeline, that run a set of curl commands like those above and check the actual output against that expected.  

The only thing that's at all complicated about these is the need to parse the JSON data using only basic shell utilities like awk: if you use tools like jsawk then of course the scripts are smaller and simpler, but in this case, I wanted to be sure that the scripts would execute on the lightweight, JEOS shell that runs the Jenkins executor processes in the CloudBees Paas platform.  I'm sure that there are shell script gurus out there who can do better, but you can see the scripts I use on GitHub at https://github.com/mqprichard/mongo-chess/tree/master/scripts.

Building the Integration Test Pipeline 

Here's how to configure the Jenkins build pipeline: you can see the full configuration here.  First, we need a Jenkins job to build the Maven project and run the JUnit tests: that's called mongo-chess (view configuration) and apart from the normal build/test configuration the only other thing we need is to set up the pipeline so that we archive the built artifacts (in this example I'm using the Jenkins Enterprise Fast Archive plugin that I described in the earlier blog, but that's just icing on the cake) and run the integration test build (mongo-chess-integration) on a successful build, as shown below:


The second part of the pipeline is the integration test itself and you can view the configuration here.  As you can see from the following screenshots, this runs after the mongo-chess build/test completes and it spins up an instance of the archived application (remember this all happens on-demand in the cloud), then calls the Execute Shell Jenkins plugin to run the integration tests.



Take a look at the console output for one of the build jobs and you'll see the result: provided the interface behavior has remained, there is no diff between the observed and expected outputs and the shell script returns zero.  The build is successful and we pass in turn to the next stage of the pipeline (mongo-chess-end2end-integration), which is where we would configure the SOASTA TouchTests for the mobile application.  More on that in an upcoming blog/webinar - for now that's just a placeholder.

Want to see this technology in action?

My friends at SOASTA and I will demo it live on Sept 27 at 10amPT on our new webinar, Automated Testing & Continuous Deployment for Mobile Apps in the Cloud. Sign up and join us!

Attend Jenkins User Conference SF - A Special Discount For You!

To find out more about this and all things Jenkins, please join us at the 2012 San Francisco Jenkins User Conference on Sunday, 30 September and enjoy a full day of Jenkins-focused learning and networking opportunities.  Register here and as a special "Thank You" for reading all the way to the bottom of my blog, enter promotional code JUC-2012SF to get a $100 discount off the full-price registration!



Mark Prichard, Senior Director of Product Management
CloudBees


Mark Prichard is Java PaaS Evangelist for CloudBees. He came to CloudBees after 13 years at BEA Systems and Oracle, where he was Product Manager for the WebLogic Platform. A graduate of St John's College, Cambridge and the Cambridge University Computer Laboratory, Mark works for CloudBees in Los Altos, CA.  Follow Mark on Twitter and via his blog Clouds, Bees and Blogs.



Follow CloudBees:

   

1 comment:

  1. Great blog!!! Informative & Knowledgeable post, Thanks for share..Mobile Application Design

    ReplyDelete