My First RADanalytics Application

Introduction

In this tutorial you will build a microservice which will calculate an approximate value for Pi when requested by HTTP. You will learn how to create applications which utilize Apache Spark and that are deployed directly from a source repository to OpenShift with a single command.

This tutorial will walk you through every step of creating, deploying and operating an Apache Spark driver application on OpenShift. You should expect to spend 30 to 60 minutes on these exercises, depending on your familiarity.

Prerequisites

Before beginning the tutorial, there are a few prerequisites which are necessary to ensure you have the proper tools and resources to complete the work.

  • Access to an OpenShift cluster

  • A terminal with the oc command available

  • A new project in OpenShift to store your work

  • The Oshinko templates available in your project (see the Get Started page for instructions)

  • An editor to work on your source files

  • An online space for a Git repository to store your code, such as GitHub or BitBucket

With these in place you are ready to begin creating your microservice.

Application overview

The SparkPi microservice that you will build during this tutorial is quite simple in its design and functionality. SparkPi is deployed as a single server pod and several Apache Spark pods. The microservice provides an HTTP server that will accept GET requests and respond with an estimation of Pi. It does this by utilizing Apache Spark to calculate the value using the summing of a circle’s area technique.

It is worth mentioning that in a normal environment you would not want to run extended calculations during an HTTP request unless they can be completed quickly. For the purposes of this tutorial however, it simplifies the implementation details. As you build further applications from the material provided here please keep this simplification of design in mind.

Regardless of the implementation language or the HTTP framework, all of the tutorial applications linked from this page have the same general architecture. This diagram shows the major components involved with deployment and usage of the microservice:

Component key

  1. Computer with a web browser or other method for making HTTP requests

  2. The "SparkPi" microservice

  3. An Apache Spark cluster

Although this is three different components, the main one that you will be concerned with throughout the tutorial is #2. You should already have a computer with the necessary tools installed for making Pi requests, editing your source files and commanding OpenShift to deploy your microservice. The Apache Spark cluster will be created automatically for you by the Oshinko source-to-image tools.

As source-to-image technology is fundamental to the operation of this tutorial, let’s take a closer look at its details. Source-to-image provides a convenient way for you to produce container images directly from your source repositories. This work is handled by language specific builder images which can ingest your source code, assemble it and then produce an application image which is ready to be deployed. In the case of the Oshinko source-to-image builders, an Apache Spark cluster will also be created dynamically for your application when it runs.

Setup your coding environment

To utilize the source-to-image workflow you will need to have a Git repository to store your work. This repository is then used by the source-to-image builders to create the final application image which will be run on OpenShift. For the purposes of this tutorial your repository will need to be accessible by the OpenShift cluster. The easiest way to do this is by using an external service like GitHub or BitBucket.

Create a new repository on your preferred platform and make a local clone of it, if you need help with this process there are instructions available in BitBucket’s documentation and GitHub’s documentation.

With your freshly minted repository you are ready to start coding!

Choose your language and technology stack

This tutorial has been developed for multiple languages and HTTP frameworks. Functionally, all these microservices work in the same manner with similar results to an end user. In this way they represent a black box style microservice which could be replaced with another component that satisfies the same input and output requirements.

Each of these language specific tutorials provide instructions on how to structure your repository and what files to create. You will also learn how each microservice works and the commands necessary to build and launch them in OpenShift. Because the source-to-image templates that you will use are language specific, you must choose a single implementation to exist in your repository. When you are finished, return here to learn about interacting with your microservice and exposing it outside of OpenShift. Please choose one of the following language and framework options:

Become a user

At this point you should have created a code repository for your microservice, populated the repository with source files, built an application image and launched that image on OpenShift. The final stage in this tutorial is to expose your microservice outside of OpenShift and begin interacting with it as a user would.

The first step in this process is to expose a route to your microservice. OpenShift contains an edge router which will associate domain name URIs with services. By default, applications you create through source-to-image have services that can expose routes which will contain their name, the project name and a hostname for the OpenShift server. You can create these routes by using the following command:

oc expose svc/sparkpi

To see the routes available in your project, run oc get routes, you should see something like this:

$ oc get route
NAME                      HOST/PORT                                      PATH      SERVICES            PORT       TERMINATION   WILDCARD
cluster-uo7wa9-ui-route   cluster-uo7wa9-ui-route-pi.10.0.1.109.xip.io             cluster-uo7wa9-ui   <all>                    None
sparkpi                   sparkpi-pi.10.0.1.109.xip.io                             sparkpi             8080-tcp                 None

You will notice that in addition to the sparkpi route we created earlier that there is also a route prefixed with cluster-. This is the route created automatically by the Oshinko source-to-image tooling and it provides access to the Apache Spark web interface. You may visit either one of these routes in your web browser to inspect the results.

In addition to accessing the SparkPi microservice through the web browser, you can use the curl tool and a little bit of command scripting to get the results. The following commands will substitute in the URI for your service and the results should look similar for you:

$ curl http://`oc get routes/sparkpi --template='{{.spec.host}}'`
Python Flask SparkPi server running. Add the 'sparkpi' route to this URL to invoke the app.

$ curl http://`oc get routes/sparkpi --template='{{.spec.host}}'`/sparkpi
Pi is roughly 3.140480

You can see that our rough approximation is close, but not quite close enough. You can try adding the scale parameter to your URL to see how it affects the outcome.

curl http://`oc get routes/sparkpi --template='{{.spec.host}}'`/sparkpi?scale=5

Continue exploring

Congratulations! You have just created and deployed your first RADanalytics application onto OpenShift. At this point you are beginning to understand the core concepts behind the Oshinko source-to-image tooling. You should investigate the other applications and examples in the Tutorials section and also revisit the Get Started page to learn how the Oshinko webui can be used to control the Apache Spark clusters in your projects.