How do I access the Apache Spark web interface?

Apache Spark provides an HTML-based web interface that displays information about the state of the processing cluster and previously executed applications.

After deploying an Apache Spark cluster with Oshinko, you will find its associated resources (Pods, Services, Routes) in the OpenShift console. Using this information you will navigate to the Spark cluster web interface.

Prerequisites

  • A web browser.

  • An OpenShift console available.

  • An Apache Spark cluster deployed by Oshinko.

Procedure

  1. Navigate your browser to the OpenShift console and select the project that contains the Apache Spark cluster.

  2. Locate the master pod for your Apache Spark cluster, this can be found by looking for a resource that is identified by the name you chose for the cluster with -m appended to it.

  3. Expand the resource view associated with the Spark master pod to expose its details. It should look similar to this:

    OpenShift console Spark

  4. Click on the link provided in the "NETWORKING" section that is associated with the route for the UI service. This service will have the name of your cluster with -ui appended to it. After clicking the link you will be navigated to the Spark master interface page which will look similar to this:

    Spark master interface

Additional resources