$ oc new-app --template=oshinko-python-spark-build-dc -p GIT_URI=[your source repo] -e SPARK_OPTIONS='--packages org.apache.spark:spark-sql-kafka-0-10_2.11:2.4.0'
You need to add --packages org.apache.spark:spark-sql-kafka-0-10_2.11:2.1.0
when running spark-shell
, spark-submit
or to SPARK_OPTIONS
for S2I. For
example, to start a new application with these options you could run the
following:
$ oc new-app --template=oshinko-python-spark-build-dc -p GIT_URI=[your source repo] -e SPARK_OPTIONS='--packages org.apache.spark:spark-sql-kafka-0-10_2.11:2.4.0'
Note In this example the Apache Spark version is 2.4.0
, you will need to
change this version information to match the version of Spark you are using.