S3 Source Example

a connectivity example

Load the client libraries...

--packages=org.apache.hadoop:hadoop-aws:2.7.3

Configure with S3 credentials (python)...

hadoopConf=spark.sparkContext._jsc.hadoopConfiguration()
hadoopConf.set("fs.s3a.access.key", "Access Key Id")
hadoopConf.set("fs.s3a.secret.key", "Secret Key")

S3 Source Example a connectivity example