S3 Source Example

Load the client libraries...

• --packages=org.apache.hadoop:hadoop-aws:2.7.3

Configure with S3 credentials (python)...

• hadoopConf=spark.sparkContext._jsc.hadoopConfiguration()
• hadoopConf.set("fs.s3a.access.key", "Access Key Id")
• hadoopConf.set("fs.s3a.secret.key", "Secret Key")