I've tried spark-submit with --driver-class-path, with --jars as well as tried this method <a href="https://petz2000.wordpress.com/2015/08/18/get-blas-working-with-spark-on-amazon-emr/" rel="nofollow">https://petz2000.wordpress.com/2015/08/18/get-blas-working-with-spark-on-amazon-emr/</a>
On using SPARK_CLASSPATH in the commandline as in
I get this error
But I'm not able to add it. How do I add postgresql JDBC jar file to use it from pyspark? I'm using EMR version 4.2
Thanks
On using SPARK_CLASSPATH in the commandline as in
Code:
SPARK_CLASSPATH=/home/hadoop/pg_jars/postgresql-9.4.1208.jre7.jar pysparkI get this error
Code:
Found both spark.executor.extraClassPath and SPARK_CLASSPATH. Use only the former.But I'm not able to add it. How do I add postgresql JDBC jar file to use it from pyspark? I'm using EMR version 4.2
Thanks