Answer by Alex Naspo for Spark on Java - What is the right way to have a...
You can use mapPartition or foreachPartition. Here is a snippet taken from Learning SparkBy using partition- based operations, we can share a connection pool to this database to avoid setting up many...
View ArticleSpark on Java - What is the right way to have a static object on all workers
I need to use a non-serialisable 3rd party class in my functions on all executors in Spark, for example:JavaRDD<String> resRdd = origRdd .flatMap(new FlatMapFunction<String, String>() {...
View Article
