SPARK Custom Partitioner Java Example

Below is an example of partitioning the data based on custom logic.
For writing a custom partitioner we should extend the Partitioner class , and implement the getPartition() method.For this example I have a input file which contains data in the format of <Continent,Country>.
I would like to re-partition the data based on the first letter of the continent.

Main Class :

Custom Partitioner Class:


Output : As can be seen , the data of the continents which start the same letter are in the same part file.

Leave a Reply

Your email address will not be published. Required fields are marked *