Author Archives: baahu

Find the sum of numbers in a text file

baahu September 30, 2016 No Comments

If we have a text file which contains numbers along with other data and we have to find the sum of those numbers , then AWK would come in handy. Ex:file.txt 10,Ajit 20,Sumit 22,Sam 35,john If the file contains above data , we could use AWK to find the sum… Read more »

Spark CombineByKey

baahu September 24, 2016 No Comments

CombineByKey() is very similar to combiner of the mapreduce framework. In the MR framework, the combiner function is called in the map phase to do a local reduction and this value is then eventually sent over to reducer, this results in large savings if network bandwith. In SPARK, groupByKey() doesnt… Read more »