Author Archives: baahu

Streaming Twitter Data into Kafka- Python code

Below is a simple python based Kafka producer which reads data from twitter and puts data into kafka topicYou will have to register with twitter to get tweets streamed into this app.After registration you will have your own access_tokens,access_token_secret,consumer_key,consumer_secret . Install tweepy and twitter libraries using below command pip install… Read more »

SPARK : How to generate Nested Json using Dataset

I have come across requirements where in I am supposed to generate the output in nested Json format.Below is a sample code which helps to do the same.The input to this code is a csv file which contains 3 columns . company name department employee name Example: google,jessica,sales google,sita,technology We… Read more »