End to End Project using Apache Spark Streaming/Apache Hadoop with Kafka

I have developed the "Real-Time Dashboard for Weather Monitoring" using Apache Spark Streaming/Apache Hadoop with Kafka.


I have followed the below reference architecture developing this application/dashboard.

Real-Time Dashboard Reference Architecture

These are modules used in this big data project.

Part 1: Architecture Overview

Part 2: Kafka Producer Module

Part 3: Spark Streaming Module

Part 4: Hive Module

Part 5: Presto Module

Part 6: REST API(service) using Flask-RESTPlus

Part 7: Dashboard using Python Dash 

Real-Time Dashboard for Weather Monitoring

End to End Project using Apache Spark Streaming/Apache Hadoop with Kafka | With Code Walkthrough "Click Here"

Happy Learning !!!