Analysis of 15 years US domestic flight data using spark in different input format (csv, parquet, sequence file, json). Different statistical calculations are done upon the dataset using PySpark and then comparison of execution time of each iteration of different input format.
Flight Data Analysis Using Spark
Rangeet Pan
Research Assistant
My research interests include distributed robotics, mobile computing and programmable matter.