Back to Projects
GWU MSBA

Flight Big Data Analysis

Processed and transformed a 30M-row flight dataset on AWS EC2 instances with optimized SQL queries and aggregation logic.

Role: Data EngineerDate: 2024
Flight Big Data Analysis - GWU MSBA project

Situation

The Data Management course final project required processing a massive flight dataset to demonstrate cloud-based big data skills.

Task

Design and implement a scalable data processing pipeline on AWS EC2 that could handle 30 million rows efficiently.

Actions

  • Set up AWS EC2 instances for distributed processing
  • Optimized SQL queries and data transformation logic
  • Designed aggregation and summarization pipelines
  • Produced analysis-ready datasets for downstream reporting

Results

30M

Rows processed

What I'd Do Next

Apply similar patterns to real-time streaming data scenarios.

Built with v0