Description

• 5+ years building enterprise-scale data solutions using Spark, Hadoop, Hive, and Scala
• Strong scripting skills (Python or Perl) and expert-level complex SQL (window functions, multi-joins)
• AWS cloud experience required (S3, EMR, Glue, Athena)
• Experience with Agile delivery, CI/CD pipelines, automated testing, and GitHub workflows
• Financial services or regulated industry experience preferred

OBJECTIVES

• Design and maintain scalable, reliable big data pipelines
• Optimize Spark/Hadoop workloads for performance, scalability, and cost efficiency
• Implement automated testing and data quality validation
• Enable analytics and data science teams with high-quality, accessible datasets
• Leverage AI-assisted tools (Copilot, ChatGPT, Q Developer) to improve development productivity

PROBLEM-SOLVING

• Diagnose and resolve Spark performance bottlenecks and data pipeline failures
• Optimize complex SQL transformations and large-scale joins
• Troubleshoot data quality, latency, and reliability issues in production
• Improve AWS workload efficiency through tuning and resource optimization
• Automate repetitive engineering tasks using AI-assisted development tools

EXPERIENCE VALIDATION

• Delivered end-to-end pipelines using Spark and Hadoop ecosystem tools
• Optimized SQL and pipeline performance with measurable improvements
• Deployed and supported AWS data workloads (EMR, Glue, Athena, S3)
• Implemented CI/CD and automated testing for data pipelines
• Used AI coding assistants and GitHub workflows in team-based development

Key Skills

Education

Bachelor's degree