Description

Design and maintain scalable, reliable big data pipelines for enterprise-scale solutions.

This role is hybrid.

Responsibilities

• Build and optimize data pipelines using Spark, Hadoop, and Hive.
• Develop complex SQL queries and Python/Perl scripts for data transformation.
• Deploy and manage AWS data workloads including S3, EMR, Glue, and Athena.
• Implement CI/CD pipelines, automated testing, and GitHub workflows.
• Troubleshoot performance bottlenecks and ensure data quality in production.

Required Skills

• 5+ years of experience building enterprise data solutions.
• Expert-level SQL with window functions and multi-joins.
• Strong proficiency in Scala, Python, or Perl.
• Hands-on experience with Spark and Hadoop ecosystems.
• Proficient in AWS services (S3, EMR, Glue, Athena).
• Experience with Agile delivery and CI/CD tools.
• Knowledge of AI-assisted development tools like Copilot.

Preferred Skills

• Experience in financial services or regulated industries.
• Proven track record of optimizing large-scale data workloads.

Key Skills

Education

Bachelor's degree