Description

You will lead test automation and data engineering efforts within an Agile squad supporting a large-scale Common Data Platform.

Responsibilities

• Design, build, and maintain automated test frameworks and reusable test components.
• Coordinate data quality assurance across ingestion, transformation, and delivery stages.
• Develop and maintain data pipelines using Spark and Airflow.
• Lead end-to-end automated testing strategies and ensure traceability between requirements and test scenarios.
• Monitor and troubleshoot data pipelines, implementing alerts and auto-remediation.
• Provide mentorship on testing methods, automation techniques, and quality assurance best practices.

Required Skills

• 7+ years of experience in test automation, data engineering, or software engineering.
• Bachelor's degree in Computer Science, Information Systems, or a related field.
• 2+ years of experience with modern data stacks including S3, Spark, Airflow, and Lakehouse architectures.
• 2+ years of experience with test automation tools such as Great Expectations, Lambda Test, Selenium, or Cucumber.
• Proficiency with SQL and data modeling concepts.
• Experience designing and managing ETL pipelines for structured, semi-structured, and unstructured data.
• Proficiency with AWS services including S3, EC2, Lambda, Data Pipeline, and CloudWatch.
• Strong understanding of data governance, information security, and architecture principles.

Preferred Skills

• Experience building data pipelines using Spark on AWS cloud platforms.
• Knowledge of CI/CD automation tools and DevOps best practices.
• Experience coaching and mentoring technical teams.

Key Skills

Education

Bachelor's degree