Description

Lead enterprise-wide automation testing for ETL, data migration, and transformation initiatives.

Responsibilities

• Architect and maintain automation testing frameworks for ETL, ELT, and data migration processes.
• Design and execute testing strategies for large-scale enterprise big data applications.
• Manage CI/CD pipelines using GitLab to support automated testing workflows.
• Optimize data processing by configuring and testing AWS Lambda functions and S3 integrations.
• Brief stakeholders and senior management on the benefits and constraints of chosen technology solutions.

Required Skills

• 6-8 years of IT experience specializing in testing and automation.
• Strong programming proficiency in Java, Groovy, and Spark Scala.
• Advanced SQL skills including Joins, Aggregations, Windowing functions, CTEs, and RDBMS schema design.
• Hands-on experience with Databricks, Structured Streaming, Delta Lake, and Delta Live Tables.
• Deep understanding of Data Lake concepts such as time travel, schema evolution, and optimization.
• Experience with AWS services including S3, Glue, and Lambda.
• Proven ability to test ETL/ELT workflows using tools like Data Migration Service.
• Experience maintaining GitLab CI/CD pipelines and monitoring via CloudWatch.

Preferred Skills

• Architecture experience within an AWS environment.
• Familiarity with using AWS tools for viewing and processing massive data at scale.

Key Skills

Education

Any graduate