Description

We are seeking an experienced AWS Data Engineer with 6+ years of experience in data engineering, cloud-based data platforms, and large-scale data processing. The ideal candidate should have strong hands-on expertise in designing, developing, and maintaining scalable data pipelines and data warehouse solutions using AWS services.

The role involves working closely with data architects, analysts, application teams, and business stakeholders to build reliable, secure, and high-performing data solutions. The candidate should have a strong understanding of ETL/ELT, data modelling, cloud data lakes, data warehousing, orchestration, and performance optimisation.

Key Responsibilities

• Design, develop, and maintain scalable batch and real-time data pipelines on AWS.
• Build and manage data lake and data warehouse solutions using Amazon S3, Redshift, Glue, and related AWS services.
• Develop ETL and ELT workflows using AWS Glue, Lambda, Python, PySpark, and SQL.
• Extract, transform, and integrate data from databases, APIs, files, applications, and third-party systems.
• Create and optimise data models for reporting, analytics, and downstream applications.
• Develop reusable data ingestion, transformation, validation, and reconciliation frameworks.
• Implement workflow orchestration using Apache Airflow, AWS Step Functions, or similar tools.
• Work with Amazon Redshift, RDS, Athena, DynamoDB, and other AWS data services.
• Implement data quality checks, error handling, audit logging, and monitoring mechanisms.
• Optimise data pipelines, SQL queries, storage formats, and cloud resource utilisation.
• Build secure data solutions using AWS IAM, KMS, VPC, Secrets Manager, and access-control policies.
• Support data migration from on-premises or other cloud platforms to AWS.
• Troubleshoot pipeline failures, performance issues, and production data defects.
• Collaborate with DevOps teams to implement CI/CD pipelines for data engineering workloads.
• Maintain technical documentation, data mappings, data dictionaries, and solution designs.
• Participate in requirement gathering, solution estimation, code reviews, testing, and deployment activities.
• Follow Agile delivery practices and participate in sprint planning, reviews, and retrospectives.
• Provide technical guidance and mentoring to junior data engineers.

Key Skills

Education

Not specified