Description

You will design and manage scalable data architectures and end-to-end technical implementations on AWS.

Responsibilities

• Build and orchestrate ETL data pipelines using AWS Glue and Step Functions.
• Develop streaming and batch processing systems using PySpark and Kafka.
• Lead technical implementations and mentor engineering teams.
• Implement data security using IAM roles, policies, and governance frameworks.
• Optimize data storage and querying through partitioning and Parquet file formats.

Required Skills

• 6+ years of experience in data engineering roles.
• Extensive experience with AWS services: EMR, S3, Redshift, Glue, Lambda, Step Functions, DynamoDB, RDS, Athena, EventBridge, API Gateway, and SNS.
• Proficiency in Python and PySpark.
• Strong SQL skills for complex querying in Redshift and Snowflake.
• Hands-on experience with Kafka for data streaming.
• Experience with DevOps tools including Docker, Terraform, and CI/CD pipelines.
• Deep understanding of data lake, data warehouse, and data lakehouse architectures.
• Knowledge of Redshift Spectrum and Athena for large-scale querying.
• Familiarity with data governance, compliance, and quality frameworks.

Preferred Skills

• Experience working within Agile methodologies.

Key Skills

Education

Any Graduate