Description

You will design and implement AWS data lake solutions for large-scale, high-volume data environments.

Responsibilities

• Design AWS data lake architectures integrating Redshift with DMS, Glue, Lambda, S3, Athena, and Airflow.
• Develop Glue ETL scripts using PySpark for data frame transformations, joins, and re-nationalization.
• Convert Oracle scripts and stored procedures into Redshift equivalents.
• Implement complex ETL scenarios including CDC and SCD logic across multiple source systems.
• Manage data manipulation and transformation using SQL programming and Redshift stored procedures.

Required Skills

• 3+ years of hands-on experience designing AWS data lake architectures.
• Proficiency in PySpark and Glue ETL scripting.
• Advanced SQL programming and Redshift stored procedure development.
• Experience with AWS services: Redshift, DMS, Glue, Lambda, S3, and Athena.
• Practical knowledge of Airflow for data orchestration.
• Experience with AWS Security protocols.
• Shell scripting skills.

Preferred Skills

• AWS certifications related to data engineering or databases.
• Experience with Docker, Parquet file formats, and Jupyter Notebooks.
• Knowledge of API Gateway, route structuring, and token-based authentication.

Key Skills

Education

Any gradudate