Description

You will design, develop, and implement ETL Glue PySpark processes to move and transform data into AWS data warehouses and data lakes like Redshift and Aurora PostgreSQL.

Responsibilities

• Develop and implement ETL Glue PySpark processes to extract, transform, and load data from various sources into AWS data warehouses and data lakes.
• Automate and orchestrate ETL processes, applications, and APIs to optimize performance across on-premises and cloud environments.
• Conduct business analysis and data validation to minimize errors, extract insights, and support business decisions.
• Provide production support, monitoring, troubleshooting, and resolving issues for ETL processes and data integrity.
• Develop comprehensive documentation covering data flows, mappings, APIs, and support procedures.

Required Skills

• 8+ years of experience in ETL development, data warehousing, AWS cloud services, and API development.
• Proficiency in Python and expertise in ETL development using tools like Informatica or DataStage.
• Advanced skills utilizing AWS services, including AWS Glue, S3, EC2, Lambda, and Step Functions.
• Proficiency in SQL across on-premises (e.g., DB2 on AIX) and AWS databases (RDS, Aurora PostgreSQL, Redshift).
• Expertise in scripting languages, including Perl Scripting and Korn Shell.
• Experience with API development using REST and SOAP frameworks.
• Experience with version control tools such as Bitbucket.
• Demonstrated experience in data analysis/validation using tools like Tableau.
• Familiarity with Agile development methodologies.

Key Skills

Education

Any Gradute