Description

You will design, build, and maintain data pipelines and ETL processes to ensure reliable data ingestion, transformation, and delivery.

Responsibilities

• Design logical data models and implement physical database structures to support web applications and reporting.
• Implement data quality, validation, security, and privacy measures to safeguard sensitive information.
• Monitor and troubleshoot data pipelines to resolve processing, performance, and reliability issues.
• Collaborate with developers, analysts, and QA teams to provide necessary datasets and resolve defects.
• Document data engineering processes, technical specifications, and best practices.

Required Skills

• 5+ years of ETL coding experience.
• Proficiency in Python and SQL for data processing and automation.
• Experience with AWS data environments including EMR, Glue, S3, DocumentDB, Redshift, RDS, and Athena.
• Experience with data warehouses (Redshift) and NoSQL stores (DocumentDB, DynamoDB, OpenSearch).
• Experience building data lakes using AWS Lake Formation.
• Experience with workflow orchestration tools like AWS Step Functions or AWS MWAA.
• Strong understanding of relational databases, including tables, views, indexes, and tablespaces.
• Experience with GitHub and CI/CD processes.
• Knowledge of SDLC and Agile methodologies.

Preferred Skills

• Certified Data Management Professional (CDMMP).
• IBM Certified Solution Developer.
• SAS Certified Data Integration Developer.
• Informatica Certified Professional (ICP).

Key Skills

Education

Any Graduate