Description

Build and maintain data pipelines and ETL processes to ensure reliable data ingestion, transformation, and delivery.

Responsibilities

• Design and optimize data models, schemas, and physical database structures to support web applications and reporting.
• Implement data quality, validation, security, and privacy measures to protect sensitive information.
• Monitor and troubleshoot data pipelines to resolve performance, processing, and reliability issues.
• Collaborate with developers, analysts, and QA teams to provide necessary datasets and resolve defects.
• Document data engineering processes, technical specifications, and best practices.

Required Skills

• 5+ years of ETL coding experience.
• Proficiency in Python and SQL for data processing and automation.
• Experience with distributed computing frameworks like Apache Spark.
• Experience with AWS data environments including EMR, Glue, S3, DocumentDB, Redshift, RDS, and Athena.
• Experience building data lakes using AWS Lake Formation.
• Experience with workflow orchestration tools such as AWS Step Functions or AWS MWAA.
• Hands-on experience with NoSQL stores including DocumentDB, DynamoDB, and OpenSearch.
• Strong understanding of relational databases, including tables, views, indexes, and tablespaces.
• Experience with GitHub, CI/CD processes, and Agile methodologies within the SDLC.

Preferred Skills

• Certified Data Management Professional (CDMMP).
• IBM Certified Solution Developer.
• SAS Certified Data Integration Developer or Informatica Certified Professional (ICP).

Key Skills

Education

Bachelor's degree