Description

You will build and manage data pipelines to process and move data across various systems.

This role is on-site.

Responsibilities

• Design, document, and integrate production and post-production tools with various departments.
• Develop event-driven data pipelines that trigger, validate, and move data to different S3 buckets.
• Standardize the vendor onboarding process to streamline data ingestion using approved cloud tools.
• Automate data validations across landing, raw, archive, and curated data layers in AWS.
• Maintain the Data Quality Framework, checking data quality rules and alerting stakeholders via email reports.

Required Skills

• 2+ years of experience developing ETL/ELT processes.
• Proficiency in AWS Glue and Redshift.
• Strong skills in PySpark, Python, and SQL.
• Working knowledge of AWS S3, SNS, and SQS.
• Familiarity with workflow tools like Jira and Code Pipeline.
• Experience managing data validation frameworks.
• Bachelor's or Master's degree in a related technical field.

Key Skills

Education

Any Graduate