Description

You will design, build, and maintain scalable data ingestion pipelines and the underlying data infrastructure.

This role is remote.

Responsibilities

• Design and implement scalable data pipelines using Python and SQL.
• Maintain and optimize Snowflake data warehouse performance, including data ingestion and query tuning.
• Extract data from APIs using Python and AWS Lambda, and automate workflows with AWS Airflow.
• Develop checks and scripts to troubleshoot data issues and enhance data quality.
• Maintain code quality via CI/CD processes using Azure DevOps.

Required Skills

• 7+ years of experience in Data Engineering, focusing on scalable data pipelines.
• Expert-level proficiency in Snowflake, including data ingestion and performance optimization.
• Strong SQL skills for writing efficient queries and optimization.
• Proficiency in Python for data extraction and API integration.
• Experience with AWS services: Lambda, Airflow, Glue, S3, SNS.
• Familiarity with ETL/ELT processes.

Preferred Skills

• Experience with Fivetran and DBT.

Key Skills

Education

Any Graduate