Description

Design, implement, and maintain scalable data platforms and orchestration workflows.

Responsibilities

• Design and develop ETL data pipelines using AWS and Apache Airflow to automate data extraction and loading into Snowflake.
• Build reusable Directed Acyclic Graphs (DAGs) in Airflow, including dynamic DAG generation and custom operator development.
• Develop data transformation pipelines using DBT to support analytics and reporting requirements.
• Implement and maintain CI/CD pipelines using Terraform, CloudFormation, and Jenkins for automated deployment.
• Optimize data pipelines, SQL queries, and AWS infrastructure for performance, scalability, and cost-effectiveness.

Required Skills

• 6+ years of experience in data engineering roles.
• Proficiency in Apache Airflow for orchestrating complex workflows and managing task dependencies.
• Advanced SQL skills and hands-on experience with Snowflake features including Snowpipes, SnowTasks, and RBAC.
• Experience developing reusable scripts and automation tasks using Python.
• Strong knowledge of DBT for data transformation and modeling.
• Experience with AWS cloud infrastructure management.
• Practical experience with CI/CD tools such as Jenkins and Terraform.
• Proficiency with version control systems like Git.
• Bachelor's degree in Computer Science, Engineering, or a related field.

Preferred Skills

• SnowPro certification.

Key Skills

Education

Bachelor's degree