Description

Lead the design and implementation of end-to-end data pipelines and scalable cloud-native storage solutions.

This role is on-site.

Responsibilities

• Design and implement end-to-end data pipelines using Python and Airflow for complex workflow orchestration and dependency management.
• Automate ETL processes using Python scripting and reusable Airflow DAG components.
• Develop and optimize high-performance SQL queries for data extraction, transformation, and reporting across large datasets.
• Integrate data from multiple sources including Alteryx, PostgreSQL, Oracle, and PL/SQL into unified data models.
• Mentor junior data engineers and oversee their technical output while ensuring data security and compliance.

Required Skills

• 12+ years of experience in data engineering or related roles.
• Expertise in Python, SQL, Pandas, and Numpy.
• Proven experience with Airflow for DAG creation, orchestration, and workflow management.
• Hands-on experience with AWS services including S3, Glue, Redshift, Aurora RDS, and CloudWatch.
• Experience with data integration from PostgreSQL, Oracle, and PL/SQL.
• Ability to design, monitor, and scale data pipelines.
• Bachelor's degree in a STEM-based field or any graduate degree.

Preferred Skills

• Experience with Databricks in an AWS environment.
• Background in building analytics tools for business performance metrics.

Key Skills

Education

Any Graduate