Description

• Design, develop, and maintain scalable ETL/ELT data pipelines using Python, DBT, and Apache Airflow.
• Build and manage Airflow DAGs for workflow orchestration, scheduling, monitoring, and dependency management.
• Develop reusable and optimized DBT models, snapshots, tests, and macros for data transformation.
• Implement data quality checks, validation frameworks, and reconciliation processes across data pipelines.
• Develop and optimize Python scripts for data ingestion, transformation, automation, and exception handling.
• Work with structured and semi-structured data formats such as JSON, CSV, and Parquet.
• Integrate data from multiple sources into enterprise data warehouses and analytics platforms.
• Monitor, troubleshoot, and resolve data pipeline failures and performance bottlenecks.
• Collaborate with Data Architects, Analysts, and Business stakeholders to understand and deliver data requirements.
• Ensure adherence to best practices around coding standards, version control, testing, and deployment.
• Participate in production support, incident resolution, and root cause analysis activities.

Qualification
Mandatory Skills

• 5+ years of experience in Data Engineering.
• Strong hands-on experience with DBT (Data Build Tool).
• Strong experience with Apache Airflow including DAG development, scheduling, monitoring, and workflow orchestration.
• Proficiency in Python for ETL development, automation, and data processing.
• Strong SQL skills with experience in writing complex queries and performance tuning.
• Experience developing and supporting enterprise-scale ETL/ELT pipelines.
• Knowledge of data modeling concepts including Star Schema and Dimensional Modeling.
• Experience with Git-based version control systems.

Preferred Skills

• Experience with Snowflake, Redshift, BigQuery, or other cloud data warehouses.
• Experience with AWS services such as S3, Glue, Lambda, or EMR.
• Exposure to CI/CD pipelines and DevOps practices.
• Experience working in Agile/Scrum environments.
• Familiarity with data quality, governance, and observability frameworks.

Nice to Have

• Snowflake Streams, Tasks, Snowpipe.
• DBT Cloud.
• PySpark or Spark.
• Terraform or Infrastructure as Code.
• Monitoring and alerting tools.
• Bachelor's degree in Computer Science, Information Technology, Engineering, or related field.

Key Skills

• DBT
• Apache Airflow
• Python
• SQL
• ETL
• ELT
• Data Engineering
• Snowflake
• AWS
• Data Warehousing
• Data Modeling
• CI/CD
• Git
• Cloud Data Platforms

Key Skills

Education

Bachelor's degree