Description

Key Skills: Python, Apache Airflow, ETL/ELT, dbt Core, SQL, Git, CI/CD, Data Pipelines, Data Management, Automated Testing

Good to Have Skills: Experience with Google Cloud Platform (GCP), Google Cloud Storage (GCS), Cloud Composer, and Kafka or other event-driven data streaming tools for enhanced data processing capabilities.

Roles & Responsibilities:

• Design, build, and maintain scalable ETL/ELT pipelines using Python and Apache Airflow for enterprise data processing.
• Own the dbt Core transformation layer, including models, tests, documentation, and deployment across the data platform.
• Manage the full data life cycle across ingestion, transformation, quality, storage, and retention processes.
• Define data management standards for data quality, cataloging, and lineage to ensure governance compliance.
• Administer and optimize the Airflow environment, including Cloud Composer where applicable for workflow management.
• Collaborate with data architects, analysts, and cross-functional teams to deliver reliable data pipelines.
• Conduct comprehensive design and code reviews to maintain high quality standards across the team.
• Mentor junior data engineers and provide guidance on best practices and technical development.
• Troubleshoot and resolve production data pipeline issues to ensure continuous data availability and reliability.

Experience Required: 4+ years of experience in data engineering with strong understanding of data management principles and end-to-end data life cycle. Experience with relational or cloud data warehouse platforms, Agile development methodologies, and writing automated tests for data pipelines and dbt models required

Key Skills

Education

Any Graduate