Description

You will design, build, and maintain scalable ETL/ELT pipelines using Python and SQL.

This role is remote.

Responsibilities

• Design and develop efficient data processing solutions and ETL/ELT pipelines.
• Implement Test-Driven Development (TDD) by writing unit tests before pipeline development.
• Manage and optimize data workflows within Azure Databricks, PostgreSQL, and MySQL.
• Ensure data quality and integrity through rigorous automated testing and validation.
• Document data models, pipelines, and test cases to ensure long-term maintainability.

Required Skills

• 8+ years of experience in data engineering.
• Strong proficiency in Python and SQL.
• Expertise in Test-Driven Development (TDD) using pytest or unittest.
• Hands-on experience with ETL/ELT processes and pipeline automation.
• Experience working with Azure Databricks.
• Knowledge of data modeling principles, including star schema and normalization.
• Proficiency with version control (Git) and CI/CD practices.
• Experience with database performance optimization.

Preferred Skills

• Experience with orchestration tools such as Airflow, Prefect, or Dagster.
• Knowledge of Azure cloud platforms.
• Exposure to containerization tools like Docker or Kubernetes.

Key Skills

Education

Any Graduate