Description

You will design and maintain scalable data pipelines and ETL processes to support analytics and data science initiatives.

This role is on-site.

Responsibilities

• Design and maintain scalable data pipelines and ETL processes for analytics and reporting.
• Automate workflows for data ingestion, transformation, and delivery using orchestration tools and CI/CD pipelines.
• Manage big data environments using Azure, Databricks, and Azure Data Factory.
• Implement monitoring and observability to identify and resolve production issues.
• Collaborate with cross-functional teams through code reviews to ensure maintainable, high-quality solutions.

Required Skills

• 5+ years of relevant experience or a Master's degree in a related field.
• Proficiency in SQL, Python, and PySpark.
• Hands-on experience with Azure, Databricks, and Azure Data Factory.
• Experience building production-grade data pipelines with monitoring and troubleshooting.
• Proficiency with Git, code reviews (PRs), and enforcing coding standards.
• Experience working in Agile environments.
• Hands-on experience with CI/CD pipelines for automated deployment and testing.
• Experience with orchestration tools like Airflow or dbt.
• Experience with containerization using Docker or Kubernetes.

Preferred Skills

• Deep understanding of Azure cloud architecture and management tools.
• Knowledge of MLOps best practices for deploying and monitoring machine learning models.

Key Skills

Education

Bachelor's degree