Description

You will design, build, and maintain scalable data pipelines and ETL processes to support analytics and data science initiatives.

Responsibilities

• Build and automate workflows for data ingestion, transformation, and delivery using orchestration tools and CI/CD pipelines.
• Manage big data environments using cloud platforms, specifically Azure, Databricks, and Azure Data Factory.
• Implement monitoring and observability to proactively identify and resolve production issues.
• Participate in code reviews and enforce coding standards to ensure maintainable, high-quality solutions.
• Ensure data integrity and security by applying data governance and quality management practices.

Required Skills

• Proficiency with SQL, Python, and PySpark for data engineering tasks.
• Hands-on experience with Azure, Databricks, and Azure Data Factory.
• Experience building and maintaining production-grade data pipelines in live environments.
• Proficiency with Git for version control and participating in code reviews.
• Experience with CI/CD pipelines for automating deployment and testing.
• Experience using orchestration tools such as Airflow or dbt.
• Experience with containerization using Docker or Kubernetes.
• Knowledge of Agile methodologies and DataOps concepts.
• Bachelor's degree in Computer Science, Engineering, Information Systems, or a related field.

Preferred Skills

• Deep understanding of Azure cloud architecture and management tools.
• Knowledge of MLOps best practices for deploying and monitoring machine learning models.
• Experience supporting data governance and data-driven business transformation initiatives.

Key Skills

Education

Bachelor's degree