Description

You will design and implement scalable data solutions within the Azure environment.

This role is remote.

Responsibilities

• Design and implement data pipelines using Azure Databricks and Azure Data Factory.
• Develop and maintain ETL/ELT workflows utilizing Apache Spark and PySpark DataFrames.
• Optimize data pipelines for efficient ingestion, transformation, and processing of large datasets.
• Implement data quality frameworks and manage data lineage across platforms.
• Apply Master Data Management (MDM) concepts to maintain data consistency and integrity.

Required Skills

• 5+ years of experience in data engineering.
• Proficiency with Azure Databricks.
• Strong command of Apache Spark and PySpark.
• Experience with Azure Data Factory.
• Proficiency in SQL.
• Strong background in Data Pipelines and ETL/ELT development.
• Knowledge of Lakehouse Architecture principles.
• Familiarity with Data Governance and Data Lineage practices.

Key Skills

Education

Any Gradute