Description

You will build and maintain scalable data engineering solutions within the Azure Databricks ecosystem.

Responsibilities

• Design and implement data pipelines using Azure Databricks and Azure Data Factory.
• Develop and maintain ETL/ELT workflows utilizing Apache Spark and PySpark DataFrames.
• Build and optimize data pipelines for ingestion, transformation, and processing of large datasets.
• Implement data quality frameworks and manage data lineage across enterprise platforms.
• Apply Master Data Management concepts to ensure data consistency and integrity.

Required Skills

• 5+ years of professional experience in data engineering.
• Expertise with Azure Databricks.
• Strong proficiency in PySpark and Apache Spark.
• Experience with SQL and building complex transformations.
• Hands-on knowledge of Azure Data Factory.
• Familiarity with Data Governance and Data Lineage principles.
• Experience implementing Lakehouse Architecture patterns.
• Proficiency in designing and developing ETL/ELT processes.

Key Skills

Education

Any Gradute