Description

You will design and implement data engineering solutions using the Azure ecosystem.

Responsibilities

• Build and manage data pipelines using Azure Data Factory and Databricks.
• Develop scalable data processing logic using Python, Scala, and PySpark.
• Implement automation via PowerShell scripting and manage Azure storage services including Blob and Gen2.
• Execute data validation processes to ensure data integrity and quality.

Required Skills

• 13+ years of experience in data engineering and data analytics.
• Expertise in Microsoft Azure data components, specifically Databricks and Data Factory.
• Strong proficiency in Python, SQL, Scala, and PySpark.
• Hands-on experience with PowerShell scripting.
• Deep understanding of Azure storage concepts, including Blob and Gen2.
• Ability to work with any graduate-level educational background.

Preferred Skills

• Experience with Great Expectations for data validation.
• Familiarity with DataDog or Splunk for data platform observability.
• Knowledge of Power BI and commercial, financial, or ERP processes.

Key Skills

Education

Any Graduate