Description

Core Responsibilities:

• Design and maintain scalable data pipelines using Databricks, Spark (PySpark/Scala), Delta Lake, and Databricks Workflows
• Build and optimize Azure Data Lake Storage Gen2 lakehouse solutions
• Implement data quality frameworks and governance policies
• Troubleshoot and optimize Databricks/Spark performance bottlenecks
• Support analytics and reporting teams with curated datasets
• Automate ingestion, transformation, and deployment processes
• Participate in Agile delivery and technical design discussions
• Mentor junior engineers and provide technical leadership
• Ensure security, compliance, SDLC, and ITSM best practices are followed

Required Experience:

• 5+ years of Data Engineering experience
• 5+ years working with Data Warehousing, Data Lakes, or Big Data platforms
• 5+ years of Python experience
• 3+ years of Databricks experience
• Strong SQL and Microsoft SQL Server expertise
• Experience within Azure environments

Preferred Certifications:

• Microsoft Certified: Azure Data Engineer
• Databricks Certified Data Engineer Associate
• Databricks Certified Data Engineer Professional

Key Technologies:

• Databricks
• Spark / PySpark / Scala
• Delta Lake
• Azure Data Lake Storage Gen2
• Python
• SQL Server
• CI/CD
• Data Governance
• Data Quality Frameworks
• Agile

Soft Skills / Environment:

• Strong communication and presentation skills
• Ability to work independently and cross-functionally
• Strategic mindset with strong problem-solving ability
• Comfortable in fast-paced enterprise environments
• Mentorship and leadership capabilities preferred

Key Skills

Education

Bachelor's degree