Description

Lead the design and implementation of scalable data engineering solutions and hybrid cloud architectures.

This role is on-site.

Responsibilities

• Build, maintain, and optimize data pipelines as workloads transition from development to production.
• Design enterprise-level architectures for cloud and on-premise platforms.
• Execute change tasks including security provisioning, schema creation, DB role creation, and DDL execution.
• Monitor Azure Synapse Analytics and optimize performance through table distribution, indexing, and partitioning.
• Manage data migration projects, specifically moving workloads from AWS EMR to Databricks.

Required Skills

• 15+ years of relevant experience in data engineering.
• Expertise with Azure, Databricks, Apache Spark, and Python.
• Experience implementing large-scale hybrid cloud platforms and applications.
• Deep knowledge of ETL technologies and various ETL tools.
• Hands-on experience with AWS EMR (Elastic MapReduce).
• Proficiency in scripting languages and CI/CD workflows.
• Ability to whiteboard enterprise architectures and create star schema models in Dedicated SQL Pools.
• Experience with Scrum management for medium to large-scale projects.

Preferred Skills

• Broad knowledge of multiple cloud platforms and their basic utilities.

Key Skills

Education

Bachelor's or Master's degree