Description

You will design and implement data engineering solutions within the Microsoft ecosystem, owning end-to-end Azure Synapse implementations.

Responsibilities

• Debug and fine-tune Spark notebooks to optimize memory consumption and processing performance.
• Manage large-scale data volumes (100s of GB) and ensure process compliance across assigned modules.
• Conduct feasibility studies, technical risk assessments, and architectural reviews.
• Provide technical guidance and implement coding standards and best practices for the team.

Required Skills

• 10+ years of experience in data engineering with strong Microsoft ecosystem focus.
• Proficiency in PySpark and writing Python code in Azure Synapse Notebooks.
• Hands-on experience with Azure Synapse end-to-end implementations.
• Strong expertise in SQL, Azure Data Lake, and data modeling.
• Working knowledge of Kusto for telemetry data processing.
• Experience with Spark structured streaming, specifically from Event Hubs.
• Recent experience (3-4 years) working within the Microsoft ecosystem.
• Solid understanding of data warehousing and data management principles.

Preferred Skills

• Knowledge of Azure DevOps, Azure WebApps, Azure Functions, and Azure Cosmos DB.
• Local Redmond residency preferred.

Key Skills

Education

Any Graduate