Description

Design and implement scalable Azure-based data engineering and analytics solutions.

Responsibilities

• Design high-performing analytics solutions on Azure following well-architected framework guidelines.
• Build pipelines to ingest and integrate structured and unstructured data from heterogeneous sources.
• Optimize code and troubleshoot PySpark notebooks to ensure performance.
• Provide hands-on technical leadership across multiple client projects.
• Collaborate with delivery leadership to ensure projects meet quality standards and timelines.

Required Skills

• Expert-level hands-on experience using Databricks across at least 4 projects.
• Proficiency in creating Databricks notebooks using Scala or Python.
• Expertise in PySpark and Spark SQL for diverse data processing use cases.
• Hands-on experience with Azure storage services including ADLS, SQL DB, Synapse, Delta tables, and Hive.
• Strong understanding of Delta Lake integration with ADLS and other storage layers.
• Experience with Databricks scheduling features for job automation.
• Experience with at least one big data project using MapReduce.
• Ability to work with any relevant Azure Analytics services.

Preferred Skills

• Databricks Certified Data Engineer or Microsoft DP-203 certification.
• Knowledge of Azure DevOps, HDInsight, and Power BI.
• Exposure to Databricks on AWS.

Key Skills

Education

Any graduate