Description

You will design and document data engineering and analytics solutions following modern data architecture principles.

Responsibilities

• Design data flows, build data strategies, and document end-to-end processes.
• Develop and deploy data pipelines for both streaming and batch data sources.
• Set up Databricks environments in Azure, including Spark clusters and scalable data engineering jobs.
• Implement data zones in Delta Lake and manage fine-grained access control.
• Guide team members and ensure architectural compliance through oversight.

Required Skills

• 5+ years of experience in data architecture or data engineering.
• Proficiency in Scala, Python, or Java.
• Hands-on experience with Azure Databricks and Apache Spark.
• Expertise in building pipelines using Structured Streaming, DataFrame API, and SQL.
• Experience with Azure Data Lake, Azure Data Factory, and Delta Lake.
• Working knowledge of Kafka and NoSQL databases.
• Experience with Azure services including Key Vault, Functions, Log Analytics, and VNET.
• Proficiency in SQL for designing database tables and schemas.
• Competency with Git, APIs, Notebooks, and Terraform.

Preferred Skills

• Familiarity with Azure Synapse, Azure Purview, Azure NSG, and data catalog tools.
• Knowledge of Data Streaming Architecture.

Key Skills

Education

Any Graduate