Description

You will lead the design and implementation of large-scale data architecture and technology roadmaps using Databricks.

Responsibilities

• Architect Databricks-specific solutions for streaming and batch use cases, including Delta Lake and cloud data warehouse migrations.
• Design Azure Data Factory processes for SQL and NoSQL data pipelines and manage Azure SQL database objects.
• Develop Spark tasks for data aggregation and transformation using Scala, Python, and Java.
• Perform root cause analysis on performance bottlenecks, data discrepancies, and system defects.
• Guide development teams on architectural approaches, data strategy, and legacy modernization.

Required Skills

• 8+ years of experience with Azure Databricks, Azure Data Factory, and PySpark.
• 5 years of experience with deployment and builds using Azure DevOps.
• 3 years of experience using Python to extract data from unstructured files like XML, PDF, and HTML.
• Proficiency in Scala, Python, Java, and SQL.
• Experience with cloud modernization, data migration, and data warehousing.
• Hands-on knowledge of data security, messaging patterns, ELT, and data wrangling.
• Experience working in an Agile development environment.
• Strong technical documentation and communication skills for stakeholder engagement.

Preferred Skills

• Databricks certification.
• Experience driving technical workshops with business and technical clients.

Key Skills

Education

Bachelor's degree