Description

You will build and maintain data pipelines using Azure Databricks and Spark.

Responsibilities

• Develop ETL processes and data workflows using Python and Spark.
• Conduct design and code reviews to ensure quality and adherence to standards.
• Perform unit and end-to-end testing to maintain application integrity.
• Manage offshore team tasks and review their code contributions.
• Review requirements and provide accurate work effort estimations.
• Collaborate with client stakeholders to define technical requirements.

Required Skills

• 5+ years of experience in data development.
• Proficiency with Azure Databricks and Spark.
• Strong programming skills in Python.
• Expertise in SQL and MS SQL Server.
• Deep understanding of ETL concepts and relational databases.
• Experience following application development documentation best practices.
• Strong written and oral communication skills for stakeholder interaction.
• Insurance domain knowledge.
• Degree in any graduate field.

Preferred Skills

• Familiarity with Azure Data Factory (ADF) frameworks.
• Experience working with XML and JSON file formats.
• Experience with Agile methodologies and JIRA.

Key Skills

Education

Any Graduate