Description

Design, develop, and maintain data pipelines for ingesting real-time claims data into cloud environments.

Responsibilities

• Design and implement scalable, secure data solutions and ETL workflows to migrate data from on-premise Hadoop sources to Azure.
• Manage the full data engineering lifecycle, including research, proof of concepts, development, testing, and deployment.
• Build diagnostic tools and processes to troubleshoot, maintain, and optimize petabyte-scale data solutions.
• Develop CI/CD workflows on Azure to support continuous deployment for data projects.
• Maintain technical documentation regarding data definitions, transformations, and processes to ensure governance.

Required Skills

• 5+ years of experience working with Azure cloud.
• Proficiency in Python or Scala.
• Hands-on experience with Azure Databricks and Azure Data Factory (ADF).
• Experience with Kafka for real-time data integration.
• Experience building UI via React or Angular.
• Knowledge of CI/CD workflows.
• Ability to migrate SSIS workflows into cloud environments.

Preferred Skills

• Experience with large-scale petabyte data processing.

Key Skills

Education

ANY GRADUATE