Description

You will design and build cloud-based big data pipelines within a scrum environment. You will own the technical direction and release processes for critical data infrastructure.

Responsibilities

• Design and architect cloud-based big data pipelines.
• Build and maintain data processing workflows using Spark and Scala.
• Manage deployment pipelines and CI/CD processes.
• Support the production stack and collaborate with SRE teams.
• Lead scrum team initiatives and technical direction.

Required Skills

• 10+ years of overall IT experience.
• 3+ years of experience with Scala.
• 3+ years of experience with Python.
• 5+ years of experience with AWS services including S3, EMR, SNS/SQS, EventBridge, Lambda, CloudWatch, MSK, Glue, and container services.
• Experience with Spark, Spark streaming, and Hadoop.
• 3+ years of experience in Architecture and Design of cloud-based big data pipelines.
• 2+ years of experience with Infrastructure as Code, specifically Terraform.
• Experience with SQL, Shell scripting, and orchestration via Airflow or MWAA.
• Experience with Git, CI/CD, and DevOps practices.
• Knowledge of Healthcare Domain and Data Standards.

Preferred Skills

• Azure experience.
• Big Data and/or Cloud certifications.

Key Skills

Education

Any Graduate