Description

You will own the stability and resolution of streaming and batch data pipelines on Databricks and AWS.

This role is remote.

Responsibilities

• Respond to PagerDuty and Datadog alerts, triaging incidents efficiently.
• Debug and resolve failures across Databricks/Spark, Kafka, Delta Lake, and PostgreSQL.
• Execute operational runbooks, apply configuration fixes, and escalate with root cause analysis.
• Maintain thorough incident documentation and postmortems to reduce recurring alert noise.
• Coordinate with downstream consumers and engineering teams during production incidents.

Required Skills

• 10+ years of experience with Databricks, Spark/PySpark, Kafka, and Delta Lake.
• Strong proficiency in Python, SQL, and PostgreSQL.
• Experience with AWS services including S3, IAM, Secrets Manager, and KMS.
• Knowledge of Datadog, PagerDuty, and production observability practices.
• Ability to distinguish between transient infrastructure issues, configuration fixes, and application defects.
• Experience with Any Graduate education level.

Preferred Skills

• Experience with Apache Flink and stateful streaming systems.
• Healthcare or HIPAA domain exposure.

Key Skills

Education

Any Graduate