Description

You will design and manage scalable data pipelines using batch and stream processing architectures.

This role is on-site.

Responsibilities

• Build and maintain data pipelines using PySpark for batch and stream processing.
• Implement event-driven programming patterns using Kafka and distributed parallel processing.
• Manage containerized workloads using Docker and Kubernetes.
• Own the deployment lifecycle through CI/CD pipelines.
• Apply SRE and platform engineering mindsets to ensure application scalability and reliability.

Required Skills

• 5 to 10 years of experience in Data Engineering.
• Proficiency in Python and PySpark.
• Extensive experience with Kafka and Spark streaming.
• Hands-on experience with AWS cloud services.
• Working knowledge of SQL and NoSQL databases, specifically DynamoDB.
• Experience with containerization using Docker and Kubernetes.
• Familiarity with CI/CD tools such as Jenkins or GitHub Actions.
• Strong analytical, debugging, and problem-solving skills.

Preferred Skills

• Experience in MLOps.

Key Skills

Education

Not specified