Description

You will design and implement scalable data pipelines and architectures within a distributed environment.

This role is on-site.

Responsibilities

• Develop batch and streaming data pipelines using Spark, Python, and Scala.
• Architect data products utilizing streaming, serverless, and microservices patterns.
• Configure Jenkins CI/CD pipelines for Docker image builds and managed Spark jobs.
• Design data models and implement data warehousing, governance, and architecture standards.
• Collaborate with product and engineering teams to influence technical direction.

Required Skills

• 7+ years of experience in data engineering.
• Proficiency with Microsoft Azure, AWS, or GCP.
• Hands-on experience with Spark, PySpark, and Databricks (including Delta Lake).
• Expertise in SQL, Python, and Scala.
• Experience with Airflow, Kafka, and Hadoop.
• Working knowledge of EMR, EC2, and Snowflake.
• Experience with Presto and SQL databases.
• Ability to build and manage Docker images and CI/CD pipelines.

Preferred Skills

• Experience architecting serverless and microservices-based data platforms.

Key Skills

Education

Any Graduate