Description

You will own the design and implementation of scalable data infrastructure.

Responsibilities

• Design and build data pipelines using Apache Spark or equivalent big data frameworks.
• Implement CDC and deduplication logic using event timestamp strategies and relevant connectors.
• Build scalable data models and Lakehouse architectures, optimizing for performance.
• Manage streaming and messaging systems, handling producers, consumers, and topic integration.
• Apply DevOps practices for automation, including Git and CI/CD workflows.

Required Skills

• 10+ years of professional experience in data engineering.
• Expertise in Python, Scala, or Java for data processing.
• Hands-on experience with Apache Kafka, Azure Event Hub, RabbitMQ, or Confluent Cloud.
• Strong proficiency in SQL and relational databases like SQL Server or Oracle.
• Experience building pipelines using Apache Spark.
• Familiarity with CDC implementation.
• Experience with cloud platforms, preferably Azure.
• Proficiency in Git and CI/CD practices.

Key Skills

Education

Any Gradute