Description

You will build and maintain large-scale data pipelines and lakehouse architectures handling terabytes to petabytes of data.

Responsibilities

• Manage the end-to-end data lifecycle from ingestion and transformation to the consumption layer.
• Design and build microservices to support data-driven applications.
• Implement real-time data ingestion using APIs, telemetry, and logs.
• Collaborate with business stakeholders to document requirements and design technical solutions.

Required Skills

• 8+ years of experience in Data Engineering.
• 3+ years of experience working within large-scale Data Lake ecosystems.
• Expertise in Databricks and Spark (Python/SQL).
• Proficiency in Python, SQL, and Scala.
• Experience with Kafka, Airflow, and Hive.
• Strong understanding of Data Lake/Lakehouse architecture.
• Ability to manipulate JSON and XML data formats.
• Experience with API development and usability.
• Knowledge of ML Ops techniques, including supervised and unsupervised learning.

Preferred Skills

• Experience with predictive and regressive analysis.
• Background in data mining and implementing traditional classifiers.

Key Skills

Education

Any gradudate