Description

You will build and manage big data pipelines within a cloud-based lakehouse environment.

This role is hybrid.

Responsibilities

• Design and define big data architectures for large-scale data processing.
• Develop and maintain data integration workflows using SQL and dimensional modeling.
• Implement data processing solutions using Spark and Databricks on AWS.
• Manage data ingestion and storage using Kafka, S3, and EMR.
• Write and optimize scripts in Python and Shell within Linux environments.

Required Skills

• 5+ years of experience in big data engineering.
• Expertise in SQL for large-scale data warehouses or lakehouse environments.
• Hands-on experience with Spark and Databricks.
• Proficiency with Presto, Athena, or Trino.
• Experience with AWS services including S3, EC2, and EMR.
• Practical knowledge of Kafka for data streaming.
• Strong command of Python and Shell scripting in Linux.

Preferred Skills

• Experience with Scala, Java, or stream processing for near real-time ingestion.
• Knowledge of Data Science, Machine Learning, or graph databases.

Key Skills

Education

Any Graduate