Description

You will architect, implement, and manage scalable data platforms to support advanced analytics and machine learning projects.

Responsibilities

• Design data solutions on Databricks, including Delta Lake, Data Warehouse, and Data Mart architectures.
• Build and implement scalable data pipelines using Spark, Kafka, Flink, and AWS Glue to ingest and process diverse datasets.
• Develop and optimize data models across SQL, NoSQL, and Data Lake storage solutions.
• Automate data workflows and deployment pipelines using Apache Airflow, Terraform, and CI/CD frameworks.
• Lead data projects, mentor junior engineers, and collaborate with data scientists and ML engineers to provide necessary infrastructure.

Required Skills

• 5-8 years of hands-on experience in data engineering and designing large-scale architectures.
• Proven expertise in Databricks, Delta Lake, Lakehouse, and Unity Catalog.
• Deep experience with AWS cloud services including EC2, RDS, S3, Athena, Glue, Lambda, EMR, Kinesis, and SQS.
• Strong proficiency with big data technologies such as Apache Spark and Kafka.
• Experience with streaming and ETL/ELT development.
• Ability to use Python, Scala, or R for data processing scripts.
• Knowledge of Apache Airflow and Talend for workflow automation.
• Experience implementing data governance, encryption, and access controls in cloud environments.
• Degree in Computer Science, Data Science, Engineering, Mathematics, Statistics, or a related quantitative discipline.

Key Skills

Education

Any Graduate