Description

Architect and manage scalable data platforms to support advanced analytics and machine learning projects.

This role is hybrid.

Responsibilities

• Design data solutions on Databricks, including Delta Lake, Data Warehouse, and Data Mart architectures.
• Build scalable data pipelines using Spark, Kafka, Flink, and AWS Glue to ingest and process diverse datasets.
• Develop and optimize data models across SQL, NoSQL, and Data Lake storage solutions.
• Automate data workflows and deployment pipelines using Apache Airflow, Terraform, and CI/CD frameworks.
• Mentor junior engineers and collaborate with data scientists and ML engineers to provide necessary infrastructure.

Required Skills

• 5-8 years of hands-on experience in data engineering and designing large-scale architectures.
• Proven expertise in Databricks, Delta Lake, Lakehouse, and Unity Catalog.
• Deep experience with AWS cloud services including EC2, RDS, S3, Athena, Glue, Lambda, EMR, Kinesis, and SQS.
• Strong proficiency with big data technologies such as Apache Spark and Kafka.
• Experience with streaming and ETL/ELT development.
• Ability to use Python, Scala, or R for data processing scripts.
• Knowledge of Apache Airflow and Talend for workflow automation.
• Experience implementing data governance, encryption, and access controls in cloud environments.

Preferred Skills

• Degree in Computer Science, Data Science, Engineering, Mathematics, Statistics, or a related quantitative discipline.

Key Skills

Education

Any Graduate