Description

You will design, develop, and implement large-scale data engineering solutions using Databricks across cloud environments.

Responsibilities

• Design and execute end-to-end data engineering solutions focusing on large-scale processing and integration.
• Develop scalable streaming and batch solutions using cloud-native components.
• Perform data transformation tasks including cleansing, aggregation, enrichment, and normalization.
• Manage Databricks administration and operations, including notebooks, clusters, jobs, and pipelines.
• Implement DataOps principles and CI/CD workflows to optimize data environment deployments.

Required Skills

• 5+ years of experience in data engineering roles.
• Proficiency in at least two major cloud platforms: AWS, Azure, or GCP.
• Expert-level proficiency in Spark Scala, Python, and PySpark.
• Extensive experience with Databricks for data migration and cloud-based implementations.
• Hands-on experience with Kafka, StreamSets, and MapReduce.
• Deep knowledge of Spark Streaming, Spark Core, Spark SQL, and data modeling.
• Experience using Apache Spark MLlib, TensorFlow, and PyTorch.
• Bachelor’s degree in Computer Science, IT, or equivalent experience.

Preferred Skills

• Databricks Engineering (Professional) certification.
• Azure Data Engineer Associate, GCP Professional, or AWS Solutions Architect Professional certifications.

Key Skills

Education

Any Graduate