Description

You will be a technical leader driving the data engineering function using big data processing frameworks.

Responsibilities

• Create and optimize data engineering pipelines for analytics projects.
• Support data and cloud transformation initiatives.
• Contribute to the cloud strategy based on technical experience.
• Develop and maintain automated ETL processes focusing on data flow, error recovery, and exception handling.
• Partner with stakeholders to define data requirements and build high-quality data ingestion and processing systems.

Required Skills

• 5+ years of experience in Big Data with an engineering background.
• Advanced experience with Apache Spark processing and Scala/Python programming.
• Advanced SQL skills, including performance tuning.
• Experience writing Spark streaming jobs integrated with Apache Kafka.
• Proficiency in Shell Scripting.
• Hands-on experience with AWS services: EMR, Glue, S3, Athena, IAM, Lambda, and CloudWatch.
• Experience using external job schedulers like Airflow.
• Bachelor’s degree required.

Key Skills

Education

Bachelor’s degree