Description

Design and maintain scalable data pipelines within a secure environment. You will own the data flow from ingestion to analysis, ensuring integrity and confidentiality.

Responsibilities

• Design and maintain scalable data pipelines using Hadoop and EMR.
• Implement data processing tasks using Python.
• Write and optimize SQL queries for complex data transformation and analysis.
• Collaborate with data scientists and engineering teams to ensure seamless data flow.
• Identify and report security risks regarding data confidentiality, integrity, and availability.

Required Skills

• 5+ years of experience in data engineering.
• Proficiency in Python for data processing.
• Extensive experience with Hadoop and EMR.
• Strong SQL skills for complex transformations.
• Understanding of data privacy, protection, and security measures.
• Bachelor's degree in a relevant field.
• Strong problem-solving skills and attention to detail.

Preferred Skills

• Familiarity with Kafka.

Key Skills

Education

Bachelor's degree