Description

You will build and automate large-scale data processing systems for batch and streaming workflows.

Responsibilities

• Develop scalable Spark data pipelines using the Airflow scheduler and executor framework.
• Design data models optimized for storage, retrieval, and specific business requirements.
• Architect logging best practices to support and influence data flow.
• Contribute to shared engineering tooling and standards to improve team productivity.
• Improve data quality by utilizing and enhancing internal automated detection tools.

Required Skills

• 8+ years of relevant industry experience.
• Bachelor’s or Master’s degree in Computer Science or equivalent practical experience.
• Strong proficiency in SQL and relational database query authoring.
• Experience designing and deploying high-performance systems with reliable monitoring.
• Proven ability to analyze large datasets to identify gaps, inconsistencies, and insights.
• Experience implementing robust logging practices within data systems.
• Strong written and verbal communication skills.

Key Skills

Education

ANY GRADUATE