Description

Design and manage scalable data pipelines and large-scale data movements for the engineering team.

Responsibilities

• Build and maintain scalable data pipelines to support business needs.
• Write and optimize complex SQL queries to process and manipulate large datasets.
• Manage the export and import of large datasets using specialized utilities.
• Develop automation scripts within Unix/Linux environments to streamline operations.
• Support deployment workflows using CI/CD principles.

Required Skills

• 7+ years of experience in data engineering and building data pipelines.
• 5+ years of extensive experience in Python programming.
• 3+ years of extensive experience in SQL.
• 3+ years of experience with Unix/Linux shell scripting.
• 3+ years of hands-on experience with AWS services.
• Proficiency in writing complex SQL queries and managing large-scale data transfers.
• Basic knowledge of CI/CD concepts and tools.
• Bachelor's degree or equivalent graduate qualification.

Key Skills

Education

ANY GRADUATE