Description

You will own the end-to-end development and maintenance of our data infrastructure.

Responsibilities

• Design, develop, and maintain ETL/ELT data pipelines.
• Build real-time and batch data processing systems.
• Implement data ingestion from various sources including APIs, databases, files, and streaming data.
• Optimize data pipeline performance and reliability.
• Design and implement scalable data architecture solutions.

Required Skills

• 5+ years of experience in data engineering.
• Proficiency in Python and SQL.
• Experience with Apache Spark and Kafka.
• Hands-on knowledge of Azure Data Factory and Synapse.
• Familiarity with data warehousing and data lake setup.
• Experience implementing data governance and security practices.
• Proficiency with Git and GitLab.
• Familiarity with Docker.

Key Skills

Education

Any Gradute