Description

You will build and maintain scalable data pipelines to ingest, process, and transform large-scale datasets.

Responsibilities

• Design, develop, and maintain data pipelines for large-scale data processing.
• Implement real-time data streaming solutions using Kafka or similar eventing technologies.
• Optimize pipelines for performance, scalability, and reliability.
• Collaborate with data scientists and analysts to implement data requirements.
• Troubleshoot pipeline issues and ensure data quality and consistency.

Required Skills

• 3+ years of experience in data engineering.
• Proficiency in Azure Databricks, SQL, and PySpark.
• Experience with real-time data streaming using Kafka or eventing solutions.
• Strong understanding of data modeling, ETL processes, and data warehousing.
• Familiarity with cloud platforms including Azure, AWS, or GCP.
• Bachelor's degree in Computer Science, Engineering, or a related field.
• Proven problem-solving skills and attention to detail.

Key Skills

Education

Bachelor's degree