Description

You will build and maintain scalable data pipelines and ETL processes within an Azure environment. You will design and optimize data architectures using Azure ML and Databricks to consolidate diverse data sources into a unified platform.

Responsibilities

• Design and optimize data architectures and workflows using Azure ML and Databricks.
• Implement integration solutions to consolidate diverse data sources into a unified platform.
• Analyze solar generation patterns to help field teams identify anomalies.
• Monitor and troubleshoot pipelines to ensure reliability, performance, and data quality.
• Collaborate with data scientists and stakeholders to define and meet data requirements.

Required Skills

• 10+ years of experience in data engineering, data integration, and ETL processes.
• Extensive hands-on experience with Azure ML and Databricks.
• Proficiency in Python, SQL, and Scala.
• Experience with big data technologies including Spark, Hadoop, and Kafka.
• Strong understanding of data engineering principles and best practices.
• Ability to convey complex technical information to non-technical audiences.
• Bachelor’s degree in Computer Science, Data Engineering, or a related field.

Key Skills

Education

Any Graduate