Description
You will design and maintain data pipelines for industrial domains, including CPG and Manufacturing.
This role is on-site.
Responsibilities
- Design and develop data pipelines using Azure Data Factory, Azure Databricks, and Azure Data Lake Gen2.
- Implement ETL/ELT processes and enforce data governance standards.
- Process large-scale data using Apache Spark and PySpark for analytics readiness.
- Integrate and consolidate data from sales, inventory, and customer databases.
- Collaborate with ML Engineers and Data Scientists to enable advanced analytics solutions.
Required Skills
- 7+ years of experience in Data Engineering within industrial domains.
- Expertise in Python, SQL, and PySpark.
- Strong knowledge of Azure Data Factory and Azure Databricks.
- Proficiency in implementing Data Governance standards.
- Experience with CI/CD for data pipeline automation.
- Familiarity with AI/ML concepts and advanced analytics integration.
- Understanding of Industry 4.0 and ISA-95 standards.
- Experience with Big Data technologies.
Preferred Skills