Description

Design and maintain scalable data pipelines for structured and unstructured data ingestion, processing, and storage.

Responsibilities

• Design and implement ETL processes to transform raw data into analysis-ready formats.
• Develop end-to-end solutions for data ingestion, cleaning, and aggregation.
• Optimize pipelines for performance and reliability based on data volume and velocity.
• Monitor and troubleshoot pipeline issues to ensure minimal workflow disruption.
• Apply data governance principles to maintain data quality, integrity, and security.

Required Skills

• 11+ years of IT experience.
• Hands-on experience with Azure Databricks and Azure Data Factory.
• Proficiency in PySpark and Python.
• Experience handling structured and semi-structured data sources including JSON and XML.
• Ability to provide end-to-end ETL solutions.
• Understanding of Snowflake, data modeling, schema design, and data governance.
• Any Graduate degree.

Preferred Skills

• Experience in Snowflake data engineering.
• Basic knowledge of Java.

Key Skills

Education

Any Gradute