Description

You will design, implement, and maintain data lakehouse solutions that integrate data storage and analytics.

Responsibilities

• Implement lakehouse architecture to integrate and optimize data storage and analytics processes.
• Develop ETL pipelines for efficient data ingestion, transformation, and loading from various sources.
• Optimize query performance by analyzing and tuning data workflows to minimize latency.
• Implement data governance policies to ensure compliance and protect sensitive data.
• Collaborate with data teams to gather requirements and deliver solutions that enhance data accessibility.

Required Skills

• 6+ years of overall IT experience.
• 4+ years of experience working with Apache Iceberg, Databricks, or Delta Lake.
• Proficiency in Python scripting and PySpark for data processing.
• Strong SQL capabilities and hands-on experience managing big data using ETL tools.
• Experience with the AWS cloud platform and its data services.
• Proficiency in BASH/Shell scripting.
• Experience working within Agile methodologies.

Preferred Skills

• Experience with Kafka and MuleSoft API.
• Understanding of healthcare data systems.

Key Skills

Education

Any Graduate