Description

You will design and implement data pipelines to integrate disparate sources into a single Data Lakehouse.

Responsibilities

• Build data quality pipelines to ensure correctness and create trusted datasets.
• Implement Data Lakehouse solutions that accurately reflect business operations.
• Perform data platform tuning, including physical data modeling, partitioning, and compaction.
• Guide data visualization and reporting efforts to align technical solutions with business objectives.

Required Skills

• 9+ years of professional experience.
• 5+ years designing and maintaining data pipeline architectures.
• 5+ years of programming in Python, ANSI SQL, PLSQL, and TSQL.
• Experience with ETL, ELT, Pub/Sub, and Change Data Capture patterns.
• Proficiency with Python packages including pandas, Numpy, Pyarrow, Pytest, Scikit-Learn, and Boto3.
• Experience implementing Data Lakehouse using Apache Iceberg or Delta Lake.
• Knowledge of Apache Airflow, Kubernetes, and S3 Object Storage.
• Solid grasp of software development practices including Design Principles, Testing, Refactoring, CI/CD, and version control.

Preferred Skills

• Experience with Dremio and Airbyte.

Key Skills

Education

Any Graduate