Description

You will design and maintain data pipeline architectures and Lakehouse solutions.

Responsibilities

• Design and implement reliable data pipelines to integrate disparate sources into a single Data Lakehouse.
• Build data quality pipelines to ensure correctness and create trusted datasets.
• Implement a Data Lakehouse solution that accurately reflects business operations.
• Perform data platform tuning, including physical data modeling, partitioning, and compaction.
• Guide data visualization and reporting efforts to align with business objectives.

Required Skills

• 5+ years of experience as a Data Engineer designing and maintaining pipeline architectures.
• 5+ years of programming experience in Python and ANSI SQL.
• 2+ years of development experience with DBT (Data Build Tool).
• Experience implementing a Data Lakehouse using Medallion Architecture with Apache Iceberg on S3 object storage.
• Experience with data modeling methods including Star Schema, Snowflake, and Data Vault.
• Proficiency with Python packages: Pandas, Numpy, Pyarrow, Pytest, Scikit-Learn, and Boto3.
• Knowledge of data integration patterns such as ELT, Pub/Sub, and Change Data Capture.
• Solid understanding of software engineering practices: Design Principles, Testing, Refactoring, CI/CD, and version control.

Preferred Skills

• Experience with Dremio, Apache Airflow, and Airbyte.
• Python UI development experience using Dash.
• Experience with Kubernetes, AWS EKS, and general AWS Cloud services.

Key Skills

Education

Any Gradute