Description

You will architect and implement real-time Data Lakehouse solutions that integrate structured and unstructured data sources.

Responsibilities

• Design and maintain Data Lakehouse architectures combining data lake flexibility with data warehouse structure.
• Build scalable ETL/ELT pipelines and distributed search systems using Trino for SQL-based querying.
• Optimize data storage, partitioning, and query performance using Apache Iceberg for schema evolution and ACID compliance.
• Implement data governance frameworks including metadata management, lineage, and role-based access control.
• Collaborate with DevOps to automate data infrastructure and monitor system throughput and latency.

Required Skills

• 5+ years of experience in data modelling, partitioning, and performance tuning.
• Hands-on experience with Apache Iceberg, Trino, Databricks, Delta Lake, or Snowflake.
• Proficiency in SQL, Python, and Scala.
• Strong experience with Apache Spark.
• Experience managing structured and semi-structured data in object storage like S3 or GCS.
• Experience with real-time data processing frameworks such as Kafka, Artemis, or Flink.
• Familiarity with data orchestration tools like Apache Airflow.
• Knowledge of machine learning pipelines and MLOps integrations.
• Experience working within Agile/Scrum teams.
• Must be eligible for up to a Top Secret Security Clearance.

Preferred Skills

• Knowledge of regulatory compliance including CCPA and CASL.

Key Skills

Education

Any Gradute