← Back to jobs

Openkyber Logo
Data Platform Engineer

Openkyber

 

Austin, TX, USA

Posted On: Just posted
Experience: 10+ years
Availability: Hybrid
Openings: 1
Category: Data Platform Engineer
Tenure: No Preference/Any
Related Jobs

No related jobs found

Description

You will architect, build, and operate the cloud-based data lakehouse platform, ensuring scalable infrastructure for analysts and downstream applications.

This role is hybrid.

Responsibilities

  • Architect and maintain the data lakehouse using Apache Iceberg, managing schema evolution, snapshots, and table compaction for performance and cost.
  • Deploy, tune, and operate Trino clusters for interactive and federated analytics, managing connectors and resource groups.
  • Build and optimize large-scale batch and streaming data processing pipelines using Apache Spark, troubleshooting shuffle bottlenecks and query plans.
  • Develop scalable data frameworks and services in Scala and Python, integrating ingestion and transformation logic with the lakehouse.
  • Design and operate platform components on Kubernetes (EKS/AKS/GKE) and provision cloud infrastructure (AWS/Azure/GCP) using IaC.

Required Skills

  • 10+ years of experience in data engineering or platform engineering.
  • Deep expertise with Apache Iceberg table formats, including ACID transactions and time-travel.
  • Strong proficiency in deploying and tuning Trino for distributed query processing.
  • Hands-on experience building large-scale data pipelines with Apache Spark.
  • Production-level coding skills in Scala and Python.
  • Experience operating containerized workloads on Kubernetes (EKS, AKS, or GKE).
  • Proficiency with cloud platforms (AWS, Azure, or GCP) and Infrastructure as Code.

Preferred Skills

  • Experience with data governance, monitoring, and observability tools for platform reliability.
  • Background in mentoring engineers and promoting software engineering best practices.

Education

Any Graduate

Related Jobs

No related jobs found

← Back to jobs