Description

You will design, implement, and optimize core data platform components including Snowflake, Databricks, and AWS S3 Data Lakes.

This role is on-site.

Responsibilities

• Manage and scale high-throughput streaming systems using Kafka or Kinesis to support real-time telemetry and low-latency analytics.
• Optimize Spark and PySpark applications for performance and cost-efficiency across large-scale distributed clusters.
• Develop Data-as-a-Service (DaaS) features to enable Business Units to onboard data and manage access with minimal friction.
• Design and automate granular access control models to ensure data security and compliance with global regulations like GDPR and CCPA.
• Ensure seamless data flow and interoperability between disparate platforms utilizing technologies like Apache Iceberg to maintain a Single Source of Truth.

Required Skills

• 6+ years of professional experience in data platform or distributed systems engineering in high-volume environments.
• Mastery of SQL and Python with expert knowledge of big data processing frameworks like Spark.
• Deep proficiency in AWS services and extensive experience with Snowflake or Databricks.
• Experience with Apache Airflow for orchestration.
• Strong background in Data Modeling and distributed systems.
• Knowledge of Kafka for real-time data ingestion.

Preferred Skills

• Experience with Infrastructure as Code (Terraform) and containerization (Docker, Kubernetes).
• Familiarity with data observability tools (Monte Carlo, DataHub, Soda) and Vector Databases (Pinecone, Weaviate, Milvus).
• Experience with LLM data tooling and preparation for fine-tuning.

Key Skills

Education

Any Graduate