Description

Design and implement large-scale data processing frameworks and data-intensive product features as a technical lead.

This role is on-site.

Responsibilities

• Build high-scale data frameworks using Apache Spark, Apache Hive, and AWS Glue.
• Develop ETL pipelines with Apache Airflow to integrate diverse data sources and sinks.
• Architect new product features and drive tool adoption across product domains.
• Contribute code for AI module feature engineering using Java, Scala, or C++.
• Translate data requirements into detailed technical designs for development and testing.

Required Skills

• 5+ years of data engineering experience with a Graduate degree, or 7+ years with a Bachelor's degree.
• Proficiency in Java, Scala, or C++.
• Experience with Apache Spark (batch and stream-processing), Kafka, and Hive.
• Hands-on experience with AWS services including Glue, Athena, Lambda, EMR, and RDS.
• Strong SQL and NoSQL skills using MySQL, Postgres, or DynamoDB.
• Experience building ETL pipelines with Apache Airflow and managing metadata.
• Proficiency in Python and Unix/Linux shells.
• Experience designing and consuming RESTful API services.
• Ability to communicate technical risks to QA and non-technical stakeholders.

Preferred Skills

• Knowledge of healthcare data and insurance claims.
• Experience using Apache Zeppelin or Presto for data validation.

Key Skills

Education

Bachelor's degree in Computer Science