Description

You will build and maintain scalable data pipelines and API integrations to handle increasing data complexity.

Responsibilities

• Develop scalable data pipelines and new API integrations to support growing data volumes.
• Design and implement data models, Spark, SparkSQL, and HiveSQL jobs to populate data assets.
• Build data quality frameworks and monitoring systems to ensure production data accuracy and availability.
• Design data integrations and evaluate open source or vendor tools for data lineage.
• Collaborate with frontend/backend engineers, product managers, and analysts to define long-term data platform architecture.
• Write unit and integration tests, maintain engineering documentation, and troubleshoot data-related issues.

Required Skills

• 8-10 years of Python and Java development experience.
• 4+ years of SQL experience.
• 4+ years of experience with schema design and dimensional data modeling.
• Experience designing, building, and maintaining data processing systems.
• Experience working with MapReduce or an MPP system at scale.
• Proficiency with AWS, PySpark, and SQL.
• Experience with Agile Software Development methodologies.
• Strong troubleshooting skills and experience with IT operations for always-available services.
• Ability to manage and communicate data warehouse plans to internal clients.

Preferred Skills

• No-SQL experience.

Key Skills

Education

Any Graduate