Description

You will design and build scalable data pipelines and ETL processes while managing Spark clusters for performance and reliability.

Responsibilities

• Design and implement scalable data pipelines and ETL processes using Java and Python.
• Manage and optimize Spark clusters for performance and reliability.
• Develop and maintain data models, schemas, and quality validation checks.
• Build and integrate REST APIs for seamless data access.
• Mentor junior team members and provide technical leadership.

Required Skills

• 5+ years of professional experience in data engineering.
• Expertise in Java and Python for building scalable code.
• Deep experience with Spark, including cluster management and performance tuning.
• Proficiency in SQL and relational databases such as Snowflake and Delta Tables.
• Experience developing REST APIs using frameworks like Flask or Spring.
• Familiarity with cloud-based data platforms, specifically Azure.
• Strong understanding of data science concepts and data modeling.

Preferred Skills

• Experience with data warehousing tools like BigQuery.

Key Skills

Education

Any Graduate