Description

You will own the design, implementation, and optimization of scalable data pipelines and ETL processes. You will ensure data reliability and accessibility for large-scale analysis.

Responsibilities

• Design and implement scalable data pipelines and ETL workflows using Java, Python, and Spark.
• Manage and optimize Spark clusters to guarantee performance and reliability.
• Develop and maintain data models, schemas, and data quality validation checks.
• Build and integrate REST APIs to provide seamless data access and consumption.
• Monitor pipeline performance to identify and resolve bottlenecks.

Required Skills

• 3+ years of professional experience in data engineering.
• Strong programming expertise in Java and Python.
• Hands-on experience with Spark, including cluster management and performance tuning.
• Proficiency in SQL and relational databases such as Snowflake and Delta Tables.
• Experience developing REST APIs using Flask or Spring.
• Familiarity with Azure cloud-based data platforms.
• Solid understanding of data science concepts and data warehousing.

Preferred Skills

• Experience with BigQuery.

Key Skills

Education

Any Graduate