Description

Build and optimize big data solutions within cloud environments, focusing on ELT pipelines and performance tuning.

This role is on-site.

Responsibilities

• Create and maintain ELT data pipelines using Python scripts and Airflow DAGs.
• Optimize pipeline performance by configuring Spark system resources.
• Debug existing Microsoft BI ETLs to translate business logic into big data pipelines via Airflow.
• Collaborate with business partners to define requirements and provide technical solutions.
• Maintain overlap with onsite teams until 11 am Pacific Time for US-based scrum ceremonies.

Required Skills

• 3-5 years of experience implementing Big Data solutions in cloud environments.
• Hands-on experience with Airflow DAG, Spark, Python, Hive, Trino, and S3 buckets.
• Experience with Microsoft BI ETLs.
• Proficiency with Agile methodologies.
• Ability to work an offshore schedule with morning overlap for US-based scrum ceremonies.
• Any Graduate degree.

Preferred Skills

• Experience with Snowflake or Databricks.
• Knowledge of the financial domain, specifically the credit card industry.

Key Skills

Education

ANY GRADUATE