Description

You will build and maintain production data pipelines and manage large-scale data transformations.

This role is on-site.

Responsibilities

• Build production data pipelines using Python, Spark, Hive, and Hadoop.
• Design schemas and implement data modeling strategies.
• Perform performance tuning for data transformations across large datasets.
• Collaborate with business and technology partners to assess data needs and prioritize tool adoption.
• Modernize existing ETL frameworks and processes through automation.

Required Skills

• 10+ years of experience in software development using a major language.
• 10+ years of experience in data engineering.
• 3+ years of experience with Python packages, specifically PySpark.
• 3 to 5 years of practical experience with relational databases, preferably Oracle or Oracle Exadata.
• Strong SQL skills and deep knowledge of relational databases.
• Hands-on experience with Spark, Hive, and Hadoop.
• Experience working within Agile methodologies.
• Proficiency with Git, Jira, and Confluence.
• Ability to write maintainable, testable code with a software engineering mindset.

Preferred Skills

• BS/MS in Computer Science, Engineering, or a quantitative discipline.
• Experience with Airflow and cloud or distributed computing.
• Knowledge of financial concepts and BI platforms like MicroStrategy or Tableau.

Key Skills

Education

Any Graduate