Description

Lead the development of production data pipelines and manage large-scale data transformations.

Responsibilities

• Build production data pipelines using Python and Spark frameworks.
• Design schemas and implement data modeling strategies.
• Tune performance for data transformations across large datasets.
• Write maintainable, testable code following software engineering best practices.
• Operate within an Agile methodology to deliver software.

Required Skills

• 10+ years of software development experience using Python.
• 5+ years of experience with PySpark and Airflow.
• 3 to 5 years of practical experience with relational databases, specifically Oracle or Oracle Exadata.
• Strong SQL skills and deep knowledge of relational database systems.
• Hands-on experience with Spark, Hive, and Hadoop.
• Proven expertise in data modeling and schema design.
• Experience managing and querying data stores such as Hadoop.
• Ability to optimize performance for large-scale data processing.

Preferred Skills

• Experience working specifically with Oracle Exadata environments.

Key Skills

Education

Any Graduate