Description

You will build and maintain the data infrastructure supporting analytics for oil and gas operations.

Responsibilities

• Design and implement reliable data pipelines to integrate disparate data sources into a single Data Lakehouse.
• Design and implement data quality pipelines to ensure data correctness and build trusted datasets.
• Design and implement the Data Lakehouse solution reflecting core business operations.
• Assist with data platform performance tuning and physical data model support, including partitioning and compaction.
• Provide guidance on data visualizations and reporting to align solutions with business objectives.

Required Skills

• 5+ years of experience as a Data Engineer designing and maintaining data pipeline architectures.
• 5+ years of programming experience in Python and ANSI SQL.
• 2+ years of development experience with DBT (Data Build Tool).
• Experience implementing a Data Lakehouse using a Medallion Architecture with Apache Iceberg on S3 Object Storage.
• Experience in various data integration patterns including ELT, Pub/Sub, and Change Data Capture.
• Experience with common Python Data Engineering packages including Pandas, Numpy, Pyarrow, Pytest, Scikit-Learn, and Boto3.
• Experience in software development practices such as Design Principles and Patterns, Testing, Refactoring, CI/CD, and version control.
• Familiarity with Data Analysis and Data Warehouse concepts.
• Proficiency with Performance Tuning techniques.

Key Skills

Education

Any Graduate