Description

You will design and maintain scalable, automated data pipelines for diverse data types, including text, imagery, logs, and geospatial data.

Responsibilities

• Transform, clean, and enrich data specifically for machine learning and AI model training.
• Collaborate with cross-functional teams to define data requirements aligned with business and scientific goals.
• Improve data quality, observability, and lineage tracking across sustainability data assets.
• Optimize pipeline performance and scalability using AWS-native technologies.
• Support ML workflows by enabling reproducible data ingestion, transformation, and feature engineering.

Required Skills

• 5+ years of professional experience as a Data Engineer.
• Proficiency in Python and SQL for data manipulation and automation.
• Experience with data pipeline frameworks like Apache Airflow, Spark, or AWS Glue.
• Expertise in PySpark and Pandas for data processing.
• Strong working knowledge of AWS services including S3, Glue, Lambda, and Athena.
• High-level understanding of common ML/AI techniques and data modeling for ML workflows.
• Knowledge of data quality, observability, and governance frameworks.

Key Skills

Education

Any Graduate