Description

You will design and maintain data infrastructure and ETL processes to integrate diverse data sources.

Responsibilities

• Design, develop, and maintain ETL processes to integrate data from various sources.
• Write efficient, scalable Python scripts to automate data processing tasks.
• Build and manage data pipelines using Apache Airflow, AWS Glue, or similar tools.
• Collaborate with software engineers to integrate data solutions using TypeScript.
• Optimize and tune data systems for performance and scalability while ensuring data quality and integrity.

Required Skills

• 5+ years of experience in data engineering roles.
• Advanced proficiency in Python.
• Hands-on experience with PySpark.
• Strong understanding of ETL design and implementation.
• Experience with stakeholder management across different organizational levels.
• Excellent written and oral communication skills.

Preferred Skills

• Experience with big data technologies including Hadoop, Spark, or Kafka.

Key Skills

Education

Any Graduate