Description

You will design, build, and maintain data pipelines and ETL processes using Python. You own the lifecycle of data extraction, transformation, and loading from diverse sources. You optimize data models for performance and scalability while ensuring data quality and consistency. You maintain technical documentation for all pipelines and processes.

Responsibilities

• Design and develop efficient, reliable data pipelines and ETL processes.
• Extract, transform, and load data from diverse sources and technologies.
• Optimize data models and architectures for performance and scalability.
• Ensure data quality and consistency across all data sources.
• Maintain comprehensive documentation for all data pipelines and processes.

Required Skills

• 5+ years of experience in data engineering.
• 5+ years of experience with Python programming.
• 5+ years of experience with data systems and complex query writing.
• Proficiency with relational databases.
• Bachelor's degree in Computer Science, Engineering, or a related field.
• Experience with ETL tools such as ADF or NIFI.

Preferred Skills

• Experience with Azure.
• Experience with NoSQL databases.
• Experience with NIFI.

Key Skills

Education

Any graduate