Description

You will design, develop, and maintain scalable data pipelines, data sets, and systems for the data infrastructure.

Responsibilities

• Develop ETL processes to transform raw data into usable formats, optimizing for cost effectiveness using partitioning, indexing, and caching.
• Design and implement data models to support business analytical and reporting needs.
• Ensure data integrity, consistency, and performance, identifying and resolving errors and performance issues.
• Perform data quality checks to verify data accuracy and consistency across systems.
• Document data pipelines, processes, and best practices for knowledge sharing.

Required Skills

• 5+ years of progressive experience in data engineering or a related field.
• Proficiency in SQL.
• Experience with ETL processes.
• Working knowledge of Python.
• Familiarity with cloud platforms such as AWS or Azure.
• Experience with big data tools like Apache (Spark, Kafka, Airflow) or Google Big Query.
• Experience with database systems including Oracle or SQL Server.
• Proficiency in version control using Git.
• Experience with UNIX environments.

Key Skills

Education

Any Graduate