Description

You will build and maintain data pipelines to enable analytics across structured and unstructured data sources.

Responsibilities

• Convert business requirements into technical designs for data ingestion, preparation, and transformation.
• Develop ETL/ELT processes and real-time streaming applications.
• Manage data movement from various sources into data warehouses and data lakes using batch and streaming methods.
• Debug production failures and implement technical solutions.
• Contribute to DevOps pipelines and processes.

Required Skills

• 5+ years of work experience in data engineering.
• Proficiency in Spark, Scala, and Shell scripting.
• Hands-on experience with GCP, BigQuery, Composer, and Airflow.
• Experience with real-time streaming and Kafka.
• Proficiency in Python and ETL development.
• Working knowledge of SQL and any DBMS.
• Understanding of Data Warehouse concepts and the Data Management life cycle.
• Experience with DevOps processes and Agile methodologies.
• Bachelor's degree or equivalent work experience.

Preferred Skills

• GCP Data Engineer certification.
• ETL/ELT developer certification.

Key Skills

Education

Bachelor's degree