Description

You will build and maintain large-scale data pipelines and engineering workflows.

Responsibilities

• Build and manage complex data pipelines to handle large datasets.
• Write complex SQL queries to export and import large volumes of data using various utilities.
• Develop scripts using Python and Unix/Linux shell environments.
• Support data movement and processing within AWS environments.
• Integrate basic CI/CD practices into the data engineering lifecycle.

Required Skills

• 7+ years of experience in data engineering and data pipelines.
• 5+ years of extensive experience in Python programming.
• 3+ years of extensive experience in SQL and Unix/Linux shell scripting.
• 3+ years of hands-on experience with AWS.
• Proficiency in SQL for complex data manipulation and utility-based data transfers.
• Working knowledge of CI/CD processes.
• Any Graduate degree.

Preferred Skills

• Experience with Kafka Streams, Spark Streaming, or building data-intensive streaming applications.
• Experience with Snowflake or other Cloud Data Warehouses.
• Knowledge of Scala, Java, or distributed computing platforms.

Key Skills

Education

Any Gradute