Description

You will engineer and automate database processes for efficiency and scale.

Responsibilities

• Develop ETL workflows using Python and Shell Scripting.
• Build and optimize Apache Spark ETL pipelines using PySpark, Scala, or Java.
• Implement Apache Airflow for ETL orchestration and monitoring.
• Develop cloud-based ETL solutions using AWS Glue and GCP Dataflow.
• Manage and optimize data warehouses including Snowflake, Redshift, BigQuery, and HDFS.

Required Skills

• 5+ years of experience in an IT consulting, analyst, programmer, developer, or engineering role.
• Proficiency in PySpark, Scala, Java, and Python.
• Experience with Shell Scripting.
• Hands-on experience with Snowflake, Redshift, and BigQuery.
• Familiarity with AWS and Dataflow.
• Experience designing and automating database-centric workflows.

Preferred Skills

• Bachelor's degree in Computer Science or related field.
• Experience with HDFS management and optimization.

Key Skills

Education

Bachelor's degree