Description
You will engineer and automate database processes for efficiency and scale.
Responsibilities
- Develop ETL workflows using Python and Shell Scripting.
- Build and optimize Apache Spark ETL pipelines using PySpark, Scala, or Java.
- Implement Apache Airflow for ETL orchestration and monitoring.
- Develop cloud-based ETL solutions using AWS Glue and GCP Dataflow.
- Manage and optimize data warehouses including Snowflake, Redshift, BigQuery, and HDFS.
Required Skills
- 5+ years of experience in an IT consulting, analyst, programmer, developer, or engineering role.
- Proficiency in PySpark, Scala, Java, and Python.
- Experience with Shell Scripting.
- Hands-on experience with Snowflake, Redshift, and BigQuery.
- Familiarity with AWS and Dataflow.
- Experience designing and automating database-centric workflows.