Description

You will design and build data pipelines and infrastructure.

Responsibilities

• Develop and optimize data processing workflows using Python and PySpark.
• Manage and query large datasets within data warehousing solutions (Redshift or Snowflake).
• Implement and maintain distributed computing solutions using tools like Spark and Hadoop.
• Work with streaming data ingestion using Kafka.

Required Skills

• 6+ years of professional experience.
• Bachelor's degree.
• 4+ years using Python, PySpark, and SQL.
• 4+ years experience with data warehousing (Redshift or Snowflake).
• 2+ years experience with distributed computing tools (MapReduce, Hadoop, Hive, EMR, Kafka, Spark).
• 4+ years experience on AWS.
• 2+ years experience with NoSQL databases.
• 2+ years experience with Databricks.

Key Skills

Education

Bachelor's degree