Description

You will design and implement cloud data pipelines, focusing on AWS and Databricks. You will optimize complex SQL queries and build real-time streams using Spark.

Responsibilities

• Design and implement data pipelines within a cloud environment.
• Develop and migrate data solutions specifically within AWS.
• Build and implement data pipelines using Databricks or similar cloud databases.
• Write complex, highly optimized SQL queries to process large volumes of data.
• Develop real-time data streams using Spark.

Required Skills

• 6+ years of professional experience designing and implementing cloud data pipelines.
• 5+ years of experience developing data solutions in AWS.
• 2+ years of experience building pipelines with Databricks or equivalent cloud databases.
• Expert-level SQL for complex query optimization on large datasets.
• Hands-on object-oriented programming with Python.
• Professional experience building real-time data streams using Spark.
• Proficiency with PySpark.
• Knowledge of architectural best practices for building data lakes.
• Bachelor's degree.

Key Skills

Education

Bachelor's degree