Description

You will own the design, development, and operation of core data pipelines.

Responsibilities

• Develop and maintain PySpark ETL pipelines for batch and incremental processing.
• Build and operate AWS Glue Spark jobs, managing configuration, scaling, retrieval, and cost optimization.
• Design event-driven data workflows triggered by S3, EventBridge, or streaming sources.
• Load and transform data into Amazon Redshift, optimizing distribution and sort keys for query performance.
• Implement dimensional data models (star/snowflake), including SCDs and data quality controls.

Required Skills

• 12+ years of professional experience in data engineering.
• Strong PySpark proficiency (Spark SQL, DataFrame operations, performance tuning).
• Hands-on experience building and operating AWS Glue Spark jobs.
• Proven experience loading and optimizing data within Amazon Redshift.
• Expertise in designing dimensional data warehouse schemas (Fact/Dimension tables).
• Familiarity with core AWS data services (S3, IAM, CloudWatch).
• Experience managing Glue Catalog and schema definitions.

Preferred Skills

• Strong production ownership mindset for debugging and failure recovery.

Key Skills

Education

Any Gradute