Description

You will design and implement end-to-end data processing solutions, focusing on event-driven data warehouse architecture and ETL development.

Responsibilities

• Architect and develop event-driven data warehouses using streaming and batch processing.
• Write and troubleshoot Python and PySpark scripts for data extraction, cleansing, and delivery.
• Implement ETL architectures including data profiling, process flow, and error handling.
• Optimize SQL queries, stored procedures, and data partitioning to improve performance.
• Lead junior engineers through code reviews and optimization training.

Required Skills

• 10+ years of software or data development experience.
• Expertise in data warehouse design, dimensional modeling, and ETL development.
• Advanced proficiency in Python, PySpark, and Java.
• Extensive experience with AWS services including Kinesis, Glue, EMR, S3, Lambda, and Athena.
• Hands-on experience designing applications with the Java Spring Framework (Spring Boot, Spring Cloud, Spring Data).
• Proficiency with streaming services like Confluent Kafka and Kinesis.
• Strong SQL and NoSQL scripting skills.

Preferred Skills

• Experience with Snowflake, SQL Server, DynamoDB, or Neo4j.

Key Skills

Education

Any Graduate