Description

You will architect, develop, test, and deploy data lake and warehouse solutions in AWS.

Responsibilities

• Design and implement data ingestion pipelines for high-volume datasets.
• Lead data migration and transformation initiatives.
• Develop and integrate software applications using established architectural patterns.
• Automate end-to-end ETL processes across the big data ecosystem.
• Collaborate with stakeholders to define the AWS application framework.

Required Skills

• 6+ years of professional experience.
• Proficiency in SQL, Spark/PySpark, and Spark Structured Streaming.
• Hands-on experience with Spark SQL and the Java Spark Framework for large-scale processing.
• Experience designing high-throughput, low-latency pipelines using Kafka.
• Expertise in Python and Java development.
• Familiarity with Linux/RHEL 8 environments.
• Experience with REST APIs and performance tuning of Spark jobs.
• Experience with AWS EMR for distributed data processing.
• Experience with Kubernetes or similar frameworks for cluster management.

Key Skills

Education

Any Graduate