Description

You will design and build data ingestion and processing pipelines to move data from SAP Hana into an Amazon S3 data lake and Redshift warehouse.

Responsibilities

• Design data ingestion pipelines using Apache NiFi, Amazon MSK, EMR, and PySpark to transfer data from Hana to S3.
• Build processing pipelines to move data from S3 into Amazon Redshift.
• Develop the data visualization layer by building datasets in Amazon QuickSight and publishing dashboards.
• Use Python and JSON to integrate with Spark and manage EMR cluster metadata for pipeline automation.
• Coordinate daily with business teams, technical PMs, and offshore AWS teams.

Required Skills

• 5+ years of experience in data architecture or engineering.
• Proficiency with AWS Cloud tools including S3, Athena, Redshift, Glue, EMR, and Data Pipeline.
• Hands-on experience with Apache NiFi, Kafka, and Amazon MSK.
• Strong programming skills in Python, Scala, and PySpark.
• Experience with SAP, ERP systems, or BW/HANA analytics.
• Working knowledge of JSON and converting ORC to Parquet formats.
• Experience using JIRA for project tracking.
• Ability to work in Plano, TX.

Preferred Skills

• Experience with AWS Cloud9.

Key Skills

Education

Any Graduate