Description

You will design, build, and maintain data pipelines and technical solutions within an Agile environment.

This role is on-site.

Responsibilities

• Build and maintain data movement processes using Python, PySpark, and AWS.
• Migrate data sets from S3 to OneLake and integrate vendor data into Snowflake.
• Collaborate with Agile teams to implement distributed microservices and full-stack solutions.
• Perform unit testing and code reviews to ensure performance and design quality.
• Support the migration of external vendor applications into the internal environment.

Required Skills

• 8+ years of professional experience in data engineering or application development.
• Expertise in Python and PySpark.
• Hands-on experience with AWS, specifically EMR and Glue.
• 3+ years of experience with distributed computing tools such as MapReduce, Hadoop, Hive, or EMR.
• 2+ years of experience with NoSQL implementations like MongoDB or Cassandra.
• 2+ years of experience with data warehousing using Redshift or Snowflake.
• 2+ years of experience with real-time data and streaming applications.
• 3+ years of experience with UNIX/Linux, including shell scripting.
• Proficiency in SQL, Scala, or Java.

Preferred Skills

• Experience with Microsoft Azure or Google Cloud.
• Previous client-side experience.

Key Skills

Education

Any Graduate