Description

You will build and maintain data pipelines using Spark and various Big Data frameworks.

Responsibilities

• Write complex SQL queries for data analytics and modeling.
• Develop and test data components using Spark, Python, or Scala.
• Manage data workflows using AWS services including Glue, S3, Athena, and EMR.
• Perform data analysis and component packing deployment within cross-functional teams.
• Automate tasks using Windows PowerShell scripting.

Required Skills

• 5+ years of experience with SQL, including complex queries and data modeling.
• 4+ years of experience with Spark.
• 3+ years of experience with modern programming languages such as Python or Scala.
• Experience with the Hadoop ecosystem, including Hive, Sqoop, and HDFS.
• Hands-on experience with Teradata and Unix environments.
• Proficiency with AWS services: Glue, S3, Athena, and EMR.
• Experience with Windows PowerShell scripting.

Preferred Skills

• Bachelor’s degree in Computer Science Engineering, a related discipline, or equivalent experience.

Key Skills

Education

Bachelor’s degree in Computer Science Engineering or related discipline or equivalent.