Description

You will design and maintain scalable big data solutions, applying SRE principles to ensure system reliability. You will own the development of ETL processes and data pipelines while collaborating with data scientists and product managers to solve complex business problems.

Responsibilities

• Design and maintain scalable, high-performance big data solutions.
• Develop and implement ETL processes for large datasets.
• Perform data analysis and profiling to ensure data quality and integrity.
• Conduct code reviews and mentor junior team members.
• Collaborate with cross-functional teams to deliver data-driven solutions.

Required Skills

• 3+ years of experience in big data engineering.
• Strong proficiency in SQL, Hive, and Spark.
• Hands-on experience with ETL processes and large data sets.
• Proficiency in Unix shell scripting and shell scripts.
• Understanding of data modeling and data warehousing concepts.
• Ability to work in shifts as required.
• BSc in Computer Science or equivalent education.

Preferred Skills

• Experience with Pig, Java, Go, NiFi, and PySpark.
• Knowledge of distributed computing, the Hadoop ecosystem, and NoSQL databases.
• Experience working on AMEX projects or within the FSS domain.

Key Skills

Education

BSc in Computer Science