Description

You will design and build big data systems using PySpark, Python, and Scala, focusing on high-performance batch and real-time processing.

This role is on-site.

Responsibilities

• Develop and maintain scalable data pipelines using PySpark, Hadoop, and Hive for batch and real-time processing.
• Refactor unstructured or legacy codebases into optimized, maintainable engineering standards.
• Apply supervised and unsupervised machine learning techniques for data preprocessing and application engineering.
• Design software systems with intense computational needs, handling large-scale datasets and data warehouses.
• Collaborate with stakeholders to articulate complex business problems and deliver structured technical solutions.

Required Skills

• Strong proficiency in Python and Scala programming languages.
• Hands-on experience with PySpark, Hadoop, Hive, and SQL.
• Experience with Unix scripting and data warehousing concepts.
• Ability to work with unstructured data and re-engineer costly compute processes.
• Understanding of machine learning techniques and data preprocessing.
• Bachelor's degree or equivalent experience with 5+ years in big data systems.

Preferred Skills

• Experience in Credit Cards and Retail Banking domains.
• Familiarity with Jenkins, AutoSys, and Service Now.

Key Skills

Education

Any Graduate