Description

You will build and maintain big data applications for processing large banking datasets.

Responsibilities

• Develop scalable data pipelines using Spark to ingest, transform, and load data from relational databases, message queues, and log files.
• Implement data cleansing, transformation, and feature engineering using Spark SQL and MLlib.
• Optimize Spark jobs for performance and efficiency in collaboration with other engineers.
• Translate analytical requirements from data scientists into production-ready Spark applications.
• Write clean, maintainable, and well-documented code while participating in regular code reviews.

Required Skills

• 6+ years of software development experience with a focus on big data technologies.
• Proven experience developing applications using Apache Spark and Scala.
• Strong understanding of distributed computing frameworks and functional programming.
• Proficiency with relational databases including MySQL and PostgreSQL.
• Solid grasp of SQL querying, data pipelines, and data warehousing concepts.
• Experience with AWS, Azure, or GCP.
• Knowledge of Docker, Kubernetes, and CI/CD pipelines.
• Any Graduate degree.

Preferred Skills

• Experience with data ingestion tools like Kafka or Flume.
• Understanding of the banking domain.
• Knowledge of machine learning libraries such as TensorFlow.

Key Skills

Education

Any Graduate