Description

Lead big data development and manage production support for the Data Management and Reporting system.

Responsibilities

• Monitor production jobs, debug failures, and resolve P1 system issues.
• Design and develop data loading strategies and shared components for cross-module use.
• Extract, stage, transform, and load data into Postgres SQL and Hive-based systems.
• Ingest data from Mainframe systems using Kafka and process large datasets with Apache Spark and Scala.
• Coordinate daily handshakes with offshore teams and provide on-call support between 5 PM and 9 PM EST, including weekend coverage.

Required Skills

• 5+ years of experience in big data environments.
• Proficiency in Scala and Apache Spark.
• Hands-on experience with Hadoop, Kafka, and Hive.
• Experience with Postgres SQL and distributed databases.
• Working knowledge of Airflow, Control M, or Oozie.
• Experience with Kubernetes or similar containerization tools.
• Strong architecture, design, and coding skills.
• Proven experience in production operations and support.
• Ability to work within Agile methodologies.

Key Skills

Education

Any Graduate