← Back to jobs

InnoMethods Corporation Logo
Data Engineer / Ab Initio Consultant

InnoMethods Corporation

 

Berkeley Heights, NJ, USA

Posted On: 7 days ago
Experience: 5+ years
Availability: Onsite
Openings: 1
Category: Data Engineer Consultant
Tenure: No Preference/Any
Related Jobs

No related jobs found

Description

You will design and implement high-performance, event-driven data pipelines for a large credit card processing system, ensuring low-latency processing and high availability.

This role is on-site.

Responsibilities

  • Create complex Ab Initio continuous flow graphs with stateful joins, sliding time windows, and aggregations.
  • Implement event-driven pipelines using Kafka, message queues, and file streams for near-real-time updates.
  • Ensure resilience of continuous flows through checkpointing and recovery to guarantee exactly-once processing.
  • Develop and monitor ETL pipelines for batch and real-time data ingestion, validation, and transformation.
  • Troubleshoot performance issues in large-scale clusters and maintain 24/7 production reliability.

Required Skills

  • Deep knowledge of Ab Initio ecosystem: GDE, EME, Conduct>It, and Continuous Flows.
  • Proficiency in Unix/Linux shell scripting, SQL, and database technologies (Oracle, Teradata).
  • Experience with Kafka, message queues, and real-time stateful services.
  • Understanding of data security, governance, and compliance standards.
  • Experience with cloud-based big data platforms and containerization (Docker, Kubernetes).
  • Knowledge of CI/CD pipelines for data engineering projects.
  • 5+ years of experience in data engineering or related field.

Preferred Skills

  • Proficiency in Java, Scala, or Python.
  • Experience with business process modeling and documentation tools.

Education

Any Graduate

Related Jobs

No related jobs found

← Back to jobs