Description

You will design and build data integration and processing systems to support business objectives and data science workflows.

Responsibilities

• Analyze data sources and APIs to design methods for data collection, filtering, and cleansing.
• Develop SQL, Hive queries, and APIs to extract data from storage.
• Collaborate with data scientists to ensure source data is properly aggregated and cleansed.
• Partner with cloud architects and DevOps to define cloud architectures and build automated data pipelines.
• Work with product managers to align technical implementation with business requirements.

Required Skills

• 5 to 10 years of relevant data engineering experience.
• Proficiency with Big Data technologies including Hadoop, Hive, and Apache Spark.
• Strong programming skills in Python and Java.
• Experience with NoSQL databases and REST APIs.
• Working knowledge of GIT and version control.
• Experience in client-facing roles with strong communication skills.
• Domain knowledge in finance, banking systems, fraud, or payments.
• Experience working within SCRUM methodology and distributed teams.

Preferred Skills

• Familiarity with HTTP and invoking web APIs.
• Exposure to machine learning engineering, NLP, and text processing.
• Experience with pipelines, job scheduling, and workflow management.

Key Skills

Education

Any Graduate