Description

job responsibilities include:

Design and develop data processing and ingestion solutions using Hadoop and related technologies (HDFS, MapReduce, Hive, Impala, PySpark) or Teradata.

Write scalable, robust, and high-performance code for data processing and analysis.

Build and manage data processing workflows and pipelines for batch and real-time data.

Develop and maintain middleware services using Python.

Create and query GraphQL-based queries.

Write optimized SQL queries considering large data sizes (up to 30-30TB).

Work with Unix shell scripting for process automation and optimization.

Collaborate on performance improvements and identify optimization opportunities.

Develop REST APIs using Python and integrate with various data systems.

Work with Graph databases and Autosys for job scheduling and automation.

Follow secure coding practices and risk management protocols.

Contribute to Agile development practices and testing concepts.

Desired Qualifications:

Bachelor’s or master’s degree in computer science, Engineering, or a related field.

3+ years of experience in Big Data engineering or a related role.

Expertise in Hadoop ecosystem tools (HDFS, MapReduce, Hive, Impala, PySpark) and Teradata.

Proficiency in Python for middleware services and developing REST APIs.

Must-Have:

We need big data experience and Teradata experience along with autopsy and unix shell scripting

Key Skills

Education

Any Graduate