Description

You will design, build, and optimize large-scale data pipelines for AML analytics and reporting.

Responsibilities

• Develop and maintain scalable data pipelines and ETL workflows using PySpark.
• Manage data warehousing and query-based analysis with Hive and Impala.
• Perform operational troubleshooting and scripting on Linux/Unix environments.
• Ensure data quality, lineage, and timely availability for compliance teams.
• Tune distributed computing performance and data partitioning on Hadoop.

Required Skills

• 10+ years of experience in data engineering.
• Proficiency in PySpark for scalable data processing.
• Strong understanding of Hive and Impala for data warehousing.
• Linux/Unix scripting and operational troubleshooting.
• Solid grasp of distributed computing, data partitioning, and Hadoop performance tuning.
• Experience with version control (Git) and CI/CD pipelines.
• Familiarity with workflow orchestration tools like Apache Airflow.

Preferred Skills

• Exposure to the ELK stack (Elasticsearch, Logstash, Kibana).
• Experience with MongoDB for semi-structured data storage.
• Functional knowledge of Banking domain and AML processes.

Key Skills

Education

Any Gradute