Description

You will own the development and optimization of complex data pipelines and SQL workflows, focusing on DB2, Hive, and Spark.

This role is on-site.

Responsibilities

• Develop and maintain complex SQL queries and stored procedures with a focus on DB2.
• Design and implement batch data processing pipelines using Apache Spark for scalable performance.
• Perform data cleanup, transformation, and validation on large datasets to ensure quality and integrity.
• Troubleshoot performance issues and optimize ETL/data workflows.
• Collaborate with cross-functional teams to define and deliver data requirements.

Required Skills

• 10+ years of experience in data engineering or related fields.
• Strong expertise in SQL and Stored Procedures; DB2 is mandatory.
• Hands-on experience with DB2 in large-scale data handling environments.
• Working knowledge of Hive and the Hadoop ecosystem.
• Experience with Apache Spark (PySpark/Scala preferred) for data processing.
• Solid understanding of batch processing frameworks.
• Proficiency in data cleansing, transformation, and validation.

Preferred Skills

• Experience with ETL tools such as Informatica or DataStage.
• Exposure to cloud platforms including AWS, Azure, or GCP.

Key Skills

Education

Bachelor's degree