Description

You will lead the development, validation, and migration of Big Data products and applications within Hadoop and Teradata clusters.

Responsibilities

• Lead the design, development, and testing of data ingestion pipelines and end-to-end ETL processes.
• Develop validation frameworks and automated test suites to ensure data integrity and transformation accuracy.
• Perform data migration and conversion validation across different applications and platforms.
• Provide technical leadership for data profiling, analysis, and the identification of data quality issues.
• Collaborate with architects and product managers to establish physical application frameworks, including libraries and modules.
• Develop tools to measure data quality and visualize anomaly patterns in source and processed data.

Required Skills

• 15+ years of experience in the Client’s Data Domain with a Data Architecture background.
• 10+ years of software development and testing experience.
• Expertise in Spark, Scala code development, and SQL within Hadoop environments.
• Proven experience developing and testing ETL and real-time data-processing systems.
• Experience with CI/CD environments and automated framework development.
• Strong understanding of software development practices and tools.
• Bachelor's degree in Computer Science or Engineering.

Preferred Skills

• Experience with Google Cloud Platform (GCP).
• Experience developing tools and utilities for monitoring and alerting.
• Familiarity with JIRA or similar project management and bug tracking tools.

Key Skills

Education

Bachelor's degree