Description

Design, build, and maintain scalable data pipelines for massive financial datasets.

This role is on-site.

Responsibilities

• Design and maintain scalable data pipelines using Snowflake, Hadoop, Spark, NiFi, and Databricks.
• Implement data architectures with governance, metadata management, and lineage tracking.
• Develop automated testing frameworks and validation scripts for ETL processes and data transformations.
• Collaborate with cross-functional teams to embed testing into the development lifecycle and support incident response.
• Mentor junior engineers and improve technical standards for data reliability and CI/CD integration.

Required Skills

• 15+ years of experience in data engineering and software development.
• Advanced SQL and experience with relational and NoSQL databases.
• Strong expertise in Snowflake, Hadoop, Spark, Databricks, and Kafka.
• Proficiency in Python for data engineering and test automation.
• Experience with orchestration tools, workflow management, and CI/CD pipelines.
• Proven experience in data testing methodologies, ETL validation, and automated testing frameworks.
• Knowledge of data profiling, anomaly detection, and statistical validation techniques.

Preferred Skills

• Experience with agile ceremonies and rapid data validation.
• Background in financial datasets.

Key Skills

Education

Any Gradute