Description

Lead a cross-functional Agile team to build and enhance a data ingestion and entity resolution platform.

This role is on-site.

Responsibilities

• Lead development of software for data ingestion and entity resolution.
• Architect application, data, and infrastructure solutions for large, complex datasets.
• Design and implement batch and real-time data pipelines for production systems.
• Build efficient storage and search functions for structured and unstructured data.
• Develop rapid user-driven prototypes to evaluate technical and architectural approaches.

Required Skills

• 4+ years of management experience in large-scale data processing and instrumentation.
• Strong coding proficiency in Java, Python, or Scala.
• Experience with big data ecosystems: Spark, Kafka, Hadoop, Airflow.
• Hands-on experience with Databricks, Delta Lake, Elasticsearch, or Snowflake.
• Proficiency with relational and NoSQL databases (DB2, MongoDB).
• Working knowledge of data formats: Parquet, CSV, JSON, XML.
• Experience with Iceberg, data modeling, and columnar databases.

Preferred Skills

• Experience with the Quantexa platform for entity resolution.
• Experience sourcing client reference data from vendors.

Key Skills

Education

Any Gradute