Description

You will lead technical efforts to build and enhance a data ingestion and entity resolution platform.

This role is hybrid.

Responsibilities

• Design and maintain batch and real-time data pipelines for production systems.
• Build efficient storage and search functions for structured and unstructured data.
• Manage large, complex datasets from diverse sources using distributed systems.
• Develop user-driven prototypes to evaluate technical options and architectural approaches.
• Collaborate with cross-functional teams to implement data and infrastructure architecture.

Required Skills

• Minimum 4 years of experience in large-scale data processing and instrumentation.
• Strong coding proficiency in Java, Python, or Scala.
• Hands-on experience with Spark, Kafka, Hadoop, Iceberg, and Airflow.
• Practical knowledge of relational and NoSQL databases including DB2 and MongoDB.
• Experience working with data formats such as Parquet, CSV, JSON, and XML.
• Deep understanding of data modeling, columnar databases, and distributed systems.
• Solid grasp of software engineering principles, including OOP, functional programming, and design patterns.
• Proven ability to explain technical concepts to non-technical stakeholders.

Preferred Skills

• Experience with Databricks, Elasticsearch, or Snowflake.
• Familiarity with the Quantexa platform for entity resolution.

Key Skills

Education

Any Gradute