Description

Lead technical efforts to build and enhance data ingestion and entity resolution platforms within a cross-functional Agile team.

This role is on-site.

Responsibilities

• Lead development of software for data ingestion and entity resolution.
• Design efficient storage and search functions for structured and unstructured data.
• Build batch and real-time data pipelines for production systems.
• Develop user-driven prototypes to evaluate technical options and architectural approaches.
• Work with large, complex datasets from diverse sources using distributed systems.

Required Skills

• 4+ years of experience in management with a focus on large-scale data processing.
• Strong coding proficiency in Python, Java, or Scala.
• Experience with big data products such as Databricks, Elasticsearch, or Snowflake.
• Hands-on experience with Kafka, Spark, Hadoop, Iceberg, or Airflow.
• Proficiency in data modeling and working with relational and NoSQL databases like DB2 and MongoDB.
• Experience handling data formats including Parquet, CSV, JSON, XML, and relational data.
• Deep understanding of software engineering principles, including OOP, functional programming, and design patterns.
• Knowledge of large-scale distributed systems.

Preferred Skills

• Experience with Databricks/Delta Lake.
• Experience with REST-based applications.
• Experience with client reference data sourcing from vendors.

Key Skills

Education

Any Gradute