Description

You will design and build scalable data infrastructure to handle complex data workflows.

This role is on-site.

Responsibilities

• Define data requirements and wrangle large volumes of structured and unstructured data.
• Build and maintain data pipelines that clean, transform, and aggregate data from disparate sources.
• Develop mechanisms to ingest, analyze, validate, normalize, and standardize data.
• Create data policies and interfaces, including synthesizing or anonymizing data.
• Implement statistical data quality procedures to support Data Scientists.

Required Skills

• 7+ years of overall IT experience.
• 5+ years in a data engineering or ETL role manipulating large datasets.
• 3+ years with Big Data tools: Hadoop, Spark, Spark SQL, Kafka, Sqoop, Hive, S3, HDFS, or Cloud platforms (AWS, GCP).
• 3+ years building, testing, and optimizing data ingestion pipelines using Tibco, IBM, or similar technologies.
• Experience with Databricks UI, managing Databricks Notebooks, Delta Lake (Python/Spark SQL), Delta Live Tables, and Unity Catalog.
• High-velocity, high-volume stream processing using Apache Kafka and Spark Streaming.

Key Skills

Education

Any Gradute