Description

Build and optimize big data pipelines, architectures, and datasets to extract value from large, disconnected data sources.

This role is on-site.

Responsibilities

• Build processes for data transformation, metadata, dependency, and workload management.
• Develop and maintain scalable data pipelines, structures, and big data stores.
• Perform root cause analysis on internal and external data to answer business questions and identify improvements.
• Manipulate and process unstructured datasets to drive analytical insights.

Required Skills

• 5+ years of experience in data engineering.
• Advanced SQL knowledge and experience with relational databases.
• Hands-on experience with Hadoop, Spark, and Kafka.
• Proficiency with AWS services including EMR, Redshift, and Athena.
• Experience with ETL tools and Informatica.
• Working knowledge of CI/CD tools.
• Experience with NoSQL databases, specifically Postgres and Cassandra.
• Experience with workflow management tools such as Azkabhan, Luigi, or Airflow.
• Experience with stream-processing systems like Storm or Spark-Streaming.

Preferred Skills

• Strong project management and organizational skills.

Key Skills

Education

Any Graduate