Description

You will design, build, and maintain scalable data pipelines and infrastructure to transform raw data into actionable formats.

This role is on-site.

Responsibilities

• Build and maintain optimal data pipeline architecture for extraction, transformation, and loading from diverse sources.
• Automate manual processes and redesign infrastructure to improve scalability and data delivery.
• Implement validation systems and monitoring to ensure data quality, accuracy, and availability.
• Write unit and integration tests, document technical workflows, and contribute to the engineering wiki.
• Perform root cause analysis on data processes to resolve technical issues and identify improvements.

Required Skills

• 5+ years of relevant experience in data engineering.
• Proficiency in Python, Java, or Scala.
• Strong expertise in SQL and relational/NoSQL databases, including Postgres and Cassandra.
• Experience with Big Data tools such as Hadoop, Spark, Kafka, and Hive.
• Hands-on experience with AWS services including EC2, EMR, RDS, and Redshift.
• Experience with workflow management tools like Airflow or Ozzie.
• Proficiency with Git, Jira, Jenkins, and Shell scripting.
• Knowledge of Agile methodology, test-driven development, and automated testing.

Preferred Skills

• Experience with stream-processing systems like Storm or Spark-Streaming.
• Experience working with Snowflake.

Key Skills

Education

Any Graduate