Description

You will build scalable data pipeline platforms and high-performing data models on big-data architecture. You will own the design and development of data solutions that coexist with legacy ETL and Data Warehouse applications.

Responsibilities

• Build scalable data pipelines using Hadoop and Apache Spark for real-time and batch processing.
• Design and develop data solutions using columnar databases with Parquet or ORC file formats.
• Collaborate with architects, data analysts, and infrastructure engineers to design big-data infrastructure.
• Support data management efforts across Development, QA, and Production environments.
• Partner with enterprise data teams to identify optimal data sourcing locations.

Required Skills

• 5+ years of experience in ETL programming.
• Proficiency in Hadoop ecosystem tools, including Apache Flume and Apache Sqoop.
• Hands-on experience with Apache Spark design and development using Scala, Java, Python, or Data Frames with RDDs.
• Strong programming skills in Java, Scala, or Python.
• Advanced SQL knowledge.
• Background in design and development of columnar databases using Parquet or ORC.
• Experience working within Agile methodologies.
• Degree in Information Technology or a related field.

Preferred Skills

• Experience working in hybrid environments with legacy ETL and modern big-data applications.

Key Skills

Education

Any Graduate