Description

You will lead the development and optimization of batch and real-time data pipelines to ensure scalability and performance.

Responsibilities

• Architect, design, and deploy data integration, streaming, and analytics solutions using Spark, Kafka, and Snowflake.
• Research technologies independently to recommend appropriate solutions and establish technical best practices.
• Collaborate with key partners across all stages of the Software Development Life Cycle.
• Lead the implementation of data integration, data quality, and data architecture standards.
• Support technology-specific standards within a global delivery model.

Required Skills

• At least 4 years of Information Technology experience.
• Extensive experience with the Hadoop ecosystem and Apache Spark.
• Hands-on experience with Kafka streaming, including topics, partitions, producers, and consumers.
• Proficiency in SQL and Unix shell scripting.
• Experience with Snowflake.
• Solid understanding of distributed computing, data partitioning, and parallel processing.
• Knowledge of Relational Modeling, Dimensional Modeling, and modeling of unstructured data.
• Experience working within Agile software development frameworks.
• Bachelor's degree or foreign equivalent, or equivalent progressive experience.

Preferred Skills

• Proficiency in Scala, Java, or Python.
• Deep experience with Spark Core, Spark SQL, Spark Streaming, and Spark MLlib.
• Experience with cluster management tools like Apache Mesos or Apache YARN.

Key Skills

Education

Bachelor's degree