Description

You will lead the Big Data Engineering team, providing architectural vision and long-term IT strategy.

Responsibilities

• Translate client requirements into technical architecture and design integrated Big Data platforms.
• Design and implement data collectors to transport data to the Big Data Platform.
• Build and manage data pipelines using Hadoop components and Spark.
• Implement monitoring solutions to track infrastructure health.
• Review existing architecture and recommend new technologies to provide business value.

Required Skills

• 5+ years of experience with Big Data, Cloudera Distribution 7.x, and RDBMS.
• Strong experience with Amazon EMR, Databricks, or Cloudera CDP.
• 4-5 years of programming experience in Python, Java, and Scala.
• Hands-on experience building pipelines with Sqoop, Hive, Solr, MapReduce, Impala, Spark, Spark SQL, and HBase.
• Expertise in REST API development using Python frameworks like Django or Flask.
• Experience with real-time analytics using Spark, Kafka, or Storm.
• Proficiency in Unix/Linux environments, including scripting and job scheduling.
• Experience with Dask, Numpy, Pandas, and Scikit-Learn.
• Knowledge of Jenkins and JIRA.

Preferred Skills

• Microservices and web service development using the Spring framework.

Key Skills

Education

Bachelor's degree