Description

Manage and support production oversight of big data platforms, ensuring stability and performance across the data architecture.

Responsibilities

• Maintain Hadoop and Spark processing systems, including NoSQL databases and small-scale developments or fixes.
• Oversee data acquisition and develop data set processes while managing resource and security requirements.
• Monitor Big Data and Hadoop applications to identify production failure scenarios and manage incidents.
• Troubleshoot application errors to prevent recurrence and ensure adherence to daily SLAs.
• Deploy and maintain Hadoop clusters, including node management and NameNode high availability configuration.

Required Skills

• 3+ years of direct experience administering Apache Hadoop frameworks including Spark, HBase, HDFS, Hive, Parquet, Sentry, Impala, and Sqoop.
• Experience with Cloudera Manager, Cloudera Navigator, and Hue.
• Proficiency in maintaining data pipeline architectures for security, scalability, and performance.
• Strong Unix/Red Hat skills.
• Knowledge of Informatica BDM, EDC, EDQ, and Axon.
• Experience with NoSQL databases and Big Data ML toolkits like SparkML.
• Understanding of distributed computing principles and Data Warehouse/RDBMS.
• Ability to use cluster monitoring tools such as Ganglia, Nagios, or Cloudera Manager.

Preferred Skills

• Python scripting experience.
• SQL knowledge.
• Experience with cloud technologies such as AWS or Azure.

Key Skills

Education

Any Graduate