← Back to jobs
Jersey City, NJ, USA
No related jobs found
· Spark Administration: Deploy, configure, monitor, tune, and troubleshoot Apache Spark environments and workloads. Support Spark job performance, resource management, capacity planning, and production operations.
· HDFS Administration: Manage and support HDFS storage, including capacity, performance, availability, permissions, replication, and troubleshooting. Strong understanding of HDFS architecture and integration with distributed compute platforms is required.
· Kafka Administration: Deploy, configure, manage, monitor, and troubleshoot Apache Kafka clusters, topics, producers, and consumers. Support performance, capacity, availability, security, and production operations.
· Monitoring and Performance Tuning: Monitor platform connectivity, health, capacity, and performance using tools such as Grafana, Splunk, and relevant platform management tools. Proactively identify and resolve performance bottlenecks across Spark, HDFS, and Kafka.
· Security and Compliance: Implement and manage security configurations, including Kerberos, SSL/TLS, authorization policies, HDFS ACLs, and enterprise IAM/LDAP integrations.
· Automation and Scripting: Develop and maintain automation using Python, Bash, or Perl for routine administration, configuration management, monitoring, and operational support.
· Troubleshooting and Support: Provide expert-level troubleshooting and technical support for Spark, HDFS, Kafka, and related big data technologies, including participation in an on-call rotation.
· Platform Administration: Support installations, patching, upgrades, migrations, and platform lifecycle activities while minimizing downtime. Cloudera Hadoop/CDP administration will be part of the role but is secondary to expertise in Spark, HDFS, and Kafka.
Requirements:
· Excellent problem-solving, analytical, and communication skills.
· Minimum 3 years of hands-on production experience administering and supporting Apache Spark, HDFS, and Kafka.
· Strong, demonstrable expertise in Spark administration, configuration, performance tuning, and troubleshooting.
· Strong, demonstrable expertise in HDFS administration, storage management, security, performance, and troubleshooting.
· Strong, demonstrable expertise in Kafka administration, topic management, producer/consumer architecture, performance, security, and troubleshooting.
· Experience with Linux operating systems and command-line tools.
· Experience with enterprise monitoring and operational tooling such as Grafana and Splunk.
· Experience with Cloudera Hadoop/CDP and Cloudera Manager is preferred but is not the primary skill requirement.
· Cloudera certifications or equivalent big data certifications are desirable but not required.
· Solid understanding of Kerberos, SSL/TLS, IAM/LDAP, and security concepts for distributed data platforms
Bachelor's degree
No related jobs found
← Back to jobs