Description

Strengths

• Core Splunk admin + architecture is strong (hybrid): Demonstrated experience with clustered on-prem Splunk Enterprise, load balancers, HA, plus Splunk Cloud in Multi-AZ, and day-to-day ops (upgrades, troubleshooting, capacity planning).
• SPL + performance troubleshooting appears strong: Mentions query optimization, internal index review, data source validation, and time-range/root-cause patterns. Also builds interactive dashboards via SPL + XML (Classic) with usability focus.
• Data ingestion breadth and engineering thinking: Credible ingestion footprint (AWS, Databricks, Snowflake, custom apps). Uses S3-based ingestion pipelines for resilience (buffering/replay) and discusses redundancy to avoid data loss/latency.
• Cribl experience (partial but real): Has configuration + upgrade experience. Not full greenfield architecture, but it’s a positive match versus “none.”
• Gov / federal context: Has government contracting experience since ~2015 and has participated in FedRAMP/FISMA processes (artifact contribution).
• IT infrastructure foundation: Helpdesk → sysadmin → DBA → security → Splunk path suggests solid underlying networking/systems/security intuition—useful for troubleshooting complex environments.
• ITSI exposure (backend): Has dealt with ITSI backend issues and real-world platform constraints (Java/RHEL compatibility), which is practical ops experience.

Weaknesses / Risks

• Role needs Splunk Cloud Admin certification
• He has federal AWS exposure and hybrid Splunk, but the requirement calls out Splunk GovCloud specifically—this wasn’t explicitly demonstrated.
• Cribl depth is limited: Experience is config/upgrade, not large-scale architecture/build-out.
• Strong AWS, but Azure/ServiceNow/Archer/Syslog-NG/Function Apps weren’t covered in the interview.
• AI requirements are only lightly met: Some chatbot/AI-assisted efficiency exploration, but no explicit AITK/MLTK experience and no clear detail on secure MCP deployment (just general AI interest).

Description of work

Sidekick Security LLC is looking for a highly qualified Splunk Engineer to become part of our skilled and diverse team. This opportunity is based in Atlanta, Georgia.

Splunk Search Processing Language (SPL) : Strong proficiency in writing complex and optimized queries is fundamental. This includes using transforming commands, eval functions, and sub searches to extract and manipulate data effectively.

Data Ingestion and Management : Skills in collecting, processing, and indexing data from diverse sources (e.g., cloud platforms, servers, applications, endpoints) are crucial. This involves managing forwarders and data inputs.

Dashboard Creation and Visualization : The ability to create meaningful reports, graphs, and interactive dashboards to communicate insights to technical and business teams is essential.

Scripting and Automation : Strong scripting skills, especially in Python, Bash, & PowerShell, are highly valued for automating data collection, processing, and other use cases provided by other teams.

IT Infrastructure Knowledge : A solid understanding of IT systems, networking, and security is necessary to effectively monitor and troubleshoot complex environments.

Hosting Environment : Strong experience in both self-hosted and Splunk GovCloud environments.

Tool Integrations: Experience with integrating with Archer, ServiceNow, Azure, and AWS.

Data Forwarding: Proficiency with managing and configuring data intakes via Syslog-NG, Cribl, AWS Lambda, & Azure Function Apps.

Basic Qualifications: Minimum knowledge, skills, abilities needed

Bachelor’s Degree in Information Systems/Computer Science or related field and 7 years of relevant work experience; OR Master’s Degree and 5 years relevant work experience

Preferred Qualifications:

Candidates with these skills will be given preferential consideration:

AI-Specific and Emerging Skills

AI Toolkit (AITK, formerly MLTK) : Expertise in using the Splunk AITK to apply predictive analytics, anomaly detection, forecasting, and clustering methods is a key AI-related skill.

Splunk MCP Server : An understanding of how to securely implement the Splunk MCP Server app, as well as its potential impact and limitations.

AI Assistant for SPL : Familiarity with using the generative AI-powered AI Assistant to generate and explain SPL queries using natural language would be nice, but not necessary since it’s not currently available in Splunk GovCloud FedRAMP Moderate or High.

Monitoring AI Application Stacks : The ability to monitor the quality, security, cost, and performance of AI agents, Large Language Models (LLMs), and underlying infrastructure would be nice to have

Key Skills

Education

Bachelor's or Master's degrees