SIEM Engineering: 15 Interview Questions on Splunk, Sentinel & SOC Operations

A Splunk deployment ingests 500 GB/day from 200 sources. Search performance is degrading — simple searches over 7 days take 10+ minutes. The index cluster has 6 peers with adequate CPU and memory.

A new firewall data source sends logs to Splunk in a custom format. The security team wants to use the Splunk ES (Enterprise Security) correlation searches, but the fields are not CIM-compliant.

A Microsoft Sentinel deployment receives Azure AD sign-in logs. The SOC wants to detect impossible travel (login from India and US within 1 hour) but the built-in analytic rule generates 500 false positives daily from VPN users.

A Splunk forwarder on a Windows domain controller stops sending Security Event Log data. The forwarder service is running, but splunkd.log shows TailReader - Could not send data to output queue.

A SOC uses Splunk Enterprise Security. A correlation search detects brute force attacks (5 failed logins in 10 minutes). During a red team exercise, the blue team notices the search only triggers after 15 minutes, missing the attack window.

A Sentinel analytic rule detects lateral movement via SMB (port 445) between workstations. The rule fires on every file share access, generating 10,000 alerts per day. Most are legitimate IT admin activity.

A Splunk ES notable event for "Excessive Data Transfer" triggers. The event shows a user downloaded 50 GB from SharePoint. The SOC manager argues this is a DLP issue, not a security incident.

A Sentinel SOAR playbook is supposed to isolate a compromised endpoint via Microsoft Defender for Endpoint. The playbook runs successfully but the device remains online because the isolation API returns "Action is already in progress."

A security team ingests firewall logs into both Splunk and Sentinel for redundancy. The same attack generates an alert in both platforms with different severity levels: Splunk rates it "critical" while Sentinel rates it "low."

A Splunk deployment must retain security logs for 7 years for compliance (PCI-DSS, HIPAA). The current hot+warm+cold architecture uses expensive SSD storage for all tiers. The storage cost is $180,000/year.

A Sentinel workspace ingests logs from 15 different cloud services (AWS, Azure, GCP, Okta, Office 365). The bill is $45,000/month, with 60% coming from verbose AWS CloudTrail and Azure Diagnostic logs.

A Splunk ES correlation search uses stats dc(src_ip) as unique_sources by dest_ip to detect port scans. In production, this search times out on indexes with 10 billion events.

A SOC needs to detect living-off-the-land binaries (LOLBins) like certutil, bitsadmin, and regsvr32 being abused for malware delivery. Standard process creation logs do not show command-line arguments.

A red team exercise reveals that the SOC takes 4 hours on average to escalate a critical alert to the incident response team. The bottleneck is manual ticket creation, evidence gathering, and stakeholder notification.

A CISO wants to measure the maturity of the SOC's detection engineering program. The team currently has 400 detection rules but no metrics on coverage, quality, or drift.