False Negative
Introduction
Today, I will share my view on the most significant and most dangerous failure that can occur in a Security Operations Center: when a true positive is classified as a false negative.
I have worked for many years in a Network Operations Center, holding various positions, including first-line, second-line, third-line, and head of NOC. If I had to summarize the challenges, I would say that the biggest bet is how, through monitoring tools and a barrage of alarms, we can distinguish which alerts are critical (true positives), what the root cause is, and which decisions need to be made immediately to restore service and limit risk. Technology helps us significantly, offering an almost real-time view of events. With the help of rules and machine learning, we can train our systems to automatically decide which actions should be taken.
Over time, as businesses grew and threats became more complex, the need for protection was no longer limited to the availability and performance of services. Data security and protection against malicious actions became equally critical. Thus, gradually, alongside the NOC, the Security Operations Center (SOC) was established, with the mission not only to ensure that everything “runs,” but also to ensure that everything “runs securely.” Despite their differences, the two departments share many similarities in their operation and the challenges they face. If we were to summarize the main similarities and differences between the two departments in a short table, in my opinion, it would be as follows:
| Characteristic | NOC | SOC |
|---|---|---|
| Main Mission | Availability, performance, and health of network/IT | Information and systems security |
| Type of Incidents | Technical issues: downtime, latency, errors | Security threats: attacks, malware, breaches |
| Tools | Network monitoring (e.g., SolarWinds, Nagios), performance dashboards, SNMP traps | SIEM, EDR, IDS/IPS, threat intelligence, DLP |
| Procedures | Troubleshooting, escalation, patching, capacity planning | Incident response, forensics, threat hunting, containment |
| KPIs | Uptime, MTTR (Mean Time to Repair), network throughput | MTTD (Mean Time to Detect), MTTR (Mean Time to Respond), number of incidents detected |
| Staff Specialization | Network engineers, system admins, IT ops | Security analysts, incident responders, threat hunters |
| Focus | Proactive and reactive management of technical issues | Proactive and reactive management of security threats |
| Frequency of Changes | Often manage changes (e.g., upgrades, migrations) | Focus on threat detection/response, not infrastructure changes |
The two departments share many similarities in their operations, resulting in similar challenges in their daily routines. Indicatively, some of the areas I believe they have in common are:
- Alert fatigue: Both centers receive a high volume of notifications, which increases the risk of essential alerts being overlooked due to overload.
- Need for 24/7 operation: Continuous monitoring necessitates shifts, which increases operational stress and the risk of staff burnout.
- Difficulty in prioritizing incidents: Many incidents seem urgent, but not all are equally critical. Proper assessment and prioritization can be challenging under pressure.
- Dependence on tools and automation: Excessive reliance on tools can lead to blind spots or loss of technical skills.
- Lack of specialized personnel: Both centers often struggle to find and retain experienced professionals.
- Collaboration with other IT teams: Resolving incidents often requires effective communication and coordination with different teams, which can lead to delays or misunderstandings.
- Change and update management: Continuous changes in infrastructure, tools, and threats require ongoing training and adaptation of procedures.
All the above lead to the biggest problem: what happens when a true positive is drowned in the noise and ignored, or when some of the criteria for an alert to be classified as a true positive by monitoring and analysis systems are not met, and it is categorized as a false negative? That’s where the nightmare begins.
What do the terms true positive, false positive, false negative, and true negative mean?
- True Positive: The system correctly detected a real threat.
- False Positive: The system raised an alarm for something that was ultimately found to be benign.
- False Negative: The system failed to detect a real threat – the worst-case scenario, as the threat goes unnoticed.
- True Negative: The system correctly ignored benign activity.
The balance between these scenarios is the “holy grail” of every NOC and SOC. Too many false positives tire the team, while false negatives leave doors open to attackers.
The Nightmare
How does the nightmare of the false negative manifest in each of the above challenges, and what measures could we take to limit the risk?
The systems monitored by SOC teams multiply geometrically, while the logs generated increase exponentially. Monitoring systems soon reach their technical limits in processing, analyzing, and managing such large volumes of alerts. At the same time, alert correlation methods do not always keep up with developments, resulting in the inability to condense multiple alerts with a common root cause into a limited number of messages.
The dynamic monitoring environment created by SOC teams presents additional challenges in establishing proper rules and in the analytical evaluation of alerts. Every day is a new major challenge that leaves no time or energy to deal with potential failures.
The equation becomes even more challenging as malicious groups adapt to new attack vectors, modify their methodologies, and exploit this dynamic environment to conceal their tracks.
If we wanted to use a football metaphor, we would say that SOC teams are like a goalkeeper who receives many shots on target every minute and must block them all. As the rate of shots increases, statistically, some will get past the goalkeeper, and the more tired they get, the more goals will be scored. For attackers, it is enough if 1 out of 1000 attempts is successful. For them, the attack is successful despite 999 failures. For SOC teams, the 999 successes do not count, as the 1 successful attack is enough to bring down the entire structure they protect.
Finally, the mix becomes even more explosive if we add failure in incident prioritization, the SOC team’s inability to absorb continuous changes, the complex change management and collaboration procedures within organizations, and the lack of training. All these create the perfect climate for essential indicators to be missed and for a SOC department to fail.
Mitigation
The ways to address the nightmare are simple and are based on strengthening the collaboration between humans and technology, placing the human at the center, not the technology.
- Dependence on tools: Tools and platforms exist to facilitate our daily routines, not to replace human decision-making. They suggest solutions, group data, and process information, but they have limits and are vulnerable to new attacks. Teams need to know the tools they use in depth, use them to confirm their assessments, and, most importantly, not trust them blindly. In an environment where zero-trust mentality prevails, the SOC must be the first to apply it. Weekly, it is essential to conduct retrospective analysis of raw data, record false positives, true positives, false negatives, and true negatives, and monitor the relevant indicators.
- Need for 24/7 operation: SOC teams operate around the clock. Their biological clock is disrupted, and the environment is extremely stressful. They should not be treated as office employees; special care must be given to both their physical and psychological rest and relief. Fatigue and burnout are the main reasons for poor judgment.
- Collaboration with other Departments: Collaboration with other departments within the organization, with direct communication channels, an agreed way of exchanging information, and a common vocabulary, strengthens the organization’s defense. Information in the NOC may be vital for the SOC and vice versa. The problem is always collective and cannot be solved by one side alone; the contribution of all involved is necessary. Knowledge and information silos must be eliminated for the defense to be resilient.
- Training: SOC team training should be included in their working hours and not be a separate activity. When you schedule training outside working hours, you create conditions that may prevent the engineer from coping. Of course, training and information are essential even outside working hours, but this should be a personal choice for each employee. It is crucial to offer a variety of training options, rather than limiting them solely to technical topics. Training in soft skills enhances communication, expression, and teamwork.
- Procedures and Playbooks: Keep them as simple as possible. In a panic and crisis, no one can refer to volumes and countless pages. Simplicity, Focus, and Testing: these should be their ingredients. This way, they are easy to memorize, straightforward, and with each test, they become more accurate.
Conclusion
Cybersecurity is never static; it is a constant battle between technological evolution, human judgment, and the creativity of attackers. The biggest challenge for every SOC is not to lose the essential in the noise, not to let that one critical indicator that can make the difference go unnoticed. Technology is a valuable ally, but humans remain the center of gravity of defense. Investing in collaboration, training, and the simplicity of procedures, as well as the ongoing search for balance between automation and human intuition, is the key to preventing the nightmare of a false negative from becoming reality.
The challenge is great, but collective effort and continuous improvement can make the difference. Because, at the end of the day, security is everyone’s business.