From Chaos to Clarity: AI-Powered Cyber Risk Management

by

Author: Denis Avetisyan

A new framework uses artificial intelligence to automatically connect real-world cyber incidents to known attack patterns and security defenses, improving threat response and risk mitigation.

This research details a novel approach leveraging fine-tuned sentence transformers and a comprehensive knowledge base to map incidents to MITRE ATT&CK techniques and applicable security controls.

Despite increasing cyber threats, translating threat intelligence into actionable security measures remains a significant challenge, particularly for organizations lacking dedicated expertise. This research, presented in ‘Operationalising Cyber Risk Management Using AI: Connecting Cyber Incidents to MITRE ATT&CK Techniques, Security Controls, and Metrics’, introduces a novel framework that automatically maps cyber incidents to specific adversary techniques and corresponding security controls using a fine-tuned sentence-transformer model and a comprehensive knowledge base called the Cyber Catalog. Achieving Spearman and Pearson correlations of 0.7894 and 0.8756 respectively, this approach demonstrably improves the accuracy and consistency of incident-technique mapping. Could this automated connection of threat intelligence to practical security measures fundamentally reshape proactive cyber risk management?

The Inevitable Escalation: Understanding Modern Threats

The digital threat landscape is undergoing a dramatic escalation, marked by both a surge in the volume and a heightened complexity of cyberattacks. Traditional security measures, designed for a comparatively static environment of predictable threats, are increasingly proving inadequate against this evolving challenge. Organizations now face a relentless barrage of incidents, ranging from opportunistic ransomware to highly targeted, nation-state-level intrusions. This rapid increase in both frequency and sophistication overwhelms existing defenses, creating a critical gap between detection and effective response. The sheer scale of modern cyberattacks necessitates a fundamental shift in security strategy, moving beyond simple preventative measures toward proactive threat hunting and rapid incident analysis.

A swift and accurate understanding of how an attack is unfolding – mapping observed actions to established adversary tactics and techniques – is now central to effective cybersecurity. Rather than solely focusing on what happened, security teams increasingly prioritize discerning how an attacker gained access, moved within a system, and ultimately achieved their objectives. This approach, informed by frameworks like MITRE ATT&CK, allows for a shift from reactive containment to proactive threat hunting and the application of targeted mitigations. By correlating incident details with known adversary behaviors, organizations can predict future actions, prioritize responses based on real-world threat intelligence, and ultimately reduce the dwell time of attackers within their networks – a critical factor in minimizing damage and preventing data breaches.

Current incident classification techniques often fall short in the face of escalating cyberattacks, creating a critical bottleneck in effective response. Many systems rely on signature-based detection or manual analysis, proving slow and imprecise when confronting novel or rapidly evolving threats. This leads to miscategorization of incidents – perhaps labeling sophisticated ransomware as simple malware – or significant delays in identifying the true scope and impact. Consequently, security teams struggle to prioritize responses, allocate resources appropriately, and implement targeted mitigation strategies, leaving organizations vulnerable for extended periods and potentially magnifying the damage caused by successful breaches. The inability to swiftly and accurately characterize incidents thus undermines the entire incident response lifecycle, highlighting the urgent need for more intelligent and automated classification solutions.

Semantic Mapping: Trading Keywords for Context

Incident reports and technique descriptions are converted into vector representations using the `Sentence Transformers` library, utilizing the `all-mpnet-base-v2` model. This model encodes text into high-dimensional vectors, where semantic similarity is reflected in the proximity of vectors in the vector space. The `all-mpnet-base-v2` model was pre-trained on a large corpus of text and is known for its balance between accuracy and computational efficiency. These generated vectors serve as the basis for downstream analysis, enabling comparisons between reports and techniques based on meaning rather than literal keyword matches.

Traditional information retrieval relies on keyword matching, which often fails to identify relevant documents when synonyms or paraphrased language are used. Utilizing sentence transformers allows for semantic search, where documents are compared based on their contextual meaning rather than exact term overlap. This approach significantly improves both accuracy – reducing false positives by identifying truly relevant results – and recall – increasing the number of relevant documents retrieved by identifying results that may not share keywords but are semantically similar. By representing text as dense vector embeddings, the system can calculate the cosine similarity between documents, effectively quantifying semantic relatedness and providing more comprehensive search results.

Fine-tuning involves adapting the pre-trained `all-mpnet-base-v2` Sentence Transformer model using a dataset of incident reports and associated technique descriptions. This process updates the model’s weights to better represent the nuances of incident analysis terminology and relationships. Specifically, we utilize a contrastive loss function during training, maximizing the similarity between embeddings of related incident reports and techniques while minimizing similarity between unrelated pairs. The resulting fine-tuned model demonstrably improves performance on downstream tasks, such as identifying relevant techniques for a given incident report, compared to using the base model directly. This targeted adaptation is crucial as general-purpose sentence transformers may lack the specialized vocabulary and contextual understanding required for accurate incident analysis.

Data & Loss: Forcing the Model to Learn the Hard Way

Data augmentation was implemented to address limitations in the size of the training dataset. This process generates modified versions of existing data points, effectively increasing the diversity and quantity of training examples without requiring new data collection. Techniques employed included variations in input phrasing and slight perturbations of existing data to create new, plausible examples. This approach helps to improve the model’s generalization ability and robustness, particularly when dealing with unseen data or edge cases, and mitigates the risk of overfitting to the limited original dataset.

Multiple Negatives Ranking Loss (MNRL) is employed to improve the model’s ability to differentiate between correct and incorrect associations of incidents with MITRE ATT&CK techniques. Traditional loss functions often treat each negative example independently; MNRL, however, presents the model with a set of incorrect technique matches alongside a single correct match. This forces the model to rank the correct technique higher than all presented negatives, thereby enhancing its discriminative power and improving the overall quality of technique prediction. The implementation prioritizes learning from the relative ranking of matches, rather than absolute scores, leading to a more robust and accurate model.

Hard negative mining is a strategy used during model training to improve performance by focusing on examples that the model consistently misclassifies. Instead of treating all negative examples equally, this technique identifies those negative instances that are most frequently and confidently predicted as positive. By prioritizing these “hard” negatives during subsequent training iterations, the model is forced to refine its decision boundaries and better distinguish between relevant and irrelevant data points. This targeted approach effectively concentrates learning efforts on areas where the model demonstrates the greatest weakness, leading to improved accuracy and generalization capabilities.

Model performance was quantitatively assessed using four distinct metrics: Mean Squared Error (MSE), Mean Absolute Error (MAE), Pearson Correlation, and Spearman Correlation. When evaluating the model’s ability to map security incidents to MITRE ATT&CK techniques, a Spearman correlation coefficient of 0.789 and a Pearson correlation coefficient of 0.876 were achieved. These results demonstrate an approximate 0.21 improvement over baseline models for both correlation metrics, indicating a stronger statistical relationship between predicted and actual technique mappings. These metrics provide a comprehensive evaluation of the model’s predictive accuracy and its ability to discern relevant patterns within incident data.

Model evaluation using regression loss functions yielded a Mean Absolute Error (MAE) of 0.135 and a Mean Squared Error (MSE) of 0.027. These values represent a three-fold reduction in error compared to baseline models; specifically, the achieved MAE and MSE are approximately one-third of the errors observed in the baseline evaluation. This indicates a substantial improvement in the model’s ability to accurately predict values, as quantified by these metrics.

The Cyber Catalog: From Alerts to Actionable Intelligence

The Cyber Catalog functions as a central knowledge base, designed to bridge the gap between detected security incidents and established cybersecurity frameworks. It achieves this by meticulously linking individual incidents – such as malware detections or suspicious network activity – to both the CIS Critical Security Controls and the MITRE ATT&CK Techniques. This interconnectedness allows security teams to rapidly understand the context of an incident, identifying which foundational controls were potentially bypassed and which specific attacker tactics, techniques, and procedures were employed. By correlating incident data with these standardized frameworks, the Cyber Catalog fosters a more proactive and informed approach to threat response and ultimately strengthens an organization’s overall security posture.

Automated incident-to-control mapping represents a significant advancement in cybersecurity responsiveness. By instantly correlating detected threats with established security frameworks – such as the CIS Critical Security Controls – systems can move beyond simple alerts and directly suggest prioritized remediation steps. This process eliminates the time-consuming manual effort of security analysts deciphering incident details and researching applicable countermeasures. Consequently, organizations can address vulnerabilities more rapidly, reducing the window of opportunity for attackers and minimizing potential damage. The ability to dynamically link specific incidents to relevant controls also facilitates a more data-driven approach to security, allowing teams to focus resources on the most critical areas and demonstrably improve their security posture.

The utility of security controls isn’t simply their presence, but their demonstrated effectiveness, and the Cyber Catalog addresses this through the integration of quantifiable Security Metrics. This system moves beyond passive compliance checks by actively measuring how well implemented controls mitigate actual threats, as evidenced by incident data. By tracking metrics like mean time to detect, mean time to respond, and control failure rates, the Cyber Catalog provides a data-driven assessment of security posture. This allows organizations to identify weaknesses, prioritize improvements, and ultimately demonstrate a return on investment for their security initiatives – shifting the focus from simply doing security to measuring security’s impact.

The pursuit of automated cyber risk management, as demonstrated by this framework connecting incidents to MITRE ATT&CK, feels predictably optimistic. It attempts to impose order on chaos, a perennial human endeavor. Ken Thompson observed, “Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it.” This applies equally to security architectures. The ‘Cyber Catalog’ and fine-tuned sentence transformers may initially map incidents effectively, but production environments invariably reveal edge cases. The system will become complex, brittle, and require constant maintenance, ultimately mirroring the very vulnerabilities it seeks to mitigate. It’s not a solution; it’s merely a more sophisticated form of technical debt.

The Road Ahead

The automation of incident mapping to frameworks like MITRE ATT&CK feels, predictably, like trading one set of problems for another. The current work addresses the tediousness of manual correlation, but any system reliant on ‘understanding’ natural language incident descriptions invites a new class of failures. Consider: the system will inevitably misclassify, and each misclassification will appear as a logical consequence of its algorithms – a comforting illusion before the inevitable cascade. Anything self-healing just hasn’t broken yet.

Future iterations will undoubtedly focus on expanding the ‘Cyber Catalog’ knowledge base. This is a Sisyphean task, because documentation is collective self-delusion. The moment a control is codified, it becomes obsolete. The real challenge isn’t breadth of coverage, but managing the inherent drift between described controls and their real-world implementations. A more interesting direction lies in accepting ambiguity. If a bug is reproducible, it suggests a stable system – the chaos is the signal, not the noise.

Ultimately, this research will be judged not by its accuracy, but by the cost of its errors. The field needs to shift from seeking perfect automation to developing robust error budgets – understanding how much misclassification is acceptable, and designing systems that fail gracefully – or, preferably, predictably. The pursuit of ‘intelligence’ is a distraction; the value lies in quantifiable reliability.

Original article: https://arxiv.org/pdf/2603.12455.pdf

Contact the author: https://www.linkedin.com/in/avetisyan/

See also:

2026-03-16 13:59