Turning Threat Data into Defenses: An AI-Powered Approach

Author: Denis Avetisyan

This research details a new system that automatically translates cyber threat intelligence into actionable firewall rules using the power of large language models and semantic reasoning.

The agentic solution leverages a defined architecture to facilitate autonomous operation and decision-making.

The architecture combines agentic AI, LLMs, and the CLIPS expert system to leverage hypernymy/hyponymy for improved intrusion detection and prevention.

Responding to the ever-increasing volume and sophistication of cyber threats demands more automated and intelligent security systems, yet maintaining trustworthiness remains paramount. This paper, ‘From Threat Intelligence to Firewall Rules: Semantic Relations in Hybrid AI Agent and Expert System Architectures’, introduces an agentic AI system that bridges the gap between raw cyber threat intelligence and actionable security controls. By leveraging large language models and focusing on hypernym-hyponym relationships, the system automatically generates CLIPS code for expert systems, effectively creating firewall rules to mitigate malicious network traffic. Could this neuro-symbolic approach represent a significant step towards truly adaptive and resilient cybersecurity infrastructure?

The Semantic Bottleneck in Cyber Threat Intelligence

Cyber Threat Intelligence (CTI) reports represent a vital cornerstone of modern cybersecurity, yet their inherent format often limits their effectiveness. While brimming with critical information regarding emerging threats, vulnerabilities, and attack vectors, these reports frequently exist as free-form, unstructured text – resembling a narrative rather than a dataset. This poses a significant challenge for automated analysis; security systems struggle to parse and interpret the nuances of natural language, hindering their ability to rapidly identify, categorize, and respond to potential threats. Consequently, valuable insights remain locked within these reports, creating a bottleneck that delays threat detection and increases an organization’s vulnerability. The inability to efficiently process this unstructured data necessitates innovative approaches to unlock the full potential of CTI and maintain a proactive security posture.

Current cybersecurity threat intelligence analysis heavily relies on human experts sifting through vast quantities of unstructured text reports. These traditional methods, while valuable, face inherent limitations in consistently and accurately extracting the nuanced semantic relationships critical for effective threat detection. Keyword searches and simple pattern matching often fail to capture the context surrounding indicators of compromise, leading to false positives or, more dangerously, missed threats. This semantic bottleneck results in delayed responses, incomplete understanding of attack campaigns, and an inability to proactively defend against evolving cyberattacks; the time lost in manual interpretation directly impacts an organization’s security posture and can allow malicious actors to exploit vulnerabilities before defenses are deployed.

The escalating digital landscape generates a relentless torrent of Cyber Threat Intelligence (CTI) data, far exceeding the capacity of manual analysis. This overwhelming volume isn’t simply a logistical challenge; it represents a critical bottleneck in effective cybersecurity. Consequently, automated methods for semantic understanding are no longer optional, but essential. These systems aim to move beyond keyword spotting and delve into the meaning of threat reports, identifying relationships between indicators, actors, and vulnerabilities. By automatically extracting and structuring this knowledge, organizations can accelerate threat detection, prioritize responses, and proactively defend against evolving cyberattacks. The pursuit of actionable intelligence, derived from this automated semantic processing, promises a shift from reactive security measures to a more predictive and resilient posture.

From Text to Structure: Advanced Semantic Extraction

Semantic extraction relies on transforming textual data into numerical vector representations to facilitate analysis and comparison. Techniques like Word2Vec and GloVe achieve this by mapping words to vectors based on their contextual similarity within a corpus; words appearing in similar contexts receive closer vector representations. More recent approaches, such as SecureBERT, leverage transformer-based models to generate contextualized word embeddings, capturing nuanced semantic meanings. These vector representations enable algorithms to quantify semantic relationships, identify relevant information, and build knowledge graphs, effectively moving beyond simple keyword matching to understand the underlying meaning of text.

Large Language Models (LLMs), specifically Qwen2.5-Coder-14B-Instruct, are demonstrably improving the accuracy of semantic extraction processes. This improvement is achieved through the implementation of Chain-of-Thought prompting, which encourages the LLM to articulate its reasoning steps, and Deterministic LLM Inference, a technique focused on generating consistent outputs for the same input. By guiding the LLM’s analytical process and ensuring predictable results, these enhancements enable more reliable identification and isolation of key information within unstructured text, exceeding the performance of standard extraction methods.

Research detailed in our paper indicates a 7% improvement in F1 score when applying a novel prompting methodology to semantic extraction from Cyber Threat Intelligence (CTI) reports. This methodology leverages hypernymy and hyponymy – hierarchical relationships between concepts – to enhance the identification of relevant text snippets for intrusion prevention systems. Comparative analysis demonstrates that this approach outperforms baseline methods in accurately extracting key information, and is foundational for constructing a comprehensive knowledge graph detailing cyber threats and their associated characteristics. The improved extraction accuracy directly contributes to more effective threat detection and mitigation strategies.

Automated Threat Response via Semantic Information Flow

Semantic Information Flow pipelines facilitate automated threat response by converting raw data insights into actionable Filtering Rules. These pipelines typically involve stages of data extraction, normalization, and enrichment, culminating in the generation of rules compatible with security infrastructure such as firewalls, intrusion detection systems, and security information and event management (SIEM) platforms. The resulting rules define criteria for identifying malicious activity – for example, blocking traffic from a known malicious IP address, or alerting on specific patterns of network behavior. This automated rule application reduces reliance on manual intervention, enabling faster response times and improved security posture by proactively blocking or mitigating threats based on derived intelligence.

Expert systems utilize rule-based logic to automate threat mitigation by employing inference engines, such as CLIPS, to process established filtering rules. These systems operate on a knowledge base consisting of facts and rules; when new data is presented, the inference engine applies the rules to the facts to derive conclusions and trigger automated responses. This process enables real-time identification of malicious activity based on predefined criteria, facilitating actions like blocking network traffic, isolating compromised systems, or alerting security personnel. The efficiency of these systems is directly related to the quality and completeness of the rule set and the speed of the inference engine in evaluating those rules against incoming data streams.

Rigorous evaluation of automated threat response systems requires several key metrics. Hamming Loss measures the fraction of incorrectly classified instances, providing an overall error rate. Top-K Accuracy assesses whether the correct threat is included within the system’s top K predictions, acknowledging that multiple relevant threats may exist. BERTScore utilizes contextual embeddings from the BERT model to evaluate the semantic similarity between predicted and actual threat descriptions. Finally, ROUGE-L, based on the longest common subsequence, assesses the quality of generated threat reports by comparing them to reference summaries; these metrics collectively provide a comprehensive assessment of system performance beyond simple accuracy, considering both classification correctness and the quality of textual outputs.

The Pursuit of Adaptability: Agentic AI and Knowledge Base Refinement

Agentic AI systems, such as those utilizing the CoALA architecture, facilitate continuous knowledge base refinement through automated learning loops. These systems employ Large Language Models to analyze new data and identify opportunities to update existing information or create new entries. The framework allows for iterative improvement of the knowledge base without requiring manual intervention for every update. This is achieved by enabling the AI to autonomously propose changes, evaluate their potential impact, and implement validated improvements, resulting in a dynamically maintained and evolving knowledge repository.

Agentic AI systems utilize Large Language Models (LLMs) to dynamically adapt to changes in threat landscapes. These LLMs are capable of processing unstructured data, such as threat intelligence reports and security blogs, to identify emerging patterns and indicators of compromise. This information is then used to refine existing filtering rules, including signature updates and behavioral analysis parameters. The LLMs’ ability to understand contextual information and semantic relationships allows for the creation of more accurate and nuanced filtering criteria, reducing false positives and improving detection rates. This automated refinement process allows security teams to respond more effectively to rapidly evolving threats without manual intervention.

Knowledge base refinement utilizes quantitative metrics to validate consistency and accuracy. Specifically, Krippendorff’s Alpha, a statistic measuring inter-annotator agreement, was employed to assess technical correctness; our results demonstrate a high alpha value, indicating strong reliability in the labeling process. Simultaneously, Spearman Correlation was used to evaluate scope calibration, quantifying the degree to which refined rules align with the intended coverage of the knowledge base; a high correlation coefficient was observed. These metrics collectively support the conclusion that the agentic AI-driven refinement process produces a consistently accurate and reliably scoped knowledge base, suitable for dynamic threat detection and filtering.

The Trajectory of Automated Cybersecurity Intelligence

The convergence of advanced semantic extraction, agentic artificial intelligence, and automated knowledge refinement is fundamentally reshaping the landscape of cybersecurity. Traditionally, threat intelligence relied heavily on manual analysis – a slow and resource-intensive process prone to human error. Now, systems can autonomously dissect vast quantities of data, not just identifying keywords, but truly understanding the meaning and relationships within the information. Agentic AI then empowers these systems to proactively investigate threats, formulate hypotheses, and even take corrective actions, all without direct human intervention. Crucially, automated knowledge refinement ensures that this intelligence isn’t static; systems continuously learn from new data, correcting inaccuracies and improving predictive capabilities. This represents a shift from reactive defense to proactive, self-improving security, promising a future where threats are anticipated and neutralized before they can cause significant damage.

The automation of cybersecurity intelligence is fundamentally reshaping threat response by diminishing the need for exhaustive manual analysis. Traditionally, security teams spent considerable time sifting through alerts, investigating potential incidents, and piecing together fragmented information – a process prone to delays and human error. Now, systems capable of autonomously extracting, correlating, and interpreting threat data can drastically reduce this workload. This acceleration isn’t simply about speed; it allows security professionals to focus on strategic initiatives and complex threats, while the automated systems handle the bulk of routine investigations. Consequently, organizations experience improved security posture through faster identification of malicious activity, quicker containment of breaches, and a more proactive defense against evolving cyber threats, ultimately minimizing potential damage and downtime.

The progression of automated cybersecurity intelligence hinges on concentrated development across several key computational areas. More efficient Large Language Models (LLMs) are needed, not simply in terms of parameter count, but in their ability to process and correlate security data with reduced computational cost and latency. Simultaneously, robust knowledge representation techniques – methods for structuring and storing cybersecurity information in a way that facilitates reasoning and inference – are paramount. Current systems often struggle with the nuance and context inherent in threat intelligence; improved knowledge representation could address this. Crucially, the field requires reliable evaluation metrics that move beyond simple accuracy scores to assess the system’s ability to detect novel threats, minimize false positives, and adapt to evolving attack landscapes. Without advancements in these three areas, the transformative potential of automated cybersecurity intelligence will remain largely unrealized.

The pursuit of automated firewall rule generation, as detailed within this study, mirrors a fundamentally mathematical endeavor. The system’s reliance on semantic relationships – specifically hypernymy and hyponymy – to translate threat intelligence into actionable security policies demands logical rigor. This aligns with the sentiment expressed by G. H. Hardy: “Mathematics may be compared to a box of tools.” The LLM, in this context, is but one tool; the true elegance lies in the construction of a provably correct system-one where the derived firewall rules are not merely functional based on testing, but logically sound due to the underlying semantic framework. The system’s architecture reflects a commitment to demonstrable correctness, exceeding simple empirical validation.

What’s Next?

The demonstrated coupling of large language models with symbolic reasoning – specifically, the CLIPS expert system – offers a tantalizing, if preliminary, glimpse of automated security orchestration. However, the system’s reliance on pre-defined semantic relationships – hypernymy and hyponymy – represents a fundamental limitation. If the knowledge graph fails to accurately reflect the nuances of a novel attack, the system will falter – a clear indication that statistical correlation, however impressive, is not understanding. The illusion of intelligence quickly dissolves when confronted with the unforeseen.

Future work must address the brittleness inherent in static knowledge bases. A truly robust system will require mechanisms for dynamic knowledge acquisition – the ability to infer semantic relationships from raw text, and, crucially, to validate those inferences. This is not merely a matter of improving natural language understanding; it demands a formalization of cybersecurity concepts, a rigorous ontology capable of supporting provable inferences. If it feels like magic that the current system ‘works’, one hasn’t revealed the invariant.

Beyond knowledge representation, the question of action remains. Generating firewall rules is a limited, if pragmatic, output. The true potential of agentic AI in cybersecurity lies in autonomous response – the ability to not only detect and prevent threats, but to adapt defenses in real-time, anticipating future attacks. Achieving this requires moving beyond rule-based systems to models capable of genuine planning and reasoning under uncertainty – a challenge that demands a return to fundamental principles of artificial intelligence.

Original article: https://arxiv.org/pdf/2603.03911.pdf

Contact the author: https://www.linkedin.com/in/avetisyan/