WINNIIO Perspectives — Whitepaper VI — Apr 2026
The AI You Can't Trust
What Happens When Someone Attacks Your Factory's Brain
Apr 2026
“You are deploying AI to make real-time decisions in your production processes. Have you tested what happens when someone attacks it? Not the network — the AI itself.”
The Attack Surface Nobody Audited
In 2021, Colonial Pipeline's operations were paralyzed for six days by a ransomware attack that cost $4.4 million in ransom and an estimated $1 billion in downstream economic impact. The attack vector was a legacy VPN credential — a network security failure. The industrial sector learned the lesson, invested in network segmentation and perimeter defense, and moved on.
What the sector has not yet reckoned with is a different class of threat — one that does not attack the network. It attacks the intelligence.
Cyberattacks against industrial control systems increased by 300 percent between 2020 and 2025, according to the Dragos Industrial Cybersecurity Report. Sweden's MSB reported more than 200 serious IT incidents in critical infrastructure in 2025 alone — a 40 percent increase from the prior year. Penetrations of steel production facilities in Europe were documented in 2024.
But the attack surface that is growing fastest is not the one most security teams are monitoring. It is the AI layer.
When an AI model makes real-time decisions about furnace temperature, press timing, or energy load distribution — decisions that previously required a human operator's judgment and were executed on equipment with direct physical safety interlocks — the AI system becomes part of the critical path. Compromise the AI, and you compromise the process.
The tools that exist to defend conventional OT/ICS environments are necessary but structurally insufficient. Network anomaly detection platforms can identify unusual traffic patterns. They cannot tell you whether the AI model controlling your heat treatment furnace has been gradually taught, over weeks of subtle data manipulation, to make decisions that will cause equipment damage or process failure under specific conditions.
Four Attacks Conventional Security Misses
- Data poisoning. The most insidious because it operates over time and leaves no obvious signature in network traffic. An attacker who can introduce manipulated readings into the sensor stream can gradually shift an AI model's behavior. The shift is small enough at each step to evade statistical outlier detection. The cumulative effect is a model that behaves correctly under normal conditions but fails in predictable ways under conditions the attacker controls.
- Adversarial inputs. Crafted sensor readings designed to exploit the specific mathematical structure of a trained AI model. A model that accurately predicts energy demand under normal operating conditions may be made to produce wildly incorrect outputs by introducing a carefully calculated perturbation — one that looks like sensor noise but is precisely targeted at the model's decision boundary.
- Model extraction. An attacker with query access to an AI system can reconstruct its underlying logic through systematic probing. For a manufacturer whose competitive advantage is encoded in AI models trained on proprietary production data, this represents direct theft of intellectual property — without any network intrusion that traditional security tools would detect.
- Multi-agent manipulation. When multiple AI agents coordinate across machines, production zones, or facilities, the compromise of a single node creates a propagation pathway. A Byzantine-faulty agent that appears to be functioning normally can inject corrupted signals into the collective decision-making of the network.
“The Claroty and Nozomi platforms watching your OT network are necessary. They are not sufficient. They cannot tell you whether the decisions your AI is making are trustworthy — only whether the traffic looks normal.”
The Compliance Pressure Is Real and It Is Now
NIS2 — implemented in Swedish law in 2025 — classifies heavy industry and steel production as “essential services.” The obligations include risk assessments for all IT and OT systems, documented incident response procedures, supply chain security controls, and evidence of resilience against both conventional and emerging attack vectors. An AI system that has not been tested against adversarial manipulation is, by definition, an unassessed risk.
Sweden's AI Strategy (2026, SEK 479 million allocated) explicitly identifies “safe and trustworthy AI” as a priority area, with focus on ensuring that AI systems in critical sectors are resistant to manipulation, traceable in their decision-making, and subject to meaningful human control.
The IEC 62443 standard series for industrial cybersecurity requires Security Level assessments for all OT systems — but currently lacks specific methodologies for AI components. This is the compliance gap that forward-looking manufacturers need to close now, before regulators close it for them with prescriptive requirements.
Building AI That Can Be Trusted
Four Structural Properties
- Adversarial robustness by design. Models should be trained with explicit resistance to known attack classes. Adversarial training — exposing models to synthetic attacks during training, using frameworks such as the PGD (Projected Gradient Descent) attack methodology — increases robustness against previously unseen adversarial inputs. This is not a one-time test; it is a continuous practice, analogous to penetration testing for conventional software.
- Physics-informed validation. Industrial processes obey physical laws that an AI model cannot override. A temperature cannot change by 200 degrees Celsius in three seconds. These physical constraints are precisely defined and perfectly knowable. An AI decision that violates them — regardless of what the sensor data appears to indicate — is wrong, and the system should refuse to execute it. This is the defense layer that no adversarial manipulation can circumvent, because it operates on invariants rather than learned patterns.
- Uncertainty quantification. A trustworthy AI system does not just produce decisions — it produces decisions with confidence estimates. When a model's internal state indicates high uncertainty — because the input pattern is unusual, because sensor readings are in conflict, or because the operating condition is outside the training distribution — the system should escalate to human review rather than proceeding autonomously.
- Secure federated learning. When multiple facilities share AI model improvements through federated learning, the federation protocol must be cryptographically protected against poisoning. Secure aggregation techniques allow model updates to be combined without any single party being able to observe individual facilities' raw training data. Byzantine fault tolerance ensures that compromised nodes cannot corrupt the shared model.
The Digital Twin as Security Instrument
Perhaps the most powerful and underutilized tool in industrial cybersecurity is one that most manufacturers already have components of: the digital twin.
A high-fidelity digital twin of a production facility is not just a planning and optimization tool. It is a safe environment in which to simulate attack scenarios before an actual attacker finds the same vulnerabilities in the real system.
Red team/blue team exercises — standard practice in IT security — have rarely been applied to industrial AI systems because the cost of simulating attacks in a live production environment is prohibitive. A digital twin eliminates this constraint. Security teams can inject realistic attack scenarios based on the MITRE ATT&CK for ICS catalog — sensor spoofing, command injection, coordinated multi-site attacks — and observe how the AI system responds, without any risk to production.
“The digital twin is the one place where you can let an attacker win — safely, repeatedly, and with full instrumentation — so that you understand your vulnerabilities before they do.”
Beyond incident simulation, a continuously operational digital twin provides a reference model for anomaly detection in the live system. When the real facility's behavior diverges from the twin's prediction — beyond the expected noise envelope — this divergence is itself a security signal. An attack designed to evade sensor-level detection may still produce a behavioral signature at the process level that the twin comparison reveals.
The Cost of Waiting
The cost of a cybersecurity incident in industrial production is not merely the ransom or the remediation expense. It is the production loss, the delivery defaults, the equipment damage, the regulatory investigation, and the reputational consequences with customers and insurers. For a facility running AI-controlled heat treatment processes, an undetected data poisoning attack that causes systematic quality failures may not be identified until finished parts fail in the field — with consequences that extend far beyond the factory floor.
The NIS2 compliance clock is running. The AI Act obligations begin in 2026. The attack frequency continues to climb. And the gap between what conventional OT security tools can detect and what AI-specific attacks require to be detected is not closing on its own.
The manufacturers who move now — who build adversarial robustness into their AI architectures, who use their digital twins as security simulation environments, who implement physics-informed validation as a systematic check on every AI control decision — will have done more than protect themselves. They will have built a security competency that is genuinely differentiated, that satisfies current and anticipated regulatory requirements, and that provides a foundation for expanding AI deployment with confidence rather than trepidation.
The question is not whether your industrial AI systems are trustworthy. It is whether you have done the work to find out.
WINNIIO AB develops trustworthy AI frameworks and digital twin security implementations for industrial manufacturing. This paper reflects independent research and analysis.
References: Dragos Industrial Cybersecurity Report 2025; MSB Annual Report 2025; NIS2 Directive (EU 2022/2555); EU AI Act (2024/1689); IEC 62443 (Industrial Cybersecurity); IEC 63278-1 / AAS Part 4 Security (IDTA, June 2025); MITRE ATT&CK for ICS (v15); Goodfellow et al. (2015); Madry et al. (2018); Bagdasaryan et al. (2020); Bonawitz et al. (Google, 2017); Castro & Liskov (1999); Nature Scientific Reports, “Digital Twins as Security Instruments” (2025).