Automated detection of network vulnerabilities using artificial intelligence
07 Nov, 2020
3 Minutes
Valentin Kiechle
German-speaking Data Scientists wanted! Join an amazing team in Karlsruhe or Remote. Apply here.
German-speaking Data Scientists wanted!
Join an amazing team in Karlsruhe or Remote. Apply here.

For the communication between computers, machines or mobile devices of all kinds, a digital network is always needed for the exchange of communication. Such networks follow a number of standardized protocols to ensure smooth communication between all participants. The basis for this is the 7-layer ISO/OSI reference model which has established itself as an idealized example structure of such a network. In most real world applications simplified but strongly related structures and protocol families can be found. The best known variant is the TCP/IP reference model, which is used in most of the internet.


Networks of this kind are not without their weak points. Incorrect protocol implementation, procedural weaknesses and, of course, the possibility of spreading malware allow malicious third parties to gain an advantage at the expense of normal communication participants. The extent of the potential damage generally depends on the specific network, its participants and the attackers objectives, but can be existentially threatening for the victims.

Security measures

IT Sicherheitsprozess en

The research branch of IT Security deals, among other things, with solutions for securing network communication. In each case, a number of possible security objectives such as confidentiality, integrity and availability can be targeted. For each such goal, there are best practices and tools as well as measures and training of employees to improve the security level. IT security as a whole should by no means be understood as a product, but as a continuous process.

New attacks and vulnerabilities are constantly being discovered and exploited, only to be patched and cleaned up afterwards. Unfortunately, this cycle allows for the incurrence of costs for damage repair on the part of potential victims. Great efforts should therefore always be put into preventive measures.

Test network security

In order to stay one step ahead of the attacker in the security of new systems, existing networks and other communication channels, automated tests against the stability and integrity of the network are recommended. This means that real attacks are simulated and carried out in practice without any intention of damage. Especially communication artifacts like protocol messages are a good entry point for the test algorithm. There are several possibilities to design such messages with attack character. These differ primarily in their prerequisites: some require a complete insight into the protocol specifications for the independent creation of new messages, some get by with far less. In theory, this is an important distinction between the approaches. In practice, however, concrete scenarios are often very complex. Many industrial plants use protocol families that are not used in the general Internet and are therefore rarely available in free data sets for public research. Furthermore, there are also company-internal protocols that are guarded as a trade secret. In all these cases no protocol specifications are available. So how can artificial messages be generated that simulate an attack?

Reverse engineering of protocols

In order to still be able to create messages according to the specifications, there are different approaches to reconstruct protocol specifications. For this purpose communication artifacts like collected sample messages are used to get insights about the general structure and flow of the underlying protocol. Traditional approaches rely on a rule-based approach and use simple algorithms to perform intermediate steps like clustering or filtering of unwanted outliers. These approaches are limited by the expert knowledge of the inventors and have problems to transfer to more modern protocols or newer versions. Artificial intelligence offers a more flexible approach for the automatic learning of protocol specifications. With a new approach, our colleague Valentin Kiechle has been able to show in his master thesis that a group of deep learning algorithms can solve the problem for an unknown protocol when working together.

Automated attack simulation

The use of the above mentioned method for reverse engineering of protocols using neural networks allows the arbitrary learning of new protocols without an explicit prior knowledge of them. With the artificial intelligences trained this way it is also possible to generate new messages. By using a further AI on a higher level, the attack character of the messages can be developed in a post-processing step. The combination of these AIs thus allows automated attack simulation on a network whose protocols are not known or available to the test expert. Thus, any system can be tested for many possible vulnerabilities. All that is needed is a corpus of normal communication data from the network, from which the AIs can learn their knowledge of the protocol.


  • Anyone who communicates digitally must use a network, which in turn uses standardized protocols
  • Security gaps in these networks can be detected by simulating attacks
  • For such simulations new messages with attack character have to be designed according to the used protocols. For this purpose the often not available protocol specifications must be known
  • Thanks to reverse engineering of protocols, these specifications can be automatically generated
  • Artificial intelligence is a flexible approach for this reverse engineering
  • The trained AI then provides a good basis for automated generation of messages for attack purposes

Keep reading