In 1975, a book was published that changed the way we approach complex problems. Inspired on how nature works “Adaptation in Natural and Artificial Systems” set the bases of genetic algorithms. The release date of this blogpost is strongly linked to that book, it is a symbolic tribute to its author, John Henry Holland, who passed out exactly two years ago. We strongly encourage everyone to embrace its legacy.
Recently our CTO, Yaniv Karta, released a blog post about the history of z9, our detection engine, and how it was successfully applied to protect Android and iOS devices from known and unknown threats using the Zero Packet Inspection (ZPI) approach. ZPI is a high efficiency intrusion detection methodology totally compliant with users’ privacy needs.
In this blogpost, we will see how our z9 engine is trained to achieve such results and how the ZPI methodology was ported to the Windows platform. Also, we will discuss how this can help to protect all our devices in an every day more connected and privacy aware world.
It’s all about data
z9 is a state of the art machine learning engine with several ground breaking optimizations. But machine learning is about data. Really. That’s what it’s all about. Your model can be, at most, as good as the information you use to train it. For this reason, a deep understanding of the behavior of the system you are modeling is mandatory to achieve industry’s standard results.
Our researchers in the zLabs team developed a series of test cases representative of different normal states of Android and iOS devices. These states must be characterized and represented with data, ZPI data. Then the fun starts. The same information must be collected while the device is being attacked (Figure 1) to register the change in the key attributes. From the perspective of detecting network intrusions, the attacks that we want to detect are network scans. Detecting and preventing network scans is the key to stop further and more serious attacks types.
The scans we use to train our engine are TCP, IP, UDP and ARP. Also, the first 3 scans can be performed over IPv4 or IPv6 addresses. Adding the normal behavior of the device, we have 8 different classes to identify. Just for completeness (and to understand future results), the classes are (from 0 to 7): NORMAL, TCP (IPv4), ARP, IP (IPv4), UDP (IPv4), TCP (IPv6), IP (IPv6), UDP (IPv6).
Figure 1. Data collection and centralization process.
Training process
Once the data is collected, the training pipeline shown in Figure 2 is followed. Each step consists of:
- Data set generation: From the collected data we create two data sets, one to train z9 and another one to test how well the generated model performs over new, unseen data.
- Feature optimization: To extract more information of the data a pre-training step is performed to extract the most relevant features to direct the training.
- Denoising: To guarantee that no noise is present in the data set an unsupervised learning step is performed to clean the data.
- Training: With the optimized attributes and denoised data set as inputs a z9 classifier is trained.
Figure 2. Training pipeline used to achieve the optimized z9 classifier.
Show time!
What can be obtained with the process described so far? The answer is state of the art, high accuracy network intrusion detection systems completely compliant with users’ privacy needs. So, let’s look at some details then.
To analyze how good a machine learning model performs, it is common to use confusion matrices, a very simple chart that relates the predicted classes (in columns) for a given data set against its true value (in rows).
CM 1. Confusion matrices for the Android network classifier.
In CM 1, such a matrix is shown for the Android classifier. Each class number corresponds with the classes showed above. The error over the test set is 0.02 % which means that we’ll need to collect almost 5000 samples (4579 to be precise) to find a single classification error. For the training set, no error is found in the confusion matrix meaning that our model learned our data.
What about iOS? On CM 2, the same results for the iOS classifier are shown. Once again, no error is found on the training data set. On the test set, 0.25% error is found, which means that 1 in 400 samples will be miss-classified. Not bad, huh?
CM 2. Confusion matrices for the iOS network classifier.
The forgotten little brother
The mobile industry is dominated by Android and iOS devices, but this doesn’t mean that ZPI approach cannot be used on other platforms. As a proof of this, ZPI was ported to Windows. The same methodology applied for Android and iOS was used, the only difference is in the way the data is collected.
For Android, the data is gathered reading the network statistics from the /proc filesystem while in iOS the sysctlbyname system call is used. On Windows, the network global statistics can be accessed using the following IP helper functions:
- GetIpStatisticsEx
- GetIcmpStatistics
- GetIcmpStatisticsEx
- GetTcpStatisticsEx
- GetIfTable
- GetUdpStatisticsEx
- GetInterfaceInfo
Using these functions, 649 different network attributes can be obtained. The data collection process was performed in the same manner as before, using the same states and attacks types.
After the training process, we end up with the confusion matrices shown in CM 3. In this case, the error over the training set is 0.09% and for the test set 0.3%. This means that for new unseen data, we will find a misclassification in about 320 sampled points. To effectively detect a network scan, a single detection is enough so even when the accuracy of this classifier is lower than for the Android and iOS cases, the overall performance can be considered very high.
CM 3. Confusion matrices for the Windows network classifier.
What about the real world?
The results shown so far seem to be promising, but let’s analyze how the classifiers perform in real devices outside a training environment. To test them, the following devices where used:
- Samsung Galaxy S6 running Android 6.
- iPhone 6 with iOS 8.
- HP envy 13 with Windows 10.
Each device was scanned using nmap, and the time until the first detection is accounted for different timing parameters and scan types. In Tables 1 and 2, the duration of the scans and the detection time is shown for each device/platform. It is easy to see that even when the -T2 flag is used (which increase the time between packets to diminish the scan fingerprint), most of the scans are detected by every classifier, outperforming every commercial IDS available in the industry.
Table 1. Scan duration and time until first detection for each classifier. The -T3 flag was used to perform the scans.
Table 2. Scan duration and time until first detection for each classifier. The -T2 flag was used to perform the scans.
What’s next?
The results shown here are only a small sample of the full capabilities of z9 combined with research and development with a strong emphasis on security and privacy.
Fortunately for us, the ZPI approach is not restricted to mobile devices or to the platforms described here. Any OS exposing network statistics can be used as a ground to develop a ZPI based IDS.
There are several advantages not described here about ZPI such as the very little needed processing power. In an IoT world in which the connectivity is increasing every day for even the smallest devices, ZPI can be extended to protect our gadgets without penalizing its battery use or invading user’s data. ZPI in conjunction with z9 can also be used to protect our home and enterprise routers from external and internal attackers, connected cars and other RT OS (such as QNX,VxWorks) can also be protected using the proposed approach. The future is bright and z9 will be there to protect you.
We will also announce our Zero Payload capabilities in regards to firmware attack and kernel attack detection without any visibility. The approach will show we can detect a large variety of attacks over multiple attack surfaces with a minimal sets of attributes and high accuracy. We will reveal more details on Zero Payload detection approach on the next series of blogposts.
Peer review process
The Zero Packet Inspection Paper is available for peer review. If you would like to participate, please, notify us at signal@zmpr.co.
