Deep Packet Inspection

Summary

Traditionally, control and regulation of Internet traffic have been managed by a firewall in the router device. However, routers can only scan the header of an IP packet which contains source, destination addresses, and some next-hop routing information.

Discussion

Why do we need Deep Packet Inspection?

How does DPI work and what are its techniques?

Traditional packet analysis tools only scanned packets at the IP and TCP layers, whereas DPI functions at the Application layer of the OSI reference model.

There are two different approaches to packet analysis - (1) Continual, full-scale traffic packet capture that requires high-speed processing and large storage arrays, which is expensive. (2) On-demand packet capture only when system compatibility issues occur (missing / damaged packets).

DPI is actually a combination of several techniques:

• Flow Tracking: Determines which packets are part of a flow between the source and destination computers. It's based on a 5-tuple identifier (SRC-IP, DEST-IP, SRC-PORT, DEST-PORT, PROTOCOL).
• Pattern Matching: String patterns (several, not just one) coded as regular expressions are matched with incoming packets. For example, regardless of port, L7-filter classifier for Linux's Netfilter can classify packets as HTTP, Jabber, Citrix, Bittorrent, FTP, etc.
• Statistical Analysis: Indicators (mean, variance) on absolute/relative packet sizes, flow rate per application.

For network traffic monitoring, analytics from Gartner estimate that flow analysis should be done 80% of the time, and packet capture with probes should be done 20% of the time.

What's the architecture of DPI deployment in a network?

DPI engines are usually deployed inline with firewalls in routers, SDN, and packet gateways. Offline packet analysis can also be performed for non-critical analysis.

DPI is a standard option in 4G LTE and 5G packet gateways (P-GWs). For instance, the backbone network of an ISP could be a 40-Gb/s system with four 10-Gb/s DPI modules.

In the dynamic service environment implied by cloud/SDN, DPI could potentially be co-located with network devices (as software running in virtual switches) or in the control layer (in the controller between applications and switches) due to its high CPU resource requirement. Real-time analytics using DPI is fed into big data analytics packages. This helps service providers understand what end users are doing and shape service offerings accordingly.

Network performances are proved by practical tests executed using real traffic in an ISP backbone network. DPI devices are designed to handle thousands of transactions each second. These devices analyse less than 1500 bytes per packet, which is not a very heavy load. But if this happens in real-time for every packet, there is network delay overhead.

What are the various other applications where DPI is used?

• Content Optimization: DPI can act as a proxy and modify contents in order to reduce still/video image quality or reformat web pages for mobile devices as per available bandwidth/device constraints so that users can enjoy content with reasonable performance.
• Billing Applications: ISP uses DPI to measure traffic volume and calibrate free and paid usages of network subscribers.
• Load Balancing: Redistribution of packet content to alternate servers in a load-balanced network to maintain uniform load across all deployed systems.
• User Behavior Analysis: Web/mobile applications can gauge subscriber behavior, assess what features are popular or which operations take too long, etc.
• Targeted Advertising: ISPs can inject advertisements into websites that match the assumed interests of the users according to their browsing habits.
• Copyright Enforcement: Automatically detect and block unauthorized sharing of music or video files on peer-to-peer platforms.
• Content Regulation: Recognize and block access to illegal/harmful content such as child-abuse websites. Censor content considered a threat to government or public stability.

If DPI analyses content in real-time, will it not slow down the network data transmission?

Many current DPI methods are resource-intensive and costly, especially for high bandwidth applications. Since it's done in real-time, it doesn't work on normal processors or switches.

DPI has become possible only in the last few years through advances in computer engineering and pattern matching algorithms. Specialized routers are now able to perform DPI. Routers armed with a dictionary of programs help identify the purposes behind the LAN and internet traffic they are routing. Vulnerability from repeat attacks from known viruses is removed. However, new viruses still pose a threat.

Interaction with legacy tools could also be a problem. Some firewalls simply aren't designed to support DPI, prompting worry about sudden performance drops or total failure of protective network systems.

In spite of these overheads, DPI is widely prevalent in network deployments because of the protection it offers from malicious or wasteful bandwidth usage by spams, viruses, and malware. By successfully blocking such spurious input requests from untrustworthy clients, the DPI servers are able to save the network from unnecessary congestion and possible DDoS attacks.

What are the negative consequences of DPI and how can misuse be prevented?

DPI is basically a packet sniffing technology on the network traffic, enabling operators to monitor what is happening in real-time. It was meant to be used for benevolent causes such as managing bandwidth, lawful surveillance, copyright enforcement, and network security.

However, DPI may be controversial from a customer privacy and net-neutrality standpoint. ISPs often use DPI servers to inspect Internet traffic to identify what traffic they want to slow down or restrict. At its root, DPI helps operators regain control over a network that primarily carries third-party applications and services by accurately identifying those applications in real-time.

Privacy infringement, corporate snooping, governmental suppression of facts and news, and advertisement implantation are some of the negative consequences of DPI. If an ISP sells streaming music, then OTT music applications are its competitors. ISPs can intentionally ignore/cause congestion or degrade the performance of the competing service.

One way to bypass DPI is to use traffic obfuscators as standalone software. These can change traffic signatures to look like traffic that isn't normally blocked by DPI.

What are the common tools used for packet analysis in DPI?

Governments of certain countries use proprietary and sophisticated DPI tools for online information censoring. The Great FireWall of China is one such example. They are implemented using a combination of flow analysers, filtering of certain IP ranges, flow redirection, URL filtering and Man-in-the-Middle techniques.

ntop, Netify Agent and libtins are open source utilities or toolkits in C/C++.

Corporates and Internet Service Providers may choose among the commonly available DPI tools:

• Wireshark: A popular free and open-source packet analyser which can be configured to be used for Intrusion Detection (ID). The tshark utility allows you to filter the contents of a pcap file from the command line to study network activity.
• Netfilter in Linux: Classifies packets as HTTP, Jabber, Citrix, Bittorrent, FTP, etc., regardless of port.
• Netflow from Cisco: Introduced on their routers to collect IP network traffic information as traffic enters/exits an interface and build Access Control Lists. It consists of a flow collector and analyser.
• SolarWinds Netflow: Network bandwidth monitoring (collection and analysis) tool. Free and paid versions available.
• Scrutinizer from Plixer: Can handle network flow analysis of Cisco and other vendors' network devices.

Milestones

Wireshark, an earlier part of the Ethereal project, is released as a free, open-source packet sniffing tool. It initially supports shallow packet inspection, only at the IP header level.

Traffic inspection solutions NetScreen (acquired by Juniper networks) are designed to be installed into firewall systems. Since the operation is expensive, it is triggered only on a need basis.

MIMESweeper, ClamAV, NetCache are some of the early open-source internet proxy caching servers introduced for scanning content to an ICAP server running anti-virus software.

The Great Firewall of China is deployed successfully. This internet censorship project commenced in 1998 for online traffic regulation in China.

DPI becomes a powerful network security tool with deployment on SDN/cloud servers.

DPI analysis tools feed network traffic data into Big Data Analytics for ISPs to derive critical insights on user behavior.