YOU ARE AT:OpinionReader Forum: The challenges of PCAP and 'big data'

Reader Forum: The challenges of PCAP and ‘big data’

Editor’s Note: In an attempt to broaden our interaction with our readers we have created this Reader Forum for those with something meaningful to say to the wireless industry. We want to keep this as open as possible, but we maintain some editorial control to keep it free of commercials or attacks. Please send along submissions for this section to our editors at: [email protected].

Packet capture file format for packet capture is the preferred medium used throughout the industry for debugging, replication, simulation, sampling, testing and analysis where a network trace is required. It conveniently facilitates offline usage, allowing engineers to meticulously navigate through a network trace in their own time categorically defining hidden handover messages, hosts, sources, destinations, interacting nodes and protocols – all in plain text. Communications service providers, cyber-security firms and system integrators use the Libpcap library for packet sniffing, monitoring, stress-testing, equipment hardening/robustness testing and intrusion detection.

For its host of practical use cases PCAP weighs heavily on the minds of manufacturers as well. It provides the “best” network model accessible, where it is not possible to directly connect to the live network 100% of the time. This means engineers can rigorously test their products prior to live network integration with a degree of confidence.

The criteria for a “good” PCAP is based on completeness, interfaces traced, decoded IEs and speed at which it can run. The typical method of obtaining a PCAP is to trace the network interface(s) in question for a definable period of time until the user is confident an adequate representation of the network traffic has been captured.

However, following the introduction of “10 gigabits per second” and the “big data” phenomena, PCAP traces today are bloated, complex and in some instances unreadable. For example, Wireshark – the de-facto third-party open source tool for PCAP interpretation – can only process a file of up to two gigabytes in size, and although it boasts an impressive 200 protocol decode library, it would struggle to function if it had to process a 10 Gbps sample. Realistically, it can process at 100 megabits per second. This means the seamless trace of 5 GB from the 10 Gbps interface cannot be processed by the industry’s most comprehensive tool.

As networks now run at multiples of 10 Gbps composed of voice over IP/voice over LTE, video, call control, messaging and traditional telephony you may require:

  • A “bulk” capture containing every protocol.
  • A “filtered” capture containing only protocols of interest.
  • A “subscriber” capture containing only the sessions associated with specific subscriber(s).

The current method in effect is to take a complete or bulk sample and attach an analysis suite to decipher, extract and reconstruct a PCAP with relevant packets. This technique invokes an abundance of logic and complex algorithms to ensure zero packet loss, correlation, accurate time stamping and intact synchronisation. This, in actuality, cannot be guaranteed, not to mention the lost time it takes to analyse a whole 5 GB, 10 Gbps PCAP file.

The alternative is to intelligently and passively process packets; pre-filtering and extracting only packets of interest in real-time. Implementing a hybrid hardware/software product utilizing de-tunnelling and DPI to ensure a refined PCAP compatible with industry-standard third-party tools and full end-to-end correlation enables the user to conduct their role efficiently.

The reduced capital/operating expense, floor space, complexity of application needed and size/data rate means that enabling equipment like this is efficient, cost-effective and portable, allowing it to be used to capture PCAPs remotely or onsite in conjunction with a tap point.

James Goodwin joined Anite in 2000 as project manager of the GPRS conformance product line. After managing the development of many Anite products through the technology progression from 2G to 3G, he joined the company’s product management team in 2007. He is now director of product management, responsible for the portfolio management of Anite’s Handset Testing products. Goodwin is a chartered engineer and holds a Ph.D. and MEng in Electrical and Electronic Engineering.

ABOUT AUTHOR

Kelly Hill
Kelly Hill
Kelly reports on network test and measurement, as well as the use of big data and analytics. She first covered the wireless industry for RCR Wireless News in 2005, focusing on carriers and mobile virtual network operators, then took a few years’ hiatus and returned to RCR Wireless News to write about heterogeneous networks and network infrastructure. Kelly is an Ohio native with a masters degree in journalism from the University of California, Berkeley, where she focused on science writing and multimedia. She has written for the San Francisco Chronicle, The Oregonian and The Canton Repository. Follow her on Twitter: @khillrcr