PAM 2.0 - Future of the Protocol Analysis Module

Posted by Terry Nelms on March 29, 2010 at 1:06 PM EDT.

The Protocol Analysis Module (PAM) is the component that performs deep packet inspection (DPI) in all of our IDS/IPS products.  We spend a lot of time optimizing PAM to ensure our products perform well on production networks.  In addition, we are always watchful for new hardware that could improve our DPI performance.  A couple of years ago, IBM Systems and Technology Group and IBM Research approached us about a project to develop a new massively multithreaded (MMT) processor, called the “Wire-Speed Processor” (WSP), targeted at high-speed network applications like PAM.  The problem, at that time, was that PAM was single-threaded and unable to take advantage of the large number of hardware threads.  PAM is a big, complex application (over 120,000 lines of executable code) so it would not be trivial to make it multithreaded.  However, we were very interested in the WSP; therefore, we started a project in X-Force Research to re-architect PAM (we call it PAM 2.0) to make it multithreaded.

The first challenge was determining how to efficiently exploit the concurrency in network traffic.  We chose to use two levels of parallelism:  packet level (every packet can be processed at the same time) and connection level (packets on different connections can be processed at the same time).  PAM 2.0 can process the link, network, and internet layer using packet level parallelism.  However, at the application layer, packet level parallelism did not make sense for TCP packets because PAM 2.0 has to process them in sequence order to maintain state.  So, for the application layer of TCP packets, we use connection level parallelism.  We believe this architecture will give us enough parallelism to fully utilize the large number of hardware threads available on WSP.

In addition, WSP has a couple of hardware accelerators that PAM 2.0 can use to further increase its performance.  The decompression accelerator will allow us to move our streaming decompression code to hardware.  This makes a huge difference in the latency and throughput of compressed streams (e.g., compressed web pages).  Also, we plan to take advantage of WSP’s regular expression accelerator for User Defined Events and Content Analyzer.  So, customers using those PAM features should get a performance boost.  Anyway, I think PAM 2.0 and WSP will make an impressive high-speed IPS (currently, it is still a research project).  If you are interested in learning more about PAM 2.0 and WSP you can read a recent IBM Journal of Research and Development article written by me and some friends from IBM Research.