VPP Next Generation Firewall
Introduction
VPP next generation firewall (NGFW) is a combination of L2-L4 ACLs, IPSec, and Snort based packet inspection.
Received packets are checked against ACL permit/deny rules, sent to Snort for packet inspection, and then forwarded or dropped accordingly. Forwarded packets are encrypted and encapsulated in an IPSec tunnel.
VPP access control list (ACL) are used as a security feature for blocking or permitting traffic based on L2-L4 layers. Once the application receives a packet, it checks whether there is any ACL permit/deny rule, and makes the decision of forwarding the packet accordingly.
ACL rules are applied to accept or deny a packet based on layer 2 to layer 4 protocols. In L3-L4 ACLs, IP address mask and layer 4 protocols of the packets are checked, to apply permit and deny rules. For L2-L3 ACLs, source MAC addresses of the packets are checked and permit and deny rules apply on specific MAC addresses along with L3 addresses. If the rules permit packet to transfer, packet is forwarded by using L3 forwarding table.
Snort is an Intrusion Detection/Prevention System (IDS/IPS). Snort uses a series of rules that help define malicious network activity. Packets matching against these rules generate alerts for the user. You can use community curated rules and you can define your own to either replace of supplement them. Beyond alerts and logging, other actions may be defined, like dropping packets.
Vectorscan is a fork of Hyperscan that runs on Arm and other architectures. It’s a regular expression matching library. As part of deep packet inspection performed by Snort, contents of the packets can be matched against regular expression patterns. The matching engine built into Snort is slow. Vectorscan works as a plugin that replaces the built-in search engine and provides a significant performance increase thanks to use of vector extensions and other optimizations.
Through use of the Snort plugin in VPP, Snort can be added to the VPP through two new graph nodes: snort-enq and snort-deq. These allow packets to pass through Snort, letting Snort perform its actions like dropping packets.
This guide explains how to combine ACLs, Snort intrusion detection/ packet inspection, and IPSec together to form a firewall.
A traffic generator can be used to produce traffic that matches one or more of the configured rules. Traffic is dropped at the first match, or passed if it matches none of the configured rules. A traffic generator can also be enabled to send encrypted IPSec packets through a tunnel. At the receiving side of the tunnel, the packets are then decrypted, checked, and then forwarded or dropped.
A traffic generator can be configured to send synthetic traffic or play back captured packets. A good packet capture to use is one containing malicious traffic.
The Malware Capture Facility Project started at the CTU University is a good source of malicious packet captures. Their samples are public and free to use under [CC-BY](https://creativecommons.org/licenses/by/2.0/) license.
A good sample of small size to try is: botnet-capture-20110810-neris.pcap
Snort requires rules to perform meaningful work. For the case of synthetic traffic, an example rules file with a single rule matching the generated traffic has been prepared. If a packet capture is provided, the community rules are used instead. These offer a more realistic workload.
Memif Connection
Shared memory packet interface (memif) is software emulated Ethernet interface, which provides high performance packet transmit and receive between VPP and user application or between multiple VPP instances.
In this setup, one memif interface is configured to connect from the VPP based traffic generator to the firewall. The firewall side uses DPDK zero-copy memif interfaces. Traffic that does not match configured rules is sent back to the traffic generator via a second memif interface. The traffic generator side uses VPP’s native memif interfaces. A second traffic generator generates encrypted packets that are sent on the second memif interface to the firewall. After decryption and inspection, the packets are forwarded out of the first interface.
Memif connection
Overview
In the memif connection scenario, the main operations of each script are as follows:
run_vpp_tg.sh
Runs a VPP instance for the traffic generator on the specified CPU cores
Creates two VPP memif interfaces in server role
Configures software traffic generators
Starts the traffic generators
run_vpp_fw.sh
Runs a VPP instance for the firewall on the specified CPU cores
Creates two DPDK memif interfaces in client role with zero-copy enabled
Brings interfaces up and set interfaces IP addresses
Adds IP route entry making the second memif interface the gateway
Adds ACL rules to memif interface
Adds Snort interfaces
Launches Snort connected to the VPP instance
traffic_monitor.sh
Monitors VPP interface counters and firewall throughput with VPP
show interfaceandshow runtimecommands
stop.sh
Stops VPP firewall and traffic generator instances and the Snort instance
For detailed usage on VPP CLI commands used in scripts, refer to the following links:
Execution
Note
This execution requires at least three isolated cores for VPP workers. Cores 2-5 are assumed to be isolated in this guide.
Quickly set up VPP firewall/traffic generator instances with synthetic traffic.
cd $NW_DS_WORKSPACE/dataplane-stack
./usecase/ngfw/run_vpp_tg.sh -c 1,2,3
./usecase/ngfw/run_vpp_fw.sh -c 1,4 -d 1,1,5
Note
Use
-hto check scripts supported options.
Snort outputs its alerts to a log directory created by run_vpp_fw.sh.
You can observe the alerts as they are written. Use the tail command printed
to the terminal by run_vpp_fw.sh.
Examine VPP firewall’s interfaces rx/tx counters and packet processing runtime:
./usecase/ngfw/traffic_monitor.sh
Below is key output:
Name Idx State MTU (L3/IP4/IP6/MPLS) Counter Count
Ethernet0 1 up 9000/0/0/0 rx packets 12
rx bytes 768
drops 9
ip4 12
Ethernet1 2 up 9000/0/0/0 tx packets 3
tx bytes 192
local0 0 down 0/0/0/0
Thread 1 vpp_wk_0 (lcore 4)
Time 3.0, 10 sec internal node vector rate 1.48 loops/sec 11379641.56
vector rates in 8.9125e0, out 9.9028e-1, drop 2.9708e0, punt 0.0000e0
Name State Calls Vectors Suspends Clocks Vectors/Call
Ethernet1-output active 3 3 0 2.90e1 1.00
Ethernet1-tx active 3 3 0 2.20e1 1.00
acl-plugin-fa-worker-cleaner-pinterrupt wa 6 0 0 1.45e1 0.00
acl-plugin-in-ip4-fa active 3 9 0 7.89e0 3.00
dpdk-input polling 35141382 12 0 3.71e6 0.00
drop active 9 9 0 5.33e0 1.00
error-drop active 9 9 0 5.44e0 1.00
ethernet-input active 5 12 0 9.92e0 2.40
interface-output active 3 3 0 8.67e0 1.00
ip4-drop active 3 3 0 7.00e0 1.00
ip4-inacl active 3 6 0 1.73e1 2.00
ip4-input active 5 12 0 8.92e0 2.40
ip4-lookup active 3 3 0 7.00e0 1.00
ip4-rewrite active 3 3 0 1.53e1 1.00
snort-deq interrupt wa 7 15 0 1.05e1 2.14
snort-enq active 8 15 0 5.73e1 1.88
unix-epoll-input polling 34284 0 0 1.10e1 0.00
Note
VPP
Ethernet0is the aliased name of the input memif interface in the example.VPP
Ethernet1is the aliased name of the output memif interface in the example.vector ratesprovide insights into the packet processing throughput of a specific node or function in VPP.Vectors/Callmeasures packet processing efficiency in VPP as operations per function call for a specific node or function.snort-enqandsnort-deqare graph nodes responsible for communicating with the Snort instance. Dropped packets will not be dequeued.
Stop all instances:
./usecase/ngfw/stop.sh
If you try and launch a new instance while the old one is still running,
the old one will be stopped first. To allow multiple instances to coexist on one
machine, a suffix is used to avoid clashes. By default this suffix will be your
username but you can specify it manually using the -s option that all
scripts accept.
Using a recorded packet capture
If the suggested packet capture file file is downloaded, it can be used with the traffic generator by passing the filename to the script.
./usecase/ngfw/run_vpp_tg.sh -c 1,2,3 -f botnet-capture-20110810-neris.pcap
Launching the firewall now will enable Snort community rules (if no rules are
specified using the -r option). You can read more about the rules on the
Snort website.
Snort community rules are downloaded automatically if needed. You can also trigger the download manually using the provided script.
./usecase/ngfw/get_community_rules.sh
By default, Snort community rules will be used unless the traffic generator
is using synthetic traffic. In that case, the example.rules file will be
used that matches some of the synthetic traffic generated. Custom rules can be
applied by passing in the rules filename using the -r option of the firewall
script.
Ethernet Connection
In this scenario the firewall is running on a machine connected to physical interfaces, with the other end connected to either a machine used as the software-based traffic generator (like VPP/TRex/TrafficGen), or a hardware traffic generator (like IXIA/Spirent Smartbits).
Ethernet connection
Find out which DUT interfaces are connected to the traffic generator.
sudo ethtool --identify <interface_name> will typically blink a light on the
NIC to help identify the physical port associated with the interface.
Get interface names and PCIe addresses from lshw command:
sudo lshw -c net -businfo
The output will look similar to:
Bus info Device Class Description
====================================================
pci@0000:07:00.0 eth0 network RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller
pci@0001:01:00.0 enP1p1s0f0 network MT27800 Family [ConnectX-5]
pci@0001:01:00.1 enP1p1s0f1 network MT27800 Family [ConnectX-5]
Of the two interfaces connected to the traffic generator, arbitrarily choose one
to be the input interface and the other to be the output interface. In this
setup example, enP1p1s0f0 at PCIe address 0001:01:00.0 is the input
interface, and enP1p1s0f1 at PCIe address 0001:01:00.1 is the output
interface.
Get the MAC address of the input interface enP1p1s0f0 connected to the
traffic generator via ip link show enP1p1s0f0. This MAC address will be used
as destination MAC address of generated traffic. Below is a sample output:
1: enP1p1s0f0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 8996 qdisc mq state UP mode DEFAULT group default qlen 1000
link/ether b8:ce:f6:10:e4:6c brd ff:ff:ff:ff:ff:ff
The user can quickly run firewall over NIC connection using the scripts located
at: $NW_DS_WORKSPACE/dataplane-stack/usecase/ngfw.
Overview
In Ethernet connection scenario, the main operations of scripts are as follows:
run_vpp_fw.sh
Runs a VPP instance for the firewall on the specified CPU cores
Attaches two DPDK interfaces on the specified PCIe addresses to VPP
Brings the interfaces up and set their IP addresses
Adds IP route entries making memif2 the gateway
Adds ACL rules to memif interface
Adds Snort interfaces
Launches Snort connected to the VPP instance
traffic_monitor.sh
Monitors VPP interface counters and firewall throughput via VPP
show interfaceandshow runtimecommands
stop.sh
Stops VPP firewall instance
Execution
Note
This execution requires at least two isolated cores. Cores 2 and 3 are assumed to be isolated in this guide.
Quickly set up VPP firewall with input/output interface PCIe addresses on specified cores for 1 flow:
cd $NW_DS_WORKSPACE/dataplane-stack
./usecase/ngfw/run_vpp_fw.sh -p 0001:01:00.0,0001:01:00.1 -c 1,2 -d 1,1,3
Note
Replace sample addresses in above command with desired PCIe addresses on DUT.
Send packets using the traffic generator on the other machine connected to the first NIC. The packets can be sourced from a pcap suggested in the introduction of this guide or any of the pcaps available publicly. A good source of these is the NETRESEC website.
IP and MAC addresses should be configured to hit or avoid ACLs as needed for testing. VPP firewall will forward packets that are accepted by ACLs and Snort to the output interface.
Examine VPP firewall’s interfaces rx/tx counters and packet processing runtime:
./usecase/ngfw/traffic_monitor.sh
Here is key output:
Name Idx State MTU (L3/IP4/IP6/MPLS) Counter Count
Ethernet0 1 up 9000/0/0/0 rx packets 17208504
rx bytes 1032510240
drops 319319
ip4 17208564
rx-miss 29474149
Ethernet1 2 up 9000/0/0/0 tx packets 319514
tx bytes 19170840
local0 0 down 0/0/0/0
Thread 1 vpp_wk_0 (lcore 2)
Time 172.7, 10 sec internal node vector rate 8.22 loops/sec 675728.01
vector rates in 2.1111e6, out 3.7183e4, drop 3.7207e4, punt 3.4738e-2
Name State Calls Vectors Suspends Clocks Vectors/Call
Ethernet1-output active 87128 6422321 0 4.02e-1 73.71
Ethernet1-tx active 87128 6422321 0 1.11e0 73.71
acl-plugin-fa-worker-cleaner-pinterrupt wa 331 0 0 6.52e1 0.00
acl-plugin-in-ip4-fa active 87302 12853702 0 1.96e0 147.23
dpdk-input polling 980738007 345357551 0 7.26e0 .35
drop active 174604 6426513 0 4.72e-1 36.81
error-drop active 174604 6426513 0 2.93e-1 36.81
error-punt active 6 6 0 2.33e2 1.00
ethernet-input active 44736127 345357551 0 1.05e0 7.72
interface-output active 87128 6422321 0 3.24e-1 73.71
ip4-drop active 87302 3213270 0 4.34e-1 36.81
ip4-inacl active 87302 9640459 0 1.31e0 110.43
ip4-input-no-checksum active 44736122 345357545 0 4.46e-1 7.72
ip4-lookup active 87302 6427189 0 4.16e-1 73.62
ip4-rewrite active 87302 6427189 0 1.43e0 73.62
punt active 6 6 0 1.13e2 1.00
snort-deq interrupt wa 87302 19276023 0 2.44e0 220.79
snort-enq active 44823424 19284215 0 1.09e1 .43
unix-epoll-input polling 956826 0 0 2.53e1 0.00
Note
VPP
Ethernet0is the aliased name of the input interface, which is at PCIe address0001:01:00.0in the example.VPP
Ethernet1is the aliased name of the output interface, which is at PCIe address0001:01:00.1in the example.
Stop VPP firewall:
./usecase/ngfw/stop.sh
Known Issues
Snort core configuration
On some machines Snort may fail to pin worker threads to higher cores (>16). Please watch the terminal for configuration errors. These will cause Snort to fail to start.