.. # Copyright (c) 2024, Arm Limited. # # SPDX-License-Identifier: Apache-2.0 ################################# VPP Next Generation Firewall ################################# ************ Introduction ************ VPP `next generation firewall`_ (NGFW) is a combination of L2-L4 ACLs, IPSec, and Snort based packet inspection. Received packets are checked against ACL permit/deny rules, sent to Snort for packet inspection, and then forwarded or dropped accordingly. Forwarded packets are encrypted and encapsulated in an IPSec tunnel. VPP access control list (ACL) are used as a security feature for blocking or permitting traffic based on L2-L4 layers. Once the application receives a packet, it checks whether there is any ACL permit/deny rule, and makes the decision of forwarding the packet accordingly. ACL rules are applied to accept or deny a packet based on layer 2 to layer 4 protocols. In L3-L4 ACLs, IP address mask and layer 4 protocols of the packets are checked, to apply permit and deny rules. For L2-L3 ACLs, source MAC addresses of the packets are checked and permit and deny rules apply on specific MAC addresses along with L3 addresses. If the rules permit packet to transfer, packet is forwarded by using L3 forwarding table. `Snort`_ is an Intrusion Detection/Prevention System (IDS/IPS). Snort uses a series of rules that help define malicious network activity. Packets matching against these rules generate alerts for the user. You can use community curated rules and you can define your own to either replace of supplement them. Beyond alerts and logging, other actions may be defined, like dropping packets. `Vectorscan`_ is a fork of Hyperscan that runs on Arm and other architectures. It's a regular expression matching library. As part of deep packet inspection performed by Snort, contents of the packets can be matched against regular expression patterns. The matching engine built into Snort is slow. Vectorscan works as a plugin that replaces the built-in search engine and provides a significant performance increase thanks to use of vector extensions and other optimizations. Through use of the Snort plugin in VPP, Snort can be added to the VPP through two new graph nodes: snort-enq and snort-deq. These allow packets to pass through Snort, letting Snort perform its actions like dropping packets. This guide explains how to combine ACLs, Snort intrusion detection/ packet inspection, and IPSec together to form a firewall. A traffic generator can be used to produce traffic that matches one or more of the configured rules. Traffic is dropped at the first match, or passed if it matches none of the configured rules. A traffic generator can also be enabled to send encrypted IPSec packets through a tunnel. At the receiving side of the tunnel, the packets are then decrypted, checked, and then forwarded or dropped. A traffic generator can be configured to send synthetic traffic or play back captured packets. A good packet capture to use is one containing malicious traffic. The Malware Capture Facility Project started at the CTU University is a good source of malicious packet captures. Their samples are public and free to use under [CC-BY](https://creativecommons.org/licenses/by/2.0/) license. A good sample of small size to try is: `botnet-capture-20110810-neris.pcap`_ Snort requires rules to perform meaningful work. For the case of synthetic traffic, an example rules file with a single rule matching the generated traffic has been prepared. If a packet capture is provided, the community rules are used instead. These offer a more realistic workload. **************** Memif Connection **************** Shared memory packet interface (memif) is software emulated Ethernet interface, which provides high performance packet transmit and receive between VPP and user application or between multiple VPP instances. In this setup, one memif interface is configured to connect from the VPP based traffic generator to the firewall. The firewall side uses DPDK zero-copy memif interfaces. Traffic that does not match configured rules is sent back to the traffic generator via a second memif interface. The traffic generator side uses VPP's native memif interfaces. A second traffic generator generates encrypted packets that are sent on the second memif interface to the firewall. After decryption and inspection, the packets are forwarded out of the first interface. .. figure:: ../images/ngfw_memif.png :align: center :width: 500 Memif connection Overview ======== In the memif connection scenario, the main operations of each script are as follows: ``run_vpp_tg.sh`` * Runs a VPP instance for the traffic generator on the specified CPU cores * Creates two VPP memif interfaces in server role * Configures software traffic generators * Starts the traffic generators ``run_vpp_fw.sh`` * Runs a VPP instance for the firewall on the specified CPU cores * Creates two DPDK memif interfaces in client role with zero-copy enabled * Brings interfaces up and set interfaces IP addresses * Adds IP route entry making the second memif interface the gateway * Adds ACL rules to memif interface * Adds Snort interfaces * Launches Snort connected to the VPP instance ``traffic_monitor.sh`` * Monitors VPP interface counters and firewall throughput with VPP ``show interface`` and ``show runtime`` commands ``stop.sh`` * Stops VPP firewall and traffic generator instances and the Snort instance For detailed usage on VPP CLI commands used in scripts, refer to the following links: - `VPP configuration dpdk section reference`_ - `VPP set interface ip address reference`_ - `VPP ip neighbor cli reference`_ - `VPP set interface state reference`_ - `VPP Snort plugin reference`_ - `VPP ACL cli`_. - `VPP ACL examples`_ Execution ========= .. note:: This execution requires at least three isolated cores for VPP workers. Cores 2-5 are assumed to be isolated in this guide. Quickly set up VPP firewall/traffic generator instances with synthetic traffic. .. code-block:: shell cd $NW_DS_WORKSPACE/dataplane-stack ./usecase/ngfw/run_vpp_tg.sh -c 1,2,3 ./usecase/ngfw/run_vpp_fw.sh -c 1,4 -d 1,1,5 .. note:: - Use ``-h`` to check scripts supported options. Snort outputs its alerts to a log directory created by ``run_vpp_fw.sh``. You can observe the alerts as they are written. Use the ``tail`` command printed to the terminal by ``run_vpp_fw.sh``. Examine VPP firewall's interfaces rx/tx counters and packet processing runtime: .. code-block:: shell ./usecase/ngfw/traffic_monitor.sh Below is key output: .. code-block:: none Name Idx State MTU (L3/IP4/IP6/MPLS) Counter Count Ethernet0 1 up 9000/0/0/0 rx packets 12 rx bytes 768 drops 9 ip4 12 Ethernet1 2 up 9000/0/0/0 tx packets 3 tx bytes 192 local0 0 down 0/0/0/0 .. code-block:: none Thread 1 vpp_wk_0 (lcore 4) Time 3.0, 10 sec internal node vector rate 1.48 loops/sec 11379641.56 vector rates in 8.9125e0, out 9.9028e-1, drop 2.9708e0, punt 0.0000e0 Name State Calls Vectors Suspends Clocks Vectors/Call Ethernet1-output active 3 3 0 2.90e1 1.00 Ethernet1-tx active 3 3 0 2.20e1 1.00 acl-plugin-fa-worker-cleaner-pinterrupt wa 6 0 0 1.45e1 0.00 acl-plugin-in-ip4-fa active 3 9 0 7.89e0 3.00 dpdk-input polling 35141382 12 0 3.71e6 0.00 drop active 9 9 0 5.33e0 1.00 error-drop active 9 9 0 5.44e0 1.00 ethernet-input active 5 12 0 9.92e0 2.40 interface-output active 3 3 0 8.67e0 1.00 ip4-drop active 3 3 0 7.00e0 1.00 ip4-inacl active 3 6 0 1.73e1 2.00 ip4-input active 5 12 0 8.92e0 2.40 ip4-lookup active 3 3 0 7.00e0 1.00 ip4-rewrite active 3 3 0 1.53e1 1.00 snort-deq interrupt wa 7 15 0 1.05e1 2.14 snort-enq active 8 15 0 5.73e1 1.88 unix-epoll-input polling 34284 0 0 1.10e1 0.00 .. note:: - VPP ``Ethernet0`` is the aliased name of the input memif interface in the example. - VPP ``Ethernet1`` is the aliased name of the output memif interface in the example. - ``vector rates`` provide insights into the packet processing throughput of a specific node or function in VPP. - ``Vectors/Call`` measures packet processing efficiency in VPP as operations per function call for a specific node or function. - ``snort-enq`` and ``snort-deq`` are graph nodes responsible for communicating with the Snort instance. Dropped packets will not be dequeued. Stop all instances: .. code-block:: shell ./usecase/ngfw/stop.sh If you try and launch a new instance while the old one is still running, the old one will be stopped first. To allow multiple instances to coexist on one machine, a suffix is used to avoid clashes. By default this suffix will be your username but you can specify it manually using the ``-s`` option that all scripts accept. Using a recorded packet capture =============================== If the suggested `packet capture file `_ file is downloaded, it can be used with the traffic generator by passing the filename to the script. .. code-block:: shell ./usecase/ngfw/run_vpp_tg.sh -c 1,2,3 -f botnet-capture-20110810-neris.pcap Launching the firewall now will enable Snort community rules (if no rules are specified using the ``-r`` option). You can read more about the rules on the `Snort website `_. Snort community rules are downloaded automatically if needed. You can also trigger the download manually using the provided script. .. code-block:: shell ./usecase/ngfw/get_community_rules.sh By default, Snort community rules will be used unless the traffic generator is using synthetic traffic. In that case, the ``example.rules`` file will be used that matches some of the synthetic traffic generated. Custom rules can be applied by passing in the rules filename using the ``-r`` option of the firewall script. ******************* Ethernet Connection ******************* In this scenario the firewall is running on a machine connected to physical interfaces, with the other end connected to either a machine used as the software-based traffic generator (like VPP/TRex/TrafficGen), or a hardware traffic generator (like IXIA/Spirent Smartbits). .. figure:: ../images/ngfw_nic.png :align: center :width: 500 Ethernet connection Find out which DUT interfaces are connected to the traffic generator. ``sudo ethtool --identify `` will typically blink a light on the NIC to help identify the physical port associated with the interface. Get interface names and PCIe addresses from ``lshw`` command: .. code-block:: shell sudo lshw -c net -businfo The output will look similar to: .. code-block:: none Bus info Device Class Description ==================================================== pci@0000:07:00.0 eth0 network RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller pci@0001:01:00.0 enP1p1s0f0 network MT27800 Family [ConnectX-5] pci@0001:01:00.1 enP1p1s0f1 network MT27800 Family [ConnectX-5] Of the two interfaces connected to the traffic generator, arbitrarily choose one to be the input interface and the other to be the output interface. In this setup example, ``enP1p1s0f0`` at PCIe address ``0001:01:00.0`` is the input interface, and ``enP1p1s0f1`` at PCIe address ``0001:01:00.1`` is the output interface. Get the MAC address of the input interface ``enP1p1s0f0`` connected to the traffic generator via ``ip link show enP1p1s0f0``. This MAC address will be used as destination MAC address of generated traffic. Below is a sample output: .. code-block:: none 1: enP1p1s0f0: mtu 8996 qdisc mq state UP mode DEFAULT group default qlen 1000 link/ether b8:ce:f6:10:e4:6c brd ff:ff:ff:ff:ff:ff The user can quickly run firewall over NIC connection using the scripts located at: ``$NW_DS_WORKSPACE/dataplane-stack/usecase/ngfw``. Overview ======== In Ethernet connection scenario, the main operations of scripts are as follows: ``run_vpp_fw.sh`` * Runs a VPP instance for the firewall on the specified CPU cores * Attaches two DPDK interfaces on the specified PCIe addresses to VPP * Brings the interfaces up and set their IP addresses * Adds IP route entries making memif2 the gateway * Adds ACL rules to memif interface * Adds Snort interfaces * Launches Snort connected to the VPP instance ``traffic_monitor.sh`` * Monitors VPP interface counters and firewall throughput via VPP ``show interface`` and ``show runtime`` commands ``stop.sh`` * Stops VPP firewall instance Execution ========= .. note:: This execution requires at least two isolated cores. Cores 2 and 3 are assumed to be isolated in this guide. Quickly set up VPP firewall with input/output interface PCIe addresses on specified cores for 1 flow: .. code-block:: shell cd $NW_DS_WORKSPACE/dataplane-stack ./usecase/ngfw/run_vpp_fw.sh -p 0001:01:00.0,0001:01:00.1 -c 1,2 -d 1,1,3 .. note:: Replace sample addresses in above command with desired PCIe addresses on DUT. Send packets using the traffic generator on the other machine connected to the first NIC. The packets can be sourced from a pcap suggested in the introduction of this guide or any of the pcaps available publicly. A good source of these is the `NETRESEC website `_. IP and MAC addresses should be configured to hit or avoid ACLs as needed for testing. VPP firewall will forward packets that are accepted by ACLs and Snort to the output interface. Examine VPP firewall's interfaces rx/tx counters and packet processing runtime: .. code-block:: shell ./usecase/ngfw/traffic_monitor.sh Here is key output: .. code-block:: none Name Idx State MTU (L3/IP4/IP6/MPLS) Counter Count Ethernet0 1 up 9000/0/0/0 rx packets 17208504 rx bytes 1032510240 drops 319319 ip4 17208564 rx-miss 29474149 Ethernet1 2 up 9000/0/0/0 tx packets 319514 tx bytes 19170840 local0 0 down 0/0/0/0 .. code-block:: none Thread 1 vpp_wk_0 (lcore 2) Time 172.7, 10 sec internal node vector rate 8.22 loops/sec 675728.01 vector rates in 2.1111e6, out 3.7183e4, drop 3.7207e4, punt 3.4738e-2 Name State Calls Vectors Suspends Clocks Vectors/Call Ethernet1-output active 87128 6422321 0 4.02e-1 73.71 Ethernet1-tx active 87128 6422321 0 1.11e0 73.71 acl-plugin-fa-worker-cleaner-pinterrupt wa 331 0 0 6.52e1 0.00 acl-plugin-in-ip4-fa active 87302 12853702 0 1.96e0 147.23 dpdk-input polling 980738007 345357551 0 7.26e0 .35 drop active 174604 6426513 0 4.72e-1 36.81 error-drop active 174604 6426513 0 2.93e-1 36.81 error-punt active 6 6 0 2.33e2 1.00 ethernet-input active 44736127 345357551 0 1.05e0 7.72 interface-output active 87128 6422321 0 3.24e-1 73.71 ip4-drop active 87302 3213270 0 4.34e-1 36.81 ip4-inacl active 87302 9640459 0 1.31e0 110.43 ip4-input-no-checksum active 44736122 345357545 0 4.46e-1 7.72 ip4-lookup active 87302 6427189 0 4.16e-1 73.62 ip4-rewrite active 87302 6427189 0 1.43e0 73.62 punt active 6 6 0 1.13e2 1.00 snort-deq interrupt wa 87302 19276023 0 2.44e0 220.79 snort-enq active 44823424 19284215 0 1.09e1 .43 unix-epoll-input polling 956826 0 0 2.53e1 0.00 .. note:: - VPP ``Ethernet0`` is the aliased name of the input interface, which is at PCIe address ``0001:01:00.0`` in the example. - VPP ``Ethernet1`` is the aliased name of the output interface, which is at PCIe address ``0001:01:00.1`` in the example. Stop VPP firewall: .. code-block:: shell ./usecase/ngfw/stop.sh Known Issues ============ Snort core configuration ------------------------ On some machines Snort may fail to pin worker threads to higher cores (>16). Please watch the terminal for configuration errors. These will cause Snort to fail to start. ********* Resources ********* #. `next generation firewall `_ #. `Snort `_ #. `Vectorscan `_ #. `botnet-capture-20110810-neris.pcap `_ #. `VPP configuration dpdk section reference `_ #. `VPP configuration reference `_ #. `VPP set interface ip address reference `_ #. `VPP ip neighbor cli reference `_ #. `VPP set interface state reference `_ #. `VPP ip route reference `_ #. `VPP cli reference `_ #. `VPP Snort plugin reference `_ #. `VPP ACL cli `_ #. `VPP L2 ACL examples `_ #. `VPP ACL examples `_