💻Deploying a FusionRT edge - NFV functionality on a SD-WAN edge🌐

💻Deploying a FusionRT edge - NFV functionality on a SD-WAN edge🌐

Step-by-step guide to deploying FusionRT on SD-WAN edge.

·

4 min read

Using Fusion SD-WAN it is possible to deploy an edge that has a floating static IP of /32 which is associated with a fully functional router and firewall using Openwrt. This software is an exceptional open-source project revolutionizing embedded operating systems based on Linux. Tailored for embedded devices, it efficiently routes network traffic, boasting a powerful lineup of components. Designed to optimize storage and memory usage, Openwrt fits seamlessly into SD-WAN edge nodes, ensuring smooth performance even with limited resources.

What sets Openwrt apart is its innovative firewall based on nft, the leading cloud-native low-level firewall, ensuring robust security measures. Configuration is a breeze with Openwrt, offering flexibility through a command-line interface (ash shell) or a user-friendly web interface (LuCI). With a whopping 8000 optional software packages available for installation via the opkg package management system, customization options are virtually endless.

But wait, there's more! Openwrt is capable of virtualization on x86 hardware using libvirt, expanding its versatility and compatibility to being used with the Fusion SD-WAN ecosystem. This solution provides for optimizing performance, enhancing security, and unlocking the full potential of SD-WAN within a small business.

First we setup the edge as follows:

This setups up the basic interfaces where br0 with be the LAN for Openwrt and br1 will be the WAN. Setup the broadband connections to provide Internet using legs as normal. The following step sets up WAN connection for Openwrt including the floating IP. The DHCP IP is also defined.

We can use the following custom DNSMASQ settings:

cache-size=4096
min-cache-ttl=900
no-resolv
# Include any extra threatblocks here
addn-hosts=/etc/extra-threatblock/tiktok
server=76.76.2.4
server=76.76.10.4
server=1.1.1.3

The setup for extra threatblocks on the edge is described below. In our example above we have blocked tiktok. It is also possible to run ControlD and and DNSgate.

There is one more setting required on the edge and that is to offload all traffic destined to the floating IP to openwrt. This is achieved using the bonding nft functionality.

# Create the file /etc/bonding/nftables/nat-prerouting-ipv4-port-forwarding.nft
ip daddr 154.61.111.91 dnat to 100.100.127.255
# where 154.61.111.91 is the floating IP and 100.100.127.255 is the DHCP assigned address of openwrt.
sudo systemctl reload bonding-nftables

So next, we need to install libvirt on the edge.

sudo apt-get update
sudo apt-get upgrade
sudo apt-get install qemu-kvm libvirt-daemon-system libvirt-clients virtinst cpu-checker libguestfs-tools libosinfo-bin

Voila! The Linux KVM solution is now up and running on the edge. We are now going to setup openwrt.

# Download the latest img
wget https://downloads.openwrt.org/releases/23.05.2/targets/x86/64/openwrt-23.05.2-x86-64-generic-ext4-combined-efi.img.gz
gunzip openwrt-22.03.5-x86-64-generic-ext4-combined-efi.img.gz .
sudo mkdir /var/lib/machines/openwrt
sudo mv openwrt-23.05.2-x86-64-generic-ext4-combined-efi.img /var/lib/machines/openwrt/
# Create the VM
sudo virt-install --os-type=generic --virt-type=kvm --name=openwrt --ram=512 --vcpus=2 --virt-type=kvm --hvm --network bridge=br0,model=virtio --network bridge=br1,model=virtio --graphics=vnc --disk path=/var/lib/machines/openwrt/openwrt-23.05.2-x86-64-generic-ext4-combined-efi.img,bus=ide --import

# Use this for console access instead of vnc
--connect qemu:///system

# Change vnc password by editing 
sudo virsh edit openwrt
# Add passwd="password" to graphics line

# The following starts openwrt when the edge boots
# Modify the units section in /lib/systemd/system/libvirtd.service to include
BindsTo=sys-devices-virtual-net-br0.device
BindsTo=sys-devices-virtual-net-br1.device
After=sys-devices-virtual-net-br0.device
After=sys-devices-virtual-net-br1.device

sudo virsh define /etc/libvirt/qemu/openwrt.xml
sudo virsh autostart openwrt
sudo apt-get install ebtables
sudo systemctl enable libvirtd

You can now add apps as normal on openwrt which include bbr, btop, mtr, SQM/CAKE, Wireguard VPN, etc. Below is the them we use FusionRT which is known as Argon.

# On openwrt
# Change ssh login by changing /etc/banner
# install argon
wget https://github.com/jerrykuku/luci-theme-argon/releases/download/v2.3.1/luci-theme-argon_2.
3.1_all.ipk -O $(basename https://github.com/jerrykuku/luci-theme-argon/releases/download/v2.3.1/luci-theme-arg
on_2.3.1_all.ipk)
wget https://github.com/jerrykuku/luci-app-argon-config/releases/download/v0.9/luci-app-argon-c
onfig_0.9_all.ipk -O $(basename https://github.com/jerrykuku/luci-app-argon-config/releases/download/v0.9/luci-
app-argon-config_0.9_all.ipk)
opkg install luci-lib-ipkg
opkg install luci-theme-argon*.ipk
opkg install luci-app-argon-config*.ipk

opkg update && opkg install bash && wget https://raw.githubusercontent.com/dylanaraps/neofetch/master/neofetch && bash neofetch

The NFV instance of OpenWRT can be configured as follows:

The edge configuration to log these events and onwards forward them as notifications are as follows:

sudo nano /etc/rsyslog.conf
# Uncomment the below
# provides UDP syslog reception
module(load="imudp")
input(type="imudp" port="514")
# Add this to the bottom of the file
$template remote-incoming-logs,"/var/log/%HOSTNAME%/%PROGRAMNAME%.log"
*.* ?remote-incoming-logs

sudo systemctl restart rsyslog

# Enable the logs to be received
# /etc/nftables/nft_remote_systemlog.nft
table ip remotesyslog {
        chain input {
                type filter hook input priority -1; policy accept;
                iifname br1 udp dport 54 counter accept
        }
}

The log files are now stored in /var/log/FusionRT

The following script can be used to hourly check the logs and send notifications.

#!/bin/bash
# Script to check NFV logs and forward notifications
# Run hourly
# Define log files
uhttpd_log="/var/log/FusionRT/uhttpd.log"
dropbear_log="/var/log/FusionRT/dropbear.log"

# Get current time and time one hour ago in epoch format
current_time=$(date +%s)
one_hour_ago=$((current_time - 3600))

# Check if modification time of each log file is within the last hour
if [[ $(stat -c %Y "$uhttpd_log") -gt $one_hour_ago ]]; then
    echo "Last 3 lines of $uhttpd_log:"
    note=`tail -n 3 "$uhttpd_log"`
    curl \
      -H "Authorization: Bearer humptydumpty" \
      -H prio:high \
      -H tags:penguin \
      -d "FusionRT web on ${HOSTNAME}: $note" \
      ntfy.sh/hadagreatfall
fi

if [[ $(stat -c %Y "$dropbear_log") -gt $one_hour_ago ]]; then
    echo "Last 3 lines of $dropbear_log:"
    note=`tail -n 3 "$dropbear_log"`
    curl \
      -H "Authorization: Bearer humptydumpty" \
      -H prio:high \
      -H tags:penguin \
      -d "FusionRT cli on ${HOSTNAME}: $note" \
      ntfy.sh/hadagreatfall
fi

Ronald Bartels works connecting Internet inhabiting things at Fusion Broadband.