IPFIX-balancer

The key task of the module is retransmission (replication and load balancing) of all IPFIX data flows exported from DPI (Fullflow/Clickstream/DNS flow): it receives IPFIX from fastDPI and then retransmits it to other hosts according to the settings. Additionally, a recording mode can be implemented. The IPFIX-balancer operates with data portions (messages) when distributing them between data collection nodes (receivers/collectors). It has two main modes of data distribution:

1. The "RoundRobin" mode distributes (balances) the IPFIX stream by sending one message to each specified host.
   It sends one message to one receiving node → collector → sends the next message to the next available receiving host → collector, and so on in a loop until all messages are processed.
2. The "All" mode replicates the IPFIX stream.
   It copies the received messages from DPI to all receiving nodes — collectors.

The module can work with both UDP and TCP, and it is possible to bind a specific traffic type handler (Fullflow/Clickstream/DNS flow) to a CPU core, ensuring load distribution across CPUs.

The balancer allows the following tasks to be performed:

1. Traffic distribution between data collection and storage nodes, in case QoE is working in "Cluster" mode, where data storage is performed on different servers.
2. Data duplication in replication mode, between data collection and storage nodes, to address redundancy or simultaneous export of the stream in different directions, or between different receiving hosts (Fullflow/Clickstream/DNS flow).
3. Redirecting the flow (Fullflow/Clickstream/DNS flow) from DPI without changing the configuration or requiring a fastdpi process restart.
   When the data stream is exported to the balancer at 127.0.0.1, running on the same host as DPI.
   In this case, if the destination address of the stream or collector address changes, you need to make these changes in the IPFIX-balancer configuration file and restart it without affecting the DPI-processed traffic.

Installation:

yum install fast_ipfix_balancer

It is recommended to install the IPFIX-balancer module on the same server where DPI is installed.

Stopping and restarting:

ipfix-balancer-stop
ipfix-balancer-restart

Viewing logs:

journalctl -xe | grep IPFIXcol2
journalctl -f | grep IPFIXcol2

Determine which CPU the balancing process belongs to. CPU is set by the parameter IPFIX_xx_TASKSET, description in the section Working in CLI → Configuration.

ps -ax -o %cpu,psr,comm | grep ipfix

To work with the IPFIX-balancer, you need to set up the appropriate equipment. To do this:

1. Go to the Administrator section → Equipment;
2. Click + above the equipment table;
3. In the window that appears, select the equipment type — IPFIX-balancer Server;
   
4. Fill in the remaining fields and save the equipment.

1. In the Administrator section → IPFIX-balancer Configuration, select the IPFIX server and add a receiver (by clicking + above the receiver table);
2. Select the value "All" in the “Balancer” field.
3. Specify Port Type, Balancer Subreceivers, Balancer Subreceiver Type.
4. It is recommended to set the same value for the Port Type and Balancer Subreceiver Type parameters.
5. Configure the other receiver parameters. Their description is provided in the section Parameter Description.

As a result, we will get two identical streams on two different hosts.

Receiver settings example for replication case:

1. In the Administrator section → IPFIX-balancer Configuration, select the IPFIX server and add a receiver (by clicking + above the receiver table);
2. Select the value "RoundRobin" in the “Balancer” field.
3. Specify Port Type, Balancer Subreceivers, Balancer Subreceiver Type.
   It is recommended to set the same value for the Port Type and Balancer Subreceiver Type parameters.
4. Configure the other receiver parameters. Their description is provided in the section Parameter Description.

As a result, we will get an even distribution of packets across all hosts.

Receiver settings example for load balancing case:

1. In the Administrator section → IPFIX-balancer Configuration, select the IPFIX server and add a receiver (by clicking + above the receiver table);
2. Select the value "Disabled" in the “Balancer” field;
3. Configure the other receiver parameters. Their description is provided in the section Parameter Description.

In this case, the IPFIX receiver works, writes data to a file, and can export IPFIX to the IP addresses specified in the "Export" field.

Receiver settings example for recording case:

The balancer configuration file is located at /var/fast_ipfix_balancer/backend/.env

The configuration consists of parameters in the IPFIX_xx_yy format, where:

xx — stream type:

• FULLFLOW — Fullflow
• CLICKSTREAM — Clickstream
• DNSFLOW — DNS flow

yy — parameter name:

• PORT_TYPE[0] — port type.
  Possible values:
  • tcp (recommended)
  • udp
• PORT[0] — any desired port, it must be unique for each stream.
  Default values:
  • 1500 for Fullflow
  • 1501 for Clickstream
• ROTATE_MINUTES[0] — Receiver dump rotation period or data upload period to the database. The Clickstream rotation period should be greater than or equal to the same period for Fullflow. When the Fullflow dump rotates, a HUP signal is sent to synchronize the streams. Do not change this parameter unnecessarily
  Default values:
  • 10 minutes for Fullflow
  • 12 minutes for Clickstream
• ROTATE_DELAY_SECONDS[0] — Dump rotation in seconds. Same as rotation in minutes, only in seconds. If this value is enabled, the value in minutes is ignored.
• FW_MAX_QUEUE_SIZE[0] — Rotation by flow count. Works simultaneously with rotation in minutes or seconds.
• DUMP_INSERT_PROCESSES[0] — Receiver(s) to which data is exported.
  Format: 10.0.0.2/9920/tcp,10.0.0.3/3440/udp.
• DPI_ID[0] — DPI number.
  Default value: -1.
  If the value is -1 or not specified, the DPI number is determined by the order of the recipient in the list.
• BALANCER[0] — Balancer type.
  Possible values:
  • RoundRobin — distributes the IPFIX stream
  • All — replicates the IPFIX stream
• BALANCER_SUB[0] — Receiver(s) to which data is distributed or replicated.
  Format: 10.0.0.2/9920,10.0.0.3/3440
• BALANCER_SUB_PROTO[0] — Port type.
  Possible values:
  • tcp (recommended)
  • udp
• TASKSET — CPU number on which the traffic handler will work.

# Fullflow is distributed among three QoE servers
IPFIX_FULLFLOW_PORT_TYPE[0]=tcp
IPFIX_FULLFLOW_PORT[0]=1500
IPFIX_FULLFLOW_BALANCER[0]=RoundRobin
IPFIX_FULLFLOW_BALANCER_SUB[0]=10.19.3.21/1500,10.19.3.23/1600,10.19.3.24/1600
IPFIX_FULLFLOW_BALANCER_SUB_PROTO[0]=tcp
IPFIX_FULLFLOW_BALANCER_TASKSET[0]=61
 
# Clickstream is distributed between two QoE servers: 10.19.3.21/10102 и 10.19.3.22/10102
IPFIX_CLICKSTREAM_PORT_TYPE[0]=tcp
IPFIX_CLICKSTREAM_PORT[0]=1501
IPFIX_CLICKSTREAM_BALANCER[0]=RoundRobin
IPFIX_CLICKSTREAM_BALANCER_SUB[0]=10.19.3.21/1501,10.19.3.23/1601
IPFIX_CLICKSTREAM_BALANCER_SUB_PROTO[0]=tcp
IPFIX_CLICKSTREAM_BALANCER_TASKSET[0]=62
 
# DNS flow is distributed between two QoE servers: 10.19.3.21/10103 и 10.19.3.22/10103
IPFIX_DNSFLOW_PORT_TYPE[0]=tcp
IPFIX_DNSFLOW_PORT[0]=1101
IPFIX_DNSFLOW_BALANCER[0]=RoundRobin
IPFIX_DNSFLOW_BALANCER_SUB[0]=10.19.3.21/1101,10.19.3.23/2101
IPFIX_DNSFLOW_BALANCER_SUB_PROTO[0]=tcp
IPFIX_DNSFLOW_BALANCER_TASKSET[0]=63