Setting DPDK parameters with Multi NUMA

Identify the NUMA node for nic cards from PCI address
# lspci -vmms XX:XX.X | grep -i NUMANode

Identify numa nodes on the server as below:

# numactl -H
available: 2 nodes (0-1)
node 0 cpus: 0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30
node 0 size: 16162 MB
node 0 free: 1468 MB
node 1 cpus: 1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31

DPDK Parameters:

Initialize and enable dpdk ports on ovs with dpdk-init:

#ovs-vsctl –no-wait set Open_vSwitch . other_config:dpdk-init=true

Set cpu pin sets for pmd-cpu-mask: (requires hex string for cpu sets)

#ovs-vsctl set Open_vSwitch . other_config:pmd-cpu-mask=3c00
* Above example, assigning 10,12 from numa node 0 and 11,13 from numa node 1

Set memory for dpdk-socket-mem:
dpdk-socket-mem=<mem allocation from numa node 0>,<mem allocation from numa node 1>

#ovs-vsctl –no-wait set Open_vSwitch . other_config:dpdk-socket-mem=1024,1024

Set CPU cores on which dpdk lcore threads should be spawned dpdk-lcore-mask: (requires hex string for cpu sets)

#ovs-vsctl –no-wait set Open_vSwitch . other_config:dpdk-lcore-mask=3c000
* Above example, assigning 14,16 from numa node 0 and 15,17 from numa node 1

Below script from Red Hat will help to convert the hexadecimal cpu mask to cpu bit mask and reverse:

Source: https://access.redhat.com/solutions/3221381

#!/usr/bin/python

import sys

def hex_to_comma_list(hex_mask):
    binary = bin(int(hex_mask, 16))[2:]
    reversed_binary = binary[::-1]
    i = 0
    output = ""
    for bit in reversed_binary:
        if bit == '1':
            output = output + str(i) + ','
        i = i + 1
    return output[:-1]

def comma_list_to_hex(cpus):
    cpu_arr = cpus.split(",")
    binary_mask = 0
    for cpu in cpu_arr:
        binary_mask = binary_mask | (1 << int(cpu))
    return format(binary_mask, '02x')

if len(sys.argv) != 2:
    print "Please provide a hex CPU mask or comma separated CPU list"
    sys.exit(2)

user_input = sys.argv[1]

try:
  print hex_to_comma_list(user_input)
except:
  print comma_list_to_hex(user_input)

Example of above used cpu sets:

# ./converter.py 10,11,12,13
3c00
# ./converter.py 3c00
10,11,12,13
# ./converter.py 14,15,16,17
3c000
# ./converter.py 3c000
14,15,16,17