Identify the NUMA node for nic cards from PCI address
# lspci -vmms XX:XX.X | grep -i NUMANode
Identify numa nodes on the server as below:
# numactl -H
available: 2 nodes (0-1)
node 0 cpus: 0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30
node 0 size: 16162 MB
node 0 free: 1468 MB
node 1 cpus: 1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31
DPDK Parameters:
Initialize and enable dpdk ports on ovs with dpdk-init:
#ovs-vsctl –no-wait set Open_vSwitch . other_config:dpdk-init=true
Set cpu pin sets for pmd-cpu-mask: (requires hex string for cpu sets)
#ovs-vsctl set Open_vSwitch . other_config:pmd-cpu-mask=3c00
* Above example, assigning 10,12 from numa node 0 and 11,13 from numa node 1
Set memory for dpdk-socket-mem:
dpdk-socket-mem=<mem allocation from numa node 0>,<mem allocation from numa node 1>
#ovs-vsctl –no-wait set Open_vSwitch . other_config:dpdk-socket-mem=1024,1024
Set CPU cores on which dpdk lcore threads should be spawned dpdk-lcore-mask: (requires hex string for cpu sets)
#ovs-vsctl –no-wait set Open_vSwitch . other_config:dpdk-lcore-mask=3c000
* Above example, assigning 14,16 from numa node 0 and 15,17 from numa node 1
Below script from Red Hat will help to convert the hexadecimal cpu mask to cpu bit mask and reverse:
Source: https://access.redhat.com/solutions/3221381
#!/usr/bin/python
import sys
def hex_to_comma_list(hex_mask):
binary = bin(int(hex_mask, 16))[2:]
reversed_binary = binary[::-1]
i = 0
output = ""
for bit in reversed_binary:
if bit == '1':
output = output + str(i) + ','
i = i + 1
return output[:-1]
def comma_list_to_hex(cpus):
cpu_arr = cpus.split(",")
binary_mask = 0
for cpu in cpu_arr:
binary_mask = binary_mask | (1 << int(cpu))
return format(binary_mask, '02x')
if len(sys.argv) != 2:
print "Please provide a hex CPU mask or comma separated CPU list"
sys.exit(2)
user_input = sys.argv[1]
try:
print hex_to_comma_list(user_input)
except:
print comma_list_to_hex(user_input)
Example of above used cpu sets:
# ./converter.py 10,11,12,13
3c00
# ./converter.py 3c00
10,11,12,13
# ./converter.py 14,15,16,17
3c000
# ./converter.py 3c000
14,15,16,17