Add support for OVS PMD load based sleeping

PMD threads constantly poll Rx queues which are assigned to them. In
order to reduce the CPU cycles they use, they can sleep for small
periods of time when there is no load or very-low load on all the Rx
queues they poll.

This can be enabled by setting the max requested sleep time (in
microseconds) for any PMD thread:

 $ ovs-vsctl set open_vswitch . other_config:pmd-maxsleep=50

With a non-zero max value a PMD may request to sleep by an incrementing
amount of time up to the maximum time. If at any point the threshold of
at least half a batch of packets (i.e. 16) is received from an Rx queue
that the PMD is polling is met, the requested sleep time will be reset
to 0. At that point no sleeps will occur until the no/low load
conditions return.

Sleeping in a PMD thread will mean there is a period of time when the
PMD thread will not process packets. Sleep times requested are not
guaranteed and can differ significantly depending on system
configuration. The actual time not processing packets will be determined
by the sleep and processor wake-up times and should be tested with each
system configuration.

Depends-on: https://review.opendev.org/c/openstack/tripleo-ansible/+/893341
Cc: Christophe Fontaine <cfontain@redhat.com>
Change-Id: I4447ed85ba246db1ad46c9715f6e8ee59376b1ea
Signed-off-by: Robin Jarry <rjarry@redhat.com>

deployment/openvswitch/openvswitch-dpdk-baremetal-ansible.yaml

@@ -148,6 +148,20 @@ parameters:
     default: ""
     tags:
       - role_specific
+  OvsPmdSleepMax:
+    description: >
+      PMD maximum sleep time, in range 0 to 10,000. Its a string with a number,
+      specifies the maximum sleep time that will be requested in microseconds
+      per iteration for a PMD thread which has received zero or a small amount
+      of packets from the Rx queues it is polling. The actual sleep time
+      requested is based on the load of the Rx queues that the PMD polls and
+      may be less than the maximum value.
+    constraints:
+      - allowed_pattern: "[0-9]*"
+    type: string
+    default: ""
+    tags:
+      - role_specific
   OvsDisableEMC:
     default: false
     description: |
@@ -196,6 +210,7 @@ resources:
               pmd_load_threshold: OvsPmdLoadThreshold
               pmd_improvement_threshold: OvsPmdImprovementThreshold
               pmd_rebal_interval: OvsPmdRebalInterval
+              pmd_sleep_max: OvsPmdSleepMax
               nova_postcopy: NovaLiveMigrationPermitPostCopy
               ddp_package: DdpPackage
             - values: {get_param: [RoleParameters]}
@@ -213,6 +228,7 @@ resources:
               OvsPmdLoadThreshold: {get_param: OvsPmdLoadThreshold}
               OvsPmdImprovementThreshold: {get_param: OvsPmdImprovementThreshold}
               OvsPmdRebalInterval: {get_param: OvsPmdRebalInterval}
+              OvsPmdSleepMax: {get_param: OvsPmdSleepMax}
               NovaLiveMigrationPermitPostCopy: {get_param: NovaLiveMigrationPermitPostCopy}
               DdpPackage: {get_param: DdpPackage}
 
@@ -265,6 +281,7 @@ outputs:
               tripleo_ovs_dpdk_pmd_load_threshold: {get_attr: [RoleParametersDpdk, value, pmd_load_threshold]}
               tripleo_ovs_dpdk_pmd_improvement_threshold: {get_attr: [RoleParametersDpdk, value, pmd_improvement_threshold]}
               tripleo_ovs_dpdk_pmd_rebal_interval: {get_attr: [RoleParametersDpdk, value, pmd_rebal_interval]}
+              tripleo_ovs_dpdk_pmd_sleep_max: {get_attr: [RoleParametersDpdk, value, pmd_sleep_max]}
               tripleo_ovs_dpdk_emc_insertion_probablity:
                 if:
                 - emc_disable

releasenotes/notes/ovs-dpdk-pmd-params-ccdf06f1b8ac5f8b.yaml

@@ -0,0 +1,5 @@
+---
+features:
+  - Added support for PMD load based sleeping. It can be configured via the
+    role specific THT ``OvsPmdSleepMax`` parameter to set
+    ``other_config:pmd-maxsleep`` in OVS.