Optimization of CPU hotspots with Progress/Control threads

[kpjeeja]

Optimization of CPU hotspots with Progress/Control threads

    - This patch adds Adaptive sleep for progress Thread/control
    thread instead of while with timeout(0ms). if CQ completion does
    not thie progress Thread will keep spinning. Same applies for
    CM Thread.
    - Add Cq Burst read instead on 1 completion read.
    - in case of Eagain for send/write/read. add adaptive sleep
    delay to avoid udealy of some ms

Signed-off-by: Jeeja KP <jeeja.kp@intel.com>

[jeromean]

We would need to ensure EFA is not broken. Atleast, we can do it theoretically by confining our changes specific to verbs. I'll provide more comments once we understand the native progress thread of OFI.