prov/shm: Revert to a lock-unlock inject pool

[zachdworkin]

Remove the parallel command-inject resources and revert to using a lock-unlock inject buffer pool.

Update the inject protocol to use the old method.

There is a performance regression when using the "new shm" command-inject parallel data structure. This is due to the sender not being able to complete its transmission until the receiver returns the sender's command to the sender's return queue. In the old lock-unlock method the sender would allocate receiver side resources, copy its data into the receiver inject buffer and then complete. The old method allows MPIs and applications to assume that their inject message transmissions will complete quickly and since the new method does not complete as quickly it is likely the reason for this regression.

In addition we need to move the inject stack above the cmd stack in the shm region. This is because most of the time the new inject protocol will not need to use the cmd stack. If we swap the ordering in the region then we have to do less jumps through memory to get the resources we need.

We can also use the ofi freestack for the cmd stack because we only need to pop and push on the sender side so we don't need the extra functionality of the smr_freestack.

[zachdworkin]

bot:aws:retest

[j-xiong]

bot:aws:retest

[zachdworkin]

bot:aws:retest

[zachdworkin]

@sunkuamzn what is the AWS failure with this PR?

[shijin-aws]

@zachdworkin There is single node unexpected message test timeout

server_command: ssh -n -o StrictHostKeyChecking=no -o ConnectTimeout=30 -o BatchMode=yes 172.31.49.36 'timeout 1800 /bin/bash --login -c '"'"' FI_HMEM=system FI_LOG_LEVEL=warn /home/ec2-user/PortaFiducia/build/libraries/libfabric/pr12109-undebug/install/fabtests/bin/fi_unexpected_msg -e rdm -M 2048 -I 5 -f efa -v -S 512 -p efa -E=9234'"'"''

client_command: ssh -n -o StrictHostKeyChecking=no -o ConnectTimeout=30 -o BatchMode=yes 172.31.49.36 'timeout 1800 /bin/bash --login -c '"'"' FI_HMEM=system FI_LOG_LEVEL=warn /home/ec2-user/PortaFiducia/build/libraries/libfabric/pr12109-undebug/install/fabtests/bin/fi_unexpected_msg -e rdm -M 2048 -I 5 -f efa -v -S 512 -p efa -E=9234 172.31.49.36'"'"''
client_stdout:

client returncode: 124
server_stdout:

server returncode: 124