RDMA/rxe: Fix incomplete state save in rxe_requester

jira KERNEL-325
cve CVE-2023-53539
Rebuild_History Non-Buildable kernel-4.18.0-553.89.1.el8_10
commit-author Bob Pearson <rpearsonhpe@gmail.com>
commit 5d122db
Empty-Commit: Cherry-Pick Conflicts during history rebuild.
Will be included in final tarball splat. Ref for failed cherry-pick at:
ciq/ciq_backports/kernel-4.18.0-553.89.1.el8_10/5d122db2.failed

If a send packet is dropped by the IP layer in rxe_requester()
the call to rxe_xmit_packet() can fail with err == -EAGAIN.
To recover, the state of the wqe is restored to the state before
the packet was sent so it can be resent. However, the routines
that save and restore the state miss a significnt part of the
variable state in the wqe, the dma struct which is used to process
through the sge table. And, the state is not saved before the packet
is built which modifies the dma struct.

Under heavy stress testing with many QPs on a fast node sending
large messages to a slow node dropped packets are observed and
the resent packets are corrupted because the dma struct was not
restored. This patch fixes this behavior and allows the test cases
to succeed.

Fixes: 3050b99 ("IB/rxe: Fix race condition between requester and completer")
Link: https://lore.kernel.org/r/20230721200748.4604-1-rpearsonhpe@gmail.com
	Signed-off-by: Bob Pearson <rpearsonhpe@gmail.com>
	Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
(cherry picked from commit 5d122db)
	Signed-off-by: Jonathan Maple <jmaple@ciq.com>

# Conflicts:
#	drivers/infiniband/sw/rxe/rxe_req.c

Author: PlaidCat

ciq/ciq_backports/kernel-4.18.0-553.89.1.el8_10/5d122db2.failed

@@ -0,0 +1,90 @@
+RDMA/rxe: Fix incomplete state save in rxe_requester
+
+jira KERNEL-325
+cve CVE-2023-53539
+Rebuild_History Non-Buildable kernel-4.18.0-553.89.1.el8_10
+commit-author Bob Pearson <rpearsonhpe@gmail.com>
+commit 5d122db2ff80cd2aed4dcd630befb56b51ddf947
+Empty-Commit: Cherry-Pick Conflicts during history rebuild.
+Will be included in final tarball splat. Ref for failed cherry-pick at:
+ciq/ciq_backports/kernel-4.18.0-553.89.1.el8_10/5d122db2.failed
+
+If a send packet is dropped by the IP layer in rxe_requester()
+the call to rxe_xmit_packet() can fail with err == -EAGAIN.
+To recover, the state of the wqe is restored to the state before
+the packet was sent so it can be resent. However, the routines
+that save and restore the state miss a significnt part of the
+variable state in the wqe, the dma struct which is used to process
+through the sge table. And, the state is not saved before the packet
+is built which modifies the dma struct.
+
+Under heavy stress testing with many QPs on a fast node sending
+large messages to a slow node dropped packets are observed and
+the resent packets are corrupted because the dma struct was not
+restored. This patch fixes this behavior and allows the test cases
+to succeed.
+
+Fixes: 3050b9985024 ("IB/rxe: Fix race condition between requester and completer")
+Link: https://lore.kernel.org/r/20230721200748.4604-1-rpearsonhpe@gmail.com
+	Signed-off-by: Bob Pearson <rpearsonhpe@gmail.com>
+	Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
+(cherry picked from commit 5d122db2ff80cd2aed4dcd630befb56b51ddf947)
+	Signed-off-by: Jonathan Maple <jmaple@ciq.com>
+
+# Conflicts:
+#	drivers/infiniband/sw/rxe/rxe_req.c
+diff --cc drivers/infiniband/sw/rxe/rxe_req.c
+index f63771207970,d8c41fd626a9..000000000000
+--- a/drivers/infiniband/sw/rxe/rxe_req.c
++++ b/drivers/infiniband/sw/rxe/rxe_req.c
+@@@ -746,9 -799,12 +748,12 @@@ int rxe_requester(void *arg
+  	pkt.mask = rxe_opcode[opcode].mask;
+  	pkt.wqe = wqe;
+  
++ 	/* save wqe state before we build and send packet */
++ 	save_state(wqe, qp, &rollback_wqe, &rollback_psn);
++ 
+  	av = rxe_get_av(&pkt, &ah);
+  	if (unlikely(!av)) {
+ -		rxe_dbg_qp(qp, "Failed no address vector\n");
+ +		pr_err("qp#%d Failed no address vector\n", qp_num(qp));
+  		wqe->status = IB_WC_LOC_QP_OP_ERR;
+  		goto err;
+  	}
+@@@ -790,17 -840,23 +789,33 @@@
+  
+  	err = rxe_xmit_packet(qp, &pkt, skb);
+  	if (err) {
+++<<<<<<< HEAD
+ +		qp->need_req_skb = 1;
+ +
+ +		rollback_state(wqe, qp, &rollback_wqe, rollback_psn);
+ +
+ +		if (err == -EAGAIN) {
+ +			rxe_run_task(&qp->req.task, 1);
+ +			goto exit;
+++=======
++ 		if (err != -EAGAIN) {
++ 			wqe->status = IB_WC_LOC_QP_OP_ERR;
++ 			goto err;
+++>>>>>>> 5d122db2ff80 (RDMA/rxe: Fix incomplete state save in rxe_requester)
+  		}
+  
+- 		wqe->status = IB_WC_LOC_QP_OP_ERR;
+- 		goto err;
++ 		/* the packet was dropped so reset wqe to the state
++ 		 * before we sent it so we can try to resend
++ 		 */
++ 		rollback_state(wqe, qp, &rollback_wqe, rollback_psn);
++ 
++ 		/* force a delay until the dropped packet is freed and
++ 		 * the send queue is drained below the low water mark
++ 		 */
++ 		qp->need_req_skb = 1;
++ 
++ 		rxe_sched_task(&qp->req.task);
++ 		goto exit;
+  	}
+  
+  	update_state(qp, &pkt);
+* Unmerged path drivers/infiniband/sw/rxe/rxe_req.c