staging/rdma/hfi1: correctly check for post-interrupt packets

At the end of the packet processing interrupt and thread handler,
the RcvAvail interrupt is finally cleared down.  There is a window
between the last packet check (via DMA to memory) and interrupt
clear-down.  The code to recheck for a packet once the RcvAVail
interrupt is enabled must ultimately use a CSR read of RcvHdrTail
rather than depend on DMA'ed memory.

This change adds a CSR read of RcvHdrTail if the memory check does
not show a packet preset.  The memory check is retained as a quick
test before doing the more expensive, but always correct, CSR read.

In the ASIC, the CSR read used to force the RcvAvail clear-down write
to complete may bypass queued DMA writes to memory.  The only correct
way to decide if a packet has arrived without an interrupt to push DMA
to memory ahead of itself is to read the tail directly after RcvAvail
has been cleared down.  It is not sufficient to just read the tail and
skip pushing the clear-down.  Both must be done.  The tail read will not
push clear-down write due to it being in a different area of the chip.

At this point, it is OK to have packet data still being DMA'ed to
memory.  This is the end of packet processing for previous packets.
If the driver detects a new packet has arrived before interrputs were
re-enabled, it will force a new interrupt and the interrupt will push
the packet DMAs to memory, where the driver will then react to the
interrupt and do normal packet processing.

Reviewed-by: Mike Marciniszyn <mike.marciniszyn@intel.com>
Signed-off-by: Dean Luick <dean.luick@intel.com>
Signed-off-by: Jubin John <jubin.john@intel.com>
Signed-off-by: Doug Ledford <dledford@redhat.com>

drivers/staging/rdma/hfi1/chip.c

@@ -8022,9 +8022,9 @@ static irqreturn_t sdma_interrupt(int irq, void *data)
 }
 
 /*
- * Clear the receive interrupt, forcing the write and making sure
- * we have data from the chip, pushing everything in front of it
- * back to the host.
+ * Clear the receive interrupt.  Use a read of the interrupt clear CSR
+ * to insure that the write completed.  This does NOT guarantee that
+ * queued DMA writes to memory from the chip are pushed.
  */
 static inline void clear_recv_intr(struct hfi1_ctxtdata *rcd)
 {
@@ -8043,15 +8043,33 @@ void force_recv_intr(struct hfi1_ctxtdata *rcd)
 	write_csr(rcd->dd, CCE_INT_FORCE + (8 * rcd->ireg), rcd->imask);
 }
 
-/* return non-zero if a packet is present */
+/*
+ * Return non-zero if a packet is present.
+ *
+ * This routine is called when rechecking for packets after the RcvAvail
+ * interrupt has been cleared down.  First, do a quick check of memory for
+ * a packet present.  If not found, use an expensive CSR read of the context
+ * tail to determine the actual tail.  The CSR read is necessary because there
+ * is no method to push pending DMAs to memory other than an interrupt and we
+ * are trying to determine if we need to force an interrupt.
+ */
 static inline int check_packet_present(struct hfi1_ctxtdata *rcd)
 {
-	if (!HFI1_CAP_IS_KSET(DMA_RTAIL))
-		return (rcd->seq_cnt ==
-				rhf_rcv_seq(rhf_to_cpu(get_rhf_addr(rcd))));
+	u32 tail;
+	int present;
 
-	/* else is RDMA rtail */
-	return (rcd->head != get_rcvhdrtail(rcd));
+	if (!HFI1_CAP_IS_KSET(DMA_RTAIL))
+		present = (rcd->seq_cnt ==
+				rhf_rcv_seq(rhf_to_cpu(get_rhf_addr(rcd))));
+	else /* is RDMA rtail */
+		present = (rcd->head != get_rcvhdrtail(rcd));
+
+	if (present)
+		return 1;
+
+	/* fall back to a CSR read, correct indpendent of DMA_RTAIL */
+	tail = (u32)read_uctxt_csr(rcd->dd, rcd->ctxt, RCV_HDR_TAIL);
+	return rcd->head != tail;
 }
 
 /*