Hi LinuxQuestions.org,
I am currently working on a program which utilizes the AF_XDP (or XDP in general) functionality in Linux (which was introduced in 4.18).
I am 90% based on this tutorial:
https://github.com/xdp-project/xdp-t...anced03-AF_XDP
The only things I changed are
- The Kernel program now filters for UDP IPv4 Packets (works)
- Some minor changes to the User-Space program so that I can receive packets from inside my network (around 200.000 Packets per Second)
The program has a second thread printing statistics about
- number of received packets
- Mbit/s
every two seconds (that's very helpful). So what I noticed is that the amount of packets stay in the same, expected range for about 2 minutes. Then the amount of received packets suddenly drops to 0 (and stays at 0). In case I restart my user-space program, the same thing happens again (first receiving packets, then no more after 2 minutes). I can also confirm, that the stream of packets is continuous.
So where does this problem come from?
There is this function
handle_receive_packets:
Code:
static void handle_receive_packets(struct xsk_socket_info *xsk)
{
unsigned int rcvd, stock_frames, i;
uint32_t idx_rx = 0, idx_fq = 0;
int ret;
rcvd = xsk_ring_cons__peek(&xsk->rx, RX_BATCH_SIZE, &idx_rx);
if (rcvd <= 0)
return;
/* Stuff the ring with as much frames as possible */
stock_frames = xsk_prod_nb_free(&xsk->umem->fq, xsk_umem_free_frames(xsk));
if (stock_frames > 0) {
ret = xsk_ring_prod__reserve(&xsk->umem->fq, stock_frames, &idx_fq);
/* This should not happen, but just in case */
while (ret != stock_frames)
ret = xsk_ring_prod__reserve(&xsk->umem->fq, rcvd, &idx_fq);
for (i = 0; i < stock_frames; i++)
*xsk_ring_prod__fill_addr(&xsk->umem->fq, idx_fq++) = xsk_alloc_umem_frame(xsk);
xsk_ring_prod__submit(&xsk->umem->fq, stock_frames);
}
/* Process received packets */
for (i = 0; i < rcvd; i++) {
const struct xdp_desc *pckt = xsk_ring_cons__rx_desc(&xsk->rx, idx_rx);
const uint64_t addr = pckt->addr;
const uint32_t len = pckt->len;
process_packet(xsk, addr, len);
xsk_free_umem_frame(xsk, addr);
xsk->stats.rx_bytes += len;
idx_rx += 1;
}
xsk_ring_cons__release(&xsk->rx, rcvd);
xsk->stats.rx_packets += rcvd;
/* Do we need to wake up the kernel for transmission */
complete_tx(xsk);
}
and my guess is that somehow the amount of allocated memory is not freed fast enough and it slowly goes to 0. In this case, there is no place left for incomming packets (e.g. the program is not able to reserve space for it) and thus no packets can be processed.
Any ideas on this?
Edit: Okay, I just checked
xsk_socket->umem_frame_free and it always stays in the range of allocated frames (in my case 2^18) +-20 even after the amount of received packets goes to 0.
Edit_2: It is always roughly the same time when received packets are 0.
