`XDP_REDIRECT` loses packets in `DRV_MODE` when run on MLX5 driver with ConnectX-6 NIC

[lucashc]

I am using a DevMap to redirect network packets from one interface to another.
To that end, I have this in my eBPF code:

/// Main map used to redirect packets out another interface
#[map]
static DEVMAP: DevMap = DevMap::with_max_entries(64, 0);

And in my main eBPF code:

#[xdp]
pub fn xdp_program(ctx: XdpContext) -> u32 {
  debug!(&ctx, "Got packet!");
  let ifindex = unsafe { (*ctx.ctx).ingress_ifindex };
  return DEVMAP.redirect(ifindex, 0).unwrap_or(xdp_action::XDP_DROP);
}

And I set in my main program:

dev_map.set(ifindex1, ifindex2, None, 0)?;
dev_map.set(ifindex2, ifindex1, None, 0)?;

This should route packets between two interfaces: net1 -> net2 and net2 -> net1

If I run this program in SKB_MODE I observe correct behaviour. If I run this program in DRV_MODE no packets are forwarded.
The debug statement does print that a packet is received and the rx_xdp_redirect in ethtool -S increases by one.
However, no packet comes out of the other interface.

Where could the packet have gone?

I already checked that:

• All interfaces are in promiscuous mode.
• The interface I receive from can receive packets

My setup is:

┌─Server───────────────┐    
│                      │    
│       ┌─CX-6───────┐ │    
│       │   XDP net2─┼─┼───┐
│       │            │ │   │
│       │   XDP net3─┼─┼─┐ │
│       │            │ │ │ │
│       └────────────┘ │ │ │
│                      │ │ │
│       ┌─CX-6───────┐ │ │ │
│       │       net4─┼─┼─┘ │
│       │            │ │   │
│       │            │ │   │
│       │       net5─┼─┼───┘
│       └────────────┘ │    
└──────────────────────┘    

[lucashc]

I also tried debugging this with xdp-dump from xdp-tools, but it complains that the loaded XDP program does not have any BTF debug information present. Is there a way in which Aya can load that information and make it discoverable for third-party tools?

[thomaseizinger]

I also tried debugging this with xdp-dump from xdp-tools, but it complains that the loaded XDP program does not have any BTF debug information present. Is there a way in which Aya can load that information and make it discoverable for third-party tools?

See #1227.