Skip to content

FRR occasionally fails to remove FDB entry following withdrawal of MACIP route from remote VTEP #21473

Open
Open
FRR occasionally fails to remove FDB entry following withdrawal of MACIP route from remote VTEP#21473
Labels
triageNeeds further investigation
@toreanderson

Description

@toreanderson
toreanderson
opened on Apr 8, 2026

Description

Note: this sometime happens after live migration of a virtual machine in an EVPN fabric, in the exact same setup described in issue #21472. Most of the time the live migration procedure ends up in a functional state (albeit after a certain amount of unnecessary oscillation, which is what that issue is about), but sometimes it gets stuck in a state where there bridge FDB entry for a withdrawn MACIP route (from the old hypervisor) is stuck, which in turn (presumably) prevents FRR on the new hypervisor from advertising a MACIP route of its own.

Version

I have observed this on FRR 8.5.3, 10.5.3 and latest Git master:

FRRouting 10.7.0-dev (frr1) on Linux(5.14.0-611.20.1.el9_7.x86_64).
Copyright 1996-2005 Kunihiro Ishiguro, et al.
configured with:
    '--build=x86_64-redhat-linux-gnu' '--host=x86_64-redhat-linux-gnu' '--program-prefix=' '--disable-dependency-tracking' '--prefix=/usr' '--exec-prefix=/usr' '--bindir=/usr/bin' '--datadir=/usr/share' '--includedir=/usr/include' '--libdir=/usr/lib64' '--libexecdir=/usr/libexec' '--sharedstatedir=/var/lib' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--sbindir=/usr/lib/frr' '--sysconfdir=/etc' '--localstatedir=/var' '--disable-static' '--disable-werror' '--enable-multipath=256' '--enable-vtysh' '--enable-ospfclient' '--enable-ospfapi' '--enable-ldpd' '--enable-pimd' '--enable-pim6d' '--enable-pbrd' '--enable-nhrpd' '--enable-eigrpd' '--enable-babeld' '--enable-vrrpd' '--enable-user=frr' '--enable-group=frr' '--enable-vty-group=frrvty' '--enable-fpm' '--enable-watchfrr' '--disable-bgp-vnc' '--enable-isisd' '--enable-doc' '--enable-rpki' '--enable-bfdd' '--enable-pathd' '--disable-grpc' '--enable-snmp' '--disable-zeromq' '--enable-pcre2posix' 'build_alias=x86_64-redhat-linux-gnu' 'host_alias=x86_64-redhat-linux-gnu' 'PKG_CONFIG_PATH=:/usr/lib64/pkgconfig:/usr/share/pkgconfig' 'CC=gcc' 'CXX=g++' 'LT_SYS_LIBRARY_PATH=/usr/lib64:'

How to reproduce

See issue #21472. Live migrate back and forth between two hypervisors until the issue occurs and the VM is unreachable due to missing MACIP routes (it might take dozens of attempts).

Expected behavior

When FRR receives a withdraw of a MACIP route that used to advertised from the old hypervisor, it should remove the bridge FDB entry with the dst field pointing to the old hypervisor, and start advertising a MACIP route of its own.

Actual behavior

See the complete log under «Additional context», below. The most interesting bit appears to be around the end:

   137	Apr 08 12:21:48.954802 bgpd[3431687]: [PAPP6-VDAWM] eth5(l5-g4-osl2) rcvd UPDATE about RD 10.0.0.2:2 [2]:[0]:[48]:[fa:16:3e:7e:e3:5a]:[32]:[185.47.41.88] label 0 l2vpn evpn -- withdrawn
   138	Apr 08 12:21:48.954812 bgpd[3431687]: [K7P35-81YTJ] default (0): Uninstalling EVPN [2]:[0]:[48]:[fa:16:3e:7e:e3:5a]:[32]:[185.47.41.88] route from VNI 10099 IP/MAC table
   139	Apr 08 12:21:48.954818 bgpd[3431687]: [PAPP6-VDAWM] eth5(l5-g4-osl2) rcvd UPDATE about RD 10.0.0.2:2 [2]:[0]:[48]:[fa:16:3e:7e:e3:5a] label 0 l2vpn evpn -- withdrawn
   140	Apr 08 12:21:48.954824 bgpd[3431687]: [K7P35-81YTJ] default (0): Uninstalling EVPN [2]:[0]:[48]:[fa:16:3e:7e:e3:5a] route from VNI 10099 IP/MAC table
   141	Apr 08 12:21:48.956377 bgpd[3431687]: [PAPP6-VDAWM] eth4(l4-g4-osl2) rcvd UPDATE about RD 10.0.0.2:2 [2]:[0]:[48]:[fa:16:3e:7e:e3:5a] label 0 l2vpn evpn -- withdrawn
   142	Apr 08 12:21:48.956386 bgpd[3431687]: [K7P35-81YTJ] default (0): Uninstalling EVPN [2]:[0]:[48]:[fa:16:3e:7e:e3:5a] route from VNI 10099 IP/MAC table
   143	Apr 08 12:21:48.960885 bash[2317639]: --- /dev/fd/63        2026-04-08 12:21:48.957422740 +0200
   144	Apr 08 12:21:48.960885 bash[2317639]: +++ /dev/fd/62        2026-04-08 12:21:48.958422740 +0200
   145	Apr 08 12:21:48.960885 bash[2317639]: @@ -1,2 +1,2 @@
   146	Apr 08 12:21:48.960885 bash[2317639]: -fa:16:3e:7e:e3:5a dev l2vni-10099 vlan 99 sticky master br-evpn static
   147	Apr 08 12:21:48.960885 bash[2317639]: +fa:16:3e:7e:e3:5a dev veth-to-ovs vlan 99 sticky master br-evpn static
   148	Apr 08 12:21:48.960885 bash[2317639]:  fa:16:3e:7e:e3:5a dev l2vni-10099 dst 10.0.0.2 self sticky static
   149	Apr 08 12:21:48.965710 zebra[3431682]: [MMX22-H2MY5] Tx RTM_DELNEIGH family bridge IF l2vni-10099(1685) VLAN 99 MAC fa:16:3e:7e:e3:5a dst 10.0.0.2 nhg 0 rem
   150	Apr 08 12:21:48.965718 zebra[3431682]: [HYEHE-CQZ9G] nl_batch_send: netlink-dp (NS 0), batch size=80, msg cnt=1
   151	Apr 08 12:21:48.965724 zebra[3431682]: [MQ5AP-2S1F5] netlink-dp (NS 0) error: No such file or directory, type=RTM_DELNEIGH(29), seq=149678738, pid=3631020340
   152	Apr 08 12:21:48.965730 zebra[3431682]: [QTT8V-3ZQ34] nl_batch_read_resp: netlink error message seq=149678738
   153	Apr 08 12:21:49.016192 zebra[3431682]: [YXPF5-B2CE0] netlink_route_multipath_msg_encode: RTM_DELROUTE 185.47.41.88/32 vrf 1686(100002)
   154	Apr 08 12:21:49.016202 zebra[3431682]: [HYEHE-CQZ9G] nl_batch_send: netlink-dp (NS 0), batch size=52, msg cnt=1

Here, the MACIPs route from the old hypervisor (10.0.0.2) are being withdrawn, but it would appear that FRR fails to remove the corresponding bridge FDB entry fa:16:3e:7e:e3:5a dev l2vni-10099 dst 10.0.0.2 self sticky static. So we end up in a situation where the FDB both contains the correct entry (with dev veth-to-ovs) and a stale one for the same MAC address. FRR does not have any MACIP entries for the MAC or IP address in its RIB and does not announce anything upstream, so the VM is unreachable.

Additional context

In order to recover from this issue, it is not sufficient to manually delete the stale entry from the bridge fdb (e.g., bridge fdb del fa:16:3e:7e:e3:5a dev l2vni-10099). Even though the FDB and neigh cache looks correct at that point, no MACIP route is being advertised. It is necessary to also delete both the correct FDB entry and the correct IP neigh cache entry (ip n del 185.47.41.88 dev irb-99; bridge fdb del fa:16:3e:7e:e3:5a dev veth-to-ovs master), so that the MAC address in question is completely absent from the bridge FDB/neigh cache. When they then get re-added, the MACIP routes show up in the RIB and are advertised into the data centre fabric.

Here is the complete log (see #21472 for a description of its contents):

     1  Apr 08 12:21:44.309619 systemd-machined[1123]: New machine qemu-416-instance-0000d1dc.
     2  Apr 08 12:21:44.421462 zebra[3431682]: [KMXEB-K771Y] netlink_parse_info: netlink-listen (NS 0) type RTM_NEWROUTE(24), len=116, seq=0, pid=0
     3  Apr 08 12:21:44.421484 zebra[3431682]: [SKNFJ-G938V] RTM_NEWROUTE ipv6 local proto kernel NS 0
     4  Apr 08 12:21:44.421497 zebra[3431682]: [J3J81-V75NW] Route rtm_type: local(2) intentionally ignoring
     5  Apr 08 12:21:44.421508 zebra[3431682]: [KMXEB-K771Y] netlink_parse_info: netlink-listen (NS 0) type RTM_NEWROUTE(24), len=116, seq=0, pid=0
     6  Apr 08 12:21:44.421519 zebra[3431682]: [SKNFJ-G938V] RTM_NEWROUTE ipv6 anycast proto kernel NS 0
     7  Apr 08 12:21:44.421530 zebra[3431682]: [J3J81-V75NW] Route rtm_type: anycast(4) intentionally ignoring
     8  Apr 08 12:21:44.421644 zebra[3431682]: [KMXEB-K771Y] netlink_parse_info: netlink-dp-in (NS 0) type RTM_NEWADDR(20), len=72, seq=0, pid=0
     9  Apr 08 12:21:44.421653 zebra[3431682]: [RGWF1-EHXT1] netlink_interface_addr_dplane: RTM_NEWADDR nsid 0 ifindex 1926 flags 0x80:
    10  Apr 08 12:21:44.421660 zebra[3431682]: [ME3M2-X6YT9]   IFA_ADDRESS   fe80::fc16:3eff:fe62:3aa2/64
    11  Apr 08 12:21:44.421667 zebra[3431682]: [P2VPT-508WP]   IFA_CACHEINFO pref -1, valid -1
    12  Apr 08 12:21:44.421756 zebra[3431682]: [K8FXY-V65ZJ] Intf dplane ctx 0x7f3fbc029e20, op INTF_ADDR_ADD, ifindex (1926), result QUEUED
    13  Apr 08 12:21:44.421764 zebra[3431682]: [MZPZA-W042K] zebra_if_addr_update_ctx: INTF_ADDR_ADD: ifindex tapd0ba6558-f4(1926), addr fe80::fc16:3eff:fe62:3aa2/64
    14  Apr 08 12:21:44.428004 zebra[3431682]: [KMXEB-K771Y] netlink_parse_info: netlink-dp-in (NS 0) type RTM_NEWLINK(16), len=1520, seq=0, pid=0
    15  Apr 08 12:21:44.428015 zebra[3431682]: [KMXEB-K771Y] netlink_parse_info: netlink-dp-in (NS 0) type RTM_NEWLINK(16), len=1520, seq=0, pid=0
    16  Apr 08 12:21:44.428369 zebra[3431682]: [KMXEB-K771Y] netlink_parse_info: netlink-dp-in (NS 0) type RTM_NEWLINK(16), len=1520, seq=0, pid=0
    17  Apr 08 12:21:44.428377 zebra[3431682]: [KMXEB-K771Y] netlink_parse_info: netlink-dp-in (NS 0) type RTM_NEWLINK(16), len=1520, seq=0, pid=0
    18  Apr 08 12:21:44.431856 zebra[3431682]: [K8FXY-V65ZJ] Intf dplane ctx 0x7f3fbc029e20, op INTF_INSTALL, ifindex (1926), result QUEUED
    19  Apr 08 12:21:44.431871 zebra[3431682]: [WV1M1-5064M] RTM_NEWLINK update for tapd0ba6558-f4(1926) sl_type 4 master 0 flags 0x11043
    20  Apr 08 12:21:44.431882 zebra[3431682]: [VJ8AQ-ZFT0K] Intf tapd0ba6558-f4(1926) PTM up, notifying clients is_up:1 pd_cleared:0
    21  Apr 08 12:21:44.431893 zebra[3431682]: [K8FXY-V65ZJ] Intf dplane ctx 0x7f3fbc028250, op INTF_INSTALL, ifindex (1926), result QUEUED
    22  Apr 08 12:21:44.431903 zebra[3431682]: [WV1M1-5064M] RTM_NEWLINK update for tapd0ba6558-f4(1926) sl_type 4 master 0 flags 0x11043
    23  Apr 08 12:21:44.431914 zebra[3431682]: [VJ8AQ-ZFT0K] Intf tapd0ba6558-f4(1926) PTM up, notifying clients is_up:1 pd_cleared:0
    24  Apr 08 12:21:44.431925 zebra[3431682]: [K8FXY-V65ZJ] Intf dplane ctx 0x7f3fbc027650, op INTF_INSTALL, ifindex (1926), result QUEUED
    25  Apr 08 12:21:44.431935 zebra[3431682]: [WV1M1-5064M] RTM_NEWLINK update for tapd0ba6558-f4(1926) sl_type 4 master 0 flags 0x11043
    26  Apr 08 12:21:44.431945 zebra[3431682]: [VJ8AQ-ZFT0K] Intf tapd0ba6558-f4(1926) PTM up, notifying clients is_up:1 pd_cleared:0
    27  Apr 08 12:21:44.431956 zebra[3431682]: [K8FXY-V65ZJ] Intf dplane ctx 0x7f3fbc02bdf0, op INTF_INSTALL, ifindex (1926), result QUEUED
    28  Apr 08 12:21:44.431967 zebra[3431682]: [WV1M1-5064M] RTM_NEWLINK update for tapd0ba6558-f4(1926) sl_type 4 master 0 flags 0x11043
    29  Apr 08 12:21:44.431978 zebra[3431682]: [VJ8AQ-ZFT0K] Intf tapd0ba6558-f4(1926) PTM up, notifying clients is_up:1 pd_cleared:0
    30  (snip, unrelated BGP advertisement about a non-local L2VNI)
    31  Apr 08 12:21:46.203319 zebra[3431682]: [KMXEB-K771Y] netlink_parse_info: netlink-dp-in (NS 0) type RTM_NEWLINK(16), len=1512, seq=0, pid=0
    32  Apr 08 12:21:46.203821 zebra[3431682]: [KMXEB-K771Y] netlink_parse_info: netlink-dp-in (NS 0) type RTM_NEWLINK(16), len=1512, seq=0, pid=0
    33  Apr 08 12:21:46.204409 zebra[3431682]: [KMXEB-K771Y] netlink_parse_info: netlink-dp-in (NS 0) type RTM_NEWLINK(16), len=1520, seq=0, pid=0
    34  Apr 08 12:21:46.204461 zebra[3431682]: [KMXEB-K771Y] netlink_parse_info: netlink-dp-in (NS 0) type RTM_NEWLINK(16), len=1520, seq=0, pid=0
    35  Apr 08 12:21:46.204469 zebra[3431682]: [KMXEB-K771Y] netlink_parse_info: netlink-dp-in (NS 0) type RTM_NEWLINK(16), len=1520, seq=0, pid=0
    36  Apr 08 12:21:46.208016 zebra[3431682]: [K8FXY-V65ZJ] Intf dplane ctx 0x7f3fbc029e20, op INTF_INSTALL, ifindex (1926), result QUEUED
    37  Apr 08 12:21:46.208034 zebra[3431682]: [WV1M1-5064M] RTM_NEWLINK update for tapd0ba6558-f4(1926) sl_type 4 master 0 flags 0x11043
    38  Apr 08 12:21:46.208046 zebra[3431682]: [VJ8AQ-ZFT0K] Intf tapd0ba6558-f4(1926) PTM up, notifying clients is_up:1 pd_cleared:0
    39  Apr 08 12:21:46.208057 zebra[3431682]: [K8FXY-V65ZJ] Intf dplane ctx 0x7f3fbc028250, op INTF_INSTALL, ifindex (1926), result QUEUED
    40  Apr 08 12:21:46.208068 zebra[3431682]: [WV1M1-5064M] RTM_NEWLINK update for tapd0ba6558-f4(1926) sl_type 4 master 0 flags 0x11043
    41  Apr 08 12:21:46.208079 zebra[3431682]: [VJ8AQ-ZFT0K] Intf tapd0ba6558-f4(1926) PTM up, notifying clients is_up:1 pd_cleared:0
    42  Apr 08 12:21:46.208089 zebra[3431682]: [K8FXY-V65ZJ] Intf dplane ctx 0x7f3fbc027650, op INTF_INSTALL, ifindex (1926), result QUEUED
    43  Apr 08 12:21:46.208101 zebra[3431682]: [WV1M1-5064M] RTM_NEWLINK update for tapd0ba6558-f4(1926) sl_type 4 master 0 flags 0x11043
    44  Apr 08 12:21:46.208111 zebra[3431682]: [VJ8AQ-ZFT0K] Intf tapd0ba6558-f4(1926) PTM up, notifying clients is_up:1 pd_cleared:0
    45  Apr 08 12:21:46.208122 zebra[3431682]: [K8FXY-V65ZJ] Intf dplane ctx 0x7f3fbc02bdf0, op INTF_INSTALL, ifindex (1926), result QUEUED
    46  Apr 08 12:21:46.208133 zebra[3431682]: [WV1M1-5064M] RTM_NEWLINK update for tapd0ba6558-f4(1926) sl_type 4 master 0 flags 0x11043
    47  Apr 08 12:21:46.208144 zebra[3431682]: [VJ8AQ-ZFT0K] Intf tapd0ba6558-f4(1926) PTM up, notifying clients is_up:1 pd_cleared:0
    48  Apr 08 12:21:46.208155 zebra[3431682]: [K8FXY-V65ZJ] Intf dplane ctx 0x7f3fbc02c820, op INTF_INSTALL, ifindex (1926), result QUEUED
    49  Apr 08 12:21:46.208166 zebra[3431682]: [WV1M1-5064M] RTM_NEWLINK update for tapd0ba6558-f4(1926) sl_type 4 master 0 flags 0x11043
    50  Apr 08 12:21:46.208187 zebra[3431682]: [VJ8AQ-ZFT0K] Intf tapd0ba6558-f4(1926) PTM up, notifying clients is_up:1 pd_cleared:0
    51  Apr 08 12:21:48.597855 bgpd[3431687]: [PAPP6-VDAWM] eth5(l5-g4-osl2) rcvd UPDATE about RD 100.82.4.2:2 [2]:[0]:[48]:[fa:16:3e:7e:e3:5a]:[32]:[185.47.41.88] label 0 l2vpn evpn -- withdrawn
    52  Apr 08 12:21:48.597879 bgpd[3431687]: [K7P35-81YTJ] default (0): Uninstalling EVPN [2]:[0]:[48]:[fa:16:3e:7e:e3:5a]:[32]:[185.47.41.88] route from VNI 10099 IP/MAC table
    53  Apr 08 12:21:48.597890 bgpd[3431687]: [PAPP6-VDAWM] eth5(l5-g4-osl2) rcvd UPDATE about RD 100.82.4.2:2 [2]:[0]:[48]:[fa:16:3e:7e:e3:5a] label 0 l2vpn evpn -- withdrawn
    54  Apr 08 12:21:48.597900 bgpd[3431687]: [K7P35-81YTJ] default (0): Uninstalling EVPN [2]:[0]:[48]:[fa:16:3e:7e:e3:5a] route from VNI 10099 IP/MAC table
    55  Apr 08 12:21:48.597909 bgpd[3431687]: [PAPP6-VDAWM] eth4(l4-g4-osl2) rcvd UPDATE about RD 100.82.4.2:2 [2]:[0]:[48]:[fa:16:3e:7e:e3:5a]:[32]:[185.47.41.88] label 0 l2vpn evpn -- withdrawn
    56  Apr 08 12:21:48.597919 bgpd[3431687]: [K7P35-81YTJ] default (0): Uninstalling EVPN [2]:[0]:[48]:[fa:16:3e:7e:e3:5a]:[32]:[185.47.41.88] route from VNI 10099 IP/MAC table
    57  Apr 08 12:21:48.597928 bgpd[3431687]: [PAPP6-VDAWM] eth4(l4-g4-osl2) rcvd UPDATE about RD 100.82.4.2:2 [2]:[0]:[48]:[fa:16:3e:7e:e3:5a] label 0 l2vpn evpn -- withdrawn
    58  Apr 08 12:21:48.597938 bgpd[3431687]: [K7P35-81YTJ] default (0): Uninstalling EVPN [2]:[0]:[48]:[fa:16:3e:7e:e3:5a] route from VNI 10099 IP/MAC table
    59  Apr 08 12:21:48.608668 ip[316985]: 185.47.41.88 dev irb-99 FAILED proto zebra
    60  Apr 08 12:21:48.608668 ip[316985]: Deleted 185.47.41.88 dev irb-99 FAILED proto zebra
    61  Apr 08 12:21:48.608668 ip[316985]: Deleted dev l2vni-10099 lladdr fa:16:3e:7e:e3:5a extern_learn NOARP
    62  Apr 08 12:21:48.610065 ip[316985]: Deleted 100.82.4.2 dev l2vni-10099 lladdr fa:16:3e:7e:e3:5a REACHABLE NOARP
    63  Apr 08 12:21:48.608697 zebra[3431682]: [NH6N7-54CD1] Tx RTM_DELNEIGH family ipv4 IF irb-99(1687) Neigh 185.47.41.88 MAC null flags 0x10 state 0x0 ext_flags 0x0
    64  Apr 08 12:21:48.610651 bridge[3486074]: Deleted fa:16:3e:7e:e3:5a dev l2vni-10099 vlan 99 extern_learn sticky master br-evpn static
    65  Apr 08 12:21:48.610651 bridge[3486074]: Deleted fa:16:3e:7e:e3:5a dev l2vni-10099 dst 100.82.4.2 self sticky static
    66  Apr 08 12:21:48.608709 zebra[3431682]: [MMX22-H2MY5] Tx RTM_DELNEIGH family bridge IF l2vni-10099(1685) VLAN 99 MAC fa:16:3e:7e:e3:5a dst 100.82.4.2 nhg 0 rem
    67  Apr 08 12:21:48.608719 zebra[3431682]: [HYEHE-CQZ9G] nl_batch_send: netlink-dp (NS 0), batch size=136, msg cnt=2
    68  Apr 08 12:21:48.608842 zebra[3431682]: [GZB4H-6FWZE] Rx NEIGH_IP_DELETE family IPv4 IF irb-99(1687) vrf vrf-5000(1686) IP 185.47.41.88
    69  Apr 08 12:21:48.608873 zebra[3431682]: [KMXEB-K771Y] netlink_parse_info: netlink-dp-in (NS 0) type RTM_DELNEIGH(29), len=72, seq=0, pid=0
    70  Apr 08 12:21:48.608885 zebra[3431682]: [KMXEB-K771Y] netlink_parse_info: netlink-dp-in (NS 0) type RTM_DELNEIGH(29), len=84, seq=0, pid=0
    71  Apr 08 12:21:48.608894 zebra[3431682]: [HM5M4-AQPPX] Rx RTM_DELNEIGH AF_BRIDGE IF 1685 VLAN 99 st 0x40 fl 0x50 MAC fa:16:3e:7e:e3:5a nhg 0 vni 0
    72  Apr 08 12:21:48.608904 zebra[3431682]: [KMXEB-K771Y] netlink_parse_info: netlink-dp-in (NS 0) type RTM_DELNEIGH(29), len=68, seq=0, pid=0
    73  Apr 08 12:21:48.608913 zebra[3431682]: [HM5M4-AQPPX] Rx RTM_DELNEIGH AF_BRIDGE IF 1685 st 0x42 fl 0x42 MAC fa:16:3e:7e:e3:5a dst 100.82.4.2 nhg 0 vni 0
    74  Apr 08 12:21:48.626749 bash[2317542]: --- /dev/fd/63        2026-04-08 12:21:48.622422716 +0200
    75  Apr 08 12:21:48.626749 bash[2317542]: +++ /dev/fd/62        2026-04-08 12:21:48.623422716 +0200
    76  Apr 08 12:21:48.626749 bash[2317542]: @@ -1 +1 @@
    77  Apr 08 12:21:48.626749 bash[2317542]: -185.47.41.88 dev irb-99 lladdr fa:16:3e:7e:e3:5a extern_learn NOARP proto zebra
    78  Apr 08 12:21:48.626749 bash[2317542]: +
    79  Apr 08 12:21:48.643993 bash[2317548]: --- /dev/fd/63        2026-04-08 12:21:48.640422717 +0200
    80  Apr 08 12:21:48.643993 bash[2317548]: +++ /dev/fd/62        2026-04-08 12:21:48.641422717 +0200
    81  Apr 08 12:21:48.643993 bash[2317548]: @@ -1,2 +1 @@
    82  Apr 08 12:21:48.643993 bash[2317548]: -fa:16:3e:7e:e3:5a dev l2vni-10099 vlan 99 extern_learn sticky master br-evpn static
    83  Apr 08 12:21:48.643993 bash[2317548]: -fa:16:3e:7e:e3:5a dev l2vni-10099 dst 100.82.4.2 self sticky static
    84  Apr 08 12:21:48.643993 bash[2317548]: +
    85  Apr 08 12:21:48.659838 zebra[3431682]: [YXPF5-B2CE0] netlink_route_multipath_msg_encode: RTM_DELROUTE 185.47.41.88/32 vrf 1686(100005000)
    86  Apr 08 12:21:48.659852 zebra[3431682]: [HYEHE-CQZ9G] nl_batch_send: netlink-dp (NS 0), batch size=52, msg cnt=1
    87  Apr 08 12:21:48.804382 bgpd[3431687]: [XXWBM-V772F] eth4(l4-g4-osl2) rcvd UPDATE w/ attr: nexthop 100.82.4.2, extcommunity RT:12834:10099 MM:0, sticky MAC, path 4282004244 4282005241 65201 39029 65202 4282005242 4282004245 4282004002
    88  Apr 08 12:21:48.804393 bgpd[3431687]: [YCKEM-GB33T] eth4(l4-g4-osl2) rcvd RD 100.82.4.2:2 [2]:[0]:[48]:[fa:16:3e:7e:e3:5a] label 10099 l2vpn evpn
    89  Apr 08 12:21:48.804398 bgpd[3431687]: [WTJQB-H1EZQ] default (0): Installing EVPN [2]:[0]:[48]:[fa:16:3e:7e:e3:5a] route in VNI 10099 IP/MAC table
    90  Apr 08 12:21:48.807339 bgpd[3431687]: [XXWBM-V772F] eth5(l5-g4-osl2) rcvd UPDATE w/ attr: nexthop 100.82.4.2, extcommunity RT:12834:5000 RT:12834:10099 Rmac:3a:83:7c:72:d4:89, path 4282004245 4282005242 65202 39029 65201 4282005241 4282004244 4282004002
    91  Apr 08 12:21:48.807347 bgpd[3431687]: [YCKEM-GB33T] eth5(l5-g4-osl2) rcvd RD 100.82.4.2:2 [2]:[0]:[48]:[fa:16:3e:7e:e3:5a]:[32]:[185.47.41.88] label 10099/5000 l2vpn evpn
    92  Apr 08 12:21:48.807352 bgpd[3431687]: [WTJQB-H1EZQ] default (0): Installing EVPN [2]:[0]:[48]:[fa:16:3e:7e:e3:5a]:[32]:[185.47.41.88] route in VNI 10099 IP/MAC table
    93  Apr 08 12:21:48.807358 bgpd[3431687]: [XXWBM-V772F] eth5(l5-g4-osl2) rcvd UPDATE w/ attr: nexthop 100.82.4.2, extcommunity RT:12834:10099 MM:0, sticky MAC, path 4282004245 4282005242 65202 39029 65201 4282005241 4282004244 4282004002
    94  Apr 08 12:21:48.807362 bgpd[3431687]: [YCKEM-GB33T] eth5(l5-g4-osl2) rcvd RD 100.82.4.2:2 [2]:[0]:[48]:[fa:16:3e:7e:e3:5a] label 10099 l2vpn evpn
    95  Apr 08 12:21:48.807367 bgpd[3431687]: [WTJQB-H1EZQ] default (0): Installing EVPN [2]:[0]:[48]:[fa:16:3e:7e:e3:5a] route in VNI 10099 IP/MAC table
    96  Apr 08 12:21:48.815847 zebra[3431682]: [MMX22-H2MY5] Tx RTM_NEWNEIGH family bridge IF l2vni-10099(1685) VLAN 99 sticky MAC fa:16:3e:7e:e3:5a dst 100.82.4.2 nhg 0 rem
    97  Apr 08 12:21:48.816069 ip[316985]: dev l2vni-10099 lladdr fa:16:3e:7e:e3:5a NOARP
    98  Apr 08 12:21:48.816069 ip[316985]: 100.82.4.2 dev l2vni-10099 lladdr fa:16:3e:7e:e3:5a REACHABLE NOARP
    99  Apr 08 12:21:48.816069 ip[316985]: 185.47.41.88 dev irb-99 lladdr fa:16:3e:7e:e3:5a extern_learn NOARP proto zebra
   100  Apr 08 12:21:48.815854 zebra[3431682]: [NH6N7-54CD1] Tx RTM_NEWNEIGH family ipv4 IF irb-99(1687) Neigh 185.47.41.88 MAC fa:16:3e:7e:e3:5a flags 0x10 state 0x40 ext_flags 0x0
   101  Apr 08 12:21:48.816422 bridge[3486074]: fa:16:3e:7e:e3:5a dev l2vni-10099 vlan 99 sticky master br-evpn static
   102  Apr 08 12:21:48.816422 bridge[3486074]: fa:16:3e:7e:e3:5a dev l2vni-10099 dst 100.82.4.2 self sticky static
   103  Apr 08 12:21:48.815860 zebra[3431682]: [HYEHE-CQZ9G] nl_batch_send: netlink-dp (NS 0), batch size=136, msg cnt=2
   104  Apr 08 12:21:48.815919 zebra[3431682]: [KMXEB-K771Y] netlink_parse_info: netlink-dp-in (NS 0) type RTM_NEWNEIGH(28), len=84, seq=0, pid=0
   105  Apr 08 12:21:48.815925 zebra[3431682]: [HM5M4-AQPPX] Rx RTM_NEWNEIGH AF_BRIDGE IF 1685 VLAN 99 st 0x40 fl 0x40 MAC fa:16:3e:7e:e3:5a nhg 0 vni 0
   106  Apr 08 12:21:48.815930 zebra[3431682]: [KMXEB-K771Y] netlink_parse_info: netlink-dp-in (NS 0) type RTM_NEWNEIGH(28), len=68, seq=0, pid=0
   107  Apr 08 12:21:48.815934 zebra[3431682]: [HM5M4-AQPPX] Rx RTM_NEWNEIGH AF_BRIDGE IF 1685 st 0x42 fl 0x42 MAC fa:16:3e:7e:e3:5a dst 100.82.4.2 nhg 0 vni 0
   108  Apr 08 12:21:48.840313 bash[2317598]: --- /dev/fd/63        2026-04-08 12:21:48.837422731 +0200
   109  Apr 08 12:21:48.840313 bash[2317598]: +++ /dev/fd/62        2026-04-08 12:21:48.837422731 +0200
   110  Apr 08 12:21:48.840313 bash[2317598]: @@ -1 +1 @@
   111  Apr 08 12:21:48.840313 bash[2317598]: -
   112  Apr 08 12:21:48.840313 bash[2317598]: +185.47.41.88 dev irb-99 lladdr fa:16:3e:7e:e3:5a extern_learn NOARP proto zebra
   113  Apr 08 12:21:48.857033 bash[2317604]: --- /dev/fd/63        2026-04-08 12:21:48.853422732 +0200
   114  Apr 08 12:21:48.857033 bash[2317604]: +++ /dev/fd/62        2026-04-08 12:21:48.854422732 +0200
   115  Apr 08 12:21:48.857033 bash[2317604]: @@ -1 +1,2 @@
   116  Apr 08 12:21:48.857033 bash[2317604]: -
   117  Apr 08 12:21:48.857033 bash[2317604]: +fa:16:3e:7e:e3:5a dev l2vni-10099 vlan 99 sticky master br-evpn static
   118  Apr 08 12:21:48.857033 bash[2317604]: +fa:16:3e:7e:e3:5a dev l2vni-10099 dst 100.82.4.2 self sticky static
   119  Apr 08 12:21:48.868336 zebra[3431682]: [YXPF5-B2CE0] netlink_route_multipath_msg_encode: RTM_NEWROUTE 185.47.41.88/32 vrf 1686(100005000)
   120  Apr 08 12:21:48.868346 zebra[3431682]: [J87BH-XW5PP] netlink_route_multipath_msg_encode: 185.47.41.88/32 nhg_id is 8768438
   121  Apr 08 12:21:48.868352 zebra[3431682]: [HYEHE-CQZ9G] nl_batch_send: netlink-dp (NS 0), batch size=60, msg cnt=1
   122  Apr 08 12:21:48.915504 python3[298104]: [bridgemanager.py:80 → ensure_fdb()] Adding static sticky FDB entry for fa:16:3e:7e:e3:5a on VLAN 99
   123  Apr 08 12:21:48.919726 ip[316985]: dev veth-to-ovs lladdr fa:16:3e:7e:e3:5a NOARP
   124  Apr 08 12:21:48.919872 bridge[3486074]: fa:16:3e:7e:e3:5a dev veth-to-ovs vlan 99 sticky master br-evpn static
   125  Apr 08 12:21:48.919742 zebra[3431682]: [KMXEB-K771Y] netlink_parse_info: netlink-dp-in (NS 0) type RTM_NEWNEIGH(28), len=84, seq=0, pid=0
   126  Apr 08 12:21:48.919751 zebra[3431682]: [HM5M4-AQPPX] Rx RTM_NEWNEIGH AF_BRIDGE IF 14 VLAN 99 st 0x40 fl 0x40 MAC fa:16:3e:7e:e3:5a nhg 0 vni 0
   127  Apr 08 12:21:48.919821 zebra[3431682]: [JJ27M-21GWC][EC 4043309181] MAC fa:16:3e:7e:e3:5a already learnt as remote sticky MAC behind VTEP 100.82.4.2 VNI 10099
   128  Apr 08 12:21:48.920561 python3[298104]: [neighmanager.py:72 → ensure_neigh()] Adding static neigh entry 185.47.41.88→fa:16:3e:7e:e3:5a on irb-99
   129  Apr 08 12:21:48.925214 zebra[3431682]: [KMXEB-K771Y] netlink_parse_info: netlink-dp-in (NS 0) type RTM_NEWNEIGH(28), len=84, seq=0, pid=2317625
   130  Apr 08 12:21:48.925384 ip[316985]: 185.47.41.88 dev irb-99 lladdr fa:16:3e:7e:e3:5a PERMANENT proto 255
   131  Apr 08 12:21:48.925223 zebra[3431682]: [Y6ZNP-9NYSB] Rx RTM_NEWNEIGH family ipv4 IF 1687 NSID 0 IP 185.47.41.88 MAC fa:16:3e:7e:e3:5a state 0x80 flags 0x0 ext_flags 0x0, proto 255
   132  Apr 08 12:21:48.944164 bash[2317633]: --- /dev/fd/63        2026-04-08 12:21:48.940422739 +0200
   133  Apr 08 12:21:48.944164 bash[2317633]: +++ /dev/fd/62        2026-04-08 12:21:48.941422739 +0200
   134  Apr 08 12:21:48.944164 bash[2317633]: @@ -1 +1 @@
   135  Apr 08 12:21:48.944164 bash[2317633]: -185.47.41.88 dev irb-99 lladdr fa:16:3e:7e:e3:5a extern_learn NOARP proto zebra
   136  Apr 08 12:21:48.944164 bash[2317633]: +185.47.41.88 dev irb-99 lladdr fa:16:3e:7e:e3:5a PERMANENT proto 255
   137  Apr 08 12:21:48.954802 bgpd[3431687]: [PAPP6-VDAWM] eth5(l5-g4-osl2) rcvd UPDATE about RD 100.82.4.2:2 [2]:[0]:[48]:[fa:16:3e:7e:e3:5a]:[32]:[185.47.41.88] label 0 l2vpn evpn -- withdrawn
   138  Apr 08 12:21:48.954812 bgpd[3431687]: [K7P35-81YTJ] default (0): Uninstalling EVPN [2]:[0]:[48]:[fa:16:3e:7e:e3:5a]:[32]:[185.47.41.88] route from VNI 10099 IP/MAC table
   139  Apr 08 12:21:48.954818 bgpd[3431687]: [PAPP6-VDAWM] eth5(l5-g4-osl2) rcvd UPDATE about RD 100.82.4.2:2 [2]:[0]:[48]:[fa:16:3e:7e:e3:5a] label 0 l2vpn evpn -- withdrawn
   140  Apr 08 12:21:48.954824 bgpd[3431687]: [K7P35-81YTJ] default (0): Uninstalling EVPN [2]:[0]:[48]:[fa:16:3e:7e:e3:5a] route from VNI 10099 IP/MAC table
   141  Apr 08 12:21:48.956377 bgpd[3431687]: [PAPP6-VDAWM] eth4(l4-g4-osl2) rcvd UPDATE about RD 100.82.4.2:2 [2]:[0]:[48]:[fa:16:3e:7e:e3:5a] label 0 l2vpn evpn -- withdrawn
   142  Apr 08 12:21:48.956386 bgpd[3431687]: [K7P35-81YTJ] default (0): Uninstalling EVPN [2]:[0]:[48]:[fa:16:3e:7e:e3:5a] route from VNI 10099 IP/MAC table
   143  Apr 08 12:21:48.960885 bash[2317639]: --- /dev/fd/63        2026-04-08 12:21:48.957422740 +0200
   144  Apr 08 12:21:48.960885 bash[2317639]: +++ /dev/fd/62        2026-04-08 12:21:48.958422740 +0200
   145  Apr 08 12:21:48.960885 bash[2317639]: @@ -1,2 +1,2 @@
   146  Apr 08 12:21:48.960885 bash[2317639]: -fa:16:3e:7e:e3:5a dev l2vni-10099 vlan 99 sticky master br-evpn static
   147  Apr 08 12:21:48.960885 bash[2317639]: +fa:16:3e:7e:e3:5a dev veth-to-ovs vlan 99 sticky master br-evpn static
   148  Apr 08 12:21:48.960885 bash[2317639]:  fa:16:3e:7e:e3:5a dev l2vni-10099 dst 100.82.4.2 self sticky static
   149  Apr 08 12:21:48.965710 zebra[3431682]: [MMX22-H2MY5] Tx RTM_DELNEIGH family bridge IF l2vni-10099(1685) VLAN 99 MAC fa:16:3e:7e:e3:5a dst 100.82.4.2 nhg 0 rem
   150  Apr 08 12:21:48.965718 zebra[3431682]: [HYEHE-CQZ9G] nl_batch_send: netlink-dp (NS 0), batch size=80, msg cnt=1
   151  Apr 08 12:21:48.965724 zebra[3431682]: [MQ5AP-2S1F5] netlink-dp (NS 0) error: No such file or directory, type=RTM_DELNEIGH(29), seq=149678738, pid=3631020340
   152  Apr 08 12:21:48.965730 zebra[3431682]: [QTT8V-3ZQ34] nl_batch_read_resp: netlink error message seq=149678738
   153  Apr 08 12:21:49.016192 zebra[3431682]: [YXPF5-B2CE0] netlink_route_multipath_msg_encode: RTM_DELROUTE 185.47.41.88/32 vrf 1686(100005000)
   154  Apr 08 12:21:49.016202 zebra[3431682]: [HYEHE-CQZ9G] nl_batch_send: netlink-dp (NS 0), batch size=52, msg cnt=1

Checklist

  • I have searched the open issues for this bug.
  • I have not included sensitive information in this report.

Metadata

Metadata

Assignees

No one assigned

    Labels

    triageNeeds further investigation

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions