IOS: EIGRP Peering Flapping, Auth Failure - %DUAL-5-NBRCHANGE: IP-EIGRP: Auth failure

This is an actual case we have raised recently with Cisco as we are having unexplained EIGRP flaps between two of our devices. It has been working for more than a year -- actually, it never had any issues when this was brought online last year.



Scenario:

EIGRP peering flaps between two devices, due to authentication failure. The output of show logging is flooded with the below syslogs repeatedly:

Nov 2 01:30:43.436 GMT: %DUAL-5-NBRCHANGE: IP-EIGRP(0) 2: Neighbor 10.10.10.2 (GigabitEthernet1/1) is down: Auth failure


Nov 2 01:30:45.040 GMT: %DUAL-5-NBRCHANGE: IP-EIGRP(0) 2: Neighbor 10.10.10.2 (GigabitEthernet1/1) is up: new adjacency

Nov 2 01:30:47.316 GMT: %DUAL-5-NBRCHANGE: IP-EIGRP(0) 2: Neighbor 10.10.10.2 (GigabitEthernet1/1) is down: Auth failure

Nov 2 01:30:48.820 GMT: %DUAL-5-NBRCHANGE: IP-EIGRP(0) 2: Neighbor 10.10.10.2 (GigabitEthernet1/1) is up: new adjacency

Topology is straightforward:

Router1 Gi1/1 <-----> Gi2/2 Router2


Router1 Gi1/1 = 10.10.10.1/24
Router2 Gi2/2 = 10.10.10.2/24

MD5 Authentication is used and the same key string is configured on both devices

Router1#show key chain MYCHAIN

 key-chain MYCHAIN
  key 1 -- text "myCiscoChain"
   accept lifetime (always valid) - (always valid) [valid now]
   send lifetime (always valid) - (always valid) [valid now]
Router1#
Router1#
Router1#
Router1# show run | begin key chain
key-chain MYCHAIN
 key 1
  key string 5 098123456SA679
...
Router1# show run int Gi1/1
interface GigabitEthernet1/1
 ip address 10.10.10.1 255.255.255.0
 ip authentication mode eigrp 2 md5
 ip authentication key-chain eigrp 2 MYCHAIN
...

Problem:
The issue was with a Level2/Severe bug with the IOS image running on one of the devices. Bug details below:

CSCdu73495 - All routes to network not seen because of invalid md5 authentication
http://tools.cisco.com/Support/BugToolKit/search/getBugDetails.do?method=fetchBugDetails&bugId=CSCdu73495

Enhanced Interior Gateway Routing Protocol (EIGRP) routes cannot be seen even when message digest algorithm 5 (MD5) is authenticated on all routers. This problem is intermittent and may occur when authentication is turned off and subsequently turned back on again. Sometimes, this problem occurs just after authentication is enabled.  

Workaround: This problem is intermittent and may be resolved by disabling and reenabling authentication a second time. This problem may automatically be resolved after a few minutes.


EIGRP Authentication problems & flaps on unrelated links


This bug is a duplicate of CSCdu73495, which causes authentication-related breakage in establishing peers, which eventually clears up on it's own after an indeterminate time. It can be triggered by bouncing peers/interfaces. You will not encounter this issue if you disable EIGRP authentication. CSCdu73495 was resolved in later versions of 12.1E IOS.

EIGRP neighbour cant be established if use MD5 authentication

C2610 EIGRP neighbour could be established via md5 authentication first time. After shut/no shut c2610 ethernet interface, it can't established any more. Via serial interface works fine.

EIGRP MD5 Authentication Breaks Neighbor Adjacencies over LANE

In a LANE environment with 3 or more devices running EIGRP, when upgrading from 12.1(6)E4 to 12.1(10)E4 on 7500's, EIGRP neighbor relationships may not be formed between devices running 12.1(10)E4. This is verified by performing a on one of the devices running 12.1(10)E4. The workaround for this scenario is to wait an unpredictable amount of time for the neighbors to converge, or remove and re-add EIGRP authentication from the interfaces on the affected devices. Also, neighbors can be statically configured in order for EIGRP to use unicast, rather than multicast.

2921-EIGRP flap due to bad TLV received on serial interface

Symptom: EIGRP flaps observed due to retransmission retry limit exceeded. Bad TLV error messages are seen in the logs. Conditions: Issue seen when 2921 replaces the 2611 device with similar configs.

Workaround: None. Apart from 2921, customer is using 2611 that works fine.


Known Affected Versions (Not a comprehensive list):
12.1(9)M

12.1(26)M
15.0M
12.1(8b)E15
12.3(12e)M
 
Fixed-In (Not comprehensive list):

12.1(10.2)M
12.2(4.2)M
12.0(30)SZ4
12.0(32)S6b
12.0(32)S7
12.0(32)SY4
12.0(32.3)S
12.1(6)E11
12.1(10.5)E
12.1(10.5)EC
12.2(4.2)PI
12.2(4.2a)DA
12.2(5.1)S
12.2(6.4)B
12.2(6.4)PB
12.2(15)BW
12.2(15)BX
12.2(15)ZN
12.0(32.11.1)SY


Workarounds:

1. Disable then re-enable EIGRP authentication;
2. Instead of MD5, use clear text authentication; or
3. Disable EIGRP authentication.

Permanent Fix:
Upgrade IOS version.

Due to intermittence/unpredictability, either use clear text authentication or disable authentication outright if IOS upgrade is not possible immediately. However, bouncing (disable/re-enable) the authentication can serve as your quick fix.