IOS: %HW_VPN-1-HPRXERR Packet Encryption/Decryption Errors

%HW_VPN-1-HPRXERR : [chars]: Packet Encryption/Decryption error, status=[int]

---

CSCdt40220 - AIM encryption produces Packet Encryption/Decryption error
http://www.cisco.com/cgi-bin/Support/Bugtool/onebug.pl?bugid=CSCdt40220&Submit=Search

Symptoms:
A router displays one of the following error messages:

HW_VPN-1-HPRXERR: Hardware VPN0/2: Packet Encryption/Decryption error, status=4612

This is a notification message seen on the console of the DECRYPTING PEER that tells the user that IPSEC packets have been received out of order. Re-ordering can occur in one of 3 places:
1. encrypting peer
2. network
3. decrypting peer

Only in rare cases can this occur in the decrypting peer.

The only known way for this to occur in the decrypting peer is for a packet to be bumped to process switch while the following packets from the same tunnel are fast or cef switched. This could happen if the packet is fragmented and needs re-assembly.

The following lists some of the common scenarios that might introduce out-of-order IPSEC packets. These scenrios are considered normal behaviors:

1. Fragmentation - the decrypting peer uses process switching to fragmented packets. To minimize the impact of this, Look-Ahead-Fragmentation should be enabled. This feature was added to IOS via CSCdw77514.

2. QoS: QoS scheduling mechanism happening after IPSec encryption could cause packets in the same IPsec SAs to be transmitted out-of-order.

3. Pak_priority: pak_priority is an internal flag set by the IOS to some of the router generated packets that are considered critical, e.g., routing updates, interface keepalives. When output interface queue is congested, router will honor the pak_priority flags to make sure the high priority packets are transmitted first. So in the GRE over IPsec and dynamic routing protocol design, the ESP packets could become out-of-order if the egress interface is congested and the router has to transmit the encrypted routing update first.

Conditions:
Either of the messages may be displayed depending on whether Authentication Header (AH) or Encapsulation Protocol (ESP) encapsulation is used. In addition, the ah_seq_fail or esp_seq_fail error counts increment in the output of the show crypto engine accelerator statistic privileged EXEC command.

Workaround:
- Set the maximum transmission unit (MTU) size of inbound streams to less than 1400 bytes.
- enable Look-Ahead-Fragmentation

---

WORKAROUND:

1. Adjust the interface MTU (preferably below 1400):

interface type mod/port
ip mtu byte


2. Adjust Fragmentation (See Pre-Fragmentation for IPSEC VPNs):

crypto ipsec df-bit clear
crypto ipsec fragmentation before-encryption

-- OR --

crypto ipsec df-bit clear
interface type mod/port
crypto ipsec fragmentation before-encryption

---

REFERENCES:

Pre-fragmentation for IPSec VPNs
http://www.cisco.com/en/US/products/sw/iosswrel/ps1839/products_feature_guide09186a0080115533.html