AWS EC2 CentOS 7 with LibreSwan IPsec Tunnel UP (Established), No Traffic flow, cannot ping Peer encryption Domain Nodes
I have LibreSwan Setup on AWS EC2 CentOS7 instance, IPsec tunnel is established with the peer (Cisco ASA). The Tunnel is up, but I no traffic. I cannot ping the nodes at the side of the peer. Here is my setup. on AWS EC2 instance
EC2 instance - running CentOS7
Public IP: A.B.C.D
Private IP: E.F.G.H
[toor@ip-E-F-G-H ~]# lsb_release -a
LSB Version: :core-4.1-amd64:core-4.1-noarch
Distributor ID: CentOS
Description: CentOS Linux release 7.5.1804 (Core)
Release: 7.5.1804
Codename: Core
[toor@ip-E-F-G-H ~]# ipsec version
Linux Libreswan 3.23 (netkey) on 3.10.0-862.3.2.el7.x86_64
[toor@ip-E-F-G-H ~]# ipsec version
Verifying installed system and configuration files
Version check and ipsec on-path [OK]
Libreswan 3.23 (netkey) on 3.10.0-862.3.2.el7.x86_64
Checking for IPsec support in kernel [OK]
NETKEY: Testing XFRM related proc values
ICMP default/send_redirects [OK]
ICMP default/accept_redirects [OK]
XFRM larval drop [OK]
Pluto ipsec.conf syntax [OK]
Two or more interfaces found, checking IP forwarding [OK]
Checking rp_filter [OK]
Checking that pluto is running [OK]
Pluto listening for IKE on udp 500 [OK]
Pluto listening for IKE/NAT-T on udp 4500 [OK]
Pluto ipsec.secret syntax [OK]
Checking 'ip' command [OK]
Checking 'iptables' command [OK]
Checking 'prelink' command does not interfere with FIPS [OK]
Checking for obsolete ipsec.conf options [OK]
Here is my ipsec.conf
config setup
logfile=/var/log/pluto.log
virtual_private=%v4:M.N.O.P/32,%v4:Q.R.S.T/32 #are encryption Domain of my peer
protostack=netkey
conn MyConnection
authby=secret
auto=start
type=tunnel
## phase1 ##
ike=aes256-sha1;modp2048
keyexchange=ike
## phase2 ##
phase2=esp
phase2alg=aes256-sha1;modp2048
compress=no
pfs=yes
left=%defaultroute
leftid=A.B.C.D #MyPublic Ip on AWS EC2
leftsourceip=E.F.G.H . #My private ip of my ec2 centos7 instance
leftsubnet=E.F.G.H/32
leftnexthop=%defaultroute
## MyPeer ##
right=I.J.K.L #Public ip of the peer - cisco asa device
rightsubnets={M.N.O.P/32,Q.R.S.T/32}
ikelifetime=28800s
salifetime=3600s
aggrmode=no
my ipsec.secrets
A.B.C.D I.J.K.L: PSK "*&^%$3434"
here is ipsec status
[toor@ip-E-F-G-H ~]# ipsec auto --status
000 "MyConnection/0x2": IKE algorithms: AES_CBC_256-HMAC_SHA1-MODP1536
000 "MyConnection/0x2": IKE algorithm newest: AES_CBC_256-HMAC_SHA1-MODP1536
000 "MyConnection/0x2": ESP algorithms: AES_CBC_256-HMAC_SHA1_96-MODP1536
000 "MyConnection/0x2": ESP algorithm newest: AES_CBC_256-HMAC_SHA1_96; pfsgroup=MODP1536
000
000 Total IPsec connections: loaded 2, active 2
000
000 State Information: DDoS cookies not required, Accepting new IKE connections
000 IKE SAs: total(1), half-open(0), open(0), authenticated(1), anonymous(0)
000 IPsec SAs: total(2), authenticated(2), anonymous(0)
000
000 #2: "MyConnection/0x1":4500 STATE_QUICK_I2 (sent QI2, IPsec SA established); EVENT_SA_REPLACE in 734s; newest IPSEC; eroute owner; isakmp#1; idle; import:admin initiate
000 #2: "MyConnection/0x1" esp.bd6f5d5@I.J.K.L esp.e6e75a6@A.B.C.D tun.0@I.J.K.L tun.0@A.B.C.D ref=0 refhim=0 Traffic: ESPin=0B ESPout=4KB! ESPmax=4194303B
000 #1: "MyConnection/0x2":4500 STATE_MAIN_I4 (ISAKMP SA established); EVENT_SA_REPLACE in 25692s; newest ISAKMP; lastdpd=0s(seq in:0 out:0); idle; import:admin initiate
000 #3: "MyConnection/0x2":4500 STATE_QUICK_I2 (sent QI2, IPsec SA established); EVENT_SA_REPLACE in 524s; newest IPSEC; eroute owner; isakmp#1; idle; import:admin initiate
000 #3: "MyConnection/0x2" esp.be78561f@I.J.K.L esp.21e1d536@A.B.C.D tun.0@I.J.K.L tun.0@A.B.C.D ref=0 refhim=0 Traffic: ESPin=0B ESPout=1KB! ESPmax=4194303B
000
000 Bare Shunt list:
000
here is my route -n cmd and cannot ping those nodes in the encryption domain of my peer
[toor@ip-E-F-G-H ~]# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 E.F.0.1 0.0.0.0 UG 0 0 0 eth0
M.N.O.P E.F.0.1 255.255.255.255 UGH 0 0 0 eth0
Q.R.S.T E.F.0.1 255.255.255.255 UGH 0 0 0 eth0
E.F.0.0 0.0.0.0 255.255.240.0 U 0 0 0 eth0
[root@ip-E-F-G-H ~]#ping M.N.O.P
no reply
[root@ip-E-F-G-H ~]#ping Q.R.S.T
no reply
here is my iptables -L, no POSTROUTING
[root@ip-E-F-G-H ~]#iptables -L
Chain INPUT (policy ACCEPT)
target prot opt source destination
ACCEPT udp -- anywhere anywhere udp dpt:isakmp
ACCEPT tcp -- anywhere anywhere tcp dpt:ipsec-nat-t
ACCEPT udp -- anywhere anywhere udp dpt:ipsec-nat-t
ACCEPT esp -- anywhere anywhere
ACCEPT ah -- anywhere anywhere
Chain FORWARD (policy ACCEPT)
target prot opt source destination
ACCEPT all -- anywhere anywhere
ACCEPT all -- anywhere anywhere
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
[root@ip-E-F-G-H ~]#iptables -t nat -L -v -n
Chain PREROUTING (policy ACCEPT 19 packets, 1024 bytes)
pkts bytes target prot opt in out source destination
Chain INPUT (policy ACCEPT 19 packets, 1024 bytes)
pkts bytes target prot opt in out source destination
Chain OUTPUT (policy ACCEPT 305 packets, 127K bytes)
pkts bytes target prot opt in out source destination
Chain POSTROUTING (policy ACCEPT 305 packets, 127K bytes)
pkts bytes target prot opt in out source destination
On AWS EC2 instance, I have disbled Source/dest check. On AWS EC2 instance security group, added these on inbound
Type Protocol Port Range Source Description
All traffic All All M.N.O.P/32
All traffic All All Q.R.S.T/32
SSH TCP 22 0.0.0.0/0
Custom Protocol AH (51) All 0.0.0.0/0
Custom Protocol ESP (50) All 0.0.0.0/0
Custom UDP Rule UDP 4500 0.0.0.0/0
Custom UDP Rule UDP 500 0.0.0.0/0
All ICMP - IPv4 All N/A 0.0.0.0/0
On AWS EC2 instance security group, added these on outbound
Type Protocol Port Range Source Description
All traffic All All 0.0.0.0/0
All traffic All All M.N.O.P/32
All traffic All All Q.R.S.T/32
Under VPC section, region for my ec2 instance.. Added this on the routing table
Destination Target Status Propagated
E.F.0.0/16 local Active No
0.0.0.0/0 igw-747d3673hsd Active No
M.N.O.P/32 eni-0fec2be5-on-ec2 Active No
Q.R.S.T/32 eni-0fec2be5-on-ec2 Active No
Also Add some Network ACLs - inbound
Rule # Type Protocol Port Range Source Allow / Deny
100 ALL Traffic ALL ALL 0.0.0.0/0 ALLOW
200 ALL Traffic ALL ALL M.N.O.P/32 ALLOW
300 ALL Traffic ALL ALL Q.R.S.T/32 ALLOW
* ALL Traffic ALL ALL 0.0.0.0/0 DENY
Also Add some Network ACLs - Outbound
Rule # Type Protocol Port Range Source Allow / Deny
100 ALL Traffic ALL ALL 0.0.0.0/0 ALLOW
200 ALL Traffic ALL ALL M.N.O.P/32 ALLOW
300 ALL Traffic ALL ALL Q.R.S.T/32 ALLOW
* ALL Traffic ALL ALL 0.0.0.0/0 DENY
That's ALL - The tunnel is up, aws ec2 centos7 with Libreswan to Cisco ASA. But No Traffic, i cannot ping or trace the nodes behind the cisco asa firewall. Any pointers with on the aws side is highly appreciated. thanks
I have got the cisco Admin to show me his side
here is the sh crypto ipsec sa peer A.B.C.D output
sh crypto ipsec sa peer A.B.C.D
peer address: A.B.C.D
Crypto map tag: TELCOCryptoMap, seq num: 690, local addr: I.J.K.L
access-list CRYPTO-NETXCELL-VPN extended permit ip host M.N.O.P host E.F.G.H
local ident (addr/mask/prot/port): (M.N.O.P/255.255.255.255/0/0)
remote ident (addr/mask/prot/port): (E.F.G.H/255.255.255.255/0/0)
current_peer: A.B.C.D
#pkts encaps: 0, #pkts encrypt: 0, #pkts digest: 0
#pkts decaps: 0, #pkts decrypt: 0, #pkts verify: 0
#pkts compressed: 0, #pkts decompressed: 0
#pkts not compressed: 0, #pkts comp failed: 0, #pkts decomp failed: 0
#pre-frag successes: 0, #pre-frag failures: 0, #fragments created: 0
#PMTUs sent: 0, #PMTUs rcvd: 0, #decapsulated frgs needing reassembly: 0
#TFC rcvd: 0, #TFC sent: 0
#Valid ICMP Errors rcvd: 0, #Invalid ICMP Errors rcvd: 0
#send errors: 0, #recv errors: 0
local crypto endpt.: I.J.K.L/4500, remote crypto endpt.: A.B.C.D/4500
path mtu 1500, ipsec overhead 82(52), media mtu 1500
PMTU time remaining (sec): 0, DF policy: copy-df
ICMP error validation: disabled, TFC packets: disabled
current outbound spi: 3F59926A
current inbound spi : D8CD619E
inbound esp sas:
spi: 0xD8CD619E (3637338526)
SA State: active
transform: esp-aes-256 esp-sha-hmac no compression
in use settings ={L2L, Tunnel, NAT-T-Encaps, PFS Group 5, IKEv1, }
slot: 0, conn_id: 266432512, crypto-map: TELCOCryptoMap
sa timing: remaining key lifetime (kB/sec): (4374000/3438)
IV size: 16 bytes
replay detection support: Y
Anti replay bitmap:
0x00000000 0x00000001
outbound esp sas:
spi: 0x3F59926A (1062834794)
SA State: active
transform: esp-aes-256 esp-sha-hmac no compression
in use settings ={L2L, Tunnel, NAT-T-Encaps, PFS Group 5, IKEv1, }
slot: 0, conn_id: 266432512, crypto-map: TELCOCryptoMap
sa timing: remaining key lifetime (kB/sec): (4374000/3438)
IV size: 16 bytes
replay detection support: Y
Anti replay bitmap:
0x00000000 0x00000001
Crypto map tag: TELCOCryptoMap, seq num: 690, local addr: I.J.K.L
access-list CRYPTO-NETXCELL-VPN extended permit ip host Q.R.S.T host E.F.G.H
local ident (addr/mask/prot/port): (Q.R.S.T/255.255.255.255/0/0)
remote ident (addr/mask/prot/port): (E.F.G.H/255.255.255.255/0/0)
current_peer: A.B.C.D
#pkts encaps: 0, #pkts encrypt: 0, #pkts digest: 0
#pkts decaps: 0, #pkts decrypt: 0, #pkts verify: 0
#pkts compressed: 0, #pkts decompressed: 0
#pkts not compressed: 0, #pkts comp failed: 0, #pkts decomp failed: 0
#pre-frag successes: 0, #pre-frag failures: 0, #fragments created: 0
#PMTUs sent: 0, #PMTUs rcvd: 0, #decapsulated frgs needing reassembly: 0
#TFC rcvd: 0, #TFC sent: 0
#Valid ICMP Errors rcvd: 0, #Invalid ICMP Errors rcvd: 0
#send errors: 0, #recv errors: 0
local crypto endpt.: I.J.K.L/4500, remote crypto endpt.: A.B.C.D/4500
path mtu 1500, ipsec overhead 82(52), media mtu 1500
PMTU time remaining (sec): 0, DF policy: copy-df
ICMP error validation: disabled, TFC packets: disabled
current outbound spi: 01DCEC29
current inbound spi : D9735EB5
inbound esp sas:
spi: 0xD9735EB5 (3648216757)
SA State: active
transform: esp-aes-256 esp-sha-hmac no compression
in use settings ={L2L, Tunnel, NAT-T-Encaps, PFS Group 5, IKEv1, }
slot: 0, conn_id: 266432512, crypto-map: TELCOCryptoMap
sa timing: remaining key lifetime (kB/sec): (4374000/3438)
IV size: 16 bytes
replay detection support: Y
Anti replay bitmap:
0x00000000 0x00000001
outbound esp sas:
spi: 0x01DCEC29 (31255593)
SA State: active
transform: esp-aes-256 esp-sha-hmac no compression
in use settings ={L2L, Tunnel, NAT-T-Encaps, PFS Group 5, IKEv1, }
slot: 0, conn_id: 266432512, crypto-map: TELCOCryptoMap
sa timing: remaining key lifetime (kB/sec): (4374000/3438)
IV size: 16 bytes
replay detection support: Y
Anti replay bitmap:
0x00000000 0x00000001
With this, the cisco device doesnt seem to have any incoming esp packets. Any pointers, as i believe it might be somewhere at the aws level.. thanks
here is the tcpdump
[toor@ip-E-F-G-H ~]# tcpdump esp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
^C
0 packets captured
0 packets received by filter
0 packets dropped by kernel
[toor@ip-E-F-G-H ~]# tcpdump port 4500
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
13:39:28.382551 IP ip-E-F-G-H.us-east-2.compute.internal.ipsec-nat-t > I.J.K.L.ipsec-nat-t: isakmp-nat-keep-alive
13:39:28.382575 IP ip-E-F-G-H.us-east-2.compute.internal.ipsec-nat-t > I.J.K.L.ipsec-nat-t: isakmp-nat-keep-alive
13:39:28.382580 IP ip-E-F-G-H.us-east-2.compute.internal.ipsec-nat-t > I.J.K.L.ipsec-nat-t: isakmp-nat-keep-alive
13:39:29.222092 IP I.J.K.L.ipsec-nat-t > ip-E-F-G-H.us-east-2.compute.internal.ipsec-nat-t: NONESP-encap: isakmp: phase 2/others ? inf[E]
13:39:29.222277 IP ip-E-F-G-H.us-east-2.compute.internal.ipsec-nat-t > I.J.K.L.ipsec-nat-t: NONESP-encap: isakmp: phase 2/others ? inf[E]
^C
5 packets captured
5 packets received by filter
0 packets dropped by kernel
[toor@ip-E-F-G-H ~]# tcpdump port 500
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
^C
0 packets captured
0 packets received by filter
0 packets dropped by kernel
here ip xfrm state
[toor@ip-E-F-G-H ~]# ip xfrm state
src I.J.K.L dst E.F.G.H
proto esp spi 0xe603c44c reqid 16389 mode tunnel
replay-window 32 flag af-unspec
auth-trunc hmac(sha1) 0x53c09c699e580292be07d7159ec6b7bee6767310 96
enc cbc(aes) 0x1e6fdc95ab9e02d1cbb690f89cca6528ff73e65696528b9615a5b09cd310cd2a
encap type espinudp sport 4500 dport 4500 addr 0.0.0.0
anti-replay context: seq 0x0, oseq 0x0, bitmap 0x00000000
src E.F.G.H dst I.J.K.L
proto esp spi 0x386e8731 reqid 16389 mode tunnel
replay-window 32 flag af-unspec
auth-trunc hmac(sha1) 0xec8ed07c653732ffba1d9b63929ac6211005dbee 96
enc cbc(aes) 0xe046bd204feb33e44b24bf1c1476a7425c8ef3318ab705feb2761ae471823d5a
encap type espinudp sport 4500 dport 4500 addr 0.0.0.0
anti-replay context: seq 0x0, oseq 0x3, bitmap 0x00000000
src I.J.K.L dst E.F.G.H
proto esp spi 0x17676d9a reqid 16393 mode tunnel
replay-window 32 flag af-unspec
auth-trunc hmac(sha1) 0x79ab780421c9e079c122006499f9d034ba4ef961 96
enc cbc(aes) 0xde3c9b22338e09c885d9dae4bc93c99fc650a41aa3a7cc8a6ac07affe72f5bd3
encap type espinudp sport 4500 dport 4500 addr 0.0.0.0
anti-replay context: seq 0x0, oseq 0x0, bitmap 0x00000000
src E.F.G.H dst I.J.K.L
proto esp spi 0x3cfbd571 reqid 16393 mode tunnel
replay-window 32 flag af-unspec
auth-trunc hmac(sha1) 0x4163513e0f66ac3656dd18569be72b51b60a3b9e 96
enc cbc(aes) 0xf4c5b790d4d509daef59c7ab50b6620230113c7d21eeaaff48e5a589733636ea
encap type espinudp sport 4500 dport 4500 addr 0.0.0.0
anti-replay context: seq 0x0, oseq 0x5, bitmap 0x00000000
here is ip xfrm policy
[toor@ip-E-F-G-H ~]# ip xfrm policy
src E.F.0.0/20 dst M.N.O.P/32
dir out priority 2464 ptype main
tmpl src E.F.G.H dst I.J.K.L
proto esp reqid 16389 mode tunnel
src M.N.O.P/32 dst E.F.0.0/20
dir fwd priority 2464 ptype main
tmpl src I.J.K.L dst E.F.G.H
proto esp reqid 16389 mode tunnel
--- omitted output -- but u get the gist --
sekayasin1 Answer
- You can check AWS VPC Flow Logs to verify if there is any blocked inbound traffic. But the SG and NACL listings you posted seem to be open enough for this to work.
- When you try to ping the other end from your side do you see the outbound ESP packets? You can verify either through the Flow Logs or use
tcpdumpon your EC2 instance. - Does the Cisco see your ESP packets coming in?
- Can the other side initiate connection to your end?
You will probably need to work with the administrator of the Cisco device to track down what really happens.
answered on Server Fault Oct 30, 2018 by 
MLu
MLuUser contributions licensed under CC BY-SA 3.0