How to debug ip xfrm rules

▲
5
▼

I created ip xfrm rules on 2 machine and trying to pass traffic through the ipsec tunnel. The packet comes into the other end, encrypted, and disappears.

I traced them through iptables, and here is the trace.

2015-11-27T14:50:21.442638+04:30 cfae kernel: [70234.667488] TRACE: raw:PREROUTING:policy:2 IN=eth0 OUT= MAC=00:0c:29:b7:97:49:00:0c:29:42:5b:e1:08:00 SRC=172.24.1.178 DST=10.60.60.31 LEN=128 TOS=0x00 PREC=0x00 TTL=61 ID=0 DF PROTO=UDP SPT=4500 DPT=4500 LEN=108 
2015-11-27T14:50:21.442665+04:30 cfae kernel: [70234.667513] TRACE: mangle:PREROUTING:rule:2 IN=eth0 OUT= MAC=00:0c:29:b7:97:49:00:0c:29:42:5b:e1:08:00 SRC=172.24.1.178 DST=10.60.60.31 LEN=128 TOS=0x00 PREC=0x00 TTL=61 ID=0 DF PROTO=UDP SPT=4500 DPT=4500 LEN=108 
2015-11-27T14:50:21.442672+04:30 cfae kernel: [70234.667580] TRACE: mangle:INPUT:policy:1 IN=eth0 OUT= MAC=00:0c:29:b7:97:49:00:0c:29:42:5b:e1:08:00 SRC=172.24.1.178 DST=10.60.60.31 LEN=128 TOS=0x00 PREC=0x00 TTL=61 ID=0 DF PROTO=UDP SPT=4500 DPT=4500 LEN=108 
2015-11-27T14:50:21.442674+04:30 cfae kernel: [70234.667589] TRACE: filter:INPUT:policy:1 IN=eth0 OUT= MAC=00:0c:29:b7:97:49:00:0c:29:42:5b:e1:08:00 SRC=172.24.1.178 DST=10.60.60.31 LEN=128 TOS=0x00 PREC=0x00 TTL=61 ID=0 DF PROTO=UDP SPT=4500 DPT=4500 LEN=108 
2015-11-27T14:50:21.471390+04:30 cfae kernel: [70234.696166] TRACE: raw:PREROUTING:policy:2 IN=eth0 OUT= MAC=00:0c:29:b7:97:49:00:0c:29:42:5b:e1:08:00 SRC=172.24.1.178 DST=10.60.60.31 LEN=128 TOS=0x00 PREC=0x00 TTL=61 ID=0 DF PROTO=UDP SPT=4500 DPT=4500 LEN=108 
2015-11-27T14:50:21.471401+04:30 cfae kernel: [70234.696184] TRACE: mangle:PREROUTING:rule:2 IN=eth0 OUT= MAC=00:0c:29:b7:97:49:00:0c:29:42:5b:e1:08:00 SRC=172.24.1.178 DST=10.60.60.31 LEN=128 TOS=0x00 PREC=0x00 TTL=61 ID=0 DF PROTO=UDP SPT=4500 DPT=4500 LEN=108 
2015-11-27T14:50:21.471408+04:30 cfae kernel: [70234.696225] TRACE: mangle:INPUT:policy:1 IN=eth0 OUT= MAC=00:0c:29:b7:97:49:00:0c:29:42:5b:e1:08:00 SRC=172.24.1.178 DST=10.60.60.31 LEN=128 TOS=0x00 PREC=0x00 TTL=61 ID=0 DF PROTO=UDP SPT=4500 DPT=4500 LEN=108 
2015-11-27T14:50:21.471409+04:30 cfae kernel: [70234.696234] TRACE: filter:INPUT:policy:1 IN=eth0 OUT= MAC=00:0c:29:b7:97:49:00:0c:29:42:5b:e1:08:00 SRC=172.24.1.178 DST=10.60.60.31 LEN=128 TOS=0x00 PREC=0x00 TTL=61 ID=0 DF PROTO=UDP SPT=4500 DPT=4500 LEN=108 
2015-11-27T14:50:21.695777+04:30 cfae kernel: [70234.920757] TRACE: raw:PREROUTING:policy:2 IN=eth0 OUT= MAC=00:0c:29:b7:97:49:00:0c:29:42:5b:e1:08:00 SRC=172.24.1.178 DST=10.60.60.31 LEN=128 TOS=0x00 PREC=0x00 TTL=61 ID=0 DF PROTO=UDP SPT=4500 DPT=4500 LEN=108 
2015-11-27T14:50:21.695801+04:30 cfae kernel: [70234.920804] TRACE: mangle:PREROUTING:rule:2 IN=eth0 OUT= MAC=00:0c:29:b7:97:49:00:0c:29:42:5b:e1:08:00 SRC=172.24.1.178 DST=10.60.60.31 LEN=128 TOS=0x00 PREC=0x00 TTL=61 ID=0 DF PROTO=UDP SPT=4500 DPT=4500 LEN=108 
2015-11-27T14:50:21.695808+04:30 cfae kernel: [70234.920847] TRACE: mangle:INPUT:policy:1 IN=eth0 OUT= MAC=00:0c:29:b7:97:49:00:0c:29:42:5b:e1:08:00 SRC=172.24.1.178 DST=10.60.60.31 LEN=128 TOS=0x00 PREC=0x00 TTL=61 ID=0 DF PROTO=UDP SPT=4500 DPT=4500 LEN=108 
2015-11-27T14:50:21.695810+04:30 cfae kernel: [70234.920875] TRACE: filter:INPUT:policy:1 IN=eth0 OUT= MAC=00:0c:29:b7:97:49:00:0c:29:42:5b:e1:08:00 SRC=172.24.1.178 DST=10.60.60.31 LEN=128 TOS=0x00 PREC=0x00 TTL=61 ID=0 DF PROTO=UDP SPT=4500 DPT=4500 LEN=108

From the traces, I see that packet is going from filter:INPUT back to raw:PREROUTING, which means it must be going through xfrm lookup/decode processing. But the packet comes out of this processing unchanged (same length, same src/dst addresses).

But "ip -s xfrm state" shows 0 falied/processed packets.

Is there any way I can trace these packets through xfrm module? Or is there a way to enable debugging for xfrm?

Here is the configuration.

    On Machine1: 
    # ip xfrm state
    src 10.60.60.31 dst 172.24.1.178
        proto esp spi 0x5fca3acb reqid 212013014 mode tunnel
        replay-window 0 
        auth-trunc hmac(sha512) 0xc2dde59416e78c96b9b0333686b3cf0b09016bb1c1d27215d3dbf7aca471304a1c25536eb2648b3dcd8946047007389d06eb3fb4c4e9379630acae51bd755b07 96
        enc cbc(aes) 0x944023fd181e13d401f80e2166654b549b758aeeb9f382a20ca967631b111ef0
        encap type espinudp sport 4500 dport 4500 addr 10.60.60.31
        sel src 0.0.0.0/0 dst 0.0.0.0/0 
    src 172.24.1.178 dst 10.60.60.31
        proto esp spi 0x5fca3acb reqid 212013014 mode tunnel
        replay-window 0 
        mark 212013014/0xffffffff
        auth-trunc hmac(sha512) 0xc2dde59416e78c96b9b0333686b3cf0b09016bb1c1d27215d3dbf7aca471304a1c25536eb2648b3dcd8946047007389d06eb3fb4c4e9379630acae51bd755b07 96
        enc cbc(aes) 0x944023fd181e13d401f80e2166654b549b758aeeb9f382a20ca967631b111ef0
        encap type espinudp sport 4500 dport 4500 addr 172.24.1.178
        sel src 0.0.0.0/0 dst 0.0.0.0/0 
    # ip xfrm policy
    src 0.0.0.0/0 dst 0.0.0.0/0 
        dir fwd priority 65535 
        tmpl src 10.60.60.31 dst 172.24.1.178
            proto esp spi 0x5fca3acb reqid 212013014 mode tunnel
            level use 
    src 0.0.0.0/0 dst 0.0.0.0/0 
        dir in priority 65535 
        tmpl src 10.60.60.31 dst 172.24.1.178
            proto esp spi 0x5fca3acb reqid 212013014 mode tunnel
            level use 
    src 0.0.0.0/0 dst 0.0.0.0/0 
        dir out priority 65535 
        mark 212013014/0xffffffff
        tmpl src 172.24.1.178 dst 10.60.60.31
            proto esp spi 0x5fca3acb reqid 212013014 mode tunnel
            level use 


    On Machine2:
    # ip xfrm policy
src 0.0.0.0/0 dst 0.0.0.0/0 
    dir fwd priority 65535 
    tmpl src 172.24.1.178 dst 10.60.60.31
        proto esp spi 0x5fca3acb reqid 212012972 mode tunnel
        level use 
src 0.0.0.0/0 dst 0.0.0.0/0 
    dir in priority 65535 
    tmpl src 172.24.1.178 dst 10.60.60.31
        proto esp spi 0x5fca3acb reqid 212012972 mode tunnel
        level use 
src 0.0.0.0/0 dst 0.0.0.0/0 
    dir out priority 65535 
    mark 212012972/0xffffffff
    tmpl src 10.60.60.31 dst 172.24.1.178
        proto esp spi 0x5fca3acb reqid 212012972 mode tunnel
        level use 
# ip xfrm state
src 172.24.1.178 dst 10.60.60.31
    proto esp spi 0x5fca3acb reqid 212012972 mode tunnel
    replay-window 0 
    auth-trunc hmac(sha512) 0xc2dde59416e78c96b9b0333686b3cf0b09016bb1c1d27215d3dbf7aca471304a1c25536eb2648b3dcd8946047007389d06eb3fb4c4e9379630acae51bd755b07 96
    enc cbc(aes) 0x944023fd181e13d401f80e2166654b549b758aeeb9f382a20ca967631b111ef0
    encap type espinudp sport 4500 dport 4500 addr 172.24.1.178
    sel src 0.0.0.0/0 dst 0.0.0.0/0 
src 10.60.60.31 dst 172.24.1.178
    proto esp spi 0x5fca3acb reqid 212012972 mode tunnel
    replay-window 0 
    mark 212012972/0xffffffff
    auth-trunc hmac(sha512) 0xc2dde59416e78c96b9b0333686b3cf0b09016bb1c1d27215d3dbf7aca471304a1c25536eb2648b3dcd8946047007389d06eb3fb4c4e9379630acae51bd755b07 96
    enc cbc(aes) 0x944023fd181e13d401f80e2166654b549b758aeeb9f382a20ca967631b111ef0
    encap type espinudp sport 4500 dport 4500 addr 10.60.60.31
    sel src 0.0.0.0/0 dst 0.0.0.0/0
ipsec
asked on Server Fault Nov 27, 2015 by Sindhura Bandi • edited Nov 27, 2015 by Sindhura Bandi

2 Answers

▲
5
▼

To debug xfrm, you can use command ip xfrm monitor.

It will show changes in database policy. And it will show the processing of each data packets.

answered on Server Fault Dec 15, 2016 by PokerFace • edited Nov 9, 2017 by pevik
▲
0
▼

http://techblog.newsnow.co.uk/2011/11/simple-udp-esp-encapsulation-nat-t-for.html

It appears that the Linux kernel will not decapsulate the packets without some instruction. We need to bind to a socket on the incoming udp port and enable udp encapsulation. Above link has the perl script to do it. After this, my traffic is working without any issues.

answered on Server Fault Nov 28, 2015 by Sindhura Bandi

User contributions licensed under CC BY-SA 3.0