Stop Iptables logging RST packets

[cryptotheslow]

I have an Ubuntu 12.04 server that I'm currently using as a gateway / router for a few LAN devices (WAN is a mobile BB dongle that presents itself as an ethernet adapter - eth1 in the below).

eth0 - 192.168.1.67 (LAN)
eth1 - 192.168.9.100 (WAN BB dongle)

Some websites seem to send a RST flagged packet after a while of inactivity. Not a problem in itself but I suspect because either the client machine or the server has already dropped the connection it hits the end of my iptables input chain and gets logged. I'd like to stop this logging of RST packets as it is forever waking the hard drive from its sleep state.

I'm no iptables expert by a very long mile, have tried to figure it out using man iptables but I just seem to keep making amendments to my saved iptables config that prevent it from loading at all. So any help or advice would be gratefully received.

Example of offending log entry:

May  6 02:48:25 ubuserver kernel: [  666.963777] NETFILT End of IN chain: IN=eth1 OUT= MAC=58:2c:80:13:92:63:58:2c:80:13:92:08:08:00 SRC=173.194.41.175 DST=192.168.9.100 LEN=40 TOS=0x00 PREC=0x00 TTL=251 ID=45620 PROTO=TCP SPT=443 DPT=58346 WINDOW=0 RES=0x00 RST URGP=0

^^ That's google sending a RST after doing a search.

iptables -S:

crypto@ubuserver:~$ sudo iptables -S
-P INPUT DROP
-P FORWARD DROP
-P OUTPUT DROP
-A INPUT -i lo -j ACCEPT
-A INPUT -p tcp -m tcp ! --tcp-flags FIN,SYN,RST,ACK SYN -m conntrack --ctstate NEW -j LOG --log-prefix "NETFILT I New not syn: " --log-level 7
-A INPUT -p tcp -m tcp ! --tcp-flags FIN,SYN,RST,ACK SYN -m conntrack --ctstate NEW -j DROP
-A INPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A INPUT -s 192.168.1.68/32 -i eth0 -j ACCEPT
-A INPUT -s 192.168.1.67/32 -i eth0 -j ACCEPT
-A INPUT -s 192.168.1.66/32 -i eth0 -j ACCEPT
-A INPUT -s 192.168.1.65/32 -i eth0 -j ACCEPT
-A INPUT -s 192.168.1.64/32 -i eth0 -j ACCEPT
-A INPUT -j LOG --log-prefix "NETFILT End of IN chain: " --log-level 7
-A INPUT -j DROP
-A FORWARD -p tcp -m tcp ! --tcp-flags FIN,SYN,RST,ACK SYN -m conntrack --ctstate NEW -j LOG --log-prefix "NETFILT F New not syn: " --log-level 7
-A FORWARD -p tcp -m tcp ! --tcp-flags FIN,SYN,RST,ACK SYN -m conntrack --ctstate NEW -j DROP
-A FORWARD -s 192.168.1.68/32 -i eth0 -o eth1 -m conntrack --ctstate NEW -j ACCEPT
-A FORWARD -s 192.168.1.66/32 -i eth0 -o eth1 -m conntrack --ctstate NEW -j ACCEPT
-A FORWARD -s 192.168.1.65/32 -i eth0 -o eth1 -m conntrack --ctstate NEW -j ACCEPT
-A FORWARD -s 192.168.1.64/32 -i eth0 -o eth1 -m conntrack --ctstate NEW -j ACCEPT
-A FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -j LOG --log-prefix "NETFILT End of FWD chain: " --log-level 7
-A FORWARD -j DROP
-A OUTPUT -j ACCEPT

/etc/iptables.sav that gets loaded at boot time:

crypto@ubuserver:~$ cat /etc/iptables.sav
# Generated by iptables-save v1.4.12 on Sat Dec  7 16:16:03 2013
*security
:INPUT ACCEPT [23135:1917052]
:FORWARD ACCEPT [94101:75161757]
:OUTPUT ACCEPT [34444:34170997]
COMMIT
# Completed on Sat Dec  7 16:16:03 2013
# Generated by iptables-save v1.4.12 on Sat Dec  7 16:16:03 2013
*raw
:PREROUTING ACCEPT [117705:77230082]
:OUTPUT ACCEPT [34444:34170997]
COMMIT
# Completed on Sat Dec  7 16:16:03 2013
# Generated by iptables-save v1.4.12 on Sat Dec  7 16:16:03 2013
*mangle
:PREROUTING ACCEPT [117706:77230134]
:INPUT ACCEPT [23162:1923868]
:FORWARD ACCEPT [94159:75165241]
:OUTPUT ACCEPT [34444:34170997]
:POSTROUTING ACCEPT [128546:109332806]
COMMIT
# Completed on Sat Dec  7 16:16:03 2013
# Generated by iptables-save v1.4.12 on Sat Dec  7 16:16:03 2013
*filter
:INPUT DROP [0:0]
:FORWARD DROP [0:0]
:OUTPUT DROP [0:0]
-A INPUT -i lo -j ACCEPT
-A INPUT -p tcp -m tcp ! --tcp-flags FIN,SYN,RST,ACK SYN -m conntrack --ctstate NEW -j LOG --log-prefix "NETFILT I New not syn: " --log-level 7
-A INPUT -p tcp -m tcp ! --tcp-flags FIN,SYN,RST,ACK SYN -m conntrack --ctstate NEW -j DROP
-A INPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A INPUT -s 192.168.1.68/32 -i eth0 -j ACCEPT
-A INPUT -s 192.168.1.67/32 -i eth0 -j ACCEPT
-A INPUT -s 192.168.1.66/32 -i eth0 -j ACCEPT
-A INPUT -s 192.168.1.65/32 -i eth0 -j ACCEPT
-A INPUT -s 192.168.1.64/32 -i eth0 -j ACCEPT
-A INPUT -j LOG --log-prefix "NETFILT End of IN chain: " --log-level 7
-A INPUT -j DROP
-A FORWARD -p tcp -m tcp ! --tcp-flags FIN,SYN,RST,ACK SYN -m conntrack --ctstate NEW -j LOG --log-prefix "NETFILT F New not syn: " --log-level 7
-A FORWARD -p tcp -m tcp ! --tcp-flags FIN,SYN,RST,ACK SYN -m conntrack --ctstate NEW -j DROP
-A FORWARD -s 192.168.1.68/32 -i eth0 -o eth1 -m conntrack --ctstate NEW -j ACCEPT
-A FORWARD -s 192.168.1.66/32 -i eth0 -o eth1 -m conntrack --ctstate NEW -j ACCEPT
-A FORWARD -s 192.168.1.65/32 -i eth0 -o eth1 -m conntrack --ctstate NEW -j ACCEPT
-A FORWARD -s 192.168.1.64/32 -i eth0 -o eth1 -m conntrack --ctstate NEW -j ACCEPT
-A FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -j LOG --log-prefix "NETFILT End of FWD chain: " --log-level 7
-A FORWARD -j DROP
-A OUTPUT -j ACCEPT
COMMIT
# Completed on Sat Dec  7 16:16:03 2013
# Generated by iptables-save v1.4.12 on Sat Dec  7 16:16:03 2013
*nat
:PREROUTING ACCEPT [1441:269589]
:INPUT ACCEPT [499:89156]
:OUTPUT ACCEPT [1213:88181]
:POSTROUTING ACCEPT [67:5034]
-A POSTROUTING -o eth1 -j MASQUERADE
COMMIT
# Completed on Sat Dec  7 16:16:03 2013

Any thoughts how to stop the seemingly belated RST packets from getting logged?

Thanks in advance

[cryptotheslow]

Hmm, on tailing /var/log/syslog on the server I notice that it's also logging RST packets from LAN clients on the FORWARD chain too, in this example a few seconds before coming to the end of viewing a YouTube clip (I presume the client had buffered the video and was just trying to close the outbound connections):

May  6 03:15:10 ubuserver kernel: [ 2272.434393] NETFILT End of FWD chain: IN=eth0 OUT=eth1 MAC=70:71:bc:be:44:06:54:04:a6:20:11:06:08:00 SRC=192.168.1.68 DST=173.194.41.142 LEN=40 TOS=0x00 PREC=0x00 TTL=63 ID=2356 DF PROTO=TCP SPT=60464 DPT=80 WINDOW=0 RES=0x00 RST URGP=0 
May  6 03:15:16 ubuserver kernel: [ 2278.404498] NETFILT End of FWD chain: IN=eth0 OUT=eth1 MAC=70:71:bc:be:44:06:54:04:a6:20:11:06:08:00 SRC=192.168.1.68 DST=173.194.41.142 LEN=40 TOS=0x00 PREC=0x00 TTL=63 ID=2357 DF PROTO=TCP SPT=60464 DPT=80 WINDOW=0 RES=0x00 RST URGP=0 
May  6 03:15:17 ubuserver kernel: [ 2279.534521] NETFILT End of FWD chain: IN=eth0 OUT=eth1 MAC=70:71:bc:be:44:06:54:04:a6:20:11:06:08:00 SRC=192.168.1.68 DST=173.194.41.142 LEN=40 TOS=0x00 PREC=0x00 TTL=63 ID=2358 DF PROTO=TCP SPT=60464 DPT=80 WINDOW=0 RES=0x00 RST URGP=0 
May  6 03:15:35 ubuserver kernel: [ 2297.104634] NETFILT End of FWD chain: IN=eth0 OUT=eth1 MAC=70:71:bc:be:44:06:54:04:a6:20:11:06:08:00 SRC=192.168.1.68 DST=173.194.41.142 LEN=40 TOS=0x00 PREC=0x00 TTL=63 ID=2359 DF PROTO=TCP SPT=60464 DPT=80 WINDOW=0 RES=0x00 RST URGP=0 
May  6 03:15:36 ubuserver kernel: [ 2298.224619] NETFILT End of FWD chain: IN=eth0 OUT=eth1 MAC=70:71:bc:be:44:06:54:04:a6:20:11:06:08:00 SRC=192.168.1.68 DST=173.194.41.142 LEN=40 TOS=0x00 PREC=0x00 TTL=63 ID=2360 DF PROTO=TCP SPT=60464 DPT=80 WINDOW=0 RES=0x00 RST URGP=0 
May  6 03:15:38 ubuserver kernel: [ 2300.484671] NETFILT End of FWD chain: IN=eth0 OUT=eth1 MAC=70:71:bc:be:44:06:54:04:a6:20:11:06:08:00 SRC=192.168.1.68 DST=173.194.41.142 LEN=40 TOS=0x00 PREC=0x00 TTL=63 ID=2361 DF PROTO=TCP SPT=60464 DPT=80 WINDOW=0 RES=0x00 RST URGP=0

[Doug S]

For TCP connections, Linux tends to use a "half-duplex" close sequence where either side of the session can initiate connection termination via a single 2 way FIN-ACK handshake (which puts the connection into the CLOSE_WAIT state), instead of a full 4 way FIN-ACK handshake. Thus, and particularly with routers and firewalls involved, it is pretty normal to end up with one side thinking the connection has been closed and the other side thinking it is still open.

While I didn't actually test it (but can if it doesn't work), here is what I would suggest:

-A INPUT -s 192.168.1.64/32 -i eth0 -j ACCEPT
-A INPUT -p tcp -m tcp --tcp-flags RST RST -j DROP
-A INPUT -j LOG --log-prefix "NETFILT End of IN chain: " --log-level 7
-A INPUT -j DROP

And similarly for your FORWARD chain.

[cryptotheslow]

Many thanks Doug (for the solution and explanation) - that seems to have cured the logging of RST packets.

iptables -S now loooks like this:

crypto@ubuserver:~$ sudo iptables -S
-P INPUT DROP
-P FORWARD DROP
-P OUTPUT DROP
-A INPUT -i lo -j ACCEPT
-A INPUT -p tcp -m tcp ! --tcp-flags FIN,SYN,RST,ACK SYN -m conntrack --ctstate NEW -j LOG --log-prefix "NETFILT I New not syn: " --log-level 7
-A INPUT -p tcp -m tcp ! --tcp-flags FIN,SYN,RST,ACK SYN -m conntrack --ctstate NEW -j DROP
-A INPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A INPUT -s 192.168.1.68/32 -i eth0 -j ACCEPT
-A INPUT -s 192.168.1.67/32 -i eth0 -j ACCEPT
-A INPUT -s 192.168.1.66/32 -i eth0 -j ACCEPT
-A INPUT -s 192.168.1.65/32 -i eth0 -j ACCEPT
-A INPUT -s 192.168.1.64/32 -i eth0 -j ACCEPT
-A INPUT -p tcp -m tcp --tcp-flags RST RST -j DROP
-A INPUT -j LOG --log-prefix "NETFILT End of IN chain: " --log-level 7
-A INPUT -j DROP
-A FORWARD -p tcp -m tcp ! --tcp-flags FIN,SYN,RST,ACK SYN -m conntrack --ctstate NEW -j LOG --log-prefix "NETFILT F New not syn: " --log-level 7
-A FORWARD -p tcp -m tcp ! --tcp-flags FIN,SYN,RST,ACK SYN -m conntrack --ctstate NEW -j DROP
-A FORWARD -s 192.168.1.68/32 -i eth0 -o eth1 -m conntrack --ctstate NEW -j ACCEPT
-A FORWARD -s 192.168.1.66/32 -i eth0 -o eth1 -m conntrack --ctstate NEW -j ACCEPT
-A FORWARD -s 192.168.1.65/32 -i eth0 -o eth1 -m conntrack --ctstate NEW -j ACCEPT
-A FORWARD -s 192.168.1.64/32 -i eth0 -o eth1 -m conntrack --ctstate NEW -j ACCEPT
-A FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -p tcp -m tcp --tcp-flags RST RST -j DROP
-A FORWARD -j LOG --log-prefix "NETFILT End of FWD chain: " --log-level 7
-A FORWARD -j DROP
-A OUTPUT -j ACCEPT

As always spending too much time log watching throws up more questions!

Here's a tail of /var/log/syslog on the Ubuntu server after I brought it back up after the change:

May  7 03:33:04 ubuserver named[1095]: command channel listening on 127.0.0.1#953
May  7 03:33:04 ubuserver named[1095]: zone 0.in-addr.arpa/IN: loaded serial 1
May  7 03:33:04 ubuserver named[1095]: zone 127.in-addr.arpa/IN: loaded serial 1
May  7 03:33:04 ubuserver named[1095]: zone 1.168.192.in-addr.arpa/IN: loaded serial 10006
May  7 03:33:04 ubuserver named[1095]: zone 255.in-addr.arpa/IN: loaded serial 1
May  7 03:33:04 ubuserver named[1095]: zone home.lan/IN: loaded serial 10000
May  7 03:33:04 ubuserver named[1095]: zone localhost/IN: loaded serial 2
May  7 03:33:04 ubuserver named[1095]: managed-keys-zone ./IN: loading from master file managed-keys.bind failed: file not found
May  7 03:33:04 ubuserver named[1095]: managed-keys-zone ./IN: loaded serial 0
May  7 03:33:04 ubuserver named[1095]: running
May  7 03:33:05 ubuserver cron[1104]: (CRON) INFO (Running @reboot jobs)
May  7 03:33:05 ubuserver kernel: [   14.803619] r8169 0000:01:00.0: eth0: link up
May  7 03:33:05 ubuserver kernel: [   15.290422] vboxdrv: Found 2 processor cores.
May  7 03:33:05 ubuserver kernel: [   15.290780] vboxdrv: fAsync=0 offMin=0x2a3 offMax=0x21db
May  7 03:33:05 ubuserver kernel: [   15.290956] vboxdrv: TSC mode is 'synchronous', kernel timer mode is 'normal'.
May  7 03:33:05 ubuserver kernel: [   15.290965] vboxdrv: Successfully loaded version 4.1.32 (interface 0x00190000).
May  7 03:33:05 ubuserver kernel: [   15.515795] vboxpci: IOMMU not found (not registered)
May  7 03:33:06 ubuserver kernel: [   15.982737] ip_tables: (C) 2000-2006 Netfilter Core Team
May  7 03:33:06 ubuserver kernel: [   16.010404] nf_conntrack version 0.5.0 (16384 buckets, 65536 max)
May  7 03:33:06 ubuserver kernel: [   16.219010] NETFILT End of IN chain: IN=eth1 OUT= MAC=58:2c:80:13:92:63:58:2c:80:13:92:08:08:00 SRC=192.36.148.17 DST=192.168.9.100 LEN=768 TOS=0x00 PREC=0x00 TTL=50 ID=42064 PROTO=UDP SPT=53 DPT=26035 LEN=748 
May  7 03:33:06 ubuserver kernel: [   16.409097] NETFILT End of IN chain: IN=eth1 OUT= MAC=58:2c:80:13:92:63:58:2c:80:13:92:08:08:00 SRC=192.36.148.17 DST=192.168.9.100 LEN=913 TOS=0x00 PREC=0x00 TTL=50 ID=42065 PROTO=UDP SPT=53 DPT=9489 LEN=893 
May  7 03:33:06 ubuserver kernel: [   16.428690] NETFILT End of IN chain: IN=eth1 OUT= MAC=58:2c:80:13:92:63:58:2c:80:13:92:08:08:00 SRC=192.36.148.17 DST=192.168.9.100 LEN=723 TOS=0x00 PREC=0x00 TTL=50 ID=42066 PROTO=UDP SPT=53 DPT=38010 LEN=703 
May  7 03:33:07 ubuserver ntpd_intres[931]: DNS 0.ubuntu.pool.ntp.org -> 178.23.121.164
May  7 03:33:08 ubuserver ntpd_intres[931]: DNS 1.ubuntu.pool.ntp.org -> 94.154.96.7
.. blah some more ntpd stuff

Question is around those NETFILT entries. I have Bind running on this box as a LAN only cacheing DNS resolver and 192.36.148.17 appears to be Verisign root DNS servers so maybe that is related maybe not as they would appear to be DNS query responses to no longer active connections that hit the end of the INPUT chain - I may be very wrong on that assumption! I guess I'll take another look at it when the server has been down for longer than just a reboot, it may have just been a delayed response from prior to reboot.

Anyway, my server drive is happily sleeping and syslog is no longer getting clogged up with RST messages and I learnt something too - I'm a happy bunny

Thanks again!

[Doug S]

Glad you got it working, and thanks so much for reporting back.

Question is around those NETFILT entries. I have Bind running on this box as a LAN only cacheing DNS resolver and 192.36.148.17 appears to be Verisign root DNS servers so maybe that is related maybe not as they would appear to be DNS query responses to no longer active connections that hit the end of the INPUT chain - I may be very wrong on that assumption! I guess I'll take another look at it when the server has been down for longer than just a reboot, it may have just been a delayed response from prior to reboot.

I run bind for my internal LAN also. I went back over a couple months of traffic and I only ever see communication with the i.root-server.net ( 192.36.148.17 ) if my side started it (which is the expected result). So yes, I would also guess it was a left over response.

[cryptotheslow]

Thanks again Doug.

Everything will be off for a good few hours while I sleep, so I'll take a peek again after that extended downtime, if similar logs show up after that then I'll have to get all wiresharky and delve deeper as that would be all manner of unexpected!

I shall report back - but for now sudo init 0 (hey don't laugh, init 0 worked for me ~20 years ago and still seems to work I'm sticking with it)