Strange process

[kurogane2]

I'm wondering the the process ".flush" its part of ubuntu because always i boot my server start this process

  647 root      20   0    7120    260    132 S  0.0  0.0   0:00.69 .flush
  648 root      20   0    7120    260    132 S  0.0  0.0   0:00.32 .flush
  649 root      20   0    7120    260    132 S  0.0  0.0   0:00.71 .flush
  650 root      20   0    7120    260    132 S  0.0  0.0   0:00.16 .flush
  651 root      20   0    7120    260    132 S  0.0  0.0   0:00.69 .flush

# lsof -p 647
COMMAND PID USER   FD   TYPE   DEVICE SIZE/OFF     NODE NAME
.flush  647 root  cwd    DIR     0,24     4096 51904513 /
.flush  647 root  rtd    DIR     0,24     4096 51904513 /
.flush  647 root  txt    REG     0,24   550448 51905654 /tmp/.flush
.flush  647 root  mem    REG      8,4          51905654 /tmp/.flush (path dev=0,24)
.flush  647 root    0u   CHR      1,3      0t0 30401832 /dev/null
.flush  647 root    1u   CHR      1,3      0t0 30401832 /dev/null
.flush  647 root    2u   CHR      1,3      0t0 30401832 /dev/null
.flush  647 root    3r  FIFO      0,8      0t0 30403964 pipe
.flush  647 root    4w  FIFO      0,8      0t0 30403964 pipe
.flush  647 root    5u  IPv4 30678977      0t0      UDP *:45939
.flush  647 root    7u  IPv4 30403967      0t0      UDP *:40772

I delete /tmp/.flush i reboot and that file appears again

[Habitual]

less /tmp/.flush

and if that looks funny, use

strings /tmp/.flush

and show us the output.

[kurogane2]

What interesting thing I found.

strings /tmp/.flush

https://www.dropbox.com/s/9j97kyq1z2tzeec/flush.txt


The most important of that file is

/etc/init.d/bluetoothdaemon
#!/bin/sh
/usr/bin/btdaemon
/etc/rc1.d/S90bluetooth
/etc/rc2.d/S90bluetooth
/etc/rc3.d/S90bluetooth
/etc/rc4.d/S90bluetooth
/etc/rc5.d/S90bluetooth
/etc/rc6.d/S90bluetooth
/tmp/.flush
/var/log/.flush

I just remove those files and i not have issue but i'm not sure if i miss files to remove also not sure how they access my server

[Habitual]

/tmp/.flush is suspect and leads me to wonder if you have a forward-facing website hosted on this same box?
Inspect your /tmp directory.

Also, this as root:

sudo stat -c%x /var/log/.flush /usr/.flush /tmp/.flush /tmp/helloworld

and report the output?

Highly suspicious.

I'd install rkhunter from repo and run it using

sudo rkhunter --update && rkhunter -c -sk

and have a look at /var/log/rkhunter.log
for "Warning" messages in that log file using

less /var/log/rkhunter.log

after it runs.

On the other hand, the output at that link could be innocuous, but the http: links in the strings suggest it's not.

You could also nuke them to orbit with

sudo rm /var/log/.flush /usr/.flush /tmp/.flush

Other more knowledgeable folks here may have more to add.

[andor3]

I've found that file too on a clients server, and it's very, very suspicious.

I've found it because my ntpd binaries where disapearing every once in a while. I monitored them with auditctl and found this:

type=SYSCALL msg=audit(29/05/14 12:02:31.120:68) : arch=i386 syscall=unlink success=yes exit=0 a0=0xff3f7fc0 a1=0x9 a2=0x80be5e0 a3=0xff3ffbe0 items=2 ppid=7508 pid=7509 auid=unset uid=root gid=root euid=root suid=root fsuid=root egid=root sgid=root fsgid=root ses=unset tty=(none) comm=.flush exe=/tmp/.flush key=ntpd

WHAT! syscall=unlink by executable .flush ? Who are you?

Let's check proccesses:

~$ ps aux|grep flush
root       198  0.0  0.0      0     0 ?        S<   12:31   0:00 [kdmflush]
root       201  0.0  0.0      0     0 ?        S<   12:31   0:00 [kdmflush]
root      1565  0.0  0.0   7168   136 ?        Ss   12:32   0:00 [flush-242:0]
root      1566  0.0  0.0   7168   136 ?        S    12:32   0:00 [flush-242:0]
root      1567  0.0  0.0   7168   136 ?        S    12:32   0:00 [flush-242:0]
root      1568  0.0  0.0   7168   136 ?        S    12:32   0:00 [flush-242:0]
root      1569  0.0  0.0   7168   136 ?        S    12:32   0:00 [flush-242:0]

~$ ps -e -T|grep flush
  198   198 ?        00:00:00 kdmflush
  201   201 ?        00:00:00 kdmflush
 1565  1565 ?        00:00:00 .flush
 1566  1566 ?        00:00:00 .flush
 1567  1567 ?        00:00:00 .flush
 1568  1568 ?        00:00:00 .flush
 1569  1569 ?        00:00:00 .flush

I uploaded it to VirusTotal and, right now, it shows nothing suspicious:

https://www.virustotal.com/en/file/1...is/1401379742/

File type is funny, because it's static, which might be usual on rootkits, to be able to run everywhere, and also, it's 32bit (80386, not even pentium optimized), while my clients machine is 64.

~$ file /tmp/.flush
/tmp/.flush: ELF 32-bit LSB  executable, Intel 80386, version 1 (SYSV), statically linked, for GNU/Linux 2.2.5, not stripped

Lets compare it to one regular system binary:

~$ file /bin/ls
/bin/ls: ELF 64-bit LSB  executable, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.24, BuildID[sha1]=64d095bc6589dd4bfbf1c6d62ae985385965461b, stripped

I've deleted it and it appears again on reboot. Although I was auditing the file to find who created it, it appears that it's created before auditctl process started, so it doesn't appear.

$ sudo stat -c%x /tmp/.flush 
2014-05-29 13:53:47.723794621 -0400
$ sudo stat -c%y /tmp/.flush 
2014-05-29 13:50:17.388910913 -0400
$ sudo stat -c%w /tmp/.flush 
-

13:50 was, approx, my reboot time.

I'm asking the client for an immediate shutdown and deleteion of the machine, as it was fairly clean and not in production, but I'm asking also for a virtual image I can analize...
Do you have any other ideas to check this machine?

[andor3]

And mine also has weird urls inside...

$ strings /tmp/.flush |grep http
http://103.20.195.254
http://115.23.172.31
http://61.33.28.194/
http://kill.et2046.com
http://sb.et2046.com
http://

And some other ips appart from that:

$ strings /tmp/.flush |awk '{match($0,/[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+/); ip = substr($0,RSTART,RLENGTH); print ip}'|grep -v '^$'
61.33.28.194
115.23.172.47
103.20.195.254
103.20.195.254
115.23.172.31
61.33.28.194

[unspawn]

I'd install rkhunter from repo

Actually due to http://rkhunter.cvs.sourceforge.net/...08&view=markup and http://rkhunter.cvs.sourceforge.net/....1&view=markup (which this is about) you'd have to get it from CVS (http://rkhunter.cvs.sourceforge.net/...nter/?view=tar) until we release again.

# lsof -p 647

Thanks for the 'lsof' output, I completely overlooked the "flush" process! That said, in your SysV start up directory you should find references to "IptabLex" and "IptabLes" and you should be able to find the following files:

/IptabLes /.IptabLex /boot/.IptabLex /boot/.IptabLes /boot/IptabLes /tmp/IptabLes /etc/rc.d/init.d/IptabLex  /etc/rc.d/init.d/IptabLes /etc/rc.d/rc0.d/S55IptabLex /etc/rc.d/rc1.d/S55IptabLex /etc/rc.d/rc2.d/S55IptabLex  /etc/rc.d/rc3.d/S55IptabLex /etc/rc.d/rc4.d/S55IptabLex /etc/rc.d/rc5.d/S55IptabLex /etc/rc.d/rc6.d/S55IptabLex  /var/lib/update-rc.d/IptabLex /delallmykkk /usr/.IptabLes /usr/IptabLes

*Note these files were dropped there by somebody with root privileges so please don't ask how to "fix" this but isolate the machine, investigate and replace it with a known clean and properly managed (access restrictions, updates, auditing and hardening) one. //EDIT: actually that was a bit daft. The 'strings' clearly show different files (same problem BTW):

/getsetup.rar /kill.txt /run.txt /tmp/helloworld /etc/init.d/bluetoothdaemon /usr/bin/btdaemon /etc/rc1.d/S90bluetooth /etc/rc2.d/S90bluetooth /etc/rc3.d/S90bluetooth /etc/rc4.d/S90bluetooth /etc/rc5.d/S90bluetooth /etc/rc6.d/S90bluetooth /tmp/.flush /var/log/.flush /usr/.flush .flush

[bashiergui]

$ strings /tmp/.flush |grep http
http://103.20.195.254
http://115.23.172.31
http://61.33.28.194/
http://kill.et2046.com
http://sb.et2046.com

If you look up all those IPs they come back to Seoul, Korea and Hong Kong.


Your strings output is interesting. TencentTraveler is in the user agent string, which is a Chinese browser. Can't think of a legitimate reason to run an executable out of the /tmp directory. It also looks like it starts the bitTorrent daemon (btdaemon) which is suspicious. Not sure what it's doing with the bluetooth stuff, that would be interesting to look into.


I personally would conclude you have been compromised at this point. I would pull the machine off the wire. If you can keep it running without being connected to the internet then you can do some digging.


Check if you have anything in /var/logs. If you do look for users created and authenticating. Grep the firewall logs for the IP addresses from the strings output. Get a time stamp for those events and then check the logs for other odd stuff happening at those times.


If the logs are gone that's a pretty solid indicator of compromise. If you've got a router in front of the machine then look through its logs for connections to the IPs from your strings output.


You could list the recent users and their bash histories, see what they've been up to. The attackers could have cleared the histories, which is another good indicator of compromise.


If you can't be down for analysis then reinstall and restore files from backup. Look at the stickies in this forum for some advice on securing the services after reinstallation so you're not compromised again.

[unspawn]

If you can't be down for analysis then reinstall and restore files from backup.

Unless it was investigated and the infection vector known FCOL don't. They'll be exposing the same loophole again.

[bashiergui]

Unless it was investigated and the infection vector known FCOL don't. They'll be exposing the same loophole again.

Hopefully the OP comes back to tell us if there are any logs to investigate.