New Internet accountability software: Net Responsibility

[mssever]

I've been working on an Internet accountability program for Linux. The idea is to log the websites you visit, then periodically e-mail that log so someone else. It also writes a log entry when it is shutdown.

I'm looking for people to test it. Any comments (or code contributions) are welcome.

Still to do:

• Make a configuration GUI (partially completed, but on hold for now)
• Increase the intelligence of the shutdown logger
• Build a web service that will provide a friendlier, configurable way for the accountability partner to view the logs

Installation

NOTE: I keep this post up to date, and it is the canonical place for installation instructions, usage, known bugs, etc. The discussion later in the thread often deals with specific issues and/or versions of Net Responsibility, so a given post might not contain current information.

1. Download Net Responsibility. You can get the latest release from the Sourceforge download page. If you're feeling adventurous, you can use Subversion to check out the latest development version:

   svn checkout https://responsibility.svn.sourceforge.net/svnroot/responsibility/trunk net-responsibility

2. If you downloaded the .deb, simply install it. If you downloaded the source tarball, extract it and follow the directions in the README file.
3. Run Applications > Internet > Configure Net Responsibility.

Running

The core of Net Responsibility is a daemon that runs in the background. It is started automatically when you boot your computer. However, if you want to control it manually for testing purposes, here's how:

• Start

  sudo net-responsibility-daemon

• Stop

  sudo /etc/init.d/net-responsibility-daemon stop

Net Accountability sends daily reports via e-mail. Be sure to set the proper e-mail addresses in the config file (/etc/net-responsibility.conf). The easiest way to accomplish this is by running Applications > Internet > Configure Net Responsibility. You can generate a report at any time by typing

sudo net-responsibility-report

Known Bugs
Both net-responsibility-daemon and net-responsibility-report print an error message similar to the following to one or more of stderr, /var/log/syslog, or /var/log/daemon.log:

net-responsibility-report[17580]: The following exception (SQLite3::BusyException) is caused by a known bug. If you know how to fix it, do tell! Otherwise, you can safely ignore it (I hope).
net-responsibility-report[17580]: /usr/lib/ruby/1.8/sqlite3/errors.rb:94:in 'check': Unable to close due to unfinalised statements (SQLite3::BusyException)
net-responsibility-report[17580]: => from /usr/lib/ruby/1.8/sqlite3/database.rb:153:in 'close'
net-responsibility-report[17580]: => from /usr/local/bin/net-responsibility-report:100:in 'initialize'
net-responsibility-report[17580]: => from /usr/local/bin/net-responsibility-report:116:in 'new'
net-responsibility-report[17580]: => from /usr/local/bin/net-responsibility-report:116

(The process name and ID vary, of course.) I haven't been able to solve this error yet, but I don't think that it's serious. Previously, I suppressed it, but I'm not comfortable with suppressing error messages.

[mhancoc7]

Originally Posted by mssever I've been working on an Internet accountability program for Linux. The idea is to log the websites you visit, then periodically e-mail that log so someone else. It also writes a log entry when it is shutdown. I'm looking for people to test it. Any comments (or code contributions) are welcome. Still to do: It doesn't actually send mail yet because I'm not sure yet what the best method to accomplish that is (suggestions are welcome) Make a configuration GUI Make a deb Make the reports more selective, perhaps by using a URL filter; also increase the intelligence of the shutdown logger Installation Install the dependencies: sudo aptitude install ruby dsniff libdaemonize-ruby Download and extract the attached file. Open a terminal, cd to the directory just created, and type sudo ./install.sh Running The core of Net Responsibility is a daemon that runs in the background. It is started automatically when you boot your computer. However, if you want to control it manually for testing purposes, here's how: Start sudo /etc/init.d/net-responsibility-daemon start Stop sudo /etc/init.d/net-responsibility-daemon stop If you want to monitor the log in real time: sudo less /var/log/net-responsibility.log Then press <Shift>F. Net Accountability sends weekly reports. Because it doesn't do real mail yet, the report will be mailed to your machine's root account. You can generate a report at any time by typing sudo net-responsibility-report

This looks very promising. I will be watching this closely.

Thanks, Jereme

[xilix]

OK, now I have tested it. The installation worked and I can check the log file.

All visited websites and their parts are listed (tried Firefox and Opera). This is a lot of stuff and would need very time-consuming reading of the log-file.
Perhaps the next step is to bundle all entries coming from one domain?

[mssever]

Originally Posted by xilix All visited websites and their parts are listed (tried Firefox and Opera).

Yes, this should get most URLs regardless of the browser used (it even catches my weather applet). It doesn't get some Gmail stuff. My theory is that Gmail must use a nonstandard port number for some of its behind-the-scenes work.

This is a lot of stuff and would need very time-consuming reading of the log-file. Perhaps the next step is to bundle all entries coming from one domain?

Did you use the report program?

sudo net-responsibility-report | less

It does that already. Also, I've added in my not-yet-released development version the ability to exclude certain domain names from the reports.

[jonathonblake]

Originally Posted by xilix All visited websites and their parts are listed (tried Firefox and Opera).

Quick questions:

* Does it track torrent connections?
* Does it track IRC connections;
* Does it track usenet connections;

If it doesn't, adding that capability might be useful.

xan

jonathon

[mssever]

Originally Posted by jonathonblake * Does it track torrent connections?

If the .torrent file is downloaded by HTTP, which is likely, then it logs that. But I don't think that it logs the actual torrent.

Does it track IRC connections;

No.

Does it track usenet connections;

No.

It also doesn't seem to log secure websites, which is, in my opinion, the biggest hole.

Those features would definitely be nice to have, but I don't know right now how to get at that data. If someone can contribute that, it would be great. Otherwise, I'll add those features as I have time to do research. Right now, I'm trying to get all the basic features working.

[xilix]

No, I didn't use it. Looks very good. Perhaps later one can make a "tree-view" (like Nautilus-Folder-Tree) in the GUI. Then the accountability partner can choose on which domains he has to look closer...

If I have understood right, the ability to exclude certain domain names is a preference that the person who reads the reports can set. This would mean, that "secure" domains are not viewed in the report. That's a great feature, too.

Do you have already an idea how to transfer the reports to the accountability partner? Do you think it is possible via E-Mail?

[xilix]

I think I know how to start the mailing process periodically. I would suggest using anacron.
It's like cron, but it doesn't assume that the machine is running continuously. It not a real deamon, but it works like one. If your computer is turned off during the intended time to send the mail, this is performed at the next session.
By default anacron can only start root-commands (which is only needed in this case, I think), but I also figured out how to start user-commands with it...

[clmorg01]

I'll download and test it as well. I'm looking forward to this.

[raijinsetsu]

These log files can get pretty extensive. Have you thought about compressing them? or are they already compressed?
Also, could you use a backend such as MySQL, pySQL, etc?
If you used a backend to store the information, it could easily be read from anywhere on the local net.