PDA

View Full Version : HOWTO: post to usenet with Ubuntu



Martje_001
February 9th, 2008, 11:33 AM
Hi,
Because it's a bit hard to post (binary files) to Usenet with Linux, I decided to make this tutorial.

Note: All commands, which you have to run in a terminal, are in code tags.

1. First install a few programs

sudo apt-get install par2 newspost rar

2. Place the files you want to upload in a folder.

3. Most files on Usenet are in rar-files. You can create these rar-files with this command:

rar a "name of rar file" -v10m -m0 "/home/example/upload/"*
Let me explain the command:
name of rar file: This is the name of the rar-file. ".rar" will automatic be added.
-v10m: Split the file(s) to a new rar-archive after 10 MB
-m0: Set compression level (0-store...3-default...5-best).
"/home/example/upload/"*: This means; rar everything in the folder '/home/example/upload'

4. Now we can create par files.

par2create -r10 -n7 "name of par file" /home/example/upload/*.rar*
-r10: Percent par-files you want to create
-n7: Number of par-files you want to create
"name of par file": I don't want to explain this ;)
/home/example/upload/*.rar*: means; Create par-files of everything in the folder /home/eample/upload, which ends with .rar

5. Upload your files

newspost -i upload.eweka.nl -u USERNAME -p PASSWORD -f noadress@gmail.com -n alt.binaries.boneless -y -s "SUBJECT" "/home/example/upload/*.part*" "/home/example/upload/*.par*"


Extra notes:
This is it. You can run commands after each one is done (with &&). Then it will be something like:

rar a "name of rar file" -v10m -m0 "/home/example/upload/"* && par2create -r10 -n7 "name of par file" /home/example/upload/*.rar* && newspost -i upload.eweka.nl -u USERNAME -p PASSWORD -f noadress@gmail.com -n alt.binaries.boneless -y -s "SUBJECT" "/home/example/upload/*.part*" "/home/example/upload/*.par*"

marxian
May 6th, 2008, 03:49 PM
I would very much like to use this, but newspost is not in the repositories of Hardy. Can I still use this program?

Martje_001
May 6th, 2008, 06:13 PM
Of course :)

i386:

wget http://nl.archive.ubuntu.com/ubuntu/pool/universe/n/newspost/newspost_2.1.1-4_i386.deb
sudo dpkg -i newspost_2.1.1-4_i386.deb

amd64:

wget http://nl.archive.ubuntu.com/ubuntu/pool/universe/n/newspost/newspost_2.1.1-4_amd64.deb
sudo dpkg -i newspost_2.1.1-4_amd64.deb

rabside
May 23rd, 2008, 06:40 PM
greate guide!
Only a question: why do you split file in rar of 10MB? isn't usefull bigger blocks? maybe 40MB?

Martje_001
May 23rd, 2008, 06:48 PM
Thank you! And, yes, it is, but this is only an example.

rabside
May 25th, 2008, 01:36 PM
Hi guys,
this is a little script to automate the upload:



#!/bin/bash

#---------- config start----------
username="username"
password="password"
server="newsserver"
temppath="path of temp dir used to post material es: /media/dati/"
newsgroup="newsgroup, es: alt.binaries.multimedia.divx.italian"
poster="poster name, es: poster"
email="email address, es: poster@gmail.com"
#---------- config end----------


error="false"
filename="$(basename $1)"
subject="${filename%.*}"



function checkLastCommand {
if [ "$1" != "0" ]; then
error="true"
fi
}

#rar split of file
rar a "$temppath$filename" -v40m -m0 $1

#checkresult
checkLastCommand "$?"

#create par
if [ "$error" != "true" ]; then
par2create -r10 -n7 "$temppath$filename" "$temppath"*.rar*
fi

#checkresult
checkLastCommand "$?"

#post
if [ "$error" != "true" ]; then
newspost -i "$server" -u "$username" -p "$password" -f "$email" -n "$newsgroup" -y -s "$subject" "$temppath"*
fi

#checkresult
checkLastCommand "$?"

#delete tmpfile
if [ "$error" != "true" ]; then
rm "$temppath"*
fi


I hope this help ;)

Martje_001
May 25th, 2008, 01:40 PM
I'm working on this myself, see this:
http://www.xs4all.nl/~mgj1/bnewspost.htm

hq197
August 29th, 2008, 01:00 PM
Martje!! Thank you for you guide. It's very helpful. Rabside, how do i use your script? I get a
Cannot create path of temp dir used to post material es: /media/dati/.rar
No such file or directorybasename: missing operand
Try `basename --help' for more information. in konsole when i run the script. Basename --help didn't help me at all. I have no idea how to adjust the script (if it indeed needs that).

hq197
August 29th, 2008, 01:02 PM
Thanks for the script but can you help me out of my error message http://ubuntuforums.org/showpost.php?p=5686659&postcount=8

rabside
August 29th, 2008, 01:08 PM
You have to change the value of variable temppath. Choose what you want.

Crafty Kisses
August 29th, 2008, 10:08 PM
Nice tutorial, very nicely typed and very smooth. :)

Stan*
October 6th, 2008, 07:58 AM
I can't upload all the rar-files in a directory. When I type "~/Desktop/up/*.rar" I get an error: "WARNING: No such file or directory". What can I do to solve this?

Thanks.

hq197
October 8th, 2008, 06:44 AM
I can't upload all the rar-files in a directory. When I type "~/Desktop/up/*.rar" I get an error: "WARNING: No such file or directory". What can I do to solve this?

Thanks.

Hi Stan,
Have u checked what directory your terminal is working from? To the left of your cursor and inside square brackets eg. [userisme@localhost Download] in this case, i am inside /home/userisme/Download/. If i passed a location like rar a example /Download/*.avi , i would get the same error message u did (unless there is a location like /home/userisme/Download/Download .

In the end, the best is to operate from inside the actual data directory, so your command starts with the command, rather than a location.

Stan*
October 8th, 2008, 06:03 PM
Thanks for your reply. I found it out already, I had to remove the quotes. Thanks anyway :)

DCJuggler
November 14th, 2008, 11:30 PM
Hi,

Thanks for this, but there is just one problem, is there anyway to open more than 1 thread (aka upload more than 1 file at a time) as i cant max my server out with just a single thread which means it takes it longer to upload...

colemar
September 21st, 2009, 06:11 PM
I bet you can figure out how to install: ksh, rar, par2, newspost.


#!/usr/bin/ksh

# tousenet

#---------- begin config ----------
SERVER=my.news.server.com
PORT=119
USERNAME=colemar
PASSWORD=password
NEWSGROUP=alt.binaries.test
EMAIL=colemarc@gmail.moc
NICKNAME=colemar
WORKBASE=/tmp/tousenet
RARVOLSIZE=50m
PARPERCENT=5
NFOMASK=*.nfo
SAMPLEMASK=*.sample.*
LISTFILESUFFIX=list.nfo
#---------- end config ----------

function errorexit {
print -u2 "ERROR: $1"
exit 1
}

function usage {
print "
tousenet <what> [<where>] [<subject>]

<what> a directory containing the binary files you want posted to usenet.
<where> an optional comma separated list of newsgroup names (maximum 5).
Default is alt.binaries.test.
<subject> an optional string to be used to build the articles subject.
Default is directory name of <what>.

Files contained in <what> (and subdirs) are assembled in a set of rar/par
files located in a work dir under the configured temp directory.
The common <prefix> for rar/par filenames is the directory name of <what>.

This program attempts to recognize whether a given <what> has already been
assembled for posting. To detect this situation, the work dir is named after
a fingerprint made from the first 8 characters of the md5 digest of <what>
contents and a helper listfile is created in the work directory.
"[fingerprint]" is also inserted at the beginning of articles subject; it
can be used later to search and track the post via binsearch.info or
other services.

If *.nfo and/or *.sample.* files exist in <what> directory, they will be
posted as such (besides in rar archive).
To let binsearch.info report these files together with rar/par files, you
should name them with the common prefix: <prefix>.nfo <prefix>.sample.mkv
The first *.nfo file is automatically renamed to <prefix>.nfo in the
work dir and posted as file number 0.

Example: tousenet ~/media/W.La.Foca alt.binaries.movies.divx 'W la foca! (1982) IMDB:tt0084880 avi divx 720x576 italiano-mp3'
"
exit
}

WHAT=$1
[[ $WHAT = "" ]] && usage
WHERE=${2:-$NEWSGROUP}

[[ $WHAT = /* ]] || WHAT=$PWD/$WHAT
[[ -d $WHAT ]] || errorexit "$WHAT should be a directory"
[[ -r $WHAT ]] || errorexit "$WHAT is not readable"
print "Source is $WHAT"

PREFIX=$(basename $WHAT)
[[ $PREFIX = *\ * ]] && errorexit "I don't like blanks in common prefix [$PREFIX]"
LISTFILE=$PREFIX.$LISTFILESUFFIX

SUBJECT="${3:-$PREFIX}"

cd $WHAT
what=$(find . -type f -printf '%P %s %T+\n')
typeset -L8 FINGERPRINT
print -- "$what" | md5sum | read FINGERPRINT junk
print "Fingerprint is $FINGERPRINT"

WORKDIR=$WORKBASE/$FINGERPRINT
print "Work dir is $WORKDIR"
mkdir -p $WORKDIR
cd $WORKDIR

if [[ $(print *) = "*" ]]; then
rar a $PREFIX -r -v$RARVOLSIZE -m0 $WHAT || errorexit "RAR archiver failed"
eval cp $WHAT/$NFOMASK $WHAT/$SAMPLEMASK .
par2 c -r$PARPERCENT $PREFIX * || errorexit "PAR file creation failed"
print -- "$what" > $LISTFILE
else
print "Work dir is not empty"
if [[ -f $LISTFILE ]]; then
print "Post already assembled in the work dir"
read answer?"Do you want to repost it? [Y/n] "
[[ $answer = @(n|N) ]] && errorexit "Aborted"
else
errorexit "The work dir contains something unexpected"
fi
fi

NFO=""
ls *.nfo | fgrep -v $LISTFILE | read infofile
if [[ $infofile > "" ]]; then
[[ -f $PREFIX.nfo ]] || mv $infofile $PREFIX.nfo
NFO="-e $PREFIX.nfo"
fi

newspost -T 3 -y -i $SERVER -z $PORT -u $USERNAME -p $PASSWORD -f $EMAIL -F $NICKNAME -n $WHERE -s "[$FINGERPRINT] $SUBJECT" $NFO *

print "Done. The assembled post is still in work dir $WORKDIR"

Hated On Mostly
January 14th, 2010, 12:28 AM
Hi,

Thanks for this, but there is just one problem, is there anyway to open more than 1 thread (aka upload more than 1 file at a time) as i cant max my server out with just a single thread which means it takes it longer to upload...


Finally a solution for people who use Linux.

http://jbinup.com/en/

hikaricore
February 5th, 2010, 04:38 AM
Finally a solution for people who use Linux.

http://jbinup.com/en/

Thanks for the link. Exactly what I as lookin for. :)

AintJoe
July 11th, 2010, 11:06 AM
I can't seem to find any ubuntu/debian package for newspost

colemar
July 24th, 2010, 10:12 PM
I can't seem to find any ubuntu/debian package for newspost

I had to compile it from the source tarball found at:
http://newspost.unixcab.org

I'm afraid there is probably no ready made package around.

rascalli
August 13th, 2010, 12:13 PM
This looks good.

Is there also a sort of webui for this ?

So that you click on a folder & then select : Upload
And then you can select all options ?


Or otherwise another script you can place on a server, so people can upload without SSH Access

tx

sanderj
August 23rd, 2010, 10:25 PM
There's no newspost package for Lucid. Here's how to compile yourself from source:

So get the source from http://newspost.unixcab.org/index.html#Download, and unpack it.
Go into the unpack directory
Try to make it (you will get an error):



sander@quirinius:/tmp/newspost-2.1.1$ make
make main CFLAGS="-O2 -Wall" LIBS=""
make[1]: Entering directory `/tmp/newspost-2.1.1'
cd base ; make CC="gcc" CFLAGS="-O2 -Wall"
make[2]: Entering directory `/tmp/newspost-2.1.1/base'
gcc -O2 -Wall -o test test.c
In file included from newspost.h:40,
from test.c:20:
utils.h:29: error: conflicting types for ‘getline’
/usr/include/stdio.h:651: note: previous declaration of ‘getline’ was here
make[2]: *** [test] Error 1
make[2]: Leaving directory `/tmp/newspost-2.1.1/base'
make[1]: *** [main] Error 2
make[1]: Leaving directory `/tmp/newspost-2.1.1'
make: *** [opt] Error 2
sander@quirinius:/tmp/newspost-2.1.1$


The solution is easy:


find . -type f -exec grep -l "getline" {} \; | xargs sed -i "s/\<getline\>/getline_/g"

Then make again, which will go OK.

AintJoe
August 25th, 2010, 05:13 AM
If anyone is interested, I posted the code here, http://github.com/joehillen/newspost

I'll create a debian package and GUI as soon as I feel like it. ;)

sanderj
August 25th, 2010, 06:09 AM
If anyone is interested, I posted the code here, http://github.com/joehillen/newspost

I'll create a debian package and GUI as soon as I feel like it. ;)

... and include in the the package the script that takes care of the rar, par and posting? ;-)

AintJoe
August 25th, 2010, 02:09 PM
...or at least add it to the gui.

sanderj
September 11th, 2010, 06:31 PM
The script seems to work for me, but my test posts do not appear in the newsgroup alt.binaries.test nor binsearch.info.


EDIT: the post did appear after 10 minutes on the newsserver, and after 20 minutes on binsearch.info.

SO: all OK!




...
Posted 10 files (13.5 MB encoded) in 2 minutes 21 seconds. 98 KB/second
Done. The assembled post is still in work dir /tmp/tousenet/f64bee3c

parapapapapa
October 21st, 2010, 02:56 PM
I have a question about making par2 files, like number 4 in the opening post.

Hi,


4. Now we can create par files.

par2create -r10 -n7 "name of par file" /home/example/upload/*.rar*-r10: Percent par-files you want to create
-n7: Number of par-files you want to create
"name of par file": I don't want to explain this :wink:
/home/example/upload/*.rar*: means; Create par-files of everything in the folder /home/eample/upload, which ends with .rar



I did this, and got:

~$ par2create -r10 filename.par2 filename.part*.rar
You must specify a list of files when creating.Then I did this and got:

~$ par2create -r10 filename*.par2 filename.part*.rar
You must specify a Recovery file.What am I doing wrong? I'm a beginner but I don't want to use the graphical version, I want to learn this.

brashley46
January 15th, 2011, 11:29 PM
Thanks for the link. Exactly what I as lookin for. :)

;) YES. Tested nd it works. THANK YOU!

sanderj
January 18th, 2011, 10:03 AM
Hi guys,
this is a little script to automate the upload:

<snip>

I hope this help ;)

Great, very helpful script. Some remarks:

- I believe the script puts the full path in the upload which cause the full path (like /home/sander/...) to be in the download. I've changed that myself by cd-ing to the working directory and changing the newspost parameter at the end
- I've modified the script to include the PID ($$) in the working directory and in the subject, so that you can recognize the post and run multiple instances at the same time.
- the script can't handle spaces in a directory (and maybe file) name. See below. That's strange, because 'basename' itself can handle it. I don't know the solution; I'm not a bash guru (maybe just some extra quotes or parentheses?)




sander@athlon64:~/newspost2$ ./postenmaar.sh /home/sander/spaties\ in\ deze\ directory\ naam/
basename: extra operand `deze'
Try `basename --help' for more information.

RAR 3.90 beta 2 Copyright (c) 1993-2009 Alexander Roshal 3 Jun 2009
Shareware version Type RAR -? for help

Evaluation copy. Please register.

Cannot open /home/sander/spaties
No such file or directory
Cannot open in
No such file or directory
Cannot open deze
No such file or directory
Cannot open directory
No such file or directory
Creating archive /home/sander/postenmaar-temp/.rar

WARNING: No files
sander@athlon64:~/newspost2$



[/CODE]

HackedServer
February 25th, 2011, 01:23 AM
Unfortunately newspost doesn't have NZB support.

After asked around I found out about newsmangler. It supports multiple connections and nzb creation.

https://github.com/madcowfred/newsmangler

sanderj
February 25th, 2011, 08:23 AM
Unfortunately newspost doesn't have NZB support.

After asked around I found out about newsmangler. It supports multiple connections and nzb creation.

https://github.com/madcowfred/newsmangler

Nice. Does this script generate par2 file? I see par2 is a commandline option, but the TODO says:

docs/TODO:* Add PAR2 generation. Read the par2cmdline source to work out how it decides


... so I wonder about par2 ...

Sharpie1
July 10th, 2011, 07:45 PM
I've been trying to post some test files with newspost but get the following error...

me@my-laptop:~/ys$ newspost -i news.mysip.com -u me@myisp.co.uk -f noadress@gmail.com -n alt.binaries.test.files -y -s TYS_Test /home/me/ys/*.part* /home/me/ys/*.par*

WARNING: duplicate filename /home/me/ys/TYSSubsplus45avi.part1.rar - IGNORING
WARNING: duplicate filename /home/me/ys/TYSSubsplus45avi.part2.rar - IGNORING
WARNING: duplicate filename /home/me/ys/TYSSubsplus45avi.part3.rar - IGNORING
WARNING: duplicate filename /home/me/ys/TYSSubsplus45avi.part4.rar - IGNORING
WARNING: duplicate filename /home/me/ys/TYSSubsplus45avi.part5.rar - IGNORING
Calculating CRCs... done

From: noadress@gmail.com
Newsgroups: alt.binaries.test.files
Subject: TYS_Test

10 Files: 89.3 MB
Yencoding 89.3 MB total and posting to news.myisp.com

Posting in 1 second... Segmentation faultNot much to go on I know, sorry...
I installed the recent deb with

sudo dpkg -i newspost_2.1.1-4_i386.debany ideas ?

cheers
Sharpie1

(I have edited personal info out of the code !)

Sharpie1
July 14th, 2011, 07:29 PM
I finally got it running:
following Martje_001's original post I installed newspost,
made a folder with the file to be posted in it
using that folder as my working directory, I issued the following commands in terminal:

rar a <filename with filetype omitted> -v25m -m0this makes 25MB rar files (with the correct usenet naming convention) of everything in the folder (remove the handles < and >)
then :
In the working directory:

par2create -r10 -n7 <filename with filetype omitted> *.rar*this creates 7 10% par files of the rars in the folder

I then removed the original file from the folder, leaving the nfo file, the rars and the pars

Then, using that folder as my working directory, I issued the following commands:

newspost -d -i news.myisp.com -f myname@myisp.com -F 'My Nickname' (include the single quotes around the Nickname)
This sets the default newsserver, email address and Name for newspost
Then issued the posting command

newspost -n alt.my.group.name -y -s Subject_Line_of_My_Posting *
***this yencodes and posts everything in the working folder so make sure your working directory is correct before posting the contents of your home folder !!!!****

cheers
Sharpie1

Ultra Cronic
July 31st, 2012, 10:53 PM
Hi,
Because it's a bit hard to post (binary files) to Usenet with Linux, I decided to make this tutorial.

Note: All commands, which you have to run in a terminal, are in code tags.

1. First install a few programs

sudo apt-get install par2 newspost rar2. Place the files you want to upload in a folder.

3. Most files on Usenet are in rar-files. You can create these rar-files with this command:

rar a "name of rar file" -v10m -m0 "/home/example/upload/"* Let me explain the command:
name of rar file: This is the name of the rar-file. ".rar" will automatic be added.
-v10m: Split the file(s) to a new rar-archive after 10 MB
-m0: Set compression level (0-store...3-default...5-best).
"/home/example/upload/"*: This means; rar everything in the folder '/home/example/upload'

4. Now we can create par files.

par2create -r10 -n7 "name of par file" /home/example/upload/*.rar*-r10: Percent par-files you want to create
-n7: Number of par-files you want to create
"name of par file": I don't want to explain this ;)
/home/example/upload/*.rar*: means; Create par-files of everything in the folder /home/eample/upload, which ends with .rar

5. Upload your files

newspost -i upload.eweka.nl -u USERNAME -p PASSWORD -f noadress@gmail.com -n alt.binaries.boneless -y -s "SUBJECT" "/home/example/upload/*.part*" "/home/example/upload/*.par*"
Extra notes:
This is it. You can run commands after each one is done (with &&). Then it will be something like:

rar a "name of rar file" -v10m -m0 "/home/example/upload/"* && par2create -r10 -n7 "name of par file" /home/example/upload/*.rar* && newspost -i upload.eweka.nl -u USERNAME -p PASSWORD -f noadress@gmail.com -n alt.binaries.boneless -y -s "SUBJECT" "/home/example/upload/*.part*" "/home/example/upload/*.par*"

Good God!! they need a GUI for this. I have waisted 4 nights sluthing the forums and google and DuckDuckGo searches and seams as though Linux is a total failure with usenet posting/reading file split Yenc management . As much as I HATE windows I miss NewsBin pro, it's hard to believe their is no Ubuntu counterpart not even close, Pan=fail/ knode=FAIL/ major missing features. especially uploading to the usenet. Files need to be split and yEnc coded automatically and nzb's created with nfo files all integrated in the upload process.
I can't believe the fantastic intelligent support for Linux now and they are so out of the loop with the Usenet. and Oh Ya support for SSL port 563 connections a must now with newsreader/writer/posters. Most of us [especially mine] have all these programs in our systems as Xterminal command line programs and craplications, The GUI developement is a disgrace for these.:(
I hate the fact that I have 1 windows box [laptop] on hand for when I want to sleuth the newsgroups. Linux can't touch Newbin pro. Reads writes manages corupt files [PAR] a nd auto eYenc upload and download, preview even tells me if the files are Password protected apon commanding a download as not to waist my time.
Auto handles nzb's [Usenet shortcut to files]. and creates them too. I'd pay a hundred dollars for a Linux version of NewsBin pro right now. NO it doesn't fly in Wine, as is most of the case.](*,):rolleyes:

Ultra Cronic
August 1st, 2012, 02:41 AM
Seems as though the linux community really needs a website possible commercial that does all of the intricate manipulations to a binary file attachment needed in order to post to usenet.
It would have to have a "Profile data base " so you enter your usenet account info over an ssl port maybe even need an IPsec key to secure it and then have a powerful web GUI to handle uploads. Of course people that are on the edge of society or in my case paranoia of privacy data mining [spying is immoral etc..] would not like it, but heck, I don't see Linux's developers as having any intention of fixing this Layman's way [GUI] to post to usenet problem. There used to be a Newsgroup service out there that had a web interface but I forget who it was.:popcorn::guitar:

Ultra Cronic
August 1st, 2012, 03:19 AM
Finally a solution for people who use Linux.

http://jbinup.com/en/

JbinUP has some qwerky demands on JAVA 2.6 from SUN.
Witch in my case may send Openshot video editor and other proggys to the sharts.
This is what happens when one depends on "other" peoples work to make "Their" program work. [As in I-PAD's JAVA crippling] Just a little system upgrade so often causes UN-intended consequences that turn programs into pile of binary bile. A more "Self Contained" attitude has been longing for GUI driven nntp file uploads in linux. Such as Putty, you can run it from a thumb-drive on another box without messing with the other box's OS to get it to work. I use it to create sslh multiplex tunnels to encrypt my internet activity entirely when using a VPN service that supports socks5 tunneling.
Not cheap but worth it. I cringe every time someone writes code that demands SUN JAVA in order for it to work, I wish linux would stick with the GNU/SDK versions of it when they need JAVA.
Sincerely; The Ultra Cronic disgruntled network technician working on his second stroke and third heart attack.
:popcorn::guitar:

Antonio19
April 13th, 2013, 05:55 PM
For those interested I found the package newspost_2.1.1-4_amd64.deb on the website debian-archive at this location :

http://archive.debian.net/etch/amd64/newspost/download
http://ftp.nl.debian.org/debian-archive/debian/pool/main/n/newspost/newspost_2.1.1-4_amd64.deb

It is working perfectly on Ubuntu Precise 12.04 amd64 ... Thanks all for the tutorials and all the good stuff.