Page 1 of 10 123 ... LastLast
Results 1 to 10 of 95

Thread: BitTorrent optimization and troubleshooting guide

  1. #1
    Join Date
    Aug 2008
    Location
    Brazil
    Beans
    12,497
    Distro
    Ubuntu Studio 12.04 Precise Pangolin

    BitTorrent optimization and troubleshooting guide

    The objective of this thread is to provide a comprehensive list of BitTorrent optimizations and troubleshooting common issues (this is a work in progress).

    Warning: I do not support downloading unauthorized content, so any questions related to copyrighted material will be ignored and I will ask a moderator to delete it. The BitTorrent technology is not designed, as many other Internet technologies, to do illegal activities. That's solely the responsibility of the user. There are many legal uses for the BitTorrent technology, including downloading Ubuntu releases, which helps to distribute the load over Canonical severs, specially the day a new version is released.
    Index
    1. Introduction
    2. How does it work
    3. Download Speeds
    4. Configurations
    5. Security
    6. FAQ
    7. Troubleshooting
    8. References


    1. Introduction

    Understanding the technology is essential, in my opinion, to get the best of it. In this tutorial I try to explain the basics as simply as possible, but this is not easy to do in a few words, specially for someone like me who doesn't have English as native language. Nevertheless, I hope you will enjoy torrenting even more after reading this tutorial and I expect to be much easier for you to optimize and troubleshoot your download speeds.

    For the purpose of simplifying this tutorial, I will be using Deluge 1.1.9 as an example when explaining configuration of clients. I consider Deluge the best BitTorrent client for Linux and Windows. Nevertheless, most of the contents of this tutorial can be applied to any client, since the concepts are the same and most clients have the same sort of basic features. You can check a list of available clients and their respective features here.

    I have included several links throughout the tutorial, most of them to Wikipedia articles, which has a lot of good info on torrents. I wish I could explain everything, but then you probably wouldn't read it. So, I will present only the essential information to let you understand the basic concepts and properly configure your Ubuntu box for torrent download. Nevertheless, further reading is strongly recommended.



    2. How does it work

    BitTorrent is the most popular peer-to-peer file sharing protocol these days. It is popular not only because of availability of all sorts of files, but because of the high speed downloads. This is possible because you don't download a file from a single source and the file is not transmitted in sequence, but broken into several small pieces. This means you can download pieces of the file from someone who has 100% of the entire file downloaded, but at the same time from someone who has only 1% of it.

    Each user sharing the same file can contribute, by uploading to you or to other peers in the swarm, as long as it has a single piece of the file and you don't have the same piece. This method greatly increases the speed, because you can download from several peers at the same time and they don't have to wait for a complete download to upload to others. Even if a peer has a limited bandwidth for uploading, your download speed can be much superior to his bandwidth, because the upload speed of all connected peers are counted to your final download speed.

    There are some very interesting differences between BitTorrent protocol and clients, compared to other popular file sharing networks, like Gnutella (Limewire, Forstwire) and eDonkey (Shareaza, aMule, Morpheus), although some of these network clients also support BitTorrent recently. For instance, with BitTorrent you don't share folders, only single files and you don't browse/search other peers computers to find what you want. You need to download a special file, with the .torrent extension, for each content you want to download. This file, usually referred as torrent, has all the information you need to connect to other peers and get pieces of the content form them. That information is embedded as metadata and includes the identification of the files included in the download (hash) and one or more trackers, which are servers responsible for listing all peers sharing the same content and allowing them to connect to each other.

    So, instead of connecting to a network of peers and then searching among their folders for the files you want, you need to browse a torrent directory to look for the torrent files or get them from a friend by e-mail, from developers web sites or any other method of direct file distribution. Once you have the torrent file loaded into your favorite BitTorrent client, it will connect to the tracker specified in the torrent, to retrieve a list o peers IP addresses sharing the content specified in the torrent. Your client will also scrape the tracker in order to add your IP address to the list, so other peers can connect to you. The tracker and the torrent doesn't have the content itself, just information about contents and peers.

    Torrent directories or indexers are web sites that allow users to upload and download torrents, but not the content itself, which is always stored in the users computers. They are usually organized into categories and provide some sort of searching feature. The torrent directory does not necessarily provides the tracker, which can be any server on the Internet, although some of them also have their own. A popular torrent directory is Mininova.org, which also provides a tracker, but only for featured contents. These are contents provided by the site partners, which are creators or distributors, thus being legal and authorized. Torrents uploaded by regular users are not guaranteed to be copyright free, although the site has a system to filter copyrighted material. So use common sense, read the torrent comments or Google the content name for additional info before downloading.

    Other kinds of useful torrent sites are just search engines like Torrentz.com. They do not allow uploading, so they don't host the torrent files. Instead, the provide a search engine like Google, that targets only torrent files and popular torrent directories. Nevertheless, they can be much more complex than Google, since they usually provide list of trackers currently active for each torrent, the number of peers, number of seeders and even a forum or comment section. They also can allow to search torrents by hash info.

    For example, the link below uses the torrent hash to link to an Ubuntu release, which is tracked by the official Canonical tracker. The part of the address in red is the torrent hash number.

    http://www.torrentz.com/60d5d82328b4547511fdeac9bf4d0112daa0ce00

    Google also indexes torrent files and provide a method for searching them, through the filetype method like below:

    ubuntu filetype:torrent

    Keep in mind that is always a good idea to browse torrent directories for additional info, to avoid fake files, which might contain malware or undesirable content. The torrent protocol is much more secure than other file sharing networks, but is not immune to people with bad intentions. For example, browsing the Torrentz link above clearly reveals that the Ubuntu file is tracked by Canonical, due to the presence of tracker address torrent.ubuntu.com. So you know that file is distributed by a trusted and official source, thus is not a fake, does not contain malicious code and it's a legal file. Nevertheless, is also essential to check the final download hash against the the one provided by the content distributor, no matter if you download directly from their site via HTTP or using BitTorrent.


    3. Download Speeds

    Closed Port

    The first thing most users suggest when questioned about connection issues is that the torrent port is not properly configured in the client or it is closed by the router/firewall. While a closed port will usually result in lower download speeds, it doesn't prevent you from connecting to some peers, downloading and uploading. Nevertheless is not recommended and impolite to join a swarm with closed ports.

    The port configured in the BitTorrent client is used only for incoming unrequested connections. So even if it is closed, your client will still be able to request connections to other peers, but won't get any requests from them. Once the connection is established by your request, the data transfer between your client and the connected peer is always allowed. This means a closed port only reduces the number of connected peers. This happens because each peer has a limited number of possible simultaneous connections, so they will not accept your connection requests all the time, but they might try to connect to you when they have a connection slot available. So, having the port opened and properly configured will increase the number of connected peers.

    Please notice that the number of connected peers does not means that they will all be uploading to you at the same time. Your client connects to a large amount of peers and keep sending them requests for content pieces. The peer being requested might not have a free upload slot at the time of request, so the client tries another connected peer. So as general rule, the more peers you can connect, bigger will be your chances of getting more file pieces at the same time and thus your download speed will be higher.

    Bandwidth Limit

    One of the most common misconceptions about download speeds is related to the wrong assumption that the connection speed advertised by your ISP is what you should see in the torrent client. For example, my DSL plan has a maximum download speed of 4 Mbps, which is advertised as "4 Mega Plan". Well this speed is not actually what I see in the torrent client, which uses another measurement, named KiB/s. To check my maximum download speed, I need to convert what is advertised from Mbps to KiB/s. So my maximum download speed is actually 488.28 KiB/s.



    Additionally to that limitation, there is the fact that BitTorrent protocol works in a way that the more you give, the more you get. After all, is called file sharing, not file leeching. This means that even considering that I have 488.28 KiB/s maximum download speed, I rarely reach that value when downloading torrents, because my upload limit is only 74 KiB/s, which will also influence my download speed.

    Tracker Health

    As you already know, if you followed this tutorial from the beginning, that a tracker is responsible for listing all peers sharing a particular torrent content. They are almost essential to let you connect to peers for downloading a torrent content.

    Sometimes the tracker for a particular torrent file can list just a few peers, while another tracker has hundreds of them. This happens because some trackers are more popular than others and the same torrent file can be uploaded to different sites using different trackers.

    There is also the possibility that you can't even connect to the tracker. In this case, you won't be able to retrieve the list of peers sharing the content and thus won't be able to download. Nevertheless, there is a technology on most modern BitTorrent clients, called DHT, that allows to find peers without a tracker server. This is particularly useful if the tracker is temporarily offline, which has been very common recently due to torrent related legal battles. Torrent trackers also go offline permanently, since it's no cheap to maintain a popular tracker.

    The situations above can be fixed by simply replacing a tracker specified in the torrent file with one that is currently active and listing a decent number of peers. Torrent search engines like Torrentz.com, provides a list of active trackers for each torrent and the corresponding number of peers and seeders, so you can easily replace a bad tracker with a new one, full of seeders.

    Fortunately, most BitTorrent clients allows to edit the trackers from a torrent file and also add multiple trackers. So if the client can't connect to the first tracker listed, it will try another one from the list until it can connect or the connection request times-out.

    Another technology available in modern BitTorrent clients, called "Peer Exchange (PEX), can also help to increase the number of connected peers. It basically allow you to connect to a peer not listed on the tracker you are using, but that is connected to another peer on it through another tracker. PEX basically connects peers from different trackers, as long as the share a peer in common.

    PEX and DHT are the reasons why sometimes your torrent client reports a lot more connected peers than the total number of peers in the swarm.

    Torrent Health

    One of the most important aspects of a torrent health is the seeder/peer ratio, which is basically the number of peers with 100% of a given torrent content, divided by the number of those with incomplete content. The bigger the ratio, the better can be your download speed, because the seeders are just uploading and not draining resources from the swarm. Let's say I join a swarm with 500 seeders and 100 peers, in which case there will be 5 uploaders for each downloader. This probably means that I will connect to some peers that will be uploading to me at full capacity, since there are enough uploaders to upload to the other 99 peers. A good ratio is at least 1:1, but 2:1 or higher are very desirable.

    A healthy torrent is not just one with a good seeder/peer ratio, but also one with a big swarm (total number of peers + seeders). Some popular torrents can have more than 50.000 peers simultaneously, but as far as I know there is no limit. Anyway, sometimes a torrent becomes virtually dead, even with hundreds of peers in the swarm. This happens when there is no more seeders in the swarm and combining all pieces of the content from all peers in the swarm you still can't complete 100% of the desired content.

    In a situation like that, you will usually see lots of peers with 99% of the file, but none with 100%, because a single piece could be missing from all of them. So unless a seeder joins this swarm, nobody will be able to complete the download. This situation can happen because a lot of people simply download the contents of the torrent and stop after completing the file. They do not keep uploading (seeding) to others that still don't have the entire file. This kind of impolite behavior is called leeching and it's definitely not good for the torrent community. It is recommended that you seed at least the same amount of pieces you have downloaded, which can be verified in BitTorrent client as "Share Ratio". A desired ratio is above 1. A ratio of 0.5 means you half uploaded half of the amount of pieces you have downloaded.

    Note: If you are downloading videos, then most of the video players are still able to play incomplete files, so sometimes is better to leave the swarm instead of waiting for that last piece.
    Sometimes the torrent simply die because of lack of interest. A dead torrent happens when there are no seeders or peers in the swarm. Unlike a game server or chat room, in which you can join and wait for another peer to join, torrents without peers are pretty much dead. The chances of finding some peers and actually downloading with decent speeds are extremely low. Nevertheless, sometimes you can find a few peers using a different tracker.

    Keep in mind that some torrent directories list the number of peers in each swarm, but they do not update that info very often, so you need to check when this value was updated, before downloading the torrent. Otherwise, you might find out that actually there are no peers sharing it when loading the torrent into the BitTorrent client.

    Other Factors

    There are several other factors that influence the download speed of BitTorrent content. It's not an exact science, but with time you gain experience and learn how to tweak your torrent client, to identify the best torrents and how to avoid bad swarms. For example, these are some other factors that can influence your download speed:

    • the bandwidth upload limit of those peers you are downloading from
    • the the number of peers also downloading from those you are downloading from
    • the number of torrents being uploaded by those you are downloading from


    For example, let's say you are downloading from me and 5 other peers. I have an upload limit of 74 KiB/s. But I'm also uploading to other 4 peers. So instead of getting 74 KiB/s from me, you only get about 18.5 KiB/s, if I upload to all peers at the same proportion. Let's say all other 5 peers you are downloading from have the same upload rate, so you will get only 92.5 KiB/s total. That is a scenario considering that I have only one active torrent, but if I'm sharing more than one torrent at the same time, then my upload speed to you will be even smaller. So as a general rule, the less torrents you share at the same time, better will be your upload and download speed for each torrent.

    Most BitTorrent clients like Deluge allows to control how many upload slots for each torrent you will use, how many torrents will be active at the same time, how many will be seeding or downloading and also how many peers you can connect at the same time.



    4. Configurations

    As already explained, we need to open a port so other peers can request connections to your BitTorrent client. If you have a router, then this is the first place to start with. If not, then proceed to the Firewall and Client Configuration topics.

    Router Configuration

    To open a port in the router you will have to forward the port or port range used by the BitTorrent client to your machine. If you don't do that, all connections attempts from remote peers reaching your router won't pass through it, even if you have only one computer in your local network. This happens because all connections to all computers on your local network reach the same router, so it needs to know if the connection on a specific port should pass through and to which machine the connection should be forwarded or they will be ignored.

    Keep in mind that a closed port in the router does not prevent you from downloading, because it only affects incoming unrequested connections. All torrent file pieces related to connections already established by your client will pass through as expected, even with a closed port, because the router is "clever" enough to know that they belong to a connection requested by the BitTorrent client on your machine.

    So, the first step is to decide which port or port range to use. The standard port range for BitTorrent traffic is 6881 to 6889. But that doesn't mean you have to use it. In fact is not even recommended, because some Internet Service Providers throttle the traffic on those ports, exactly because they are used for BitTorrent.

    Warning: you should consult your ISP contract to see if there are any restrictions related to BitTorrent usage and incoming ports. If not, then you can safely use a different port or port range, because they do not have the rights to throttle your connection if this not specified in the contract.
    It is recommended to use any single port or port range between 49152 and 65535, but it should not be in use by another server in your machine. To check if there are any ports being used, run the following command in a terminal:

    Code:
    netstat -plntu
    Most modern routers and BitTorrent clients offers an automatic method of port-forwarding, called UPnP. All you have to do is enable UPnP on the router and the BitTorrent client and configure the ports you want to use in the client. There are some security risks involved in this method, like a for example a malicious web site gaining access to your router settings and redirecting web pages you visit to phishing sites. So some people prefer to forward the ports manually as explained below.

    To manually forward the port from the router you also will need to know the internal IP address of your machine. To do that, run the following command in a terminal and look for the IP value next to inet addr: in the results. It should be something like 192.168.x.x.

    Code:
    ifconfig
    Note: it is recommended to use static internal IP if you have more than one computer on your local network. You can do that from the Network Manager. Basically a static IP means that every time you connect to the router you will get the same IP, even if another computer is already connected to it or not. You need to configure a static IP for each computer in your local network.
    Once you have your internal IP address and have chosen a port or port range to forward, then visit your router settings page to proceed with the port forwarding. There are too many router models, so this will not be explained here, but you can get specific instructions for your router model at http://portforward.com


    Firewall Configuration

    If you have firewall rules enabled, then you also need to open the port in the firewall for incoming connections. You should allow all incoming connections on the port or port range selected to be used by the BitTorrent client. Additionally, you need to allow incoming traffic on the loopback device.

    Deluge has a daemon and a frontend, that allows you to connect remotely from any client machine. On a default standalone installation, Deluge launches the daemon and connects with the frontend automatically, through the loopback device, so you don't even notice the separation. It uses port 58846 by default and another random port, both listening on the default loopback address, 127.0.0.1. If you start Deluge and it shows a blank interface, is probably due to blocked connections on those ports.

    Additionally, if you use UPnP, then you need to allow incoming traffic from the IP address of your router on port 1900.

    In regard to outgoing connections, you should allow all traffic, since Deluge will use random ports for requesting connections to other peers. Gufw firewall manager does not even allow you to block outgoing connections, which is perfectly normal. Nevertheless, if you use Firestarter or creates your own iptables rules, then you might have to create a rule to allow all outgoing connections.

    If you don't have any firewall rules enabled or don't know if you have, then probably there isn't anything else to do in regard to the firewall. Ubuntu comes with a firewall installed and running by default, but it has no rules, because there is no need to block traffic, since Ubuntu also comes without any server running. This means all traffic will reach your computer on any port, but all ports are virtually closed because there aren't any applications listening to incoming connections (server) on any of them. When you run a BitTorrent client, which is a server, it will use the port or port range as expected, because the firewall won't block them. To decide if you need a firewalled BitTorrent connection or not, read the Security section.


    Client Configuration

    Network Settings: incoming ports

    Opening the ports on Deluge is pretty simple, no matter if you have a router or not and if you are using UPnP or not. To do that open Deluge and select "Edit >> Preferences" from the menu. Then select the "Network" category.

    If you do not have a router or are using the UPnP method, then select the option "Use random ports" in the "Incoming Ports" configuration. Enable the UPnP option in the "Network Extras" if you have a router with UPnP enabled. You can also use a port range when enabling UPnP, then you can have more control over which ports will be chosen every time you start Deluge. Please notice that only one port is chosen from the range on every start.

    Note: if you have firewall rules, then do not use random ports. You need to specify the same port or port range allowed in your firewall.
    If you are forwarding the port from your router manually or if you have firewall rules, then you need to specify the port range in the "Incoming Ports" configuration. If you decided to use a single port instead of a range, which I prefer, then put the same value in both fields. Also leave the option "Use random ports" unticked.

    To test if the port is properly forwarded and configured in the client, see section 7.2 Troubleshooting Ports.


    Other Network Settings

    Outgoing Ports - although Deluge also offers an option to configure the Outgoing Ports, usually there is no need to do it, unless you have very restrictive outgoing firewall rules and need a specific range. So leave the option "Use random ports" enabled for outgoing connections.

    For TOS option visit this discussion, but you should be fine with the default value.

    NAT-PMP is another method of automatic port-forwarding used with routers that support the NAT port mapping protocol (Apple products, for example). You can leave it unticked if you are using UPnP or manual forwarding.

    LSD - can be used to connect to peers on your local network

    DHT and PEX - see the Download Speeds section for explanations

    Encryption - there are a lot misconceptions about what the encryption options do. Some people believe it helps to hide your identity. That's not true. The encryption simple makes hard for a third-party (aka your ISP) to detect the type of traffic generated by your BitTorrent application, in order to throttle your torrent speed.

    Warning: you need to check if there are any restrictions related to BitTorrent traffic in your ISP contract. If not, then is safe to use encryption to avoid traffic shaping, because if it is not in the contract, then they don't have the rights to slow you down.
    For the encryption options I recommend using Forced for both "Inbound" and "Outbound" traffic, Full Stream for the "Level" and tick "Encrypt entire stream". This way you will have the highest level of encryption possible. Keep in mind that if a peer trying to connect to you does not have encryption enabled or his BitTorrent client it's not compatible with encryption, then it's connection requests will be rejected. This could eventually prevent you from connecting to any peers on very small swarms. Nevertheless, most modern clients are indeed compatible and most people use encryption, so you should be fine most of the time.

    Bandwidth Settings

    The bandwidth settings allow you to tweak Deluge in order to maximize the download speed. First check your bandwidth limits at http://www.speedtest.net. If you need to convert your speed, than change the settings for the reports or use a conversion tool from here

    According to Deluge FAQ, you should start with these settings:

    Maximum Connections: 200
    Maximum Download Speed (Kib/s): -1
    Maximum Upload Speed (Kib/s): 80% of upload speed limit of your bandwidth
    Maximum Upload Slots: 4
    Maximum Half-Open Connections: -1
    Maximum Connection Attempts per Second: 20
    Maximum Connections is the maximum number of peers Deluge will be able to connect at the same time. Please keep in mind that this not represents all peers actively uploading to you, because after the establishment of the connection with another peer, the client still needs to request file pieces. Since everyone has a limited upload bandwidth, they also limit the number of Maximum Upload Slots. This means that, if everyone followed the recommendations above, each peer would be capable of uploading to another 4 peers at the same time (4 upload slots). So most of the peers among those 200 connections will be connected to you but not uploading. Most of the time, the peers that are actively uploading to you keeps changing, so having a lot of peers connected will maximize the number of active downloading connections. Nevertheless, you should exaggerate with these settings, otherwise you could end up clogging your network and actually reducing your download speed. Some experimenting is required. I use 150, but I usually do not have more than 2 torrents active at the same time and only increase to 250 when both are being downloaded.

    The Maximum Download Speed set as -1 means no limit. I use that setting and never have issues, but it could slow down your regular network activity, like web browsing. If you experience this kind of issue, then put a value a little bit lower than the download speed limit of your bandwidth, like 95% of it.

    Maximum Upload Speed should be about 80% of the upload speed limit of your bandwidth, otherwise it will clog you network and reduce your download speed.

    Maximum Upload Slots also depends on the upload speed limit of your bandwidth. There are some formulas to calculate this value like "1 + ( upload speed / 6)" but I prefer to use common sense. Consider a value that will give at least a good fraction of your bandwidth to each peer you upload simultaneously. If you have 50 KiB/s upload limit, then use 5 upload slots, so each peer could get about 10KiB/s (if you upload at the same rate to all of them). If you use 50 slots in this scenario, then the upload speed for each peer will be extremely reduced.

    Maximum Half-Open Connections and Maximum Connection Attempts per Second determine the outbound concurrent connection attempts. If you use unlimited value (-1) for the first one as suggested, then you should be conservative with the second value, since too many half-open connections could cause your router to restart or even block the network completely. I believe is better to limit the number of connection attempts per second, then reducing the half-open connections. This way, new attempt connections won't be put on hold if there are already too many half-open connections, but will avoid firing too many connections at the same time. The default suggested values should be enough, which in this case means it would take 10 seconds to try to connect to 200 peers.

    Per Torrent Bandwidth Usage

    These settings allows to distribute the bandwidth between each torrent. If you have limited the number of active torrents to just one or two, then leave this unchanged. Otherwise, change the default values so each torrent gets a decent fraction of your bandwidth.

    Keep in mind that setting these values to lower than the global settings will limit your download/upload speeds, if you have just one active torrent or if you still do not reach your bandwidth limit, after combining the usage of all of them. So if you do not have several torrents active all the time, it might be a good idea to not limit the bandwidth on a per torrent basis.



    5. Security

    Whenever you are connected to the outside world, you are prone to attacks. This is true even for the most simple web browsing activities and also for peer-to-peer file sharing. Nevertheless, sharing files impose additional security issues.

    As already explained, the BitTorrent protocol is much more secure than other file sharing networks, because you do not grant anyone the rights to browse your folders. But a torrent client is essentially a server and thus will be listening to a port for incoming connections. This means anyone can connect to you without your request, as long as they are using a compatible BitTorrent client and sharing the same file through the same tracker. This is obviously wanted, to maximize the number of potential connections with other peers, since the download speed depends on being connected to several of them, as already explained in the Download Speed section. Nevertheless, some people might try to use this opened pathway to your computer to gain unauthorized access to other resources on your machine.

    In a normal situation, the BitTorrent client will not allow any access to your computer other than transferring pieces of the file specified in the torrent metadata, between both connected peers. Nevertheless, security holes exists in any kind of software. So, if your torrent client has a security vulnerability, it could be used to gain access to other resources on your machine. Fortunately, there are lot's of people in the open source community that tries to find those flaws in applications and alert the community about possible risks. To minimize the risks you should always keep your system and torrent client updated with the latest patches.

    Having an updated system is essential for any Internet activity, but does not guarantee your immunity, since an attacker could be exploiting a security flaw that hasn't been discovered or fixed by the application developers, which are known as "zero-day exploits". To protect your machine against "zero-day attacks", you can use Apparmor to limit what resources and permissions are granted to your BitTorrent client, thus limiting what an attacker could do if he gains full control of your client. More about Apparmor at http://ubuntuforums.org/showthread.php?t=1008906


    Firewall

    Do I need a firewall to protect my machine while sharing files? Probably not. Essentially, a firewall just limits the access to your machine from the outside world based on port number or the IP of the remote machine. But since you want to connect to the most number of peers you can, to maximize your download speed, it makes no sense to block access to your BitTorrent port by closing it with a firewall, otherwise other peers won't be able to request connections to your client and the number of peers connected to you will be reduced. Using a firewall to protect other ports not in use by the BitTorrent client or any other server is redundant, because a port without a listening service is essentially a closed port. As far as I know, there is no possibly way to breach your computer through a closed port.

    If you have other servers running, like a ssh server for example, then you might want to use firewall rules to block connections from the outside world to the ssh port, while still allowing computers on your local network to connect. Nevertheless, if you have a router, you can do that through the router firewall, instead of your machine.


    Blocklists

    Most torrent clients like Deluge and Transmission, offers a feature called blocklist plugin. It works like a p2p firewall, by rejecting connections coming from untrusted known IP addresses. These known bad IP addresses are collected by some users and organizations engaged in Internet security and distributed for free, as lists of IP ranges. There are several different types of blocklists, targeting specific threats, since they are not used only for p2p.

    Although there is a lot of controversy about blocklists effectiveness, because IP addresses keep changing all the time and some publishers add more ranges than necessary, there are several well known addresses that you probably don't want to connect to you. Besides, you can customize these lists and create your own, which can be very useful.

    There are also standalone applications that uses the same blocklists, called ip blockers. There are two of them for Ubuntu that I'm aware of, moblock and iplist. Although I prefer moblock, because it also handles my firewall rules, they both work very well. If you used PeerGuradian on Windows, then you probably will prefer iplist, because it has a very similar interface. Nevertheless, moblock can also be controlled with a graphical interface frontend called mobloquer.

    Standalone ip blockers are better than blocklist plugins, because they protect your entire network, by acting at the firewall level, not only the p2p connections. Nevertheless, there is nothing that could prevent you from using both. Most people consider this redundant, but I like for example to use different lists for p2p activity, so using both gives me more flexibility.

    You can get several blocklists from Bluetack and I-Blocklist, although all standalone ip blockers and BitTorrent blocklist plugins offers some sort of automatic download and update from these sources.

    Keep in mind that more blocklists is not necessarily better. Some of those lists will block several valid peers and thus could reduce your download speeds considerably.


    Fake files, malware and other threats

    Ubuntu has it's own trusted repositories of applications, that should be used as the preferred method of installation. Nevertheless, sometimes you want to download Ubuntu itself or other applications like games, that are also distributed via direct download or BitTorrent. No matter which method you use, whenever you are downloading applications from outside the repositories, you should always check the final file hash number, to verify it's validity. Most publishers will offer hash numbers on their download site, so you can compare them with the one from your downloaded file.

    Most torrent directories and search engines offers some sort of validation method, like marking torrents uploaded by trusted users or distributors. They also provide comment sections for each torrent, so you can read other users comments about the downloaded content. Fake files will be usually spotted pretty fast and moderators of some sites are very effective in cleaning up the junk. Some sites also provide a method of identifying copyrighted material and automatically removing them, so these sites are preferred to avoid legal issues.

    Nevertheless, there is always people with bad intentions uploading junk to BitTorrent sites. So use common sense and the site tools before downloading any torrent. For example, browsing this Torrentz link clearly reveals that the Ubuntu file is tracked by Canonical, due to the presence of tracker address torrent.ubuntu.com and thus is an original file.




    For more information on security visit the Ubuntu Security tutorial.



    6. FAQ


    My logs shows a lot of connection attempts after stopping a torrent or closing the BitTorrent client. Should I be worried?

    When you close the BitTorrent client, you probably will keep receiving several connection attempts for a while. This is normal and they are called "ghost packets". This happens because trackers and BitTorrent clients do not update the list of peers sharing a particular torrent very often. As already explained in previous sections, your BitTorrent client scrapes the tracker to add your IP address, so other peers can connect to you. This procedure is done in a determined interval, which is the same used to retrieve the list of peers from it, so your client can connect to other peers. When you close your client, your IP number will still be listed in the tracker and other peers clients as sharing the file, therefore they will still try to connect to you, not knowing that your client is already closed.

    There is nothing to worry about those connections attempts, because the are just BitTorrent connections and since your client is closed, your port is virtually closed too. So these connections will be ignored. Nevertheless, you can make them stop by simply restarting your router or modem.

    Sometimes you get "ghost packets" even when you haven't used a BitTorrent client for a while. This can happen even if you just connected your modem or router and it's due to the way IP addresses are assigned. Most Internet Service Providers assign home users with a dynamic IP, which means you get a "new" IP address every time you connect to them. This IP is not actually "new", but it's part of an IP range available to the ISP, that is shared among all it's costumers. By shared, I don't mean you will be using the same IP as another dude simultaneously, but since most costumers are not connected at the same time, the ISP does not have a single IP for each one of them. Instead, they assign the first available IP when a someone connects. Sometimes you get an IP number that has been just released by someone else and if he was engaged on some Internet activity that rely on accepting incoming requests, like torrenting, then you will probably get his "ghost packets".



    7. Troubleshooting

    Answer the questions below and follow their directions. These questions are designed to eliminate possible issues until only one is left.


    7.1 Speed and connection issues

    #1 Are you able to download?

    • If the answer is "YES, I can download, but the speed is slow": proceed to question #2.
    • If the answer is "NO, I can't download anything": skip to question #9.


    #2 Does the torrent has dozens or hundreds peers?

    • If the answer is YES: proceed to question #3.
    • If the answer is NO: check the Torrent Health topic on the Download Speeds section.


    #3 Does the client connects to dozens or hundreds of peers but the speed is slow?

    • If the answer is "YES, there are lots of connected peers, but the speed is slow": skip to question #6.
    • If the answer is "NO, the client connects only to a few peers": proceed to question #4.


    #4 Is the incoming port used by the BitTorrent client properly opened?

    • If the answer is "I don't know how to open ports": refer to the Configurations section to learn how to do it.
    • If the answer is "I have opened it, but I'm not sure if it is working": check the section 7.2 Troubleshooting Ports.
    • If the answer is "YES, I have already tested it": proceed to question #5.


    #5 Are you using blocklists plugins or standalone ip blockers?

    • If the answer is "YES": check the blocked IP log to see if there are too many entries. You might be blocking too many IP ranges, that could be preventing you to connect to most peers.
    • If the answer is "NO, I'm not using such a thing": check the Bandwidth Settings section of the tutorial. Your client settings might be limiting the number of allowed connected peers. Also try to disable encryption, to see if the number of connected peers improve.


    #6 Does the torrent has a good seeder/peer ratio?

    • If the answer is "I have no idea": check the Torrent Health topic in the Download Speeds section.
    • If the answer is "NO, the ratio is lower than 1:1": try another tracker or another torrent with more seeders, or give some time to build up speed. It's normal to experience slower speeds when the number of peers is higher than the number of seeders.
    • If the answer is "YES, the ratio is 1:1 or higher": proceed to question #7.


    #7 Are you using standard BitTorrent ports (6881-6889)?

    • If the answer is "YES": your ISP might be throttling your connection on the specified port. Try to change the torrent port to any port in the range 49152 to 65535.
    • If the answer is "NO, I use a different port or port range": proceed to question #8


    #8 Have you checked if your Internet connection speed is normal?

    • If the answer is "I don't know how to do that": visit http://www.speedtest.net
    • If the answer is "YES, the connection is fine": try downloading another torrent from a reliable tracker, like official Ubuntu torrents, to see if it behaves the same way. If it does, then your ISP might be slowing you down according to protocol or time of day. You could try to enable full encryption or download another time to see if the speed improves.


    #9 Does the client connect to any peers?

    • If the answer is "NO": proceed to question #10
    • If the answer is "YES, but nothing happens": go back to question #6.


    #10 Do you have active firewall rules?

    • If the answer is "NO": it's probably a tracker issue, so check the Tracker Health topic in the Download Speeds section.
    • If the answer is "YES": check if you have allowed all outgoing connections in the firewall manager. Refer to the Firewall Configuration section for further instructions.



    7.2 Troubleshooting Ports

    Deluge and most BitTorrent clients offers a method of checking if the configured incoming port is open or not. But this method relies on sending a request to the developers site and waiting for a remote scan. So if the developer site is not responding, Deluge could tell you the port is closed, when in fact it could be opened. Besides, it doesn't give you any additional info, so it's hard to troubleshoot using this kind of tool.

    To test if the port forwarding is working properly and if your client is properly configured follow these steps:

    1 - Open Deluge client and check which port it is using for incoming requests. To do that, go to "Edit >> Preferences >> Network" and check the value after "Active Port:", in the "Incoming Ports" section.
    2 - Go to http://canyouseeme.org and put the port number in the form, after "What Port?" and click "Check".

    Possible results, explanations and solutions:

    Error: I could not see your service on xxx.xx.xx.xxx on port (xxxx)
    Reason: Connection timed out
    The port is stealth. It means the connections are dropped (ignored) before reaching the BitTorrent client. Check your router and firewall settings/log.

    • If the firewall log shows an entry for the test scan, then you need to open the port with a new firewall rule or disable the firewall.
    • If the test scan is not logged by the firewall, then the problem is probably in the router port forwarding settings or your ISP is blocking the connection on the selected port. If you use UPnP, don't forget to allow incoming traffic from your router IP address on port 1900, otherwise the automatic port-forwarding might fail.


    Error: I could not see your service on xxx.xx.xx.xxx on port (xxxx)
    Reason: Connection refused
    The port is closed. This means the client is not accepting connections on that port or the firewall is rejecting the connection attempts. Check your client port configuration and make sure the firewall has a rule to allow inbound tcp (also udp if you use DHT) connections on the selected port. Temporarily disabled the firewall completely might help to troubleshoot.

    Success: I can see your service on xxx.xx.xx.xxx on port (xxxx)
    Your ISP is not blocking port xxxx
    The port is open, which means it is properly forwarded by the router, the firewall is not dropping or rejecting connection attempts and the client is properly configured. Additionally, your ISP is not blocking connections on the port.




    8. References

    Last edited by lovinglinux; March 19th, 2010 at 02:48 PM. Reason: Added information about loopback device in the firewall configuration

  2. #2
    Join Date
    Aug 2009
    Beans
    8

    Re: BitTorrent optimization and troubleshooting guide

    Nice instructions. I'm using this software Bittorrent for downloading some movies and softwares. This is the first I encountered such optimization, but I can do this in the future.

  3. #3
    Join Date
    Aug 2008
    Location
    Brazil
    Beans
    12,497
    Distro
    Ubuntu Studio 12.04 Precise Pangolin

    Re: BitTorrent optimization and troubleshooting guide

    Quote Originally Posted by careertargetph View Post
    Nice instructions. I'm using this software Bittorrent for downloading some movies and softwares. This is the first I encountered such optimization, but I can do this in the future.
    Thanks. Keep in mind that it's not finished yet. I'm still writing it.

  4. #4
    Join Date
    Aug 2008
    Location
    Brazil
    Beans
    12,497
    Distro
    Ubuntu Studio 12.04 Precise Pangolin

    Re: BitTorrent optimization and troubleshooting guide

    Thanks to the moderators for fixing the typo on the title and moving it to the Tutorial & Tips forum.

  5. #5
    Join Date
    Aug 2008
    Location
    Brazil
    Beans
    12,497
    Distro
    Ubuntu Studio 12.04 Precise Pangolin

    Re: BitTorrent optimization and troubleshooting guide

    I think the tutorial is ready. Any feedback will be much appreciated.

    If you still have issues after reading the tutorial and following the troubleshooting section, then post a description of your issue, so I can analyze your situation. Please also provide which steps you took to troubleshoot.

  6. #6
    Join Date
    Dec 2008
    Location
    NoCo
    Beans
    Hidden!
    Distro
    Ubuntu 10.04 Lucid Lynx

    Re: BitTorrent optimization and troubleshooting guide

    Thanks for the explanations. I'm sure this will help lots of users. However, I have one question that was not addressed and can't seem to find any information on it anywhere.

    I use Deluge and have one port dedicated to torrenting. The same port is open on my router and in my gufw/ufw firewall rules. I am also using the Deluge IP block list.

    When I start Deluge the IP block list loads and then I watch /var/log/messages: UFW immediately starts blocking incomming connections to random ports.

    I have no torrent loaded, I have deleted all .torrent files, and have restarted cable modem and router before this test. These incomming hits will continue until I restart the computer and/or network.

    I read the information about ghost packettes but there should be no ghost packettes since I had no torrent loaded in my Deluge. Are these ghost packets from the last time I downloaded a torrent?

    I can't figure this one out.

    Cheers,
    brook

    ps after more testing the only way to stop the incoming hits is to turn off the cable modem, routers, machine, restart the network, then restart the machine. the hits are coming into ports other than the designated open port in deluge and firewall, thus being blocked by firewall.
    Last edited by brookie; September 18th, 2009 at 07:00 PM. Reason: new test results added:

  7. #7
    Join Date
    Aug 2008
    Location
    Brazil
    Beans
    12,497
    Distro
    Ubuntu Studio 12.04 Precise Pangolin

    Re: BitTorrent optimization and troubleshooting guide

    Quote Originally Posted by brookie View Post
    I use Deluge and have one port dedicated to torrenting. The same port is open on my router and in my gufw/ufw firewall rules. I am also using the Deluge IP block list.

    When I start Deluge the IP block list loads and then I watch /var/log/messages: UFW immediately starts blocking incomming connections to random ports.

    I have no torrent loaded, I have deleted all .torrent files, and have restarted cable modem and router before this test. These incomming hits will continue until I restart the computer and/or network.
    Since your router is only allowing incoming traffic on one port and you don't have any torrent loaded, these connections must be internal traffic on the loopback device. Deluge has a daemon and a frontend, that allows you to connect remotely from any client machine. But on a default standalone installation, Deluge launches the daemon and connects with the frontend automatically, so you don't even notice the separation. Nevertheless, it still uses the local network to do that.

    So, After starting Deluge, open a terminal and run this command:

    Code:
    netstat -plntu
    You will see something like this:

    Code:
    Active Internet connections (only servers)
    Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
    tcp        0      0 127.0.0.1:52715         0.0.0.0:*               LISTEN      23142/python    
    tcp        0      0 0.0.0.0:54194           0.0.0.0:*               LISTEN      23122/python    
    tcp        0      0 127.0.0.1:58846         0.0.0.0:*               LISTEN      23122/python    
    tcp6       0      0 :::54194                :::*                    LISTEN      23122/python
    udp       0      0 0.0.0.0:1900               0.0.0.0:*            LISTEN      23122/python
    Port #58846 is the default port for connecting the frontend to the daemon. This can be changed in the "Daemon" settings, although I do not recommend. In the example above, port #54194 is the one I choose for incoming torrent connections, so it's the one forwarded in the router. Port #52715 is also used by Deluge internal traffic. This port s random and changes on every start. Port #1900 is used for communication with the router and is related to UPnP. It only shows up if you have UPnP enabled.

    So after doing that, check if the firewall blocked connections are incoming on those ports.

    To fix this you will have to enable traffic on the loopback device. If you use UPnP, then you need to allow incoming connections from your router IP address on port 1900.

    BTW, I totally forgot about these settings, but I have already edited the Firewall configuration section to include this info. Thanks for reminding me.
    Last edited by lovinglinux; September 18th, 2009 at 09:00 PM.

  8. #8
    Join Date
    Dec 2008
    Location
    NoCo
    Beans
    Hidden!
    Distro
    Ubuntu 10.04 Lucid Lynx

    Re: BitTorrent optimization and troubleshooting guide

    Thanks for the reply lovinglinux.

    I got Deluge bandwidth settings tuned in a little better according to your tutorial and will test later. I figured out what the UFW block messages were in my logs as well.

    I upgraded gufw to version 0.20.7 which has an option to add a service or program on the preconfigured tab. It looks for installed programs and it saw deluge. I added it and it only added my deluge port for TCP but not UDP. Therefore, UFW was blocking all UDP attempts for my Deluge opened port.

    Once I added my deluge port for UDP as well as TCP all the blocks stopped. I guess UFW has been blocking the ghost packets on UDP when I opened it with no torrent present. I originally got the UDP/TCP firewall info from the uTorrent forums when I was using windows and uTorrent. By the way, this is not in your tutorial so I don't know if you want to add it or not.

    After adding UDP for my Deluge port the block attempts are gone from my logs, so it seems that these are ghost packets since I have no torrents loaded in Deluge at the moment and I deleted the .torrent files after seeding.

    One more thing, I adjusted my bandwidth settings under Global Bandwidth Usage and just noticed that there are also Per Torrent Bandwidth Usage. More settings to tinker with.

    Also, Speedtest.net has a cool settings in the upper left hand corner where you can select kilobits, kilobytes, megabits, megabytes for measurement so no need to convert. I assume that using kilobytes for the measurement is the same as the deluge setting for KiB/s.


    From Speedtest.net:
    Avg. DL Speed = 7.65 Mb/s = 933.84 KiB/s
    Avg. UL Speed = 3.03 Mb/s = 378.4 KiB/s

    My Deluge bandwidth Settings:
    Global Bandwidth Usage:
    Maximum Connections: 200
    Maximum Upload Slots: 30
    Maximum Download Speed (KiB/s): -1
    Maximum Upload Speed (KiB/s): 302, (378.4 KiB/s x 80% = 302.72)
    Maximum Half-Open Connections: 50
    Maximum Connection Attempts per Second: 20

    Ignore limits on local network, checked
    Rate limit IP overhead, checked

    Per Torrent Bandwidth Usage:
    Maximum Connections: -1
    Maximum Upload Slots: -1
    Maximum Download Speed (KiB/s): -1
    Maximum Upload Speed (KiB/s): 50

    Deluge default settings here:
    http://dev.deluge-torrent.org/wiki/F...tingstodefault

    Have a great weekend!
    Cheers,
    brookie

  9. #9
    Join Date
    Aug 2008
    Location
    Brazil
    Beans
    12,497
    Distro
    Ubuntu Studio 12.04 Precise Pangolin

    Re: BitTorrent optimization and troubleshooting guide

    Quote Originally Posted by brookie View Post
    Once I added my deluge port for UDP as well as TCP all the blocks stopped. I guess UFW has been blocking the ghost packets on UDP when I opened it with no torrent present. I originally got the UDP/TCP firewall info from the uTorrent forums when I was using windows and uTorrent. By the way, this is not in your tutorial so I don't know if you want to add it or not.
    The UDP connections only appear if you use DHT. In the tutorial I say that you should allow all connections in the incoming port, which means tcp and udp. Perhaps I should be more specific about this.

    Quote Originally Posted by brookie View Post
    Also, Speedtest.net has a cool settings in the upper left hand corner where you can select kilobits, kilobytes, megabits, megabytes for measurement so no need to convert. I assume that using kilobytes for the measurement is the same as the deluge setting for KiB/s.
    Yep, you are correct. I guess I wasn't clear enough about the possibility of changing the settings on speedtest.net. Nevertheless, I think it is interesting to provide a converter.


    Quote Originally Posted by brookie View Post
    One more thing, I adjusted my bandwidth settings under Global Bandwidth Usage and just noticed that there are also Per Torrent Bandwidth Usage. More settings to tinker with.
    Yep, there are already instructions about them in the tutorial.

    Quote Originally Posted by brookie View Post

    From Speedtest.net:
    Avg. DL Speed = 7.65 Mb/s = 933.84 KiB/s
    Avg. UL Speed = 3.03 Mb/s = 378.4 KiB/s

    My Deluge bandwidth Settings:
    Global Bandwidth Usage:
    Maximum Connections: 200
    Maximum Upload Slots: 30
    Maximum Download Speed (KiB/s): -1
    Maximum Upload Speed (KiB/s): 302, (378.4 KiB/s x 80% = 302.72)
    Maximum Half-Open Connections: 50
    Maximum Connection Attempts per Second: 20

    Ignore limits on local network, checked
    Rate limit IP overhead, checked

    Per Torrent Bandwidth Usage:
    Maximum Connections: -1
    Maximum Upload Slots: -1
    Maximum Download Speed (KiB/s): -1
    Maximum Upload Speed (KiB/s): 50

    Deluge default settings here:
    http://dev.deluge-torrent.org/wiki/F...tingstodefault

    Have a great weekend!
    Cheers,
    brookie
    Your settings look fine to me.

  10. #10
    Join Date
    Aug 2008
    Location
    Brazil
    Beans
    12,497
    Distro
    Ubuntu Studio 12.04 Precise Pangolin

    Re: BitTorrent optimization and troubleshooting guide

    I have decided to give Ktorrent a try and I'm quite impressed. I guess it will be my default BitTorrent client for now.

Page 1 of 10 123 ... LastLast

Tags for this Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •