How-To: BroadCatching using RSS + BitTorrent to automatically download TV shows
posted November 23rd 2004 1:08PM
Over the last couple week's TiVo let slip that they're going to start adding banner ads when we try to fast-forward commercials (and to make things even more annoying, Congress is considering a bill that might make fast-forwarding through the commercials on DVDs illegal). So we've pretty much decided we're not going to use a TiVo anymore and that instead we'll go with other solutions (for now) like a Media Center PC, as well as rolling some of our own ways to download TV shows. For this week's How-To we're going to show you how you can use RSS and BitTorrent to find and download your favorite shows, automatically, using free software. There are a lot of ways to do this, we cover one example that worked well for us, and there are a couple great tutorials we spotted that are basically the same but offer other ways of downloading, other plugins and more information. You can check them out here and here. If you're already a pro, be sure to post up your set up and experiences in the comments area.
RSS and BitTorrentBefore we get started, it might be a good idea to introduce a few topics if you're not familiar with them. We'll be
using RSS and BitTorrent to look for and download TV shows.
RSS also known as "Really Simple Syndication" is a
way to publish information so other computers can read it in a simple and standard way. In the context of this How-To
there are RSS feeds that will tell another application where and when a new TV show is available.
BitTorrent is a way to transfer files over the web, instead of
download the same file over and over from a central server, which will become slow or busy with high demand, BitTorrent
uses the bandwidth of all the people downloading the file and sends parts to everyone as it downloads, so the more
people who are downloading the better, and the more you upload, the faster you get your files (it definitely ends the
free rider problem of other p2p applications). For downloading TV shows, this of course is a perfect match, the more
popular a show is, the easier it is to get.
Disclaimer
Make sure you check with your local, state and country's laws—In some cases you may be downloading (and uploading)
copyrighted material. So far, there hasn't been a legal case that we know of where broadcast TV networks have stopped
or wanted to stop BitTorrent downloads of their shows, but we have heard that pay networks like Showtime, HBO, etc.
have sent letters to some individuals for sharing recorded TV. This is all new territory for the most part, and it will
be interesting to see what happens and if the TV networks realize that this could be their future distribution
chain.
The future of TV?
While your average TV watcher is probably not going to set
up RSS feeds and BitTorrent clients, many people are, and there is a lot of interest in using these technologies solely
for the purpose of downloading and watching TV shows.
(Mr. Quirk reminding us of a possible future recently. If you remember the old Max Headroom shows TV companies were
locked in a ratings war fought in real-time, changes in viewer numbers affected the stock price and net worth. Stopping
channel hopping was vital. One company invented "blipverts" that fed the adverts at high speed. Trouble was: fat
viewers [couch potatoes] were exploding and the TV company wanted to keep that quiet.)
The future of TV isn't certain, perhaps we'll have set top boxes that do what we're about to describe, i.e. look for
shows you like, download them in a decentralized way over high speed networks, delivering high-definition picture and
sound. All the parts are here, it just needs to get simpler.
Getting started
This How-To is fairly straight forward, we're going to download a Java engine to run a BitTorrent client, along with a
RSS plugin which will look for the TV shows we tell it to look for. For our example we're going to use a PC (Windows
XP) but all of this work on Mac OS as well and possibly Linux, but we didn't have a chance to test that out on the
Linux box here.
Installing the Java Run Time Engine
The first thing we need to do is install a Java Run Time Engine, this allows you to run Java based applications on
your specific operating system, for our example, we're using Windows XP.
You may or may not have a JRE already installed, even if you do, it's a good idea to get the latest one, Java 2
Platform Standard Edition 5.0 which you can
download here. Once downloaded and installed (a reboot may be required) we need to install the BitTorrent client,
Azureus.
Installing Azureus, the BiTorrent client
Azureus a full featured bittorrent with multiple download options, queuing and priority, pause/resume seeding as well
as detailed information about your uploadind and downloading status.
Download and install Azureus from here. Run Azureus and click through the settings wizard, this will walk you through any configuration needed. For the most part, clicking through next with all the defaults will work on most systems.
Installing the RSS Plugin for Azureus
Once Azureus is installed and running, we'll now need to install a RSS plugin. The RSS plugin will watch sites and
allow you so search as well as download for specific shows you're looking for.
There are a couple RSS plugins, for our example we're going to use RSS Import, which can be
downloaded here.
Another one to try out is RSS import Feed Scanner, there's a link to a tutorial
for using this one at the end of the How-To.
Unzip the file and place the "RSSImport" folder in the plugin directory of Azureus, it's most likely located here,
where you installed Azureus.
C:\Program Files\Azureus\plugins
Now start Azureus. If Azureus is running, then close the application and start it again.
In the top menu, choose Tools > Option. On the left side expand the + box next to Plugins
Click "Activate RSS Import Plugin".
Under RSS Channels to import, type in a RSS feed, for our example we're going to use:
http://www.tvtorrents.net/rss.php
You may also want to try:
http://www.btefnet.com/backend.php
http://varchars.com/rss/suprnova-tv.rss
If you add more than one use a ; to seperate them (no spaces).
The "Filter for channel entries" field is where you add the names of the shows you're looking for, basically you type
the name of the show and subsitute spaces in the name with periods, like this...
viva.la.bam
malcolm.in.the.middle
australian.idol
These are "regular expressions" you can add other filters, such as SVCD oe HDTV versions. Many torrents come in
different flavors of encoding, HDTV quality, DiVx, mpeg, etc...
For our example, we're on the prowl for some good ole' Cylons action in Battlestar Galactica. So we're going
to use...
battlestar.galactica
If we only wanted the SVCD version, we'd use this
battlestar.galactica.*svcd*
Another option might be to download the HDTV PDTV or DSRip versions, to do this, use (thanks Rick).
battlestar.galactica.+([ph]dtv|dsrip)
For more on this, make sure to check out the RSS Feed Scanner forums
here.
And, if you want to learn more about regular expressions,
here is a list
of handy resources to get started. RSS Import uses this type of filter called regex (regular expressions).
As you learn these, you can test out your filters here.
Downloading
Once the TV shows your specify are discovered, they will automatically start to download.
You can check out the status as well as all the other information available as you download.
It can take hours for some to download, so many users have a computer on all the time that just downloads in the background with TV shows eagerly waiting to be seen at the end of each day.
As more portable video players hit the market, we expect to see a lot of people then transferring their TV shows to
these devices, right now converting is a little tricky, but if there's enough interest, we'll do a How-To on that as
well.
And there you go, please don't watch Desperate Housewives, that's all we ask, thanks.
Phillip Torrone can be reach via his personal site: http://www.flashenabled.com
Jen, once you download the .torrent file, open it in your bittorrent client. That will initiate the download of the .avi.
F, I had the same problem. I went back and defined all my directories under Options/Files. Then went back and re-enabled the RSS Importer Plugin. That seems to work.
I got the whole thing working on my Debian linux box, BTW. Runs great! Just need some more RSS feeds ... any suggestions?
i have a problem with the Incoming TCP Listen Port 6881 after i run the test i get the error messege "Testing port 6881...NAT Error" what other port number should i use?
i have a problem with the Incoming TCP Listen Port 6881 after i run the test i get the error messege "Testing port 6881...NAT Error" what other port number should i use?
Ok i got the Incoming TCP port problem worked out. I had to open the ports in my router but now i get a messege saying "Warning! Please set a default directory and activate it to use this plugin! Plugin deactivated" whenever i try to use the rss importer. can someone please help me out.
I tried following the directions and had problems similar to those posted by Kathy, Ron, and Ben above with this RSS import plug-in. The directions in the article are pretty useless. Is there a way to just download the torrent file to Azureus directly if you can find it somewhere out there on the web from some torrent site without doing this RSS thingie? All I want to do is download the last episode of Battlestar Galactica (Litmus) because I missed it during its two showings on SciFi channel. Why is this bit torrent thing so darn hard? So many people are doing it, you'd think it would be easy.
When i'm trying to download all that i can see is a text which says: Warning! Please set a default directory and activate it to use this plugin! and stuff like that. What should I do?! Can anyone help?? :)
If you get the "Please set a default directory..." message try this. (I had that happen and it took me a couple hours to find the fix)
Click on -> Tools -> Options -> Files
Then Check the "Save to default Data Directory" And type in a place you want to store the files. Good Luck!
OK, I tried to set this up, and have several issues to take care of. First I get the NAT error. What can I do to fix this? Second, I got the default directory error, but found the instructions above and fixed it. Now all I get when I run the plugin is "plugin deactiaved"
What should I do?
Thanks!
Alright, I did some research on fixing the NAT problem, but ended up just fixing it myself after some fiddling around. I'm using Win XP Pro SP2 with the build in firewall, so this only applies to those like me.
What you have to do is View Network Connections in the control panel, single-click on your LAN or Wireless connection, click where it says Change Windows Firewall settings (on left hand menu), Uncheck "Don't Allow Exceptions", click on the Exceptions tab, click on Add Port, name it whatever you want, port = 6881, leave it as TCP, then click OK.
Voila, you're all set.
Alright, I did some research on fixing the NAT problem, but ended up just fixing it myself after some fiddling around. I'm using Win XP Pro SP2 with the build in firewall, so this only applies to those like me.
What you have to do is View Network Connections in the control panel, single-click on your LAN or Wireless connection, click where it says Change Windows Firewall settings (on left hand menu), Uncheck "Don't Allow Exceptions", click on the Exceptions tab, click on Add Port, name it whatever you want, port = 6881, leave it as TCP, then click OK.
Voila, you're all set.
Hey Guys, can anyone help? I need to know how to protect my identity on the web when using Azureus so that I don't get sent to jail or something for downloading movies and TV etc?
Any help would be great.
You can view an updated list of working Bit Torrent sites here: http://www.jodm.com/jodm/index.php?p=43
Really sorry for the newb question but how do I create a default directory
Click on -> Tools -> Options -> Files
Then Check the "Save to default Data Directory" And type in a place you want to store the files. Good Luck!
^from above.
i have my default folder set...yet it still tells me its not...does anyone know how to get this working?
ok i got a problem it keeps downloading shows not on my filter
hello,
I download BitTorrent. now I am looking for BitTorrent plugins that my download speed will be fast. But is this "RSS" only for Azureus. Can i pick it for BitTorrent as well? Or if not which will be best for BitTorrent?
this rss importer doesn't seem to work very well for me, please check if i've done the settings correcly:
in the "rss channel to import" field: http://www.rokanova.com/rss.php
And the "filter for channel entries": Family.Guy.+([ph]dtv|dsrip);Numb3rs.+([ph]dtv|dsrip);Andromeda.+([ph]dtv|dsrip);stargate.+([ph]dtv|dsrip)
Ok, I downloaded everything and I have placed the RSS Import folder (the one with the five .jar files) in C:Program FilesAzureusplugins, why won't it show up under plugins in the options sidebar in Azureus. Am I not following the instructions? Does anyone else have this problem? What's the solution?
"So far, there hasnt been a legal case that we know of where broadcast TV networks have stopped or wanted to stop BitTorrent downloads of their shows, but we have heard that pay networks like Showtime, HBO, etc. have sent letters to some individuals for sharing recorded TV."
Bad bad news, http://www.afterdawn.com/news/archive/6433.cfm, the MPAA of all organisations have decided to go after a number of sites and the people responsible for bringing us hard to get tv shows in torrent form. Personally this cuts me off from my only way to see these shows, and also somewhat makes your method useless (because there are very few shows still being uploaded).
Oh well.
Are there any RSS addresses currently working? Or have they all gone offline?
gr8 article. u forgot to mention tht torrent as a technology is a better alternative only for newer downloads - thts coz u`ll hardly find enuff seeds or seeders for older stuph!
as for the video conversion, tutorial, i`m looking forth to tht! i can help out if u want any details on video compression using virtualdub - divx & xvid encoding.
http://www.mininova.org has an rss feed that is still up (lots of tv torrents on there). Other than that I don't know.
I have just looked into tvtorrents.com, you have to sign up adn share back to remain a member (meaning you can't just delete your downloads untill u have a 1-1 share ratio). And it looks like you need to donate for the rss feed however, it looks as if its filtered by them, and they dont say an amount you have to donate.
Alright, i downloaded this program knowing you can get an ipod hack to convert your ipod photo into a video and play tv shows on it...now that i cant find it im having trouble....so my question is, does anyone know where to find that hack?
The SwarmTvProject http://swarmtv.etv.cx/ would like to point out the following rss feed..
http://www.legaltorrents.com/rss.xml
All nice and legal !
http://www.legaltorrents.com/rss.xml
It looks like the portion on the bill that would make fast-forwarding of commercials illegal has been removed: http://www.wired.com/news/politics/0,1283,65796,00.html?tw=wn_tophead_7
You may want to wait to see how (and if) Tivo implements the FF Ad's before you stop using your Tivos. Maybe it won't be as obnoxious as people are fearing.
The popup ads probably wont be bad. The fact that TiVo has sold you out is the problem. I have a spare pc box in a cupboard somewhere which I will be digging out. And I never watch TV.
Why don't you just setup a MythTV box and enjoy the same functionality, but with more freedom?
sweet! i just set this up on my server looking for old 'whose line is it anyways?' episodes
This is great and all until you get a nice cease and desist letter.
you'll also need to start your collections from scratch... which kinda stinks.
Well, I guess you could just cancel the downloads of the ones you already have.
nevermind. *lays down*
I have had this set up for a couple weeks now and don't find it terribly useful. The feeds really don't have a whole lot of shows that I'm interested in. And the torrent sites that do have a lot of the shows I want, don't have RSS feeds. It's a good setup in theory, but util there are more feeds available, you are forced to watch West Wing and CSI:Miami.
I don't see how this replaces tivo? Can you pause live tv? Get shows only with Jeri Ryan with it? I would think home grown PVRs would be a better idea if the FF ad thing bothers you that much. That way at least you have the functionality of a PVR.
Zak also raises an interesting point. As it was posted in the last podcast comments..
"BitTorrent is not safe, anyone can see your IP and that studios like MGM and Paramount are already going after people who download tv shows like Stargate Atlantis & Star Trek Enterprise.
Skeptical?
Check out the postings here:
http://yro.slashdot.org/comments.pl?sid=128409&threshold=1&c
And even here:
http://www.pvrblog.com/pvr/2004/11/how_to_never_mi.html
"
Your guide doesn't work. I know this isn't tech support, but the plugins don't appear in the program. You might want to take a look and make sure you guys didn't skip a step or something.
I've been using this setup for a number of months with great success. I'm a UK resident so we have to wait months for some of our favourite TV shows to get broadcast over here. So while this may be no use to Americans, some of us in other countries do find this rather neat.
I've been using RSSFeed (http://rssfeed.zyth.net) instead of the suggested plugins as it offers greater configuration and easier creation of filters.
There is a problem with the latest version of the plugin that stop you from being able to add filters or rss feeds. To over come the problem install an older version add at least one feed and filter, then install the newest version keeping the old options file intact. Hope this helps.
The Intellectual Property Protection Act never attempted to make skipping ads and other promotional announcements illegal when viewing movies. It did attempt to make skipping ads illegal when recording movies, or at least that's the way I read the bill before that section was taken out of it.
Ok, this is going to sound really stupid but after you type in the name of the show you are looking for, how do you initiate the search? I did not see any way to do this and the guide didn't say (or maybe I missed it). Sorry for the newb question.
This article's "battlestar.galactica.*[dsrip].*[hpdtv]" example for snagging "the HDTV PDTV or DSRip versions" is actually totally wrong. (Using parenthesis instead of brackets would have been closer.)
To match HDTV, PDTV, or DSRIP, I'd use "battlestar.galactica.+([ph]dtv|dsrip)"
Marty -
If you are using RSS Importer, then you have to set the plug-in to Active by checking the "Activate RSS Importer" check box in Preferences for the plug-in. Then it will load on the interval you set in the options for the plug-in.
lenn
I use this to supplement Tivo. I only have a single-tuner one, which is a problem since certain timeslots seem to have two or three programs I want to watch (mainly on Tuesdays). Before, I Tivo'd one and then had to watch the ohter live or record it on my VCR. Now I can have it sitting magically on my computer the next morning. That's progress.
Buttress is another great java broadcatching app
http://buttress.sailes.co.uk/
This rocks! I found out about this a couple of weeks ago, and I must say that I am very impressed. It's really nice to be able to watch shows in HD without having to spend the many hundreds of dollars that DISH wants for an HD receiver. Thank you LazyWeb!!
Broadcatching is cool, except it relies on the premise that the show you want to see was recorded and seeded by someone out in BitTorrent-land. I love the idea and its a great way to grab an episode I may have missed once and a while, and it's free. But for a couple hundred bucks I set up SnapStream on my PC and have all the TV I'll ever need.
One problem remaining with this technique is managing duplicates. Tivo is really good at knowing what shows are duplicates or re-runs.
Back to the eternal debate between text based matching versus metadata based processing.
nice idea. Hopefully to get it to work.
This is fine, but for those who want a settop box with tivo functionality, show sharing, commercial skip without touching a button, and no ads, go to ebay and pick up a replaytv 5000 series. Replaytv has been around as long as tivo, but has never sold its service to 3rd party advertizers. After you pick up your replaytv (the best one is the 5000 series, eg. 5040, 5080, etc., not the 5500 series) go to poopli.com and join the community.