PDA

View Full Version : Browser Scrape



Ramthebuffs
September 15th, 2007, 06:50 PM
I'm trying to scrape(php) some things off of espn and I can't get it to work. It works great if I run the script through a browser, but when I cron it nothing. I've tried curl and the troy wolf scraper class and both work in a browser and not alone. Any ideas on what I need to do?

UbuWu
September 16th, 2007, 12:06 AM
I don't know much about php, but with python you could easily do that using beautifulsoup or mechanize.

nanotube
September 16th, 2007, 02:20 AM
I don't know much about php, but with python you could easily do that using beautifulsoup or mechanize.

+1 for python/mechanize. haven't used beautifulsoup myself - but just looked up the project and it seems pretty useful, too.