Results 1 to 8 of 8

Thread: how to wget range.... say 1to 30?

  1. #1
    Join Date
    Aug 2007
    Beans
    134

    Question how to wget range.... say 1to 30?

    Hi,

    I read it once and cannot find it again...

    i want to download pages 1 to 30
    i.e. wget www.whatever.com/folder/1.html where the 1 is replaced [1 to 30]

    thanks for this...

    p.s. go ubuntu...

  2. #2
    Join Date
    Oct 2004
    Location
    Pennsylvania
    Beans
    1,698

    Re: how to wget range.... say 1to 30?

    Quote Originally Posted by Hopeless View Post
    Hi,

    I read it once and cannot find it again...

    i want to download pages 1 to 30
    i.e. wget www.whatever.com/folder/1.html where the 1 is replaced [1 to 30]

    thanks for this...

    p.s. go ubuntu...
    Code:
    $ for i in $(seq 30); do echo "http://www.example.com/page$i.html" ; done | wget -i -
    The URLs are created in a loop and piped to wget.

  3. #3
    Join Date
    Mar 2007
    Location
    Your Closet
    Beans
    380
    Distro
    Ubuntu 10.04 Lucid Lynx

    Re: how to wget range.... say 1to 30?

    I think this is the shortest possible answer.
    Code:
    wget www.whatever.com/folder/{1..30}.html
    You could also use the return value from wget so you don't have to specify an ending value.
    Code:
    i=1; while wget "www.whatever.com/folder/$((i++)).htm" do :; done
    Last edited by bashologist; August 29th, 2007 at 09:06 PM.
    ...

  4. #4
    Join Date
    Aug 2007
    Beans
    134

    Exclamation Re: how to wget range.... say 1to 30?

    hi,

    i'm having problems...

    i type in:

    Code:
    wget --mirror -U Mozilla 'www.website.com/forum/forum.asp?FORUM_ID=35&sortfield=lastpost&sortorder=desc&whichpage={1..52}'
    18:03:36 (13.08 KB/s) - `www.website.com/forum/forum.asp?FORUM_ID=35&sortfield=lastpost&sortorder =desc&whichpage={1..52}' saved [107250/107250]

    Removing www.website.com/forum/forum.asp?FORUM_ID=35&sortfield=lastpost&sortorder =desc&whichpage={1..52} since it should be rejected.

    my .wgetrc file:

    Code:
    robots = off
    randomwait = on
    reject =  *CAT_ID=2,*CAT_ID=12,*FORUM_ID=11,*FORUM_ID=34,*FORUM_ID=44,*FORUM_ID=51,*FORUM_ID=53,*FORUM_ID=5,*FORUM_ID=57,*FORUM_ID=6,*FORUM_ID=7,button_logout.gif,catbg.gif,fishtank.swf,footerback.gif,headbg.gif,headerback.gif,icon_bar.gif,icon_blank.gif,icon_folder_closed_topic.gif,icon_folder.gif,icon_folder_hot.gif,icon_folder_locked2.gif,icon_folder_locked_f.gif,icon_folder_new.gif,icon_folder_new_topic2.gif,icon_folder_new_topic.gif,icon_folder_open.gif,icon_folder_open_topic.gif,icon_folder_sticky.gif,icon_folder_sticky_locked.gif,icon_go_up.gif,icon_group.gif,icon_lastpost.gif,icon_mi_10.gif,icon_mi_11.gif,icon_mi_13.gif,icon_mi_14.gif,icon_mi_1.gif,icon_mi_2.gif,icon_mi_4.gif,icon_mi_6.gif,icon_mi_7.gif,icon_mi_8.gif,icon_mi_9.gif,icon_minus.gif,icon_navmenu_guestbook.gif,icon_navmenu_help.gif,icon_navmenu_home.gif,icon_navmenu_member.gif,icon_navmenu_news.gif,icon_navmenu_search.gif,icon_navmenu_url.gif,icon_pencil.gif
    accept = topic.asp*,forum.asp* -A gif,jpg,pdf
    load_cookies = ~/.mozilla/firefox/xjg5nmlx.default/cookies.txt
    limit-rate = 20K 
    no_parent = on
    what im i doing wrong?

    thanks...

  5. #5
    Join Date
    Mar 2007
    Location
    Your Closet
    Beans
    380
    Distro
    Ubuntu 10.04 Lucid Lynx

    Re: how to wget range.... say 1to 30?

    If you're gonna test your code you should use echo:
    Code:
    echo 'www.website.com/forum/forum.asp?FORUM_ID=35&sortfield=lastpost&sortorder=desc&whichpage='{1..10}
    Everything inside single quotes is literal; Nothing will expand, you can't even enter a newline in there easily.

    Here's another example with single quotes:
    Code:
    echo 'blah'{1..2}'blah'
    ...

  6. #6
    Join Date
    Aug 2007
    Beans
    134

    Re: how to wget range.... say 1to 30?

    sweet !!

    + thanks for the echo tip !!!!

    byyyyyye 4 now....

  7. #7
    Join Date
    Aug 2007
    Beans
    134

    Question Re: how to wget range.... say 1to 30?

    sorry me again...

    it saves topic.asp ok but not the forum.asp pages;


    i.e. i don't see these pages in the folder yet....
    forum.asp?FORUM_ID=35&sortfield=lastpost&sortorder =desc&whichpage=1
    forum.asp?FORUM_ID=35&sortfield=lastpost&sortorder =desc&whichpage=2
    etc....

    i appreciate your help...
    Last edited by Hopeless; August 30th, 2007 at 11:29 AM.

  8. #8
    Join Date
    Aug 2011
    Beans
    1

    Post Re: how to wget range.... say 1to 30?

    Uhm.. are you all crzy?

    just use "wget www.whatever.com/folder/{1..30}.html"


    Cheers!°


    ----------------------------edit----------------------------

    Daym didnt see second Post that clearly.. nevermind!

    cheers!
    Last edited by Bend0r; August 30th, 2011 at 10:48 PM.

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •