Results 1 to 6 of 6

Thread: Search an entire state for homes - Is a Web Crawler the right way to go?

  1. #1
    Join Date
    Aug 2013
    Beans
    209

    Search an entire state for homes - Is a Web Crawler the right way to go?

    I'm getting close to retiring and am going to move to Florida. I don't really care where, just to the state somewhere. Currently, there is no way I know of to search the entire state for homes. The best you can do is area by area. I don't know much about web crawlers but I was wondering if I could make one that would search each area for the type of home I want and then have all the results come back to me.

    Is a web crawler the best thing to use for something like this?

    Please don't recommend to me to just narrow down my search to specific areas. I could do that IF I WANTED TO but I DON'T.

  2. #2
    Join Date
    Sep 2006
    Location
    France.
    Beans
    Hidden!
    Distro
    Ubuntu Mate 16.04 Xenial Xerus

    Re: Search an entire state for homes - Is a Web Crawler the right way to go?

    Moved to Cafe, not a support request.
    | My old and mostly abandoned blog |
    Linux user #413984 ; Ubuntu user #178
    J'aime les fraises.
    Nighty night me lovelies!

    | Reinstalling Ubuntu ? Please check this bug first ! |
    | Using a ppa ? Please install ppa-purge from universe, you may need it should you want to revert packages back |
    | No support requests / username changes by PM, thanks. |
    [SIGPIC][/SIGPIC]

  3. #3
    Join Date
    Jun 2013
    Beans
    17

    Re: Search an entire state for homes - Is a Web Crawler the right way to go?

    Quote Originally Posted by PopsTheSailor View Post
    I'm getting close to retiring and am going to move to Florida. I don't really care where, just to the state somewhere. Currently, there is no way I know of to search the entire state for homes. The best you can do is area by area. I don't know much about web crawlers but I was wondering if I could make one that would search each area for the type of home I want and then have all the results come back to me.

    Is a web crawler the best thing to use for something like this?

    Please don't recommend to me to just narrow down my search to specific areas. I could do that IF I WANTED TO but I DON'T.
    My parents were thinking of moving to Florida, because the homes were so cheap. Unfortunately distance played a big factor. I think they used realtor.com a lot. Easiest way to search is type in a city and state, e.g. tampa, fl, and a max price, like $20,000 and it will show you properties close to it. There are also other sites such as zillow.com. Good luck!

  4. #4
    Join Date
    Apr 2014
    Beans
    5

    Re: Search an entire state for homes - Is a Web Crawler the right way to go?

    This is a good idea if you have the time to make it. You could write a small crawler in Python, collect as much data as possible (text, images).
    Then from this database of pages, you could extract terms such as price $, location etc - or simply browse by image

  5. #5
    Join Date
    Aug 2013
    Beans
    209

    Re: Search an entire state for homes - Is a Web Crawler the right way to go?

    I'm wondering if I don't really understand what a web crawler can do. I want to filter what comes back, not bring everything back and then filter. It would be MUCH LESS date to download. Is that possible? I just don't know anything about web crawlers.

  6. #6
    Join Date
    Sep 2005
    Location
    Dallas, Tx
    Beans
    266
    Distro
    Xubuntu 12.10 Quantal Quetzal

    Re: Search an entire state for homes - Is a Web Crawler the right way to go?

    You have to bring back the data to filter it...there's no magical way you can run an algorithm on data you don't have. Typically you capture the page as an XML and mine the xml for the data you want. If you don't know how to program at all, learning and getting this u to date could take weeks.

    ...But no real need to use a web crawler for this...

    Alot of these real estate sites have APIs which allow you direct access to their listing if you register with them as a developer. You can use the api to direct pull what you want.

    Just be aware that no matter what path you do gown is most likely to take you much, much longer than just going and searching every major city in florida.
    Desktop
    Antec 300 Illusion, Athlon II X4 640, 8GB DDR3 1333, EarthWatts430W, Radeon HD 6670 1GB
    Laptop
    Lenovo Thinkpad Z61T, T7200@2.00GHz, 2GB DDR2, 100gb HD, TPM Encryption

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •