Sunday, June 2, 2013

Craigslist Bot

I coded a Craigslist bot that scrapes all the new listings for a specific category you're looking in and auto-emails each of the vendors.  It was an interesting exercise in crawling that operates similarly to my C search engine here:

Why? I'm tired of browsing all the Craigslist postings and emailing each person. I thought it'd be a fun solution to auto-email the vendors, as I was going to email the same thing anyway.

In Python, BeautifulSoup parses the page for the Craigslist links. It saves them to a postgres database so that it won't duplicate-email the same URL.  It also "intelligently" remembers the name of the Craigslist posting so if somebody creates a duplicate with the same name to readvertise, it won't double-email them.

I've left it on overnight and it makes browsing Craigslist a lot easier.: