Chris Towle

Computational Craftsman

Car Sales Web Crawler

Purpose: To create a web crawler to parse three unique sites for relevant data

I was tasked with creating a web crawler to assist a car dealership. Using provided log-in credentials, the crawler would gather the target data and dump it to a central database.

Technologies Used

Languages: Python, MySQL

Hardware:
Dual Core Linux PC.

Software Technologies:
Used threads to run parallel crawlers, with roughly 9900 cars (full spec + images) crawled per hour. With an eight core computer this could be bumped up to a theoretical 39600 cars per hour.
A huge improvement in performance was achieved by using an alternative Python interpreter (PyPy), this alone gave a 5x performance gain.
Numerous techniques were used to get past each site's quirks, such as Header Spoofing, Cookie Spoofing, and Session Time overrides.

LEMP Stack Web Server

Purpose: A home server to aid in web based projects

Technology: LEMP stack (Linux Nginx MySQL PHP)

To allow remote system access and home automation, I created a locally hosted web server. This system is capable of ftp, web hosting, SSH control and acts as a general purpose lab for testing.
You are currently communicating with it.

Flash Projects

Here are a few working examples of my previous work

Three Stone Studios, Interactive Booklet: Layered gradients were used to create the illusion of bending and depth. The feel of the page movement was an important aspect for my client.

No Homes for Gnomes: Was developed under a 48 hour time restraint at the Toronto Indie Jam (T.O.Jam). I acted as an AS3 programmer and game play designer. This version has not been modified since the 48 hour release.