About our crawler

For dealership web teams and website providers who have seen us in their logs.

Powersports Watch keeps a daily record of the inventory powersports and motorsports dealerships publish on their websites, so dealers can see how their market moves: what is on each lot, how long it has been there, and when prices change. To do that, a small crawler visits dealership websites once each morning.

How it identifies itself

Every request carries this User-Agent, so it can be told apart from a person:

Mozilla/5.0 (compatible; PowersportsWatch/1.0; +private inventory tracker; contact: [email protected])

What it reads

• The public inventory pages a shopper would browse: inventory listings, inventory sitemaps and the pages for individual units.
• On a few website platforms, the same public inventory search the site's own pages use.

It never signs in, never submits enquiries, quotes or credit applications, and never reaches anything behind a login. It records units, prices and dates, not anyone's personal information.

When, and how much

• Once a day, starting at 6:00 a.m. Mountain time. A run usually finishes within half an hour.
• At most one request every 0.6 seconds to any one website, one at a time.
• Where a site has a page per unit, each unit's page is read about every third morning, plus new units on the day they appear. The inventory list itself is read daily.
• If a site answers with an error, the crawler waits and tries again at most three times, and waits much longer (30 seconds or more) when told it is asking too often. A site that keeps failing is visited less, not more.

Questions, or want us to slow down or stop?

Write to [email protected] or use our contact form and tell us the website address. We will reply within two business days.