Deep Scan
- Thomas Timbul
There isn’t an official API to quickly get all products from a storefront, so scanning with the Amazon Browser is an automated version of what you’d otherwise have to pay a person long hours to do: Browsing a store or search results page by page, collecting all the ASINs and basic data, before feeding them in to Price Checker for analysis.
Deep scan is enabled by default - see below for cancelling it.
We developed our deep scan in order to overcome the primary Amazon imposed limitations:
Amazon Limitations
Search results on Amazon are limited to either 400 pages or 2400 products. In some browse trees there are 24 results per page, others have 16 or 48.
This means that if you were to just analyse the front page of a certain store, you’d at most only get the first 2400 to 9600 items. Frustrating and limiting on storefronts that can have 10k+ products at all times.
Popularity Considerations
In some cases an argument can be made that stores may carry items that aren’t actually very popular. If you are mostly looking for fast sellers instead of long tail items, it may be sufficient to focus on the top 2400 or 9600 items. When sorted by “Featured” these will usually be the most ‘popular’/'best selling'/'Amazon is eager to promote' items.
Duplicates
Some products may be linked to from multiple different departments or categories. This happens when there is an overlapping interest. For instance, baby nappy rash cream may appear in both “Baby” and “Health and Personal Care” and/or “Health and Beauty” categories, which makes perfect sense from a marketing perspective, as potential buyers looking for different goods are incentivised to look.
As an analogy, a supermarket may place barbecue meats in the meat aisle, but additionally in a special summer barbecue seasonal aisle. They will also be placing refreshing drinks there, but which can also be found in the usual drinks aisle.
However, for our deep scan this means that links to an individual product may be encountered more than once. As a result the initial number of results will be over-estimated.
By default PC2 will remove such duplications and only return each unique ASIN once. It is possible to uncheck “remove duplicates” before you start, if you wish to check which products feature in multiple places.
Brands and Departments
These are the main ways in which to refine and subdivide search results into smaller chunks. The resulting search, refined to a particular brand or department, will each have its own limit of up to 400 pages or 2400/9600 products.
Deep scan will attempt to perform this step automatically, effectively bypassing these limitations and producing more thorough results.
However, this comes at the expense, naturally, of a potentially much longer run.
Amazon have stopped reporting accurate product counts on a brand-based deep scan. Price Checker will underreport the estimated number of results.
Cancelling Deep scan
You can cancel deep scan simply by cancelling or closing it when it runs:
If you change your mind, check the “auto-grab” box to run deep scan again:
