To scrape Amazon product data, collect the details you need from product pages and save them in a table. Names, prices, ratings and seller information can support competitor research or a product watchlist. The important part is keeping the exact product and offer clear.
For example, a renewed phone in one condition is a different offer from the same phone in another condition. Colors, sizes, pack quantities and sellers can also change the price. Comparing the nearest numbers on two pages can give you the wrong result.
I built the Python example around Unwrangle, a third-party service that returns Amazon product data. It accepts US product URLs, saves selected fields to CSV and keeps the full response for checking. A second short program exports separate seller offers into their own rows.
Use your own account for live requests, or export a saved response without another call. Begin with a small list of familiar products before using the data for price alerts, catalog research or a recurring competitor report.
Which Amazon Product Details Should You Collect?
Define the product and the offer separately
Keep the product URL and the selected ASIN, the identifier commonly found in an Amazon /dp/ URL. Also preserve the selected variation and seller information when the source provides them. A parent product family should not collapse several sizes or colors into one price history.
For a basic comparison, start with title, brand, price, currency and source URL. Add ratings, specifications or images when the analysis needs them. A list of every possible field can distract from whether the actual price and identity are correct.
| Question | Data to keep | Common misleading shortcut |
|---|---|---|
| Is this competitor cheaper? | Matching variant, pack, seller and price conditions | Comparing similar titles only |
| What does the brand sell? | Product identity, attributes and variants | Treating one selected option as the entire family |
| Has a price changed? | The same offer context at another time | Comparing different sellers as one series |
| What do reviewers discuss? | Review source, sample and coverage | Calling the detail-page sample all reviews |
I would use that field list when comparing vendors, rather than accept a general claim that a service handles Amazon.
Choose product details or listing discovery
A detail request starts from a known product. A search collector starts from a query or category and needs pagination and URL discovery. Those are different stages, even if a platform offers both.
If you already have twenty ASINs to watch, direct product requests may be sufficient. If you are researching an assortment, establish how candidates enter the list and what the search source covers. A detail exporter cannot retrieve products it was never given.
Keep the discovery query or category with the candidate URL. Later, that lets you distinguish a real assortment gap from a search filter or page range that simply excluded the item.
Which Amazon Product Scraping Method Fits Your Project?
Managed APIs reduce retrieval work
I read Unwrangle's Amazon detail documentation to check the URL-based request and returned detail object. I selected named fields from that object and kept the original response for optional data.
I also reviewed SerpApi's Amazon product API as an alternative. Its parameters and schema differ, so I would compare output samples, supported marketplaces and usage terms before adapting an export written for another provider.
A managed service saves you the work of fetching pages and extracting fields. Try it with products you actually need. Check separately if you want all seller offers, a full review history or information for a particular delivery location.
A scraper you run yourself gives you more to maintain
For direct collection, inspect the actual HTML and rendered page before choosing selectors or embedded data. Layouts and representations vary. A generic JSON-LD parser only helps if the document contains the Product objects and fields your project needs.
An example that reads a known DOM element can be useful for a supported page, but it needs testing against the product types in your sample. Do not assume the first currency-looking string is the purchase price. Reference prices, installments and promotional amounts can be nearby.
I would choose this route for a specific need, such as field or client control the managed option lacks. For a small job, consider the time spent maintaining it alongside hosting and connection costs. A short request script is only one part of a recurring dataset.
How Does Scraping Amazon Product Data With Python Work?
Prepare a US URL or an ASIN watchlist
Install Requests with python -m pip install requests and set your own UNWRANGLE_API_KEY. The script below accepts Amazon US product URLs, a file containing those URLs, or a saved response. Its domain check is for amazon.com; another marketplace needs documented support and an appropriate adaptation.
This example uses the third-party amazon_detail endpoint. It does not use an Amazon-owned seller API, and it does not need a separate proxy plan in front of the hosted service. A live request consumes the allowance of your own provider account.
Save the collector and selected-field exporter
Save this as amazon_product.py. It retains the product identity, selected variation, seller, price fields and buying offers alongside collection status and time. Nested values stay JSON text; the complete response is saved for inspection.
"""Collect product rows with source context; --json makes no network request."""
import argparse, csv, hashlib, json, os
from datetime import datetime, timezone
from pathlib import Path
from urllib.parse import urlparse
import requests
SITE = "amazon"
PRODUCT_FIELDS = ['asin', 'name', 'brand', 'price', 'price_reduced', 'currency', 'seller_name', 'delivery_zipcode', 'variant', 'buying_offers', 'url']
META_FIELDS = ["status", "source_kind", "requested_url", "requested_store",
"retrieved_at", "exported_at", "error"]
def now():
return datetime.now(timezone.utc).isoformat()
def cell(value):
return json.dumps(value, ensure_ascii=False) if isinstance(value, (dict, list)) else value
def product_row(bundle):
meta = bundle.get("_collection", {})
payload = bundle.get("response", bundle)
row = {k: meta.get(k) for k in META_FIELDS}
row.update(source_kind=meta.get("source_kind", "undated_saved_response"),
requested_url=meta.get("requested_url", payload.get("url")),
exported_at=now(), status="ok", error="")
detail = payload.get("detail")
if payload.get("success") is not True or not isinstance(detail, dict) or not detail:
row.update(status="failed", error="No successful product detail object")
else:
row.update({k: cell(detail.get(k)) for k in PRODUCT_FIELDS})
return row
def collect(url, store, folder):
host = (urlparse(url).hostname or "").lower()
domain = SITE + ".com"
if urlparse(url).scheme != "https" or not (host == domain or host.endswith("." + domain)):
raise ValueError("Use an HTTPS product URL on " + domain)
key = os.getenv("UNWRANGLE_API_KEY")
if not key:
raise ValueError("Set UNWRANGLE_API_KEY before a live request")
params = {"platform": SITE + "_detail", "url": url, "api_key": key}
if SITE == "target":
if not store:
raise ValueError("Target live requests require --store")
params["store_no"] = store
try:
response = requests.get("https://data.unwrangle.com/api/getter/", params=params, timeout=90)
except requests.RequestException as error:
raise ValueError(type(error).__name__) from None
if not response.ok:
raise ValueError("API HTTP " + str(response.status_code))
try:
payload = response.json()
except ValueError:
raise ValueError("Non-JSON API response") from None
bundle = {"_collection": {"source_kind": "live_api_response", "requested_url": url,
"requested_store": store if SITE == "target" else None, "retrieved_at": now()},
"response": payload}
safe = json.dumps(bundle, ensure_ascii=False, indent=2).replace(key, "[redacted]")
folder.mkdir(parents=True, exist_ok=True)
name = hashlib.sha256((url + str(store)).encode()).hexdigest()[:16] + ".json"
(folder / name).write_text(safe, encoding="utf-8")
return json.loads(safe)
def main():
parser = argparse.ArgumentParser()
source = parser.add_mutually_exclusive_group(required=True)
source.add_argument("--url")
source.add_argument("--urls", type=Path, help="UTF-8 file, one URL per line")
source.add_argument("--json", type=Path, help="A saved response or wrapped collection")
parser.add_argument("--store")
parser.add_argument("--output", type=Path, default=Path(SITE + ".csv"))
args = parser.parse_args()
if args.json:
bundles = [json.loads(args.json.read_text(encoding="utf-8"))]
rows = [product_row(b) for b in bundles]
else:
urls = args.urls.read_text(encoding="utf-8-sig").splitlines() if args.urls else [args.url]
urls = list(dict.fromkeys(u.strip() for u in urls if u.strip()))
if not urls:
parser.error("The input URL list is empty")
if not os.getenv("UNWRANGLE_API_KEY") or (SITE == "target" and not args.store):
parser.error("Set UNWRANGLE_API_KEY; Target also requires --store")
rows = []
for url in urls:
try:
rows.append(product_row(collect(url, args.store, args.output.with_suffix(".raw"))))
except ValueError as error:
rows.append({"status": "failed", "requested_url": url,
"requested_store": args.store if SITE == "target" else None,
"exported_at": now(), "error": str(error)})
with args.output.open("w", encoding="utf-8-sig", newline="") as handle:
writer = csv.DictWriter(handle, fieldnames=META_FIELDS + PRODUCT_FIELDS)
writer.writeheader()
writer.writerows(rows)
failed = sum(row["status"] != "ok" for row in rows)
print(str(len(rows)) + " rows; " + str(failed) + " failed; " + str(args.output))
if failed:
raise SystemExit(1)
if __name__ == "__main__":
main()
Run one product, using the provider's documentation URL as an input example:
python amazon_product.py --url "https://www.amazon.com/Apple-iPhone-13-128GB-Blue/dp/B09LNX6KQS/" --output amazon-run1.csv
The live path creates a CSV and an amazon-run1.raw directory. Keep a new output name for each collection period. A failed product remains a failed row; a blank price does not become zero or yesterday's value.
Re-export without pretending the data is fresh
Use an actual saved JSON filename from the raw directory:
python amazon_product.py --json amazon-run1.raw/your-saved-file.json --output amazon-reexport.csv
This mode makes no request. Saved collection metadata stays attached; an unwrapped response without a collection time keeps that time blank. A new export timestamp does not establish a new Amazon observation.
I ran the exporter against the provider's public example for ASIN B09LNX6KQS. The output kept the selected Blue/128GB/Unlocked variation, seller information and buying-offer array. I checked that saved response offline; I did not measure a current Amazon price or access-success rate.
Why Can the Scraped Amazon Price Look Wrong?
Compare offer condition before choosing a price
I found a clear example in the documented Amazon response: the renewed iPhone had separate buying offers with different condition labels and sellers. I would keep those offers separate rather than pick the lowest number as the product's universally correct price.
| Value in the documentation sample | What it describes | What not to infer |
|---|---|---|
ASIN B09LNX6KQS |
The selected product identity | Every variation in the family |
377.00, Refurbished - Excellent |
One listed condition/seller offer | The price of every renewed unit |
323.36, Refurbished - Good |
Another listed condition/seller offer | A price drop for the first condition |
Delivery ZIP 90001 |
The sample's returned destination context | Your own delivery destination |
I use those documentation values to show why I keep product identity, condition and seller beside the amount; they are not today's offers. I would also keep purchase price, reference price and conditional offers separate.
Export buying offers as their own table when that is your analysis
The main CSV keeps buying_offers as JSON so it does not silently discard conditions. For a price-comparison sheet, make one row per offer instead. Save the following as amazon_offers.py and supply a saved response from the first program:
"""Make one row per documented Amazon buying offer from a saved response."""
import csv, json, sys
from pathlib import Path
bundle = json.loads(Path(sys.argv[1]).read_text(encoding="utf-8"))
payload = bundle.get("response", bundle)
detail = payload["detail"]
fields = ["asin", "condition", "seller", "price", "currency"]
with Path(sys.argv[2]).open("w", encoding="utf-8-sig", newline="") as handle:
writer = csv.DictWriter(handle, fieldnames=fields)
writer.writeheader()
for offer in detail.get("buying_offers", []):
writer.writerow({"asin": detail.get("asin"), "condition": offer.get("offer_type"),
"seller": offer.get("seller"), "price": offer.get("price"),
"currency": detail.get("currency")})
python amazon_offers.py amazon-run1.raw/your-saved-file.json amazon-offers.csv
When I ran the documentation sample offline, I got two offer rows with the condition and seller kept separate. Do not pair every variation with every offer unless the response shows that they belong together. That would create product-and-price combinations the source never supplied.
A regional proxy setting does not select Amazon's delivery destination. For location-sensitive work, choose a source with the required destination controls and inspect the returned evidence. If the response lacks a usable price or condition, leave the observation incomplete rather than making it look comparable.
PuraRoute for Ongoing Amazon Product Research
For businesses using their own Amazon scraper, PuraRoute provides dynamic residential proxies as well as static residential and static native proxies. That gives you options for repeated product checks and longer-running work with an assigned IP. Choose the service around the marketplaces, products and price comparisons your research needs, then test it on a small real job.
Support the Amazon research you actually need
Dynamic residential proxies can suit repeated checks of a product list, with available location choices for the markets you follow. Static residential proxies are an option when your tool needs an assigned IP for longer. For projects that need a more consistent local residential connection, static native proxies are another option. For an Amazon price project, try the exact ASINs and offers you want to compare. Check that the seller, condition and price fields are useful before adding more products.
Review the spend as your product list grows
PuraRoute's traffic-usage information helps you see what ongoing collection costs. Compare that cost with usable product and offer records from your sample. Then choose how often the list really needs checking. A weekly catalog review and a price-alert service have different needs, so the plan should follow the work rather than a larger package being the default purchase.
How Do You Save Amazon Product Data to CSV Repeatedly?
Run a small URL list with explicit failures
Create amazon-urls.txt with one Amazon US product URL per line. Use one URL for each selected ASIN you intend to observe, then run:
python amazon_product.py --urls amazon-urls.txt --output amazon-watchlist-run1.csv
The program removes exact duplicate lines and runs sequentially. Each intended URL receives a row, so a failed job cannot quietly shrink the dataset. Inspect status before calculating a price change. Raw responses are retained for successful HTTP/JSON retrievals, including a provider response that reports failure.
For the next collection period, use a new output name. Match selected ASIN, condition, seller, currency and delivery context before comparing amounts. If those change, retain the distinction; it is not necessarily the same seller changing the price of the same offer.
Add search discovery when the question starts with an assortment
For a fixed watchlist, explicit URLs are enough. For category or keyword research, use a documented discovery path such as Amazon product search and keep its query and page scope with the candidate list.
A search dataset and a detail dataset have separate coverage questions. Did discovery find the relevant ASIN? Did detail retrieval return the required offer? Adding more requests for known URLs does not fix a discovery list that omitted the products you care about.
Choose a frequency that fits the question and source terms. A weekly assortment review is different from a short-lived price alert. Keep source observations, later exports and analysis times separate so reprocessing old JSON cannot create an apparent new market change.
How Do You Use Amazon Data for Prices and Catalog Research?
Compare the same offer at different times
For a price history, keep the selected ASIN, seller, condition and currency beside the amount. A different seller or renewed condition can explain a change that looks like a discount. If you care about delivery to a particular place, keep the available delivery information too.
In that documentation example, I kept the two renewed-phone offers separate because their condition labels and prices differed. I would use the offer export to compare those choices; a report containing only the lowest number would hide the difference.
| Research goal | Useful fields | What the data does not tell you by itself |
|---|---|---|
| Competitor pricing | Product, condition, seller and the chosen price | Why a seller changed an offer |
| Catalog comparison | Product identity, brand, attributes and variants | That every family member has the same price |
| Review research | A clearly identified review source and sample | That a detail-page sample covers all reviews |
| Price alerts | A fresh successful check of the same offer | That an old saved response is today's price |
Expand only after the sample answers the business question
For a fixed watchlist, explicit product URLs may be enough. For an assortment study, check how the search stage finds products and which pages or filters it covers. A product absent from an incomplete discovery list should not be reported as absent from Amazon.
Review unusual changes before acting on them. If prices disappear from several products together, compare the raw responses with the exporter. The issue may be a changed field or incomplete collection rather than a market-wide event.
I would start with the product and offer details the report needs, then add a separate variant, offer or review table when the analysis requires it. This keeps the spreadsheet usable for the people making pricing or catalog decisions, instead of filling it with fields no one knows how to interpret.
Frequently Asked Questions About Scraping Amazon Product Data
Does a product-detail response include every customer review?
Do not assume that it does. A response can contain a rating summary or selected reviews without covering the full review history. If you need that dataset, investigate the provider's review-specific source, pagination and field coverage. Keep the product-detail sample separate from a complete review collection.
Can I use this script for Amazon UK unchanged?
The example checks for Amazon US URLs. A different marketplace needs the provider's supported input and an appropriate update to the script's domain check. Confirm the returned currency and market as well. Relabeling the output or changing only a spreadsheet heading would not establish UK collection.
Are arrays in the CSV lost?
No. The example serializes selected nested objects and arrays as JSON text inside cells. That keeps their original relationships available for later processing. If you need one row per variation or seller offer, create a separate linked table instead of expanding unrelated arrays together and accidentally multiplying records.
Do I need pandas for the export?
The example uses Python's built-in CSV and JSON modules with Requests for retrieval. Pandas can help with later analysis but is not required for the initial one-product export. Start with a file you can inspect before adding more transformation tools or a scheduled reporting pipeline.
Can a public price tell me another seller's actual sales?
A product price or availability observation does not establish completed sales or revenue. Treat any separate estimated-sales field according to its source and definition. Do not calculate actual business performance from a public product-page snapshot as though the transaction history had been observed.
