To scrape Walmart product data, collect information such as item names, prices, sellers and availability, then save it in a table. Start by deciding whether you need online offers or information from particular stores. A national product page may be useful for pricing research without answering a local-pickup question.
The seller matters too. A Walmart product page can show a marketplace seller's offer. If that seller or the selected pack changes, a new price may describe a different purchase from the one you intended to compare.
I built the Python example around Unwrangle, a third-party product-data service. It accepts Walmart US product URLs, writes selected fields to CSV and saves the full response. It also supports a list of products and keeps failed requests visible.
Start with a few familiar items and inspect the results. Once the item, seller and price make sense, you can build a watchlist or recurring report. Local inventory work needs a source that actually supports the stores and fields you want.
Which Walmart Data Does Your Project Need?
Online offers and local inventory are different datasets
An online product record can include title, price, brand, images and seller information. A local inventory question adds a store, fulfillment method and observation time. Decide which one you need before comparing scrapers.
| Business question | What the observation needs | What is insufficient |
|---|---|---|
| What is the displayed online offer? | Item, variant, seller and price | A title with an unlabeled amount |
| Is pickup available nearby? | The actual store and pickup state | National shipping availability |
| Which stores have clearance? | Verified item-store price observations | The same page labeled with several store names |
| How does another seller compare? | Identified seller offers for the same item | Assuming the displayed seller is always Walmart |
I checked Walmart's pricing explanation, which says store and online prices can differ. For local research, I would choose a source that supplies the store information from the start, rather than add a store label afterward.
Choose fields you can use in the final comparison
For a first export, keep item identity, name, price-related fields, currency, seller and URL where returned. Add ratings or specifications if the business question needs them. Review text and individual author information are a separate collection decision, not a necessary addition to every price sheet.
I would choose a tool for the actual report: catalog research and clearance alerts can need different fields. Instead of asking only whether a provider "supports Walmart," I would ask about the item, seller and local fields the report depends on.
Which Walmart Product Scraping Method Should You Use?
Use a documented endpoint for a shorter path to structured data
I read Unwrangle's Walmart detail documentation to check its URL and item-ID inputs. For the example, I chose a full URL and exported the returned detail object, rather than assume one HTML selector would work across Walmart product pages.
A managed service does more of the collection work for you. Test a few of your products and check the price, seller and any required store fields. Compare the cost before expanding the list.
I chose a stable set of product and observation fields for the exporter. I kept selected nested values as JSON text and saved the full response separately, leaving the returned seller or variation details available for a more detailed table later.
A direct collector must inspect the document Walmart returns
If you operate retrieval yourself, check whether the response is the product page or a verification document. A successful status code alone does not establish that product data arrived.
Some Walmart documents expose application data in a __NEXT_DATA__ script; others or later versions may differ. Inspect the actual document and field structure before choosing a parser. A generic Product JSON-LD example is not evidence that all Walmart pages supply that representation.
When I read these scraper discussions, I noticed that users separated missing fields and wrong-location prices from blocked requests. I would diagnose those separately too: fixing access does not prove that a returned price belongs to the store you need.
How Do You Scrape Walmart Product Data With Python?
Prepare the online-offer input and your account
Install Requests with python -m pip install requests, then set your own UNWRANGLE_API_KEY. Start with a Walmart US product URL. This example retrieves online product details through the documented walmart_detail endpoint; it does not configure a local store or claim local-clearance coverage.
The script supports a single URL, a file of URLs or a saved response. Live requests use your provider account's allowance. JSON mode works locally, without another API request or an additional proxy purchase.
Save the collector and context-preserving exporter
Save this as walmart_product.py. It keeps item ID, seller, selected variation, price and the returned stock indicator beside collection status and time. The full response remains available for investigating unusual offers.
"""Collect product rows with source context; --json makes no network request."""
import argparse, csv, hashlib, json, os
from datetime import datetime, timezone
from pathlib import Path
from urllib.parse import urlparse
import requests
SITE = "walmart"
PRODUCT_FIELDS = ['id', 'name', 'price', 'list_price', 'currency', 'seller_name', 'in_stock', 'selected_variant', 'offer_text', 'url']
META_FIELDS = ["status", "source_kind", "requested_url", "requested_store",
"retrieved_at", "exported_at", "error"]
def now():
return datetime.now(timezone.utc).isoformat()
def cell(value):
return json.dumps(value, ensure_ascii=False) if isinstance(value, (dict, list)) else value
def product_row(bundle):
meta = bundle.get("_collection", {})
payload = bundle.get("response", bundle)
row = {k: meta.get(k) for k in META_FIELDS}
row.update(source_kind=meta.get("source_kind", "undated_saved_response"),
requested_url=meta.get("requested_url", payload.get("url")),
exported_at=now(), status="ok", error="")
detail = payload.get("detail")
if payload.get("success") is not True or not isinstance(detail, dict) or not detail:
row.update(status="failed", error="No successful product detail object")
else:
row.update({k: cell(detail.get(k)) for k in PRODUCT_FIELDS})
return row
def collect(url, store, folder):
host = (urlparse(url).hostname or "").lower()
domain = SITE + ".com"
if urlparse(url).scheme != "https" or not (host == domain or host.endswith("." + domain)):
raise ValueError("Use an HTTPS product URL on " + domain)
key = os.getenv("UNWRANGLE_API_KEY")
if not key:
raise ValueError("Set UNWRANGLE_API_KEY before a live request")
params = {"platform": SITE + "_detail", "url": url, "api_key": key}
if SITE == "target":
if not store:
raise ValueError("Target live requests require --store")
params["store_no"] = store
try:
response = requests.get("https://data.unwrangle.com/api/getter/", params=params, timeout=90)
except requests.RequestException as error:
raise ValueError(type(error).__name__) from None
if not response.ok:
raise ValueError("API HTTP " + str(response.status_code))
try:
payload = response.json()
except ValueError:
raise ValueError("Non-JSON API response") from None
bundle = {"_collection": {"source_kind": "live_api_response", "requested_url": url,
"requested_store": store if SITE == "target" else None, "retrieved_at": now()},
"response": payload}
safe = json.dumps(bundle, ensure_ascii=False, indent=2).replace(key, "[redacted]")
folder.mkdir(parents=True, exist_ok=True)
name = hashlib.sha256((url + str(store)).encode()).hexdigest()[:16] + ".json"
(folder / name).write_text(safe, encoding="utf-8")
return json.loads(safe)
def main():
parser = argparse.ArgumentParser()
source = parser.add_mutually_exclusive_group(required=True)
source.add_argument("--url")
source.add_argument("--urls", type=Path, help="UTF-8 file, one URL per line")
source.add_argument("--json", type=Path, help="A saved response or wrapped collection")
parser.add_argument("--store")
parser.add_argument("--output", type=Path, default=Path(SITE + ".csv"))
args = parser.parse_args()
if args.json:
bundles = [json.loads(args.json.read_text(encoding="utf-8"))]
rows = [product_row(b) for b in bundles]
else:
urls = args.urls.read_text(encoding="utf-8-sig").splitlines() if args.urls else [args.url]
urls = list(dict.fromkeys(u.strip() for u in urls if u.strip()))
if not urls:
parser.error("The input URL list is empty")
if not os.getenv("UNWRANGLE_API_KEY") or (SITE == "target" and not args.store):
parser.error("Set UNWRANGLE_API_KEY; Target also requires --store")
rows = []
for url in urls:
try:
rows.append(product_row(collect(url, args.store, args.output.with_suffix(".raw"))))
except ValueError as error:
rows.append({"status": "failed", "requested_url": url,
"requested_store": args.store if SITE == "target" else None,
"exported_at": now(), "error": str(error)})
with args.output.open("w", encoding="utf-8-sig", newline="") as handle:
writer = csv.DictWriter(handle, fieldnames=META_FIELDS + PRODUCT_FIELDS)
writer.writeheader()
writer.writerows(rows)
failed = sum(row["status"] != "ok" for row in rows)
print(str(len(rows)) + " rows; " + str(failed) + " failed; " + str(args.output))
if failed:
raise SystemExit(1)
if __name__ == "__main__":
main()
Use your intended item or this provider-documentation input:
python walmart_product.py --url "https://www.walmart.com/ip/14601806068" --output walmart-run1.csv
The live path creates a CSV and a walmart-run1.raw directory. Use a new output name for another collection period. Failed jobs stay in the CSV as failed; a missing price does not become zero or a copied value from a previous run.
Inspect a saved result before using it for an alert
Re-export an actual JSON file from that directory:
python walmart_product.py --json walmart-run1.raw/your-saved-file.json --output walmart-reexport.csv
I kept the saved retrieval time unchanged. If an unwrapped response contains no collection time, I leave it blank rather than label an old response with today's observation date.
In my offline check of the public documentation sample, the export kept item ID 14601806068, seller Dreame Tech, price 499.99 and the returned stock flag. I use those values to check the exporter, not to report a current Walmart offer, local stock or retrieval-success rate.
Why Can a Walmart Price Differ From the One You See?
Start by checking whether the sample represents a local store at all
The program above does not send a store parameter. Its output should be used as online product/offer information unless further returned evidence establishes another context. It is not a local-inventory collector merely because your project is interested in nearby stores.
I found the same concern in this scraper discussion: wrong-store data can look like a successful collection. I would check whether the response identifies the store I need, rather than accept a plausible amount.
For a five-store clearance project, require a source with documented store controls and verify the actual returned fields. Five copies of the same national offer with different spreadsheet labels are not five store observations. A nearby proxy exit does not supply a store-selection capability that the data source lacks.
Read seller and price fields together
When I read Unwrangle's Walmart documentation example, I found Dreame Tech listed as the seller, with purchase price 499.99 and separate list price 899.99. I would keep both distinctions visible: the seller need not be Walmart, and the reference amount need not be what a buyer pays.
| Export field | What to inspect | Wrong inference to avoid |
|---|---|---|
id |
The exact Walmart item | Similar titles identify the same item |
seller_name |
The source's displayed seller | Every product-page offer is sold by Walmart |
price, list_price |
Purchase and reference amounts separately | Selecting an unlabeled amount as the comparison price |
selected_variant |
The represented variation | Every option shares the same amount |
in_stock |
The returned stock flag in its source context | It describes every local store |
status, retrieved_at |
Whether this observation succeeded and when | A failed retrieval is a retailer stockout |
I use those historical documentation values to check the mapping, rather than issue a current-price alert. For recurring work, match item, variation, seller and relevant context before calculating a change. A seller switch is not automatically Walmart changing its own price.
PuraRoute for Walmart Product and Price Research
If your business runs its own Walmart scraper and needs paid proxies, PuraRoute is worth trying. It offers dynamic residential proxies, static residential proxies and static native proxies. You can choose around repeated product checks, market-specific research or longer-running work that needs an assigned IP, then test the service with the Walmart data you actually use.
Choose a service for your Walmart research
Dynamic residential proxies can suit a changing product list and repeated online-price checks. Static services provide an assigned IP for tools that need one over a longer period. Available location choices can help when your research concerns a particular market. Test with familiar Walmart items and check their sellers and prices; if the project concerns local stores, check that your data source supplies the required store information.
See what regular collection costs before expanding
PuraRoute provides traffic-usage information to help you track spending as more products are added. Compare that cost with the useful records your test produces. Decide whether the business needs hourly, daily or less frequent checks, then choose the plan accordingly. This makes the proxy purchase easier to judge against the price-monitoring or catalog work it is meant to support.
How Do You Add Walmart Search Results and More Products?
Turn an existing watchlist into a small batch
Put one Walmart US product URL on each line of walmart-urls.txt, then run:
python walmart_product.py --urls walmart-urls.txt --output walmart-watchlist-run1.csv
The script removes exact duplicate input lines and processes the list sequentially. Failed products remain visible. Compare the intended URL count with result statuses before using the table, rather than assuming a smaller successful file covered everything.
Use a new output name for another period. Retain the item, selected variation and seller when joining snapshots. If the current job fails, leave it failed; a stored price from yesterday is not a new check merely because it can be exported again.
Use search for discovery, then details for the selected items
If you start from a keyword instead of a watchlist, the documented Walmart search endpoint is a separate discovery path. Preserve the query, filters and page scope with the candidate URLs. Check the current pagination before describing a single page as the complete marketplace.
A product missing from the candidate list is a discovery problem. A listed candidate with no returned price is a detail/coverage problem. Keeping those outcomes separate makes it easier to decide whether to change the search, field mapping or source.
Scale only after the initial rows answer the online-offer question. A local-stock project still needs a store-capable source; the URL batch option does not add that capability. Choose scheduling and collection frequency around useful freshness and the provider's current terms.
How Do You Build Useful Walmart Price Monitoring?
Make the alert about the purchase you want to follow
Decide whether you are tracking Walmart's offer, a named marketplace seller or any acceptable seller. Then keep the item and pack size clear. A switch to another seller or a different bundle can produce a lower amount without making the original offer cheaper.
| Alert requirement | What to keep with the price |
|---|---|
| A particular product | Item ID and the selected variation or pack |
| A particular seller | The returned seller name and any relevant seller details |
| A price ceiling | The current purchase-price field and currency |
| A fresh result | Successful status and collection time |
| A store-specific need | Actual returned store evidence from a source that supports it |
For an appliance watchlist, I would match the offer before calculating a price change. Another seller appearing with a different bundle would not, by itself, make me send a sale alert. Keep the reference or list price separate from the amount being tracked.
Review missing values before sending a report
If a product's price is blank, inspect the saved response. A missing field may reflect incomplete data or an unusual offer. It should not become a zero-price alert or automatically mean the product has been discontinued.
For a recurring job, check how many intended products produced useful rows and which ones failed. A file with fewer rows can look tidy while hiding gaps. The batch script keeps failures visible so they can be retried or investigated without silently using old values.
Choose a checking frequency the business will use. A daily competitor report and a time-sensitive stock alert need different freshness. Add more products and more frequent checks when they improve those decisions, with the running cost and source's terms in view.
Frequently Asked Questions About Scraping Walmart Product Data
Does this example collect aisle locations?
It exports the product-detail fields returned by the chosen source. Do not assume that aisle information is included, especially for a specific store. If that field is required, verify the source's current coverage and store context before choosing it. A national product description does not establish a local aisle.
Can I supply only an item ID?
The provider documents an item-ID input in addition to a full URL. This script uses the URL path to keep its interface simple. If you adapt it to item IDs, follow the documented parameter and preserve the returned identity rather than assuming every number in a Walmart link is interchangeable.
Will the CSV contain one row per variant?
The initial export writes one selected product row with collection context. The full response retains any arrays for later inspection rather than automatically expanding them into independent rows. For variation analysis, create a linked variant table after inspecting the returned structure. Do not duplicate the product price across every variation unless the source actually supports that mapping.
Can I use the file in Excel without pandas?
Yes. Python's built-in CSV module writes the file, and the example uses UTF-8 with a byte-order marker for spreadsheet compatibility. Import product identifiers as text. Use a dataframe library later if the analysis benefits from it; it is not required for the initial API-to-CSV export.
Is a product with a missing price discontinued?
A missing field alone does not establish discontinuation. The source may omit a value, the offer may be unusual or the collection may be incomplete. Look for an explicit product or availability statement and compare the raw response. Keep extraction failure separate from a real retailer status in the dataset.
