How to scrape Amazon
Amazon is an online store accessible at amazon.com. Developers typically collect product information from search results, listings, and detail pages.
21 of 34 of our checks passed in the last 7 days (61.8%).
Last check 2026-11-02 22:35 UTC · last pass 2026-11-02 15:03 UTC · checked with our own requests, never customer data.
Run the code on this page with your own key: 5,000 free credits for 7 days, no card. Get a free API key →
Extraction recipe
Search and listing pages contain product overviews and filtering options. Detail pages provide individual product information. The recipe below is a set of Scrape API extract rules: we ran it through our API on two real Amazon pages with the settings on this page and kept only the fields that came back on both. It is checked again every week, and hidden if it stops working.
Search and listing pages
Validated 2026-10-30| Field | Type | Example |
|---|---|---|
products | list of 3+ items | |
products[].url | URL | https://www.amazon.com/Bluetooth-Headphones-Cancelling-Earphones-Wa... |
products[].title | text | Wireless Earbuds, 2026 Bluetooth 5.4 Headphones Bass Stereo Ear Bud... |
products[].image | image URL | https://m.media-amazon.com/images/I/61aSuH86kYL._AC_UY218_.jpg |
products[].rating | number (as text) | 4.7 |
Examples from www.amazon.com/s on 2026-10-30, shortened.
Sample output (real, trimmed)
{
"products": [
{
"url": "https://www.amazon.com/Apple-Cancellation-Translation-Headphones-High-Fidelity/dp/B0FQFB8FMG/ref=sr_1_1?dib=eyJ2IjoiMSJ9.9r8IGamzvSO3jL0UEWb1BaWP1X08eynVu6CR...",
"title": "Apple AirPods Pro 3 Wireless Earbuds with Active Noise Cancellation",
"image": "https://m.media-amazon.com/images/I/61solmQSSlL._AC_UY218_.jpg",
"rating": "4.4"
},
{
"url": "https://www.amazon.com/JBL-Vibe-Beam-Wireless-Headphones/dp/B0BQPNMXQV/ref=sr_1_2?dib=eyJ2IjoiMSJ9.9r8IGamzvSO3jL0UEWb1BaWP1X08eynVu6CRNJ7f9P6EzmT-l2dMsWUvYB...",
"title": "JBL Vibe Beam - True Wireless Earbuds - Black",
"image": "https://m.media-amazon.com/images/I/41+1Csr1pSL._AC_UY218_.jpg",
"rating": "4.2"
},
{
"url": "https://www.amazon.com/HAOYUYAN-Bluetooth-Headphones-Waterproof-Running%EF%BC%88Black%EF%BC%89/dp/B0HJ1QBMNL/ref=sr_1_3?dib=eyJ2IjoiMSJ9.9r8IGamzvSO3jL0UEWb1...",
"title": "HAOYUYAN Wireless Earbuds, Sports Bluetooth Headphones, 80Hrs Playtime Ear Buds with LED Power Display, Noise Canceling Headset, IPX7 Waterproof Earphones fo...",
"image": "https://m.media-amazon.com/images/I/71QZW38j75L._AC_UY218_.jpg",
"rating": "4.5"
}
]
}
extract_rules (1 rule)
{
"products": {
"selector": "div.sg-col-20-of-24.s-result-item",
"type": "obj",
"multiple": true,
"children": {
"url": {
"selector": "a.a-link-normal",
"type": "link"
},
"title": {
"selector": "h2.a-size-medium span",
"type": "text"
},
"image": {
"selector": "img.s-image",
"type": "image"
},
"rating": {
"selector": "span.a-size-small.a-color-base",
"type": "text"
}
}
}
}
Code
Complete scripts: set the UJEEBU_API_KEY environment variable and run them as they are. Each one sends the settings on this page and the extraction recipe, prints the extracted JSON, and prints the credits the request cost (the Ujb-Credits response header) on stderr. A non-200 answer stops the script with the API's error message.
Search and listing pages
Pages through 3 result pages, one at a time, and stops early when a page has no results.
pip install ujeebu-python==0.1.5 # Scrape Amazon search and listing pages with the Ujeebu Scrape API.
# Settings and extract rules validated on real pages on 2026-10-30 (Ujeebu site guide).
# Uses the official SDK ujeebu-python 0.1.5: pip install ujeebu-python==0.1.5
# Run: UJEEBU_API_KEY=your_key python3 amazon_listing_sdk.py
import json
import os
import sys
import time
from urllib.parse import parse_qsl, urlencode, urlsplit, urlunsplit
import requests
from ujeebu_python import UjeebuClient
API_KEY = os.environ.get("UJEEBU_API_KEY")
START_URL = "https://www.amazon.com/s?k=wireless+earbuds"
PAGES = 3
SETTINGS = {
"js": True,
}
EXTRACT_RULES = {
"products": {
"selector": "div.sg-col-20-of-24.s-result-item",
"type": "obj",
"multiple": True,
"children": {
"url": {
"selector": "a.a-link-normal",
"type": "link",
},
"title": {
"selector": "h2.a-size-medium span",
"type": "text",
},
"image": {
"selector": "img.s-image",
"type": "image",
},
"rating": {
"selector": "span.a-size-small.a-color-base",
"type": "text",
},
},
},
}
def page_url(page):
if page == 1:
return START_URL
parts = urlsplit(START_URL)
query = dict(parse_qsl(parts.query))
query["page"] = str(page)
return urlunsplit(parts._replace(query=urlencode(query)))
def scrape(client, url):
# with extract_rules, client.scrape sends a JSON POST and the API answers {"result": {...}}
try:
response = client.scrape(url, params={**SETTINGS, "extract_rules": EXTRACT_RULES})
except requests.RequestException as error:
sys.exit(f"Request failed for {url}: {error}")
if response.status_code != 200:
try:
message = response.json().get("message", response.text)
except ValueError:
message = response.text
sys.exit(f"HTTP {response.status_code} for {url}: {str(message)[:300]}")
print(f"Ujb-Credits: {response.headers.get('Ujb-Credits')}", file=sys.stderr)
return response.json()["result"]
if not API_KEY:
sys.exit("Set the UJEEBU_API_KEY environment variable.")
client = UjeebuClient(api_key=API_KEY)
for page in range(1, PAGES + 1):
result = scrape(client, page_url(page))
print(json.dumps(result, indent=2, ensure_ascii=False))
if not result.get("products"):
break
time.sleep(2) # be polite: one page at a time
npm install @ujeebu-org/ujeebu-sdk@0.1.2 // Scrape Amazon search and listing pages with the Ujeebu Scrape API.
// Settings and extract rules validated on real pages on 2026-10-30 (Ujeebu site guide).
// Uses the official SDK @ujeebu-org/ujeebu-sdk 0.1.2: npm install @ujeebu-org/ujeebu-sdk@0.1.2
// Run: UJEEBU_API_KEY=your_key node amazon_listing_sdk.mjs
import { UjeebuClient } from '@ujeebu-org/ujeebu-sdk';
const API_KEY = process.env.UJEEBU_API_KEY;
const START_URL = "https://www.amazon.com/s?k=wireless+earbuds";
const PAGES = 3;
const SETTINGS = {
js: true,
};
const EXTRACT_RULES = {
products: {
selector: "div.sg-col-20-of-24.s-result-item",
type: "obj",
multiple: true,
children: {
url: {
selector: "a.a-link-normal",
type: "link",
},
title: {
selector: "h2.a-size-medium span",
type: "text",
},
image: {
selector: "img.s-image",
type: "image",
},
rating: {
selector: "span.a-size-small.a-color-base",
type: "text",
},
},
},
};
function pageUrl(page) {
if (page === 1) return START_URL;
const url = new URL(START_URL);
url.searchParams.set("page", String(page));
return url.toString();
}
async function scrape(client, url) {
let res;
try {
// client.scrape POSTs JSON when extract_rules is set (scrapeWithRules would send it as a GET query, which the API ignores)
res = await client.scrape(url, { ...SETTINGS, extract_rules: EXTRACT_RULES });
} catch (err) {
const status = err.response?.status ?? 'error';
const message = err.response?.data?.message ?? err.message;
console.error(`HTTP ${status} for ${url}: ${String(message).slice(0, 300)}`);
process.exit(1);
}
console.error(`Ujb-Credits: ${res.headers['ujb-credits']}`);
return res.data.result;
}
if (!API_KEY) {
console.error('Set the UJEEBU_API_KEY environment variable.');
process.exit(1);
}
const client = new UjeebuClient(API_KEY);
const pause = (ms) => new Promise((resolve) => setTimeout(resolve, ms));
for (let page = 1; page <= PAGES; page++) {
const result = await scrape(client, pageUrl(page));
console.log(JSON.stringify(result, null, 2));
if (!result?.products?.length) break;
await pause(2000); // be polite: one page at a time
}
pip install requests # Scrape Amazon search and listing pages with the Ujeebu Scrape API.
# Settings and extract rules validated on real pages on 2026-10-30 (Ujeebu site guide).
# Run: UJEEBU_API_KEY=your_key python3 amazon_listing.py (pip install requests)
import json
import os
import sys
import time
from urllib.parse import parse_qsl, urlencode, urlsplit, urlunsplit
import requests
API_URL = "https://api.ujeebu.com/scrape"
API_KEY = os.environ.get("UJEEBU_API_KEY")
START_URL = "https://www.amazon.com/s?k=wireless+earbuds"
PAGES = 3
SETTINGS = {
"js": True,
}
EXTRACT_RULES = {
"products": {
"selector": "div.sg-col-20-of-24.s-result-item",
"type": "obj",
"multiple": True,
"children": {
"url": {
"selector": "a.a-link-normal",
"type": "link",
},
"title": {
"selector": "h2.a-size-medium span",
"type": "text",
},
"image": {
"selector": "img.s-image",
"type": "image",
},
"rating": {
"selector": "span.a-size-small.a-color-base",
"type": "text",
},
},
},
}
def page_url(page):
if page == 1:
return START_URL
parts = urlsplit(START_URL)
query = dict(parse_qsl(parts.query))
query["page"] = str(page)
return urlunsplit(parts._replace(query=urlencode(query)))
def scrape(url):
response = requests.post(
API_URL,
headers={"ApiKey": API_KEY},
json={"url": url, **SETTINGS, "extract_rules": EXTRACT_RULES},
timeout=180,
)
if response.status_code != 200:
try:
message = response.json().get("message", response.text)
except ValueError:
message = response.text
sys.exit(f"HTTP {response.status_code} for {url}: {str(message)[:300]}")
print(f"Ujb-Credits: {response.headers.get('Ujb-Credits')}", file=sys.stderr)
return response.json()["result"]
if not API_KEY:
sys.exit("Set the UJEEBU_API_KEY environment variable.")
for page in range(1, PAGES + 1):
result = scrape(page_url(page))
print(json.dumps(result, indent=2, ensure_ascii=False))
if not result.get("products"):
break
time.sleep(2) # be polite: one page at a time
// Scrape Amazon search and listing pages with the Ujeebu Scrape API.
// Settings and extract rules validated on real pages on 2026-10-30 (Ujeebu site guide).
// Run: UJEEBU_API_KEY=your_key node amazon_listing.mjs (Node.js 18 or later)
const API_URL = 'https://api.ujeebu.com/scrape';
const API_KEY = process.env.UJEEBU_API_KEY;
const START_URL = "https://www.amazon.com/s?k=wireless+earbuds";
const PAGES = 3;
const SETTINGS = {
js: true,
};
const EXTRACT_RULES = {
products: {
selector: "div.sg-col-20-of-24.s-result-item",
type: "obj",
multiple: true,
children: {
url: {
selector: "a.a-link-normal",
type: "link",
},
title: {
selector: "h2.a-size-medium span",
type: "text",
},
image: {
selector: "img.s-image",
type: "image",
},
rating: {
selector: "span.a-size-small.a-color-base",
type: "text",
},
},
},
};
function pageUrl(page) {
if (page === 1) return START_URL;
const url = new URL(START_URL);
url.searchParams.set("page", String(page));
return url.toString();
}
async function scrape(url) {
const res = await fetch(API_URL, {
method: 'POST',
headers: { ApiKey: API_KEY, 'Content-Type': 'application/json' },
body: JSON.stringify({ url, ...SETTINGS, extract_rules: EXTRACT_RULES }),
signal: AbortSignal.timeout(180000),
});
const text = await res.text();
if (res.status !== 200) {
let message = text;
try {
message = JSON.parse(text).message ?? text;
} catch {}
console.error(`HTTP ${res.status} for ${url}: ${String(message).slice(0, 300)}`);
process.exit(1);
}
console.error(`Ujb-Credits: ${res.headers.get('ujb-credits')}`);
return JSON.parse(text).result;
}
if (!API_KEY) {
console.error('Set the UJEEBU_API_KEY environment variable.');
process.exit(1);
}
const pause = (ms) => new Promise((resolve) => setTimeout(resolve, ms));
for (let page = 1; page <= PAGES; page++) {
const result = await scrape(pageUrl(page));
console.log(JSON.stringify(result, null, 2));
if (!result?.products?.length) break;
await pause(2000); // be polite: one page at a time
}
#!/usr/bin/env bash
# Scrape Amazon search and listing pages with the Ujeebu Scrape API.
# Settings and extract rules validated on real pages on 2026-10-30 (Ujeebu site guide).
# Run: UJEEBU_API_KEY=your_key bash amazon_listing.sh (curl 7.76 or later)
set -euo pipefail
curl -sS --fail-with-body --max-time 180 -D headers.txt \
-X POST 'https://api.ujeebu.com/scrape' \
-H "ApiKey: ${UJEEBU_API_KEY:?Set the UJEEBU_API_KEY environment variable.}" \
-H 'Content-Type: application/json' \
--data-binary @- <<'JSON'
{
"url": "https://www.amazon.com/s?k=wireless+earbuds",
"js": true,
"extract_rules": {
"products": {
"selector": "div.sg-col-20-of-24.s-result-item",
"type": "obj",
"multiple": true,
"children": {
"url": {
"selector": "a.a-link-normal",
"type": "link"
},
"title": {
"selector": "h2.a-size-medium span",
"type": "text"
},
"image": {
"selector": "img.s-image",
"type": "image"
},
"rating": {
"selector": "span.a-size-small.a-color-base",
"type": "text"
}
}
}
}
}
JSON
echo
grep -i '^ujb-credits' headers.txt >&2
<?php
// Scrape Amazon search and listing pages with the Ujeebu Scrape API.
// Settings and extract rules validated on real pages on 2026-10-30 (Ujeebu site guide).
// Run: UJEEBU_API_KEY=your_key php amazon_listing.php (PHP 7.4 or later with curl)
$apiKey = getenv('UJEEBU_API_KEY') ?: exit("Set the UJEEBU_API_KEY environment variable.\n");
$startUrl = 'https://www.amazon.com/s?k=wireless+earbuds';
$pages = 3;
$settings = [
'js' => true,
];
$extractRules = [
'products' => [
'selector' => 'div.sg-col-20-of-24.s-result-item',
'type' => 'obj',
'multiple' => true,
'children' => [
'url' => [
'selector' => 'a.a-link-normal',
'type' => 'link',
],
'title' => [
'selector' => 'h2.a-size-medium span',
'type' => 'text',
],
'image' => [
'selector' => 'img.s-image',
'type' => 'image',
],
'rating' => [
'selector' => 'span.a-size-small.a-color-base',
'type' => 'text',
],
],
],
];
function scrape(string $url, string $apiKey, array $settings, array $extractRules)
{
$ch = curl_init('https://api.ujeebu.com/scrape');
$headers = [];
curl_setopt_array($ch, [
CURLOPT_POST => true,
CURLOPT_RETURNTRANSFER => true,
CURLOPT_TIMEOUT => 180,
CURLOPT_HTTPHEADER => ['ApiKey: ' . $apiKey, 'Content-Type: application/json'],
CURLOPT_POSTFIELDS => json_encode(['url' => $url] + $settings + ['extract_rules' => $extractRules]),
CURLOPT_HEADERFUNCTION => function ($ch, $line) use (&$headers) {
if (strpos($line, ':') !== false) {
[$k, $v] = explode(':', $line, 2);
$headers[strtolower(trim($k))] = trim($v);
}
return strlen($line);
},
]);
$body = curl_exec($ch);
if ($body === false) {
fwrite(STDERR, 'Request failed: ' . curl_error($ch) . "\n");
exit(1);
}
$status = curl_getinfo($ch, CURLINFO_RESPONSE_CODE);
if ($status !== 200) {
$message = json_decode($body, true)['message'] ?? $body;
fwrite(STDERR, "HTTP $status for $url: " . substr((string) $message, 0, 300) . "\n");
exit(1);
}
fwrite(STDERR, 'Ujb-Credits: ' . ($headers['ujb-credits'] ?? '?') . "\n");
return json_decode($body, true)['result'];
}
for ($page = 1; $page <= $pages; $page++) {
$parts = parse_url($startUrl);
parse_str($parts['query'] ?? '', $query);
$query['page'] = (string) ($page);
$url = $page === 1 ? $startUrl : $parts['scheme'] . '://' . $parts['host'] . ($parts['path'] ?? '/') . '?' . http_build_query($query);
$result = scrape($url, $apiKey, $settings, $extractRules);
echo json_encode($result, JSON_PRETTY_PRINT | JSON_UNESCAPED_SLASHES | JSON_UNESCAPED_UNICODE), "\n";
if (empty($result['products'])) {
break;
}
sleep(2); // be polite: one page at a time
}
go mod init scraper && go get github.com/ujeebu/ujeebu-go@v0.1.8 // Scrape Amazon search and listing pages with the Ujeebu Scrape API.
// Settings and extract rules validated on real pages on 2026-10-30 (Ujeebu site guide).
// Uses the official SDK github.com/ujeebu/ujeebu-go v0.1.8: go mod init scraper && go get github.com/ujeebu/ujeebu-go@v0.1.8
// Run: UJEEBU_API_KEY=your_key go run amazon_listing.go
package main
import (
"encoding/json"
"errors"
"fmt"
"net/url"
"os"
"strconv"
"time"
ujeebu "github.com/ujeebu/ujeebu-go"
)
const startURL = "https://www.amazon.com/s?k=wireless+earbuds"
const pages = 3
// The extract rules of the recipe, as the JSON the API takes.
const extractRulesJSON = `{
"products": {
"selector": "div.sg-col-20-of-24.s-result-item",
"type": "obj",
"multiple": true,
"children": {
"url": {
"selector": "a.a-link-normal",
"type": "link"
},
"title": {
"selector": "h2.a-size-medium span",
"type": "text"
},
"image": {
"selector": "img.s-image",
"type": "image"
},
"rating": {
"selector": "span.a-size-small.a-color-base",
"type": "text"
}
}
}
}`
var extractRules map[string]any
func fail(format string, args ...any) {
fmt.Fprintf(os.Stderr, format+"\n", args...)
os.Exit(1)
}
func short(s string) string {
if len(s) > 300 {
return s[:300]
}
return s
}
func params(target string) ujeebu.ScrapeParams {
p := ujeebu.ScrapeParams{URL: target}
p.JS = true
p.ExtractRules = extractRules
return p
}
func pageURL(page int) string {
if page == 1 {
return startURL
}
u, err := url.Parse(startURL)
if err != nil {
fail("Bad start URL: %v", err)
}
q := u.Query()
q.Set("page", strconv.Itoa(page))
u.RawQuery = q.Encode()
return u.String()
}
func scrape(client *ujeebu.Client, target string) map[string]any {
// the int is the credits the request cost (the Ujb-Credits header), not an HTTP status
resp, credits, err := client.Scrape(params(target))
if err != nil {
var apiErr *ujeebu.APIError
if errors.As(err, &apiErr) {
fail("HTTP %d for %s: %s", apiErr.StatusCode, target, short(apiErr.Message))
}
fail("Request failed for %s: %v", target, err)
}
fmt.Fprintf(os.Stderr, "Ujb-Credits: %d\n", credits)
result, _ := resp.Result.(map[string]any)
return result
}
func printJSON(v any) {
enc := json.NewEncoder(os.Stdout)
enc.SetEscapeHTML(false)
enc.SetIndent("", " ")
if err := enc.Encode(v); err != nil {
fail("Cannot print the result: %v", err)
}
}
func main() {
apiKey := os.Getenv("UJEEBU_API_KEY")
if apiKey == "" {
fail("Set the UJEEBU_API_KEY environment variable.")
}
if err := json.Unmarshal([]byte(extractRulesJSON), &extractRules); err != nil {
fail("Bad extract rules: %v", err)
}
client, err := ujeebu.NewClient(apiKey, ujeebu.WithTimeout(180*time.Second))
if err != nil {
fail("%v", err)
}
for page := 1; page <= pages; page++ {
result := scrape(client, pageURL(page))
printJSON(result)
items, _ := result["products"].([]any)
if len(items) == 0 {
break
}
time.Sleep(2 * time.Second) // be polite: one page at a time
}
}
SDK tabs use the official Ujeebu SDKs (ujeebu-python 0.1.5, @ujeebu-org/ujeebu-sdk 0.1.2, github.com/ujeebu/ujeebu-go v0.1.8); there is no PHP SDK, so the PHP tab calls the API with curl. For agents and scripts, the same status and settings are at /scrape/amazon.json.
Try it yourself
Run this on Amazon with your own key
Get a free API key, set UJEEBU_API_KEY and run any script above as it is. 5,000 free credits for 7 days, no card.
URL patterns and pagination
What the URLs look like, read from the pages our checks fetched.
Search and listing pages
- Search or listing URL
https://www.amazon.com/s?k={query}- Next pages
- add
page=2,page=3and so on. Page 2:https://www.amazon.com/s?k=wireless+earbuds&page=2 - Validated on
https://www.amazon.com/s?k=wireless+earbudshttps://www.amazon.com/s?k=backpack
Settings that work today
From our health checks over the last 7 days: the cheapest settings that return the real page most of the time.
| Endpoint | /scrape |
|---|---|
js | true |
proxy_type | advanced |
wait_for | 8000 |
device | mobile |
| Cost | up to 10 credits a request (only HTTP 200 and 404 are billed) |
| Learned | js + wait 8000 + mobile on advanced (found by our retry layer) |
Our engineers are looking at this site.
For search and listing pages, change the query or the page parameter in the URL to paginate, and fetch the pages one after another rather than in bursts. Collect detail page URLs from listing pages first, then fetch each detail page. Our checks pass with JavaScript rendering on (js=true) and a wait of 8 seconds (wait_for=8000) before the snapshot: keep both.
Troubleshooting
The failures our own checks of amazon.com ran into over the last 7 days, and what got the page in the same check.
| What happened | Times | What fixed it |
|---|---|---|
| A different page than expected | 17 | None of the settings we tried got past it in 7 days. |
| The site blocked the request (HTTP 403 or a block page) | 1 | None of the settings we tried got past it in 7 days. |
A failed request is not billed: only HTTP 200 and 404 responses cost credits.
What people build with Amazon data
- Catalog and assortment trackingFollow which products appear for a search or a category, and in which position.More on this use case
- Ratings and reviews trackingWatch how ratings and review counts move over time for your products and competitors.More on this use case
Limits and etiquette
- Public pages only. Pages behind a login, and data shown only to signed-in users, are out of scope.
- Keep the rate modest: fetch pages one after another or with low concurrency, and cache what you already have.
- robots.txt of www.amazon.com (read 2026-10-27): public pages allowed for generic crawlers; no crawl delay set.
- Read Amazon's terms of service before you collect data at scale, and do not collect personal data you have no basis to process. This guide is not legal advice.
Questions
Can you scrape Amazon with Ujeebu?
Partly. 21 of 34 checks of public Amazon pages passed in the last 7 days; most failures were a different page than expected. Retry failed pages later, and use the settings on this page.
Which settings work for Amazon?
Call /scrape with js=true, proxy_type=advanced, wait_for=8000, device=mobile. Our retry layer found that this site needs "js + wait 8000 + mobile on advanced".
How many credits does a Amazon request cost?
With these settings a request costs up to 10 credits. Only responses with HTTP 200 or 404 are billed: a blocked or timed-out request is free.
Which fields can I extract from Amazon?
With the extraction recipes on this page, validated on real Amazon pages through our API: search and listing pages: url, title, image, rating. Pass the recipe as extract_rules in a POST to /scrape and the API returns those fields as JSON.
Is it allowed to scrape Amazon?
Its robots.txt allows generic crawlers on the page types in this guide. Collect public pages only, read Amazon's terms of service, keep your request rate modest and do not collect personal data you have no basis to process. This is not legal advice.
How fresh is this page?
The status and settings come from our own health checks of amazon.com, which run around the clock. Last check: 2026-11-02 22:35:47 UTC; last pass: 2026-11-02 15:03:59 UTC.