E-commerce · shopify.com

How to scrape Shopify stores

Shopify stores (shopify.com) is an online store. This guide is built from our own checks of its public pages: what the pages contain, whether our requests get through right now, and the request settings that worked.

Ujeebu scrapes any public website. Not listed here? Try any URL in the Playground or ask us about a site.

Working 5 credits a page, about $0.50 per 1,000 on Pro

5 of 6 of our checks passed in the last 7 days (83.3%).

Last check 2026-11-05 17:03 UTC · last pass 2026-11-05 11:34 UTC · checked with our own requests, never customer data.

Try it in the Playground

No recipe yet for Shopify stores: use the settings below, or ask us.

JSON js=false, response_type=raw 1 credit a page

Shopify JSON endpoints: every product of a store as JSON

Shopify stores share one theme engine but not one HTML layout, so a single CSS recipe would not hold across stores. Most stores answer two JSON URLs instead: send them to the Scrape API with js=false and response_type=raw and you get the store's own JSON back, 1 credit a page.

/products.json?limit=250&page={page} Every published product of the store, 250 a page (the most Shopify allows). Page 1, 2, 3... until an empty list.
/collections/{handle}/products.json?limit=250&page={page} The products of one collection (the handle is the last part of the collection URL, for example /collections/mens-shoes). /collections/all/ is the whole catalogue.

curl -s 'https://api.ujeebu.com/scrape?url=https%3A%2F%2Fwww.allbirds.com%2Fproducts.json%3Flimit%3D250%26page%3D1&js=false&response_type=raw' \
  -H "ApiKey: $UJEEBU_API_KEY"

Fields of each product: id, title, handle (the product URL is /products/<handle>), vendor, product_type, tags, created_at, updated_at, published_at, variants[] (id, title, sku, price, compare_at_price, available, option1 to option3), images[] (src, width, height), options[] (name, values), body_html (description).

Stores we checked: www.allbirds.com, www.deathwishcoffee.com, kith.com, colourpop.com, www.tentree.com.

Some stores turn these URLs off or rate limit them (a 403, 404 or 429 from the store): use the product pages then.
There is no search on this URL: the store domain is free, and our search definitions only build URLs on a fixed host (or one checked subdomain label). Loop over page until the products list is empty.
Prices are strings in the store currency; available tells whether a variant can be bought.
Without response_type=raw the answer is the JSON inside an HTML page (the browser's JSON viewer): keep raw to parse it directly.

Run the code on this page with your own key: 5,000 free credits for 7 days, no card. Get a free API key →

What you can get

The API returns the rendered HTML of the page. Select the fields you need with extract rules (CSS selectors) or parse the HTML on your side.

Public pages Working

Page types we check: Public pages.

Code

Complete scripts: set the UJEEBU_API_KEY environment variable and run them as they are. Each one sends the settings on this page, prints the start of the page, and prints the credits the request cost (the Ujb-Credits response header) on stderr. A non-200 answer stops the script with the API's error message.

No extraction recipe for these pages yet: the script fetches the page with the working settings. Add extract rules to get JSON.

shopify_page_sdk.py pip install ujeebu-python==0.1.5

# Fetch a Shopify stores page with the Ujeebu Scrape API.
# Settings from our health checks (Ujeebu site guide). Prints the start of the page HTML.
# Uses the official SDK ujeebu-python 0.1.5: pip install ujeebu-python==0.1.5
# Run: UJEEBU_API_KEY=your_key python3 shopify_page_sdk.py

import json
import os
import sys
import time

import requests
from ujeebu_python import UjeebuClient

API_KEY = os.environ.get("UJEEBU_API_KEY")
URLS = [
    "https://www.allbirds.com/products.json?limit=30",
]
SETTINGS = {
    "js": True,
}


def scrape(client, url):
    try:
        response = client.scrape(url, params=SETTINGS)
    except requests.RequestException as error:
        sys.exit(f"Request failed for {url}: {error}")
    if response.status_code != 200:
        try:
            message = response.json().get("message", response.text)
        except ValueError:
            message = response.text
        sys.exit(f"HTTP {response.status_code} for {url}: {str(message)[:300]}")
    print(f"Ujb-Credits: {response.headers.get('Ujb-Credits')}", file=sys.stderr)
    return response.text


if not API_KEY:
    sys.exit("Set the UJEEBU_API_KEY environment variable.")

client = UjeebuClient(api_key=API_KEY)

for i, url in enumerate(URLS):
    if i:
        time.sleep(2)  # be polite: one page at a time
    html = scrape(client, url)
    print(len(html), "bytes")
    print(html[:500])

shopify_page_sdk.mjs npm install @ujeebu-org/ujeebu-sdk@0.1.2

// Fetch a Shopify stores page with the Ujeebu Scrape API.
// Settings from our health checks (Ujeebu site guide). Prints the start of the page HTML.
// Uses the official SDK @ujeebu-org/ujeebu-sdk 0.1.2: npm install @ujeebu-org/ujeebu-sdk@0.1.2
// Run: UJEEBU_API_KEY=your_key node shopify_page_sdk.mjs

import { UjeebuClient } from '@ujeebu-org/ujeebu-sdk';

const API_KEY = process.env.UJEEBU_API_KEY;
const URLS = [
  "https://www.allbirds.com/products.json?limit=30",
];
const SETTINGS = {
  js: true,
};

async function scrape(client, url) {
  let res;
  try {
    res = await client.scrape(url, SETTINGS);
  } catch (err) {
    const status = err.response?.status ?? 'error';
    const message = err.response?.data?.message ?? err.message;
    console.error(`HTTP ${status} for ${url}: ${String(message).slice(0, 300)}`);
    process.exit(1);
  }
  console.error(`Ujb-Credits: ${res.headers['ujb-credits']}`);
  return String(res.data);
}

if (!API_KEY) {
  console.error('Set the UJEEBU_API_KEY environment variable.');
  process.exit(1);
}

const client = new UjeebuClient(API_KEY);
const pause = (ms) => new Promise((resolve) => setTimeout(resolve, ms));

for (const [i, url] of URLS.entries()) {
  if (i) await pause(2000); // be polite: one page at a time
  const html = await scrape(client, url);
  console.log(html.length, 'bytes');
  console.log(html.slice(0, 500));
}

shopify_page.py pip install requests

# Fetch a Shopify stores page with the Ujeebu Scrape API.
# Settings from our health checks (Ujeebu site guide). Prints the start of the page HTML.
# Run: UJEEBU_API_KEY=your_key python3 shopify_page.py   (pip install requests)

import json
import os
import sys
import time

import requests

API_URL = "https://api.ujeebu.com/scrape"
API_KEY = os.environ.get("UJEEBU_API_KEY")
URLS = [
    "https://www.allbirds.com/products.json?limit=30",
]
SETTINGS = {
    "js": True,
}


def scrape(url):
    response = requests.post(
        API_URL,
        headers={"ApiKey": API_KEY},
        json={"url": url, **SETTINGS},
        timeout=180,
    )
    if response.status_code != 200:
        try:
            message = response.json().get("message", response.text)
        except ValueError:
            message = response.text
        sys.exit(f"HTTP {response.status_code} for {url}: {str(message)[:300]}")
    print(f"Ujb-Credits: {response.headers.get('Ujb-Credits')}", file=sys.stderr)
    return response.text


if not API_KEY:
    sys.exit("Set the UJEEBU_API_KEY environment variable.")

for i, url in enumerate(URLS):
    if i:
        time.sleep(2)  # be polite: one page at a time
    html = scrape(url)
    print(len(html), "bytes")
    print(html[:500])

shopify_page.mjs

// Fetch a Shopify stores page with the Ujeebu Scrape API.
// Settings from our health checks (Ujeebu site guide). Prints the start of the page HTML.
// Run: UJEEBU_API_KEY=your_key node shopify_page.mjs   (Node.js 18 or later)

const API_URL = 'https://api.ujeebu.com/scrape';
const API_KEY = process.env.UJEEBU_API_KEY;
const URLS = [
  "https://www.allbirds.com/products.json?limit=30",
];
const SETTINGS = {
  js: true,
};

async function scrape(url) {
  const res = await fetch(API_URL, {
    method: 'POST',
    headers: { ApiKey: API_KEY, 'Content-Type': 'application/json' },
    body: JSON.stringify({ url, ...SETTINGS }),
    signal: AbortSignal.timeout(180000),
  });
  const text = await res.text();
  if (res.status !== 200) {
    let message = text;
    try {
      message = JSON.parse(text).message ?? text;
    } catch {}
    console.error(`HTTP ${res.status} for ${url}: ${String(message).slice(0, 300)}`);
    process.exit(1);
  }
  console.error(`Ujb-Credits: ${res.headers.get('ujb-credits')}`);
  return text;
}

if (!API_KEY) {
  console.error('Set the UJEEBU_API_KEY environment variable.');
  process.exit(1);
}

const pause = (ms) => new Promise((resolve) => setTimeout(resolve, ms));

for (const [i, url] of URLS.entries()) {
  if (i) await pause(2000); // be polite: one page at a time
  const html = await scrape(url);
  console.log(html.length, 'bytes');
  console.log(html.slice(0, 500));
}

shopify_page.sh

#!/usr/bin/env bash
# Fetch a Shopify stores page with the Ujeebu Scrape API.
# Settings from our health checks (Ujeebu site guide). Prints the start of the page HTML.
# Run: UJEEBU_API_KEY=your_key bash shopify_page.sh   (curl 7.76 or later)
set -euo pipefail

curl -sS --fail-with-body --max-time 180 -D headers.txt \
  -X POST 'https://api.ujeebu.com/scrape' \
  -H "ApiKey: ${UJEEBU_API_KEY:?Set the UJEEBU_API_KEY environment variable.}" \
  -H 'Content-Type: application/json' \
  -o page.html --data-binary @- <<'JSON'
{
  "url": "https://www.allbirds.com/products.json?limit=30",
  "js": true
}
JSON
head -c 500 page.html
echo
grep -i '^ujb-credits' headers.txt >&2

shopify_page.php

<?php
// Fetch a Shopify stores page with the Ujeebu Scrape API.
// Settings from our health checks (Ujeebu site guide). Prints the start of the page HTML.
// Run: UJEEBU_API_KEY=your_key php shopify_page.php   (PHP 7.4 or later with curl)

$apiKey = getenv('UJEEBU_API_KEY') ?: exit("Set the UJEEBU_API_KEY environment variable.\n");
$urls = [
    'https://www.allbirds.com/products.json?limit=30',
];
$settings = [
    'js' => true,
];

function scrape(string $url, string $apiKey, array $settings)
{
    $ch = curl_init('https://api.ujeebu.com/scrape');
    $headers = [];
    curl_setopt_array($ch, [
        CURLOPT_POST => true,
        CURLOPT_RETURNTRANSFER => true,
        CURLOPT_TIMEOUT => 180,
        CURLOPT_HTTPHEADER => ['ApiKey: ' . $apiKey, 'Content-Type: application/json'],
        CURLOPT_POSTFIELDS => json_encode(['url' => $url] + $settings),
        CURLOPT_HEADERFUNCTION => function ($ch, $line) use (&$headers) {
            if (strpos($line, ':') !== false) {
                [$k, $v] = explode(':', $line, 2);
                $headers[strtolower(trim($k))] = trim($v);
            }
            return strlen($line);
        },
    ]);
    $body = curl_exec($ch);
    if ($body === false) {
        fwrite(STDERR, 'Request failed: ' . curl_error($ch) . "\n");
        exit(1);
    }
    $status = curl_getinfo($ch, CURLINFO_RESPONSE_CODE);
    if ($status !== 200) {
        $message = json_decode($body, true)['message'] ?? $body;
        fwrite(STDERR, "HTTP $status for $url: " . substr((string) $message, 0, 300) . "\n");
        exit(1);
    }
    fwrite(STDERR, 'Ujb-Credits: ' . ($headers['ujb-credits'] ?? '?') . "\n");
    return $body;
}

foreach ($urls as $i => $url) {
    if ($i > 0) {
        sleep(2); // be polite: one page at a time
    }
    $html = scrape($url, $apiKey, $settings);
    echo strlen($html), " bytes\n", substr($html, 0, 500), "\n";
}

shopify_page.go go mod init scraper && go get github.com/ujeebu/ujeebu-go@v0.1.8

// Fetch a Shopify stores page with the Ujeebu Scrape API.
// Settings from our health checks (Ujeebu site guide). Prints the start of the page HTML.
// Uses the official SDK github.com/ujeebu/ujeebu-go v0.1.8: go mod init scraper && go get github.com/ujeebu/ujeebu-go@v0.1.8
// Run: UJEEBU_API_KEY=your_key go run shopify_page.go

package main

import (
	"errors"
	"fmt"
	"os"
	"time"

	ujeebu "github.com/ujeebu/ujeebu-go"
)

var urls = []string{
	"https://www.allbirds.com/products.json?limit=30",
}

func fail(format string, args ...any) {
	fmt.Fprintf(os.Stderr, format+"\n", args...)
	os.Exit(1)
}

func short(s string) string {
	if len(s) > 300 {
		return s[:300]
	}
	return s
}

func params(target string) ujeebu.ScrapeParams {
	p := ujeebu.ScrapeParams{URL: target}
	p.JS = true
	return p
}

func scrape(client *ujeebu.Client, target string) string {
	// the int is the credits the request cost (the Ujb-Credits header), not an HTTP status
	resp, credits, err := client.Scrape(params(target))
	if err != nil {
		var apiErr *ujeebu.APIError
		if errors.As(err, &apiErr) {
			fail("HTTP %d for %s: %s", apiErr.StatusCode, target, short(apiErr.Message))
		}
		fail("Request failed for %s: %v", target, err)
	}
	fmt.Fprintf(os.Stderr, "Ujb-Credits: %d\n", credits)
	return resp.HTML
}

func main() {
	apiKey := os.Getenv("UJEEBU_API_KEY")
	if apiKey == "" {
		fail("Set the UJEEBU_API_KEY environment variable.")
	}
	client, err := ujeebu.NewClient(apiKey, ujeebu.WithTimeout(180*time.Second))
	if err != nil {
		fail("%v", err)
	}

	for i, target := range urls {
		if i > 0 {
			time.Sleep(2 * time.Second) // be polite: one page at a time
		}
		html := scrape(client, target)
		fmt.Println(len(html), "bytes")
		fmt.Println(html[:min(len(html), 500)])
	}
}

SDK tabs use the official Ujeebu SDKs (ujeebu-python 0.1.5, @ujeebu-org/ujeebu-sdk 0.1.2, github.com/ujeebu/ujeebu-go v0.1.8); there is no PHP SDK, so the PHP tab calls the API with curl. For agents and scripts, the same status and settings are at /scrape/shopify.json.

Try it yourself

Run this on Shopify stores with your own key

Get a free API key, set UJEEBU_API_KEY and run any script above as it is. 5,000 free credits for 7 days, no card.

Get a free API key or try these settings in the Playground first →

Settings that work today

From our health checks over the last 7 days: the cheapest settings that return the real page most of the time.

Endpoint	`/scrape`
`js`	`true`
Cost	up to 5 credits a request (only HTTP 200 and 404 are billed; blocked, timed-out and failed requests are free)

Each public page is one request: change the path in the URL to fetch another one. Our checks run with JavaScript rendering on (js=true): keep it on. Start with the default (rotating) proxy, the cheapest one that passed most of its attempts in our checks.

Try it in the Playground

Proxies we tried on Shopify stores (1), attempts and passes over 7 days

Proxy	Attempts	Passed
Default (rotating)	6	4

Highlighted: the cheapest proxy that passes most of our checks of this site.

Troubleshooting

The failures our own checks of shopify.com ran into over the last 7 days, and what got the page in the same check.

What happened	Times	What fixed it
The site blocked the request (HTTP 403 or a block page)	1	None of the settings we tried got past it in 7 days.

Limits and etiquette

Public pages only. Pages behind a login, and data shown only to signed-in users, are out of scope.
Keep the rate modest: fetch pages one after another or with low concurrency, and cache what you already have.
robots.txt of www.allbirds.com (read 2026-11-04): public pages allowed for generic crawlers; no crawl delay set.
Read Shopify stores's terms of service before you collect data at scale, and do not collect personal data you have no basis to process. This guide is not legal advice.

Questions

Can you scrape Shopify stores with Ujeebu?

Yes. 5 of 6 checks of public Shopify stores pages passed in the last 7 days with the settings on this page.

Which settings work for Shopify stores?

Call /scrape with js=true. On the default (rotating) proxy, 4 of 6 attempts passed in the last 7 days.

How many credits does a Shopify stores request cost?

With these settings a request costs up to 5 credits. Only responses with HTTP 200 or 404 are billed: a blocked or timed-out request is free.

Is it allowed to scrape Shopify stores?

Its robots.txt allows generic crawlers on the page types in this guide. Collect public pages only, read Shopify stores's terms of service, keep your request rate modest and do not collect personal data you have no basis to process. This is not legal advice.

How fresh is this page?

The status and settings come from our own health checks of shopify.com, which run around the clock. Last check: 2026-11-05 17:03:41 UTC; last pass: 2026-11-05 11:34:20 UTC.