Resources
The deep web of web-scraping reality.
Things we've learned in six years of running scraping infrastructure for thousands of teams. Honest writing, no SEO bait, no listicles.
How the modern web actually defends itself
CAPTCHAs in 2026
Cloudflare Turnstile, reCAPTCHA v3, hCaptcha: what triggers them and how solvers route around them.
Browser fingerprinting
JA3/JA4, canvas, WebGL, HTTP/2 frame order. Why curl gets blocked and headed Chromium doesn't.
Behavior-based blocking
Mouse jitter, scroll velocity, timing distributions. The shape of "human."
Account-level signals
Why some sites only block authenticated requests, and how session pinning fixes it.
HTTP errors decoded
403 vs 429 vs 503
Same symptom, "I'm blocked", three very different fixes.
Why your status-200 is a lie
How sites return a 200 with a CAPTCHA HTML body. Detection patterns.
Geo-block heuristics
When your "blocked" is actually "wrong country", a debugging trace.
Hidden retry-after headers
The polite ones tell you exactly when to come back. Read them.
Proxy infrastructure
Datacenter vs residential
When the cheap option works, when it doesn't, and how to decide.
Geo-targeting accuracy
Why "US proxy" isn't enough and you sometimes need city-level.
Rotation strategies
Per-call vs sticky sessions vs custom pools. Cost trade-offs.
Auto-proxy with failover
One parameter that retries through multiple proxy tiers until success.
For LLM teams specifically
HTML → Markdown without losing structure
Tables, code, headings: the things naive converters destroy.
Chunking strategies that actually work
Semantic vs fixed vs sliding-window. With retrieval-quality numbers.
Provenance + citation patterns
Storing source URL, fetch date, content hash for traceable RAG.
Training data licensing
What we can and can't legally help with. Honest.
Want a deep dive on something we haven't written yet? Tell us. We usually write the most-asked stuff first.