Build web filters, secure web gateways, DNS filtering, parental controls and ISP-grade filtering on a 102M-domain offline categorization database with a real-time API fallback for the long tail. Local lookups, a purpose-built web-filtering taxonomy, and a COPPA layer for child-safe profiles — we supply the classification backbone, so you can focus on enforcement.
A filtering product is judged on two failure modes at once. Block a health-education page and a school files a ticket; let a fresh gambling mirror through and an enterprise churns. With well over 100 million active domains — and new ones registered, repurposed and abandoned every day — keyword lists and manually curated categories cannot keep both failure rates down. And in inline deployments the verdict has to arrive before the page does.
String matching cannot tell a breast-cancer charity from adult content, or a chemistry syllabus from a drugs marketplace. Health and education traffic becomes collateral damage, and every false block is a support ticket — or a lost school district.
Casino mirrors, proxy-avoidance sites and adult hosts rotate domains precisely because static lists lag. Anything registered after your last list refresh is invisible to it, and that unrated tail is exactly where policy violations concentrate.
A DNS resolver or inline gateway cannot hold a connection open while a cloud classifier thinks. If every verdict costs a network round-trip, your filter becomes the slowest hop on the network — the categorization has to already be there.
Crawling, rendering and classifying a hundred million domains — then keeping the results current — is a full-time ML and infrastructure operation. It is our entire product; for a filtering vendor it is a distraction from the actual product.
The architecture that works in production is simple: answer almost every lookup from a local copy of our classified-domain database, and send only the genuinely unknown tail to the real-time API — caching the answer so each unknown is paid for once.
The URL database ships as a file you load into your own infrastructure: local lookups with no per-query latency, no rate limits, and — critical for enterprise and ISP buyers — no browsing data leaving the perimeter. See how the offline database works.
The web-filtering taxonomy covers the categories policies are actually written in — adult, violence, weapons, gambling, drugs, and malware/phishing-adjacent classes — alongside benign ones like health and education, so you can allow the clinic and block the casino. The full IAB taxonomy and expanded tiers are there when you need finer granularity.
For domains and URLs not yet in the database, the real-time categorization API fetches and classifies the live page on demand — one POST, machine-learning classification with confidence scores, path-level when you need it (a specific URL, not just the domain).
Parental-control and school deployments get a second signal: COPPA risk scoring tells you whether a site is child-directed and whether it runs ad tracking on children — so a "kids mode" can be stricter than a category verdict alone.
POST to /api/iab/iab_web_content_filtering.php with
query, api_key and data_type=url. Selecting the
web-content-filtering taxonomy on the same endpoint returns filtering categories.
import requests
BLOCKED = {"Adult", "Gambling", "Weapons", "Drugs"}
def category_for(domain):
# 1. Local lookup: 102M-domain offline database
row = local_db.get(domain)
if row:
return row["category"], "local"
# 2. Fallback: real-time API for the unknown tail
resp = requests.post(
"https://www.websitecategorizationapi.com"
"/api/iab/iab_web_content_filtering.php",
data={"query": f"https://{domain}",
"api_key": API_KEY,
"data_type": "url"},
timeout=30,
)
top = resp.json()["classification"][0]
local_db.put(domain, top) # pay for each unknown once
return top["category"], "api"
category, source = category_for("lucky-spins-777.example")
if category in BLOCKED:
serve_block_page(category)
{
"classification": [
{"category": "Gambling", "confidence": 0.96},
{"category": "Video Gaming", "confidence": 0.03}
],
"language": "en",
"status": 200
}
Confidence scores let you tune the trade-off yourself: enforce at 0.9+ for a light-touch enterprise profile, or at 0.6 with review for a strict school profile. Try any URL interactively in the live demo dashboard before writing a line of code.
coppa = requests.post(
"https://www.websitecategorizationapi.com/api/coppa/score.php",
data={"query": f"https://{domain}", "api_key": API_KEY},
).json()
if profile == "kids" and coppa["risk_level"] in ("HIGH", "CRITICAL"):
serve_block_page("child-safety")
Categories map cleanly onto the profiles filtering products already sell. A typical starting matrix — every cell is your call, because you receive categories and confidences, not opaque verdicts:
| Filtering category | Typical content | Enterprise profile | Child-safe profile |
|---|---|---|---|
| Adult | Pornography, explicit content | BLOCK | BLOCK |
| Gambling | Casinos, betting, lotteries | BLOCK | BLOCK |
| Weapons | Firearms sales, explosives | POLICY | BLOCK |
| Drugs | Recreational drug promotion | BLOCK | BLOCK |
| Violence / Hate | Gore, extremist content | BLOCK | BLOCK |
| Health & Medicine | Clinics, patient education | ALLOW | ALLOW |
| Education | Schools, courseware, reference | ALLOW | ALLOW |
The last two rows are the point. Because classification is content-based rather than keyword-based, medical and educational sites resolve to their own categories instead of tripping adult or drugs rules — which is exactly where keyword filters bleed support tickets. Browse the complete set at filtering_categories.php.
You re-sync the database on a schedule and let the real-time API cover anything newer than your last sync. Because API answers are cached locally, fallback volume stays a small fraction of total lookups — the pattern shown in the code above.
Both. The offline database is domain-level, which is what DNS filtering and gateway fast paths need. The real-time endpoint accepts full URLs with data_type=url, so an HTTPS-inspecting proxy can get path-level verdicts — useful for large mixed-content platforms.
The web-filtering taxonomy if you sell filtering: it is organized around block/allow decisions. The IAB taxonomy suits products that also serve analytics or advertising use cases. Both run on the same endpoint and database, so you can carry both.
Send us a sample of your resolver or gateway logs and get back a categorized extract from the 102M-domain database — coverage rate, category distribution, and the uncategorized tail the API would absorb.
Try the Live Demo Request a Sample Read the API Docs