AI agent security depends on the quality of the domain intelligence feeding into it. Our 102 million domain database delivers five distinct intelligence signals — IAB content categories, web filtering classifications, page-type labels, OpenPageRank reputation scores, and global popularity rankings — that your agent security stack can consume to make informed, real-time access decisions without model inference.
Most AI agent security implementations rely on static blocklists, prompt-based restrictions, or post-hoc monitoring — none of which provide the structured domain intelligence needed for proactive, real-time protection.
Today's AI agent security architectures suffer from a fundamental intelligence deficit. When an agent is about to navigate to a URL, the security layer has almost no information about the destination. It may check the URL against a static blocklist — a list that was compiled weeks or months ago and contains only previously identified threats. It may try to infer the site's nature from the domain name string — a heuristic that fails for the vast majority of domains. Or it may allow the navigation and attempt to analyze the page content after loading — an approach that exposes the agent to whatever the page contains before any security check completes.
A domain intelligence feed transforms your agent's security posture from reactive to proactive. Instead of waiting for something bad to happen and then responding, the intelligence feed provides structured data about every domain before the agent connects to it. Our 102M domain database delivers five intelligence signals per domain: IAB content categories (up to Tier 4 with 700+ categories), web filtering classifications (Malware, Phishing, Adult, Gambling, etc.), page-type labels (login, checkout, admin, API, documentation, etc.), OpenPageRank reputation scores (0-10), and global popularity rankings.
These five signals combine to create a comprehensive domain profile that your security layer can evaluate in microseconds. A domain classified as "Technology & Computing" with a PageRank of 7 and a global rank in the top 10,000 is almost certainly safe. A domain with no IAB category, a PageRank of 0, and no global ranking is suspicious and should be blocked or flagged for review. The intelligence is deterministic, pre-computed, and available without any external API call when deployed locally.
Each signal provides a distinct dimension of domain awareness for your security stack
The IAB Content Taxonomy v3 assigns each domain up to four tiers of content classification. This signal tells your security layer what the domain is about — Technology, Finance, Healthcare, Shopping, etc. Use it to scope agent access to task-relevant content categories and block categories that fall outside the agent's mandate. 700+ category labels ensure granular content awareness.
Dedicated security-oriented categories that map directly to threat policies: Malware, Phishing, Spam, Adult, Gambling, Weapons, Drugs, Hate Speech, and more. These classifications power hard-block rules that are non-negotiable for enterprise deployments. A domain classified as "Malware" is blocked unconditionally, regardless of any other signal.
Twenty-plus page-type classifications — login, checkout, admin, settings, pricing, careers, contact, documentation, API reference, blog, and more — provide interaction-level intelligence. Even when a domain's content category is approved, its login page, checkout flow, or admin panel may be off-limits. Page-type labels enable page-level security granularity that domain-level blocking alone cannot provide.
Production-ready snippets to consume domain intelligence in your agent security stack
import http.client
import json
class DomainIntelligenceEvaluator:
"""Evaluates domains using five intelligence signals."""
THREAT_CATEGORIES = [
"Malware", "Phishing", "Spam", "Adult",
"Gambling", "Weapons", "Illegal Content"
]
SENSITIVE_PAGE_TYPES = [
"login", "checkout", "admin", "settings", "signup"
]
def __init__(self, api_key, min_pagerank=2, max_rank=5000000):
self.api_key = api_key
self.min_pagerank = min_pagerank
self.max_rank = max_rank
self.conn = http.client.HTTPSConnection(
"www.websitecategorizationapi.com"
)
def get_intelligence(self, target_url):
payload = (
f"query={target_url}"
f"&api_key={self.api_key}"
f"&data_type=url"
f"&expanded_categories=1"
)
headers = {
"Content-Type": "application/x-www-form-urlencoded"
}
self.conn.request(
"POST",
"/api/iab/iab_web_content_filtering.php",
payload,
headers
)
res = self.conn.getresponse()
return json.loads(res.read().decode("utf-8"))
def compute_risk_score(self, target_url):
data = self.get_intelligence(target_url)
risk = 0
signals = {}
# Signal 1: Web Filtering Category
filtering = data.get("filtering_taxonomy", [])
for f in filtering:
cat = f[0].split("Category name: ")[1]
if cat in self.THREAT_CATEGORIES:
risk += 100
signals["web_filter"] = f"THREAT: {cat}"
break
else:
signals["web_filter"] = "Clean"
# Signal 2: Page Type
page_type = data.get("page_type", "unknown")
if page_type in self.SENSITIVE_PAGE_TYPES:
risk += 40
signals["page_type"] = f"Sensitive: {page_type}"
else:
signals["page_type"] = page_type
# Signal 3: Reputation Score
pagerank = float(data.get("open_pagerank", 0))
if pagerank < self.min_pagerank:
risk += 30
signals["reputation"] = pagerank
# Signal 4: Popularity Rank
global_rank = int(data.get("global_rank", 0) or 0)
if global_rank == 0 or global_rank > self.max_rank:
risk += 20
signals["popularity"] = global_rank
# Signal 5: IAB Categories
iab = [c[0].split("Category name: ")[1]
for c in data.get("iab_classification", [])]
signals["iab_categories"] = iab
decision = "allow" if risk < 30 else (
"review" if risk < 70 else "block"
)
return {
"url": target_url,
"risk_score": min(risk, 100),
"decision": decision,
"signals": signals
}
# Usage
evaluator = DomainIntelligenceEvaluator(api_key="your_key")
result = evaluator.compute_risk_score("https://example.com")
print(f"Risk: {result['risk_score']} — Decision: {result['decision']}")
class IntelligenceFeedConsumer {
constructor(apiKey, config = {}) {
this.apiKey = apiKey;
this.minPageRank = config.minPageRank || 2;
this.threatCategories = config.threatCategories || [
"Malware", "Phishing", "Spam", "Adult", "Gambling"
];
this.sensitivePages = config.sensitivePages || [
"login", "checkout", "admin", "settings"
];
}
async evaluateDomain(targetURL) {
const resp = await fetch(
"https://www.websitecategorizationapi.com" +
"/api/iab/iab_web_content_filtering.php",
{
method: "POST",
headers: {
"Content-Type": "application/x-www-form-urlencoded"
},
body: new URLSearchParams({
query: targetURL,
api_key: this.apiKey,
data_type: "url",
expanded_categories: "1"
})
}
);
const data = await resp.json();
const signals = {
webFilter: data.filtering_taxonomy?.[0]?.[0]
?.replace("Category name: ", "") || "Unknown",
pageType: data.page_type || "unknown",
pageRank: parseFloat(data.open_pagerank || 0),
globalRank: parseInt(data.global_rank || 0),
iabCategories: (data.iab_classification || [])
.map(c => c[0]?.replace("Category name: ", ""))
};
let riskScore = 0;
if (this.threatCategories.includes(signals.webFilter))
riskScore += 100;
if (this.sensitivePages.includes(signals.pageType))
riskScore += 40;
if (signals.pageRank < this.minPageRank) riskScore += 25;
if (signals.globalRank === 0) riskScore += 20;
return {
url: targetURL,
riskScore: Math.min(riskScore, 100),
decision: riskScore < 30 ? "allow" :
riskScore < 70 ? "review" : "block",
signals,
timestamp: new Date().toISOString()
};
}
}
Purpose-built domain databases for AI agent filtering. Includes IAB categories, 20+ page types, reputation scores, and popularity rankings. One-time purchase with perpetual license.
10 Million Domains with Page-Type Intelligence
One-time purchase: Perpetual license | Optional Updates: $1,599/year
20 Million Domains with Full Intelligence Suite
One-time purchase: Perpetual license | Optional Updates: $2,999/year
50 Million Domains with Complete Intelligence Suite
One-time purchase: Perpetual license | Optional Updates: $4,999/year
Also available: Enterprise URL Database up to 102M domains from $2,499. View all database tiers →
Search any IAB or Web Filtering category to see how many domains are in our 102M Enterprise Database — the same data your domain intelligence feed will provide.
How 102 million domains from our main Enterprise Database are distributed across IAB v3 taxonomy classifications
Spanning Tier 1 through Tier 4 classifications from our 102M Enterprise Database
Charts display domain counts for the top 50 out of 700+ categories in our 102M Enterprise Database. To check the number of domains for the remaining 650+ categories, use the Category Counter tool above .
Traditional cybersecurity relies on threat intelligence feeds — curated lists of known malicious IPs, domains, and file hashes that security tools use to detect and block known threats. AI agent security requires a different kind of intelligence. Agents do not just need to know which domains are malicious; they need to know what every domain is — its content category, its purpose, its reputation, and its page-level structure. This broader intelligence requirement is what domain intelligence feeds provide.
A threat intelligence feed tells you that domain X is associated with malware distribution. A domain intelligence feed tells you that domain X is classified as "Technology & Computing > Software > Antivirus," has a PageRank of 6, ranks globally at position 45,000, and its current page is a "download" page type. With this richer context, your security layer can make nuanced decisions: the domain itself is legitimate, but the agent should not interact with download pages — so allow browsing but block file downloads.
Each intelligence signal in the 102M domain database addresses a different dimension of domain awareness. IAB content categories answer the question "what is this domain about?" with up to Tier 4 granularity across 700+ categories. Web filtering classifications answer "is this domain safe?" with security-focused labels like Malware, Phishing, and Spam. Page-type labels answer "what is this specific page for?" with 20+ functional labels. OpenPageRank scores answer "how authoritative is this domain?" with a 0-10 numerical rating. Global popularity rankings answer "how widely visited is this domain?" with a position among all active domains worldwide.
Individually, each signal provides partial awareness. Combined, they create a comprehensive domain profile that enables sophisticated security decisions. A domain with a high-threat web filtering category is blocked regardless of other signals. A domain with an approved IAB category but a low PageRank might be flagged for review. A domain with a "login" page type is blocked even if everything else checks out. The multi-signal approach eliminates the blind spots that any single signal would leave.
Domain intelligence feeds integrate naturally with existing security infrastructure. For SIEM platforms (Splunk, Sentinel, Elastic), the domain intelligence data enriches agent activity logs with category and reputation context. Instead of seeing raw URLs in the log, the SOC analyst sees "Agent visited example.com [Technology & Computing, PageRank: 7, Rank: 12,500, Page: documentation]" — a far more informative log entry that enables faster triage.
For SOAR platforms (Palo Alto XSOAR, Swimlane, Tines), domain intelligence triggers automated playbooks. When an agent visits a domain with a "Phishing" web filtering classification, the SOAR platform can automatically quarantine the agent session, notify the security team, capture a screenshot, and create an incident ticket — all within seconds of the navigation event. This automated response is only possible because the domain intelligence feed provides the structured classification data that playbook conditions require.
OpenPageRank scores provide a numerical measure of domain authority that correlates strongly with domain legitimacy. Domains with PageRank scores of 7 or above are almost exclusively legitimate, well-maintained websites operated by established organizations. Domains with scores of 1 or below are disproportionately likely to be newly registered, parked, or low-quality sites that agents should approach with caution.
Incorporating reputation scores into agent security decisions creates a trust gradient that blocklists cannot provide. Instead of a binary allow/block decision, the agent's security layer can implement tiered trust: high-reputation domains get full access, medium-reputation domains get read-only access, and low-reputation domains are blocked. This granularity reduces false positives — legitimate but lesser-known domains are not blocked outright but are restricted to safe interaction modes.
A domain's global popularity ranking — its position among all active domains based on real user traffic data from the Google Chrome User Experience Report — serves as a powerful proxy for safety. Domains in the top 100,000 are, by definition, heavily visited by real users and monitored by security vendors. The probability of a top-100K domain being an active malware distributor is extremely low because such domains are quickly identified and blacklisted by the security community.
For agent security, this means that popularity ranking can serve as a secondary safety signal. An agent navigating to a domain ranked in the top 1 million is statistically safer than an agent navigating to an unranked domain. When combined with content category and web filtering signals, popularity ranking helps identify the "unknown unknowns" — domains that are not explicitly on any blocklist but whose lack of popularity suggests they may be newly registered or purpose-built for malicious activity.
The most sophisticated agent security implementations combine all five intelligence signals into a composite risk score. The scoring formula assigns weights to each signal based on its predictive power for the specific deployment context. For a financial services agent, web filtering category might carry the highest weight (50%), followed by page type (20%), reputation (15%), popularity (10%), and IAB category (5%). For a marketing research agent, IAB category might be weighted highest because the primary concern is task-scope compliance rather than threat avoidance.
The composite risk score produces a single numerical value — say 0 to 100 — that the policy engine evaluates against three thresholds: allow (0-29), review (30-69), and block (70-100). This three-tier model provides the nuance that binary allow/block systems lack. The "review" tier is particularly valuable for domains that are ambiguous — they might be safe, but the intelligence signals are mixed, and a human should confirm before the agent proceeds.
Domain intelligence is most valuable when it is current. The 102M domain database is refreshed quarterly, which means that new domains are classified, existing classifications are updated to reflect content changes, and domains that have gone offline are removed. For the interval between database updates, the real-time API provides on-demand classification for any domain not in the local database. This hybrid model ensures that your agent security posture is never more than one API call away from a classification for any domain on the internet.
Security operations centers (SOCs) deploying AI agents at enterprise scale need domain intelligence to extend their existing monitoring and response capabilities to agent traffic. Without domain intelligence, agent web activity is a blind spot — the SOC can see that the agent made HTTP requests but has no context about what those requests were for or whether they represent a security concern.
Chief Information Security Officers (CISOs) evaluating agentic AI deployments need domain intelligence as part of their risk assessment framework. The intelligence feed provides the data that risk scoring models require to quantify the exposure created by agent web access. Without this data, the risk assessment defaults to worst-case assumptions that may block productive agent deployments entirely.
AI platform vendors building agent orchestration products need domain intelligence feeds to differentiate their security offerings. Enterprise buyers expect domain-level visibility and control as table stakes for any agent platform. The 102M domain database provides this capability as a turnkey data product that vendors can integrate without building their own classification infrastructure.
Five intelligence signals per domain, 102 million domains classified, sub-millisecond local lookups. Transform your agent security from reactive to proactive.