AI agents browsing the web inevitably encounter login pages, SSO portals, and authentication screens. Without page-type detection, they may attempt to interact with credential forms — triggering account lockouts, security alerts, and compliance violations. Page-type classification from a 102M domain database gives your agent harness the ability to identify and block auth pages before the agent ever reaches them.
A URL like accounts.google.com/signin or login.microsoftonline.com tells a human everything. To an AI agent, it is just another URL to fetch.
Every website of any significance has a login page. When an AI agent is tasked with research, data collection, or competitive analysis, it follows links across dozens of websites. Some of those links — in navigation menus, footer sections, or inline text — point to login pages. The agent does not recognize the semantic meaning of a login form. It sees input fields, buttons, and text. Depending on its framework and prompt, it may attempt to fill those fields, click the login button, or simply render the page and scrape its contents.
Our 102M domain database includes page-type classification for every domain — identifying login pages, signup forms, SSO portals, admin panels, checkout flows, and 15+ other page types. When your agent's middleware queries the database before navigating to a URL, the response includes the page-type label. A simple rule — "if page_type equals login or signup, block navigation" — prevents the agent from ever reaching an authentication surface.
This detection is pre-computed and deterministic. It does not rely on the agent parsing the page's HTML to identify form fields. It does not require the agent to understand what a login form looks like. The database has already classified the page type offline using content analysis, URL pattern matching, and structural page features. Your agent gets a definitive answer — "this is a login page" — before it makes the HTTP request, not after it has already loaded the page and potentially interacted with it.
Three mechanisms that identify and prevent agent interaction with login and auth surfaces
The database identifies pages classified as "login" across all 102 million domains. This includes standard login forms (/login, /signin, /auth), SSO portals (accounts.google.com, login.microsoftonline.com), OAuth endpoints, and custom authentication screens. A single blocking rule prevents your agent from navigating to any of these pages, regardless of the domain they belong to.
Registration and signup pages present the same risks as login pages — input fields that agents may attempt to fill, terms of service checkboxes that agents may click, and payment forms embedded in premium signup flows. The database classifies these as "signup" page types, allowing you to block them with the same policy rule you use for login pages.
Beyond login pages, the database detects admin panels and settings pages — interfaces that typically require authentication to access and that agents should never reach. Pages classified as "admin" or "settings" represent management interfaces where any agent interaction could modify system configurations, user permissions, or account settings.
Production-ready snippets for preventing AI agents from reaching authentication pages
import http.client
import json
from datetime import datetime
class AuthPageBlocker:
"""Blocks AI agents from accessing login, signup,
and authentication pages using page-type detection."""
AUTH_PAGE_TYPES = [
"login", "signup", "admin", "settings",
"checkout" # Also block payment flows
]
def __init__(self, api_key):
self.api_key = api_key
self.conn = http.client.HTTPSConnection(
"www.websitecategorizationapi.com"
)
self.blocked_attempts = []
def check_for_auth_page(self, target_url):
"""Returns (is_auth_page, page_type, details)"""
payload = (
f"query={target_url}"
f"&api_key={self.api_key}"
f"&data_type=url"
f"&expanded_categories=1"
)
headers = {
"Content-Type": "application/x-www-form-urlencoded"
}
self.conn.request(
"POST",
"/api/iab/iab_web_content_filtering.php",
payload,
headers
)
res = self.conn.getresponse()
data = json.loads(res.read().decode("utf-8"))
page_type = data.get("page_type", "unknown")
is_auth = page_type in self.AUTH_PAGE_TYPES
if is_auth:
self.blocked_attempts.append({
"url": target_url,
"page_type": page_type,
"timestamp": datetime.utcnow().isoformat(),
"action": "blocked"
})
return is_auth, page_type, {
"categories": data.get("iab_classification", []),
"domain_rank": data.get("global_rank", None)
}
# Usage in agent middleware
blocker = AuthPageBlocker(api_key="your_api_key")
is_auth, ptype, info = blocker.check_for_auth_page(
"https://accounts.google.com/signin"
)
if is_auth:
print(f"BLOCKED: {ptype} page detected — agent "
f"navigation prevented")
else:
print("Page type safe — agent may proceed")
class LoginPageDetector {
constructor(apiKey) {
this.apiKey = apiKey;
this.authPageTypes = new Set([
"login", "signup", "admin",
"settings", "checkout"
]);
this.detectionLog = [];
}
async isAuthPage(targetURL) {
const response = await fetch(
"https://www.websitecategorizationapi.com" +
"/api/iab/iab_web_content_filtering.php",
{
method: "POST",
headers: {
"Content-Type": "application/x-www-form-urlencoded"
},
body: new URLSearchParams({
query: targetURL,
api_key: this.apiKey,
data_type: "url",
expanded_categories: "1"
})
}
);
const data = await response.json();
const pageType = data.page_type || "unknown";
const isAuth = this.authPageTypes.has(pageType);
this.detectionLog.push({
url: targetURL,
pageType: pageType,
isAuthPage: isAuth,
action: isAuth ? "blocked" : "allowed",
timestamp: new Date().toISOString()
});
return {
blocked: isAuth,
pageType: pageType,
reason: isAuth
? `Authentication page detected (${pageType})`
: "Non-auth page — navigation permitted"
};
}
}
// Usage
const detector = new LoginPageDetector("your_api_key");
const result = await detector.isAuthPage(
"https://login.microsoftonline.com"
);
console.log(result.reason);
Purpose-built domain databases for AI agent filtering. Includes IAB categories, 20+ page types, reputation scores, and popularity rankings. One-time purchase with perpetual license.
10 Million Domains with Page-Type Intelligence
One-time purchase: Perpetual license | Optional Updates: $1,599/year
20 Million Domains with Full Intelligence Suite
One-time purchase: Perpetual license | Optional Updates: $2,999/year
50 Million Domains with Complete Intelligence Suite
One-time purchase: Perpetual license | Optional Updates: $4,999/year
Also available: Enterprise URL Database up to 102M domains from $2,499. View all database tiers →
Search any IAB or Web Filtering category to see how many domains have login pages your agents might encounter.
How 102 million domains from our main Enterprise Database are distributed across IAB v3 taxonomy classifications
Spanning Tier 1 through Tier 4 classifications from our 102M Enterprise Database
Charts display domain counts for the top 50 out of 700+ categories in our 102M Enterprise Database. To check the number of domains for the remaining 650+ categories, use the Category Counter tool above .
Login pages are the most ubiquitous interactive surface on the web. According to HTTP Archive data, over 85% of websites in the top 1 million have at least one login or authentication endpoint. This means that for any browsing task involving multiple websites, an AI agent has a near-certain probability of encountering at least one login page. Without page-type detection, each encounter is an uncontrolled event where the agent's behavior is unpredictable.
The risk is not theoretical. Early deployments of browser-using AI agents — including OpenAI's Operator, Anthropic's Computer Use, and various open-source frameworks — have all documented incidents where agents attempted to interact with login forms. In some cases, agents filled username fields with search queries. In others, agents clicked "forgot password" links, triggering password reset emails. In the most concerning cases, agents with stored credentials in their environment submitted those credentials to the wrong login form entirely.
The naive approach to blocking login pages is URL pattern matching — block any URL containing "/login," "/signin," "/auth," or "/sso." This approach catches perhaps 40% of login pages. It misses the other 60% because login pages use an enormous variety of URL patterns. Salesforce uses "/secur/login_portal.htm." Microsoft uses "/common/oauth2/authorize." Many SaaS applications use "/session/new" or "/users/sign_in." WordPress sites use "/wp-login.php." Custom-built authentication systems use whatever URL path the developer chose — "/portal," "/access," "/welcome," "/start," and thousands of other variations.
Page-type detection in the 102M domain database solves this problem comprehensively. The classification system analyzes the actual content and structure of pages — form fields, button labels, page titles, meta tags, and structural HTML patterns — to determine the page type. It does not rely on URL patterns alone. The result is a page-type label that correctly identifies login pages regardless of their URL structure, including custom authentication systems, OAuth flows, and enterprise SSO portals.
Login pages are just the most obvious authentication surface. The full spectrum includes signup/registration pages (which present similar risks — form fields that agents may fill, terms of service checkboxes they may click), password reset pages (which may trigger email notifications to real users), multi-factor authentication pages (where agent interaction could lock out real users from their MFA tokens), session management pages (where an agent could inadvertently terminate active sessions), and API key management pages (where an agent could expose or rotate production credentials).
Our database classifies all of these under related page types — login, signup, admin, and settings — that collectively cover the authentication and account management surface. A single blocking rule that targets these four page types creates a comprehensive shield against agent interaction with any authentication-related interface.
A critical architectural decision is when to detect login pages: before the agent navigates (pre-navigation) or after the page loads (post-navigation). Post-navigation detection examines the loaded page's DOM, looking for login form elements, password fields, and authentication-related text. This approach is accurate but too late — the agent has already loaded the page, and in some cases, has already begun interacting with it.
Pre-navigation detection using the domain database checks the page type before the HTTP request fires. The agent's middleware intercepts the navigation intent, queries the database with the target URL, and blocks the request if the page type is login, signup, admin, or settings. The page never loads. The agent never sees the form fields. There is no opportunity for interaction with authentication elements. This is the fundamental advantage of database-backed page-type detection — it operates at the intent layer, not the rendering layer.
Enterprise AI agents encounter authentication pages from three primary sources. First, navigation links — most website navigation menus include a "Login" or "Sign In" link. Agents following navigation structures will encounter these links on virtually every commercial website. Second, redirect chains — some websites redirect unauthenticated users to login pages automatically. An agent requesting a page behind authentication may find itself on a login page via a 302 redirect, even though the original URL did not indicate an auth page. Third, in-page links — blog posts, documentation pages, and marketing sites frequently include "Sign up free" or "Start your trial" CTAs that link to registration forms.
All three sources are handled by pre-navigation page-type detection. The database classifies the destination URL regardless of how the agent discovered it — whether through menu navigation, redirect resolution, or inline link following. The blocking decision is the same: check the page type, block if it is an authentication surface.
Single Sign-On (SSO) portals and OAuth authorization endpoints are the most challenging authentication pages to detect because they exist on third-party domains that the agent may not expect to encounter. An agent navigating a SaaS application may be redirected to login.microsoftonline.com, accounts.google.com, or auth0.com for authentication — domains that are completely separate from the original navigation target. Without page-type detection that covers these identity provider domains, the agent would not know it has landed on a login page.
Our database includes page-type classifications for all major identity providers — Microsoft Entra ID, Google Workspace, Okta, Auth0, OneLogin, Ping Identity, and hundreds of enterprise SSO portals. The agent does not need to understand the OAuth protocol or recognize SSO redirect patterns. The database has already classified these endpoints as login pages, and your blocking rule handles them identically to any other login page.
Once deployed, measure the effectiveness of your auth page blocking by tracking four metrics. First, the blocking rate — what percentage of agent navigation attempts are blocked due to login/auth page detection. A healthy rate is typically 3-8% of total navigation events, reflecting the natural density of login pages on the web. If the rate is above 15%, the agent's task may need refinement to reduce exposure to authentication surfaces.
Second, track false positives — pages blocked as login pages that are actually not authentication surfaces. The database's pre-computed classification has a low false positive rate (under 1%), but monitoring helps identify any edge cases in your specific browsing patterns. Third, track false negatives — login pages that the agent successfully navigated to without being blocked. These represent gaps in the database's page-type coverage, which can be addressed by reporting the URLs to the classification service for inclusion in the next database update.
Fourth, track downstream impact — did auth page blocking reduce the number of security alerts from SSO systems? Did it eliminate credential-related incidents from agent deployments? These outcome metrics demonstrate the business value of page-type detection beyond the technical blocking statistics.
Deploy page-type detection to prevent your AI agents from ever reaching authentication surfaces. Pre-classified across 102 million domains. Zero agent-side parsing required.