Performance Optimization
Following these best practices ensures your integration performs optimally while minimizing costs and maximizing reliability. Whether you're building a real-time filtering system or a batch enrichment pipeline, these guidelines help you get the most from the API.
Implement Caching
Website categories typically remain stable over time. Implement caching to reduce redundant API calls and improve response times for frequently accessed domains.
Do
- Cache results for 24-48 hours for most use cases
- Use shorter TTLs (1-4 hours) for real-time applications
- Implement cache warming for known high-traffic domains
Don't
- Query the same domain repeatedly without caching
- Cache indefinitely without refresh mechanism
- Ignore cache hit rates in monitoring
Use Batch Processing
When processing multiple domains, use batch endpoints rather than individual requests. Batch processing is more efficient and cost-effective for bulk operations.
// Efficient: Batch request
POST /v1/batch/categorize
{ "domains": ["example1.com", "example2.com", "example3.com"] }
// Inefficient: Multiple individual requests
GET /v1/categorize?domain=example1.com
GET /v1/categorize?domain=example2.com
GET /v1/categorize?domain=example3.com
Normalize Domain Inputs
Ensure consistent domain formatting before API calls to maximize cache efficiency and avoid duplicate requests for the same domain.
function normalizeDomain(input) {
return input
.toLowerCase()
.replace(/^https?:\/\//, '')
.replace(/^www\./, '')
.split('/')[0];
}
// "HTTPS://WWW.Example.COM/page" → "example.com"
Error Handling
Implement Retry Logic
Network issues and transient errors happen. Implement retry logic with exponential backoff for resilient integrations.
async function fetchWithRetry(url, options, maxRetries = 3) {
for (let i = 0; i < maxRetries; i++) {
try {
const response = await fetch(url, options);
if (response.ok) return response.json();
if (response.status === 429) {
await delay(Math.pow(2, i) * 1000);
continue;
}
throw new Error(`HTTP ${response.status}`);
} catch (error) {
if (i === maxRetries - 1) throw error;
await delay(Math.pow(2, i) * 1000);
}
}
}
Respect Rate Limits
Monitor your API usage and implement proper rate limit handling to avoid service interruptions. See our rate limits guide for details.
Do
- Monitor rate limit headers in responses
- Implement request queuing for high-volume scenarios
- Use webhooks for async notification when available
Don't
- Ignore 429 responses and continue hammering
- Retry immediately after rate limit errors
- Build systems that assume unlimited capacity
Security Best Practices
Secure Your API Keys
Protect your API credentials to prevent unauthorized access and unexpected charges. See our authentication guide for detailed security guidance.
Do
- Store keys in environment variables
- Use secrets management services
- Rotate keys periodically
- Use separate keys for dev/prod environments
Don't
- Commit keys to version control
- Expose keys in client-side code
- Share keys between applications
- Log API keys in application logs