URLertBot

Automated Security Scanner Policy

URLertBot is the automated security scanner for URLert.com. We analyze URLs to detect malicious content, including phishing, scams, and malware distribution.

Purpose & Mission

The primary purpose of URLertBot is Public Safety. We scan websites to identify threats that endanger internet users, organizations, and platforms.

  • Category: Phishing, Scam, & Malware Detection
  • Benefit: Our data protects the internet ecosystem by:
    • Protecting Users: Verifying links in emails or messages (e.g., detecting fake banking login pages) before a user clicks.
    • Securing Platforms: Helping public sharing services and platforms identify if users are hosting malicious data or bad links.
    • Assisting Owners: Alerting domain owners if their infrastructure has been compromised or injected with malicious code.

Identification

URLertBot identifies itself clearly in the HTTP User-Agent header.

User-Agent: Mozilla/5.0 (compatible; URLertBot/1.0; +https://urlert.com/bot)

Verification (IP Addresses)

To verify that traffic is genuinely from URLert, checking the User-Agent is not sufficient as it can be spoofed. We provide a strictly defined list of IP addresses that our official bot operates from.

You can validate our bot using our Public IP List:

Any request claiming to be URLertBot that does not originate from an IP in this list should be considered spoofed.

Behavior & Ethics

We strive to be a "Good Citizen" of the internet. Our scanning behavior is designed to be non-intrusive.

  • Robots.txt Compliance: URLertBot respects the standard robots.txt exclusion protocol.
  • Traffic Volume: We do not index entire websites. We only fetch specific URLs that have been submitted for security analysis.
  • Abuse Prevention: Our bot is not used for credential stuffing, inventory scalping, or aggressive scraping. It is strictly a read-only analysis tool used to detect threats.

Controlling URLertBot

Site owners can control how URLertBot interacts with their site using robots.txt.

To block URLertBot:

User-agent: URLertBot
Disallow: /

Note: Blocking security scanners may prevent you from receiving timely alerts if your site is compromised or hosting malicious content.

Contact

If you believe URLertBot is malfunctioning, accessing sensitive data, or causing issues for your infrastructure, please contact our engineering team immediately.