Active Vacancies Crawler Bot Verification
Active Vacancies operates an ethical, serverless web crawler and verification engine that aggregates publicly available career listings directly from official ATS APIs (Greenhouse, Lever, and Ashby). Continuous hourly crawling powers our 24/7 automated job alert radar, allowing candidates to be among the first applicants to newly published, verified openings that match their resumes.
Official User-Agent Identifiers
ActiveVacanciesBot/1.0 (+https://activevacancies.com/bot; bot@activevacancies.com)Dispatched on scheduled cron intervals to retrieve public JSON feeds and job boards from ATS partner platforms.
ActiveVacanciesVerificationBot/1.0 (+https://activevacancies.com/bot; bot@activevacancies.com)Executes lightweight HEAD and byte-limited GET probes to verify if previously indexed openings are still active.
Network Origin & Dedicated Static Egress
All outbound crawler traffic originates from our secured Google Cloud VPC in the us-central1 region, routed exclusively through dedicated Google Cloud NAT gateways with reserved static public IP addresses.
Google Cloud VPC + Cloud NAT
Region: us-central1 (Iowa)
crawler.activevacancies.com
Static Egress: 35.224.134.244
Crawl Governance & Politeness Policy
- Strict Token-Bucket Rate Limiting: Crawler requests are throttled through Cloud Tasks queues to a maximum of 1–2 requests per second per ATS host.
- HTTP 429 & Backoff Compliance: Our workers immediately halt and back off exponentially when an origin returns HTTP 429 (Too Many Requests) or HTTP 503.
- Bandwidth Conservation: Verification workers request HTTP HEAD first. If GET is needed, requests specify
Range: bytes=0-4096to inspect only header closures without downloading asset payloads. - Read-Only Public Data: The crawler never executes POST/PUT mutations, never submits application forms, and never bypasses authentication walls.
Webmaster Inquiries & Crawl Rate Adjustments
If you represent an employer, career portal, or ATS vendor and would like to adjust the crawl frequency, verify crawler legitimacy, or request exclusion from our index, our operations team will accommodate your request promptly.