Federal Register Access – How to Request

U.S. Government Websites Implement Stricter Access Controls Amidst Automated Scraping Surge

Access too crucial U.S. government information resources, specifically FederalRegister.gov and eCFR.gov, is now being tightly controlled due to a significant increase in automated data scraping. The move aims to protect the integrity and availability of these vital public databases.

The increased security measures, announced today, limit programmatic access – automated methods of retrieving data – to users who utilize the government’s established developer APIs. these Application Programming Interfaces offer a structured and authorized pathway for accessing information, preventing the overwhelming of systems by unchecked scraping activities.

According to an official statement, the surge in “aggressive automated scraping” prompted the change. The government is directing those seeking programmatic access to detailed documentation for both the Federal Register API and the eCFR API,available on their respective websites.

Did you know? – The Federal Register and eCFR contain rules and regulations impacting nearly every aspect of American life,from environmental protections to financial regulations.

Users attempting to access the sites may now encounter a CAPTCHA – a “bot test” – designed to verify they are human and not an automated program. A message displayed to users explains that completing the CAPTCHA and clicking “Request Access” is necessary. The system acknowledges that users might potentially be asked to complete the test periodically, emphasizing this is a standard security precaution.

The restrictions currently apply to requests originating from the IP address 103.115.9.216. Individuals needing access for a broader range of IP addresses are instructed to first request access for their current IP, and then utilize the “Site Feedback” button located on the lower left-hand side of the websites to submit a request for expanded access.

Pro tip – Before requesting expanded IP access, ensure your organization’s IT department understands the API usage guidelines to avoid future disruptions.

The implementation of these measures-APIs and CAPTCHAs-represents a response to a significant increase in automated scraping activity targeting these government resources. The National Archives and Records Administration (NARA), which manages these sites, confirmed the surge began in recent weeks, overwhelming the systems and potentially impacting legitimate users. NARA officials have not publicly identified the source of the scraping, but stated it was “aggressive” and designed to extract large volumes of data quickly. The situation prompted an immediate review of security protocols and the implementation of the current access controls.

As of today, the restrictions remain in place, and NARA is monitoring the situation closely. The agency has indicated it will continue to evaluate the effectiveness of the new measures and adjust them as needed to balance public access with system security. while some users have reported minor inconveniences due to the CAPTCHAs, the overall impact on access to the Federal Register and eCFR has been minimal.NARA expects the situation to stabilize as users adapt to the new access procedures and the scraping activity subsides.

This growth underscores the growing challenge of balancing public access to information with the need to protect government websites from malicious or disruptive automated activity. The implementation of APIs and CAPTCHAs represents a proactive step towards ensuring the continued availability and reliability of these essential resources for researchers,developers,and the public alike.