Google search results and OpenAI’s Responses API matched just 26.2% of the pages cited by ChatGPT during tests of its GPT-5.5 Instant setting, according to a preprint published on October 11 by researchers at the National University of Singapore and Xi’an Jiaotong University.
How Researchers Tracked ChatGPT Citations Across Four Model Versions
Researchers recorded what searches ChatGPT executed, which web pages resulted from those queries, and which specific pages the AI ultimately selected as sources. The evaluation used 80 questions. The dataset comprised queries spanning four distinct request types: recommendations, fact lookups, information synthesis, and analysis.
The study tested four versions of ChatGPT, running each of the 80 questions 20 times on GPT-5.3 Instant and GPT-5.5 Instant with web search enabled. GPT-5.2 Instant and GPT-5.4 Thinking had fewer questions run due to being decommissioned prior to completion of data collection between March 29 and July 29, 2026. To create substitutes, researchers retrieved up to five pages of Google’s organic results per question through SerpAPI, yielding about 43 results per question on average. They also submitted 20 queries per question to OpenAI’s Responses API within three days of running the exact same queries on ChatGPT.
API and Google Search Results Compared Against ChatGPT Web App Citations
Combining Google search results and OpenAI’s Responses API revealed a gap when matched against ChatGPT’s outputs. Across the evaluated runs, the external tools found 28.3% of the pages cited by GPT-5.3 Instant and 26.2% of those cited by GPT-5.5 Instant. At the domain level, the combination captured 50.3% of cited sites for GPT-5.3 Instant and 48.7% for GPT-5.5 Instant, treating subdomains as separate sites.
Google search results alone accounted for 9.4% of cited pages for GPT-5.3 Instant and 6.4% for GPT-5.5 Instant, while covering 26.1% and 22.8% of cited sites respectively. OpenAI’s API source list accounted for 22.9% of cited pages for the GPT-5.3 model and 22.5% for GPT-5.5. When comparing the API’s own answers directly to ChatGPT’s selections, GPT-5.5 Instant cited 23.3% of the pages the API cited, compared to 16.4% for GPT-5.3 Instant.
Prompt Rewriting and Language Shifts During Model Execution
The study documented how ChatGPT transforms original inputs before querying external providers. OpenAI’s help documentation notes that the chatbot typically converts a user prompt into one or more specific search terms.
Furthermore, individual runs showed high variability: 44.5% of distinct pages returned for a given question appeared in only one of the 20 GPT-5.5 Instant runs, while 40.8% appeared in a single run of GPT-5.3 Instant.
Recommendations for Monitoring AI Search Visibility
The preprint authors say monitoring strategies should be re-evaluated when the model behind ChatGPT changes. The findings underscore that tracking tools relying solely on API queries or standard search engine rankings may not fully replicate what users see inside the ChatGPT web application.