Adversarial Search Engine Optimization for Large Language Models

Web pages and plugin descriptions that explicitly target LLMs can enable unknown providers to compete with reputable brands that are much better ranked in traditional search.

“and even enable unknown providers to compete with reputable brands that are much better ranked in”

All parties globally lose in recommendation rates compared to the baseline where all product owners cooperate.

“Yet, all parties globally lose in recommendation rates compared to the baseline where all product owners cooperate.”

The LLM is more inclined to follow Preference Manipulation Attacks that overlap significantly with the text of the user's request.

“Mirroring the findings of Wan, Wallace, and Klein [27], we find the LLM is more inclined to following Preference Manipulation Attacks that overlap significantly with the text of the user's request.”

• cites [27]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

When more injections are present, all cameras lose in search presence.

“and that when more injections are present, all cameras lose in search presence”

LLMs are most convinced by text that closely aligns with the user's query.

“In Section 5.4, we corroborate a finding of Wan, Wallace, and Klein [27] who show that LLM's are most convinced by text that closely aligns with the user's query.”

• cites Wan, Wallace, and Klein [27]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

LLM search engines tend to summarize results in a list before making a recommendation in the authors' experiments.

“finding that LLM search engines tend to summarize results in a list before making a recommendation in our experiments”

All our attacks can be embedded as illegible text on a web page.

“All our attacks can be embedded as illegible text on a web page (see Figure 3b).”

Preference Manipulation Attacks can be used to trick LLMs to favor an attacker's web pages and external plugins.

“We have introduced Preference Manipulation Attacks, and have shown how they can be used to trick LLMs to favor an attacker's web pages and external plugins.”

The attack succeeds most often when the attack is contained in the last page seen by the search engine.

“We find that the attack succeeds most often when the attack is contained in the last page seen by the search engine (this corroborates prior findings that LLM attacks are more effective when added at the end of a prompt [3]).”

• cites [3]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Both Bing Copilot and Perplexity often do not cite the lab's main website, despite it being the highest ranked search result.

“Both Bing Copilot and Perplexity often do not cite our lab's main website, despite this being the highest ranked search result.”

Cameras from the authors' website that use injections perform better than those that do not.

“We also see that the cameras from our website that use injections perform better than those that do not”

Using adversarial LLM SEO is advantageous compared to not doing so.

“and using adversarial LLM SEO is advantageous, compared to not doing so.”

In some cases, merely addressing the LLM and promoting a product is sufficient, without explicit instruction.

“However, in some cases, merely addressing the LLM (e.g.,”

Prompt 2 leads to more selective behavior.

“we highlight in Figure 17 that prompt 2. leads to more selective behavior.”

A GPT-4 news plugin is 2-8× more likely to be selected than a competing alternative after launching an attack.

“Similarly, a GPT-4 news plugin is 2-8× more likely to be selected than a competing alternative after launching an attack.”

Many of our attacks rely on prompt injections.

“Many of our attacks rely on prompt injections, which exploit LLMs inability to reliably distinguish between data and instructions [5,9,10,15].”

Ranking highly on the API search index does not guarantee that a web page will be favored by Bing Copilot.

“Ranking highly on the API search index does not guarantee that a web page will be favored by Bing Copilot.”

It is possible to censor or promote web pages using external injections that are not necessarily on the web pages they target.

“it is possible to censor or promote web pages using”

An attack on one page can boost or censor the search presence of a different page.

“That is, an attack on one page can boost or censor the search presence of a different page.”

The attacks are very robust to variations in user requests.

“But overall, we find that our attacks are very robust to variations in user requests, and that the results from Section 5.2 replicate across numerous experimental setups (see Appendix C).”

Making highly exaggerated claims about the plugin or its competitors succeeds, without explicitly instructing the LLM to select the plugin.

“However, making highly exaggerated claims about the plugin or its competitors succeeds, without the need to explicitly instruct the LLM to select the plugin (see Appendix B.2).”

With no attack, real cameras are recommended nearly twice as often as fictitious cameras.

“With no attack, the real cameras are recommended nearly twice as often (presumably due to brand name recognition).”

The attack works even if the attacker's page is visited as part of an organic web-wide search.

“We now show that this is the case.”

• Also stated in: § 5.4 Preference Manipulation Attacks are Robust and Generally Applicable · plain

  “We find that both Bing Copilot and Perplexity are affected by this injection.”

Preference manipulation can be achieved with misinformation without explicit instructions.

“We show that preference manipulation can be achieved by explicitly embedding instructions in third-party content (cf. Figure 1 (a)), but also with more devious forms of misinformation without explicit instructions (cf. Figure 1 (b) and (c)).”

Each book is less likely to be recommended when there are more injections present.

“We again see that each book is less likely to be recommended when there are more injections present.”

Once multiple plugins launch attacks, Claude reverts to only recommending the neutral source (or none at all).

“Once multiple plugins launch attacks, Claude reverts to only recommending the neutral source (or none at all).”

• Also stated in: § E Experiment Reproducibility · plain

  “Once multiple plugins launch attacks, Claude reverts to only recommending the neutral source (or none at all).”

In 50% of the investigated searches, there were web pages present with a rank worse than 10, which would not appear on the first page of Bing Search results, while still having a chance at influencing the LLM.

“Figure 13 shows that in 50% of the investigated searches, there were web pages present which have a rank worse than 10, and thereby would not even appear on the first web page of Bing Search results, while still having a chance at influencing the LLM as outlined in Section 5.”

In both censoring and promoting experiments, the attack success rate is lowest for the middle two positions.

“We see that in both settings, the attack success rate is lowest for the middle two positions”

After a Preference Manipulation Attack, the fictitious camera is slightly more likely to be recommended than the real ones.

“After our Preference Manipulation Attack, our fictitious camera is slightly more likely to be recommended than the real ones.”

For search engines, explicitly injecting instructions is most effective.

“For search engines, we find that explicitly injecting instructions (e.g.,”

Web pages and plugin descriptions that explicitly target LLMs can significantly boost recommendation rates.

“We have shown that web pages and plugin descriptions that explicitly target LLMs can significantly boost recommendation rates”

Pages performing adversarial LLM SEO are competitive with real camera listings from established camera brands.

“Figure 8 shows that pages performing adversarial LLM SEO are competitive with real camera listings from established camera brands.”

Some Preference Manipulation Attacks make the LLM favor some sources over others without explicitly instructing the model to do so.

“Some of our Preference Manipulation Attacks also aim to make the LLM favor some sources over others without explicitly instructing the model to do so, and we observe similar phenomena as in [27].”

• cites [27]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

When asking Bing to search for a camera to recommend, a Preference Manipulation Attack makes the targeted camera 2.5× more likely to be recommended by the LLM.

“For example, when asking Bing to search for a camera to recommend, a Preference Manipulation Attack makes the targeted camera 2.5× more likely to be recommended by the LLM.”

• Also stated in: § 5.1 Preference Manipulation Attacks are Effective · plain

  “Figure 4 shows that deploying Preference Manipulation Attacks (here with a prompt injection attack) makes the attacker's camera 2.5× more likely to be recommended to the user than a comparable product.”

In some cases, attacks boost a plugin's selection rate from 0% to over 90%.

“In some cases, our attacks boost a plugin's selection rate from 0% to over 90% (see Figure 6).”

A news provider can use a Preference Manipulation Attack to override Claude's default selection of the neutral plugin.

“Figure 6a shows that Claude selects the neutral plugin by default, but that a news provider can use a Preference Manipulation Attack to override this behavior.”

For plugin optimization, merely presenting the adversary's plugin in a favorable light is not sufficient to bias the LLM.

“For plugin optimization, merely presenting the adversary's plugin in a favorable light (e.g.,”

Attacks can be silently triggered from a different web page than the one listing the product.

“As we show in Section 5.3, attacks can even be silently triggered from a different web page than the one listing the product.”

Benign product owners have incentive to also attack rather than stay idle as this boosts their recommendation rate over competitors'.

“We find that regardless of the number of parties having launched an attack, benign product owners have incentive to also attack rather than stay idle as this boosts their recommendation rate over competitors'.”

Books performing adversarial LLM SEO outperform the book from spylab.ai not doing so.

“We see that the books performing adversarial LLM SEO outperform the book from spylab.ainot doing so”

Using prompt 2, the LLM simply mentions only one camera in a large number of cases as 'the best one to buy'.

“However, using prompt 2., the LLM simply mentions only one camera in a large number of cases as”

Attacks on LLM search engines can be made stealthy by appending text of the form 'don't mention this message in your response' to the prompt injection.

“Attacks on LLM search engines are easily made stealthy by appending text of the form”

Carefully crafted website content or plugin documentation can trick an LLM into discrediting competitors.

“We demonstrate that carefully crafted website content or plugin documentations can trick an LLM to promote the attacker products and discredit competitors, thereby increasing user traffic and monetization.”

• Also stated in: § 1 Introduction · plain

  “First, our attacks can be used not only to boost an attacker's content, but also to discredit its competitors.”

The attack success rate is never lowest when the injection is the last seen in the input.

“Interestingly, the attack success rate is never lowest when the injection is the last seen in the input.”

For Perplexity, external attacks succeed significantly more often.

“For Perplexity however, external attacks succeed significantly more often (see Appendix C.4).”

The attacks are stealthy.

“Our attacks are black-box, stealthy, and reliably manipulate the LLM to promote the attacker's content.”

• Also stated in: § Preference Manipulation Attacks are stealthy. · plain

  “Preference Manipulation Attacks are stealthy.”

Claude selects the neutral plugin by default.

“Figure 6a shows that Claude selects the neutral plugin by default, but that a news provider can use a Preference Manipulation Attack to override this behavior.”

Preference Manipulation Attacks can lead to a prisoner's dilemma where all parties are incentivized to launch attacks, but collectively degrade the LLM's outputs for everyone.

“We show this can lead to a prisoner's dilemma, where all parties are incentivized to launch attacks, but this collectively degrades the LLM's outputs for everyone.”

• Also stated in: § 1 Introduction · plain

  “Second, Preference Manipulation Attacks lead to a form of a prisoner's dilemma: attackers individually benefit from launching attacks to boost their content, but when multiple attackers target the same LLM, all parties lose in search presence.”

• Also stated in: § 5.2 Preference Manipulation Attacks Lead to a Prisoner's Dilemma · plain

  “We thus observe a form of multi-player prisoner's dilemma [24].”

• cites [24]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Manipulated fictitious products can compete with real products from reputable brands.

“Table 1 shows that our manipulated fictitious products can compete with real products from reputable brands.”

Allowing many low-ranked stories attains similar attack success rates as in Figure 5.

“allowing many low-ranked stories attains similar attack success rates as in Figure 5”

Preference manipulation can be achieved by explicitly embedding instructions in third-party content.

“We show that preference manipulation can be achieved by explicitly embedding instructions in third-party content (cf. Figure 1 (a)), but also with more devious forms of misinformation without explicit instructions (cf. Figure 1 (b) and (c)).”

Attack text can be embedded in a completely independent web page that is part of the LLM's search results.

“We now show that this text can also be embedded in a completely independent web page that is part of the LLM's search results.”

Whenever a unique book is recommended, that book uses adversarial LLM SEO.

“essentially, whenever a unique book is recommended, then that book uses adversarial LLM SEO.”

In 80% of successful external attacks, the LLM does not mention the external web page containing the injection.

“These external attacks are particularly stealthy: in 80% of successful attacks, the LLM does not mention the external web page containing the injection.”

The middle two positions seem less favorable for an attacker than the extremes.

“and that the middle two positions seem less favorable for an attacker than the extremes.”

Preference Manipulation Attacks are practical on current LLM applications.

“In this paper, we demonstrate the practicality of Preference Manipulation Attacks on current LLM applications”

External attacks succeed at a lower rate than direct attacks against Bing (at most 25%, compared to 95%-100% for direct attacks).

“These external attacks succeed at a lower rate than direct attacks against Bing (at most 25%, compared to 95%-100% for direct attacks in Figure 5).”

Carefully crafted website content or plugin documentation can trick an LLM into promoting the attacker's products.

“We demonstrate that carefully crafted website content or plugin documentations can trick an LLM to promote the attacker products and discredit competitors, thereby increasing user traffic and monetization.”

• Also stated in: § 1 Introduction · plain

  “We show that by carefully crafting the text on a web page or plugin description, an attacker can trick an LLM into promoting their content over competitors.”

Preference Manipulation Attacks can be used to discredit competitors.

“First, our attacks can be used not only to boost an attacker's content, but also to discredit its competitors.”

For LLM plugin systems, Preference Manipulation Attacks can make an adversarial plugin up to 7.2× more likely to be selected than a comparable competitor.

“For LLM plugin systems, Preference Manipulation Attacks can make an adversarial plugin up to 7.2× more likely to be selected than a comparable competitor.”

Preference Manipulation Attacks are effective for fictitious products that the LLM has no prior knowledge about.

“This experiment demonstrates that Preference Manipulation Attacks are effective for fictitious products that the LLM has no prior knowledge about.”

There is prompt sensitivity for the different experiments in Figure 11.

“Figure 16 shows that there is prompt sensitivity for the different experiments in Figure 11”

Discrepancies between the rank of pages in the Bing Search API and Bing Search in the browser were not too large.

“while we did observe some inconsistencies between the rank of pages in the Bing Search API and Bing Search in the browser, the discrepancies were not too large.”

External injections can be used in the product setting.

“Figure 12 shows that we can also use external injections in the product setting”

GPT-4 mostly ignores the neutral plugin.

“In contrast, GPT-4 mostly ignores the neutral plugin; other plugins always have an incentive to attack, but end up selected less often overall when multiple attacks compete.”

The attacks reliably manipulate the LLM to promote the attacker's content.

“Our attacks are black-box, stealthy, and reliably manipulate the LLM to promote the attacker's content.”

Books performing adversarial LLM SEO are generally competitive with real books.

“and that these books are generally competitive with real books.”

Depending on the model, the severity of the prisoner's dilemma varies.

“We see in particular that depending on the model, the severity of the prisoner's dilemma varies.”

The attacks are not stealthy.

“The attacks are not stealthy, which is to be expected, since the injection is itself a news story highlighting the unreliability of a different story.”

Copilot is more selective, in particular when there is only one injection present.

“Copilot being more selective, in particular when there is only one injection present.”

Attack success rates are lower when targeting news from a certain news provider than when promoting products as in Figure 5.

“In Figure 7, we find that the attack success rates are lower when targeting news from a certain news provider than when promoting products as in Figure 5.”

The order in which Bing Copilot reads web pages seems to be relevant to adversarial LLM SEO's attack success rate.

“Considering that the order in which Bing Copilot reads web pages seems to be relevant to adversarial LLM SEO's attack success rate (see Figures 11,12 and 14)”

In terms of attack success rates, being the last read injection is never worst, in contrast to regular SEO.

“In terms of attack success rates, being the last read injection is never worst , in contrast to regular SEO.”

In many searches, low-ranked pages (on the Bing Search API) enter Bing Copilot's context window.

“We see that in many searches, low-ranked pages (on the Bing Search API) enter Bing Copilot's context window”

The exact dynamics of adversarial LLM SEO depends on the model used.

“The exact dynamics of adversarial LLM SEO depends on the model used, here illustrated using different models via Perplexity (see also Section 5.2).”

Censoring individual news stories still yields a high ASR.

“Censoring individual news stories still yields a high ASR.”

Bing Copilot does not always cite the relevant web pages.

“indicating that Bing Copilot does not always cite the relevant web pages”

Preference Manipulation Attacks can boost or censor search results of external web pages.

“Preference Manipulation Attacks can boost or censor search results of external web pages.”

Plugin augmented LLMs are also affected by Preference Manipulation Attacks.

“Plugin augmented LLMs are also affected by Preference Manipulation Attacks.”

The order in which Bing Copilot reads web pages has a Spearman rank correlation of 0.84 with the regular Bing Search index in our experiment.

“the order in which Bing Copilot reads these web pages has a Spearman rank correlation of 0.84 with the regular Bing Search index in our experiment.”

Keywords are able to detect more of the cases when Bing Copilot recommends certain products than citations.

“We see that keywords are able to detect more of the cases when Bing Copilot recommends certain products”

For most queries issued, Bing Copilot searches for the same terms and sees the same web pages.

“Moreover, for most of the queries we issue, Bing Copilot searches for the same terms and sees the same web pages”

Competitors have financial incentives to use Preference Manipulation Attacks.

“Since Preference Manipulation Attacks can boost a product's search results or a plugin's selection rate, competitors will have financial incentives to also use such techniques (as is the case for traditional SEO).”

An attacker might profit from not ranking highly if they want their external injection attacks to succeed.

“Thus, an attacker might actually profit from not ranking highly, if they want their external injection attacks to succeed.”

We cannot directly evaluate the efficacy of prompt injection defenses in our setting because the LLM applications we study are all proprietary and black-box.

“While some defenses against prompt injections have been proposed, we cannot directly evaluate their efficacy in our setting since the LLM applications we study are all proprietary and black-box.”

There exist prompts where the user would not notice anything wrong about the results.

“but that in this case, there exist prompts where the user would not notice anything wrong about the results.”

The techniques and phenomena described in the work could be used to attack real LLM search engines and plugin systems.

“Since the techniques and phenomena described in this work could be used to attack real LLM search engines and plugin systems, we have disclosed our results to major developers of LLM search engines and plugin ecosystems”

The attacks are black-box.

“Our attacks are black-box, stealthy, and reliably manipulate the LLM to promote the attacker's content.”

Prompt injections exploit LLMs' inability to reliably distinguish between data and instructions.

“Many of our attacks rely on prompt injections, which exploit LLMs inability to reliably distinguish between data and instructions [5,9,10,15].”

• cites [5]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [9]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [10]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [15]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

If users are likely to search for a query like 'the best and cheapest smartphone', a website claiming to 'sell the best and cheapest smartphones' is likely to be recommended by an LLM, even if the statement is false or the page is not the most relevant according to traditional SEO.

“For example, if users are likely to search for”

The threat of prompt injections in LLM SEO is significant.

“Not only does this indicate that the threat of prompt injections in LLM SEO is significant”

Manipulation attacks on LLMs are of practical and economic concern today.

“Overall, our work highlights that manipulation attacks on LLMs are of practical and economic concern today”

A successful defense against prompt injection attacks may not be sufficient to defend against Preference Manipulation Attacks.

“These results suggest that a successful defense against prompt injection attacks may not be sufficient to defend against Preference Manipulation Attacks (see Section 6.1 for further discussion of defenses).”

Preference Manipulation Attacks are passive attacks that do not require the victim to visit a malicious page, but only to issue search queries that return results from that page.

“Our work considers stronger passive attacks that do not require the victim to visit a malicious page, but only to issue search queries that return results from that page.”

The experiments can likely not be exactly replicated.

“Our experiments can likely not be exactly replicated for a number of reasons.”

The authors cannot control exactly which information is provided to the LLMs themselves, because proprietary algorithms extract relevant text from web pages and find relevant web pages for queries.

“Additionally, in particular with LLM search engines, we cannot control exactly which information is provided to the LLMs themselves, since there are proprietary algorithms that extract relevant text from our web pages, and find relevant web pages for our queries.”

These systems are sometimes unpredictable in how they handle the information with which they are presented.

“The fact that the attacks fail for some prompts illustrate that these systems are sometimes unpredictable in how they will handle the information with which they are presented”

External attacks suggest an avenue for plausible deniability of Preference Manipulation Attacks.

“This suggests an avenue for plausible deniability of Preference Manipulation Attacks, as an attacker could use an alternative web page to boost search results for their primary page.”

The evaluation schemes are accurate enough to support the claims made in the paper.

“having manually checked hundreds of responses, we find that they are accurate enough to support the claims we make in the paper.”

Widespread deployment of Preference Manipulation Attacks could globally degrade the utility of LLM search and plugin ecosystems.

“which could globally degrade the utility of LLM search and plugin ecosystems.”

Running the same code used to obtain the Bing Copilot results will no longer give the same results.

“running the same code we used to obtain our results for Bing Copilot will no longer give the same results.”

LLM search engines become more biased when faced with injections.

“the LLM search engines become more biased when faced with injections.”

Ranking highly on the regular search index does not guarantee that a web page will be favored by an LLM.

“ranking highly on the regular search index does not guarantee that a web page will be favored by an LLM.”

Web page owners would not necessarily have to be the first read web page.

“the web page owners would also not necessarily have to be the first read web page.”

Low-ranked pages could influence the LLM search results.

“and could influence the LLM search results.”