I’ve been heavily testing Bing AI Chat since last week and it’s been fascinating to see how it works both in the search results and in chat mode. As I explained in my post about the “code red” that triggered thousands of “code reds” at publishers, AI in the SERPs is one of the most important, and monumental changes, we have ever seen in the search industry. And it can potentially have a huge impact on publishers across the web. The financial impact could be massive.

During Microsoft’s presentation in Seattle, they showed off Bing Chat in the Edge sidebar, but I hadn’t seen it live during my testing… until yesterday! This is where Bing Chat can act as an AI companion while you browse the web (sitting in the Edge sidebar). For example, you can use Bing Chat to answer questions based on the content you are viewing in a tab you have open in Edge. And that opens up a ton of opportunities to have Bing AI help you as you browse the web. For example, you can ask it to summarize the article you are reading, pick out the most important information, understand who the author is, and more.

Well, yesterday I took one more look at the sidebar in Edge to see if Bing Chat would show up, and I’m glad I did! I quickly noticed I had access to Bing Chat in the Edge sidebar! I shared a Twitter thread with some findings based on my testing, but I wanted to document those tests here on my blog.

Below I’ve included several screenshots of using Bing Chat in the Edge sidebar. Again, it’s like an AI companion as you browse the web. I’ll keep testing and adding to the this post.

Permission to access browser web content from the Edge sidebar:
First, Bing Chat in the Edge sidebar asked if it can access browser web content to “get the best experience with relevant responses”. This is so you can select text for highlighting assistance, generate page summaries, and more.

Summarizing an article:

As you can see, although I asked Bing Chat to summarize my article, it also pulled in information based on other sites that were writing about the overall topic. The good news is that my site was the core citation, however, there were several other citations listed there, including Bing citing itself (which I have reported before). But that Bing citation led to a search result with my blog post winning the featured snippet. So, I’m cool with that. :)

Understanding more about the author of an article or blog post:
Based on pulling up some articles from some top people in the SEO industry, and our interest in Google and Bing understanding more about authors of content, I decided to ask Bing Chat who the author was based on the content I was reading in Edge. Bing Chat did not do very well… at all. You can see it doesn’t know Lily Ray wrote the post and it believes a journalist from The Atlantic wrote Barry Schwartz’s post.

Although, I have seen Bing Chat in the Edge sidebar sometimes respond with the correct author information. For example, when I asked Bing Chat in the Edge sidebar who wrote my blog post. It responded correctly:

I tweeted about the author situation on Saturday and it was great to see Fabrice Canel from Microsoft say that he passed that feedback along to the Bing Chat team.

https://twitter.com/facan/status/1627109124919140353

Selecting text and taking action (like code):
In the Bing presentation in Seattle, Microsoft demonstrated selecting code on a page and asking Bing Chat to refine it. So I searched for a PHP code example, highlighted it, and Bing Chat asked me what I want to do with that text. I could ask Bing Chat to explain it, revise, it, expand it, or ignore it. When I chose “Revise”, Bing Chat responded that it needed to move on… so that functionality isn’t active yet. But good to know that’s coming soon.

Room for ads in the Edge sidebar (and more content modules):
When using Bing Chat proper (not in the Edge sidebar), there is plenty of room for ads in the citations and below the answer. For example, ads typically have shown above the citations when hovering over parts of the answer and additional modules of content can show up under the answer. The modules can be shopping results, news articles, local listings, etc. That also opens up a ton of opportunity for ads to show up while working with Bing Chat.

Well, there is obviously less room in the Edge sidebar, but there still are opportunities for ads and content modules. For example, when I was asked Bing Chat in the Edge sidebar to summarize an article about the Samsung Galaxy S23, it provided shopping results under the response. And clicking that launched a fresh SERP for the query, “Samsung Galaxy S23”. If you focus on ads, keep an eye on what Bing is doing in the sidebar.

Collapsing citations behind a “Click to view” button. Oh no…
In my post covering the potential impact of AI chat to publishers, I explained how Microsoft vowed to continue to link downstream to publishers. Satay Nadella even said they shouldn’t be crawling websites unless they link to them. That was amazing to hear, but I noticed something in the Edge sidebar that was concerning. For some queries, Bing Chat did provide citations in the response, but then collapsed the bottom citations behind a “Click to view” button. That essentially hid the websites that were being cited (which makes it harder to click those links to visit the sites). I’ll provide feedback to Microsoft about this, but it’s clearly something all publishers should watch closely.

Using “Compose Mode” in the Edge sidebar to generate content:
In the Edge sidebar, there are three tabs. One is labeled “Chat”, which I’ve been explaining in this post, but the second tab is labeled “Compose”. I dug in a bit and that’s where you can have Bing Chat generate content. You can tell it the style you want, the format, the length, and then have Bing Chat generate the content for you.

One you generate the content, you can copy it to use elsewhere, or you could click a button labeled “Add to site”. That was very interesting to see… so maybe Edge will create content for you and then let you quickly add that content to the page you are viewing? That button doesn’t work yet, so I’ll keep an eye on it. But again, very interesting to see that.

Here is a short blog post I asked Bing Chat in the Edge sidebar to write about how to throw a knuckleball:

And just for the heck of it, I ran that content through a few AI content detection tools like GPTZeroX. Yep, it sees the content as AI-generated. Just a a quick warning if you are going to use Bing Chat for publishing content.

Tracking Bing Chat traffic in Google Analytics:
I noticed a new referral in Google Analytics the other day, which was edgeservices.bing.com. I also saw traffic started arriving via that referral right after Bing Chat was announced in early February. After testing visits from both Bing Chat and Bing Chat in the Edge sidebar, I realized that new referral was traffic from Bing Chat in the Edge sidebar. Traffic from Bing Chat proper shows up as Bing Organic (so there is a distinction in Google Analytics). Also, Bing’s Fabrice Canel confirmed to me on Twitter that the traffic was in fact from Bing Chat in the Edge sidebar.

That’s interesting, since Fabrice Canel also announced that Bing Chat data would be available in Bing Webmaster Tools at some point. But now we also have the ability to track sidebar traffic in GA (via the new referral). Just keep that in mind when analyzing traffic in GA.

For example, here is traffic from Bing Chat in the Edge sidebar showing up in GA:

Bing Chat in the Edge sidebar opens key opportunities for Microsoft (and You)
As you can see based on the examples I provided above, having an AI assistant browsing the web with you, that’s ready to answer questions about the content you are viewing, is almost like having a rudimentary version of Jarvis at your side. I covered how Jarvis is where AI chat in the search results is heading, and this is an interesting first step. Now back to my testing… I’ll try to update this post with more examples soon.

GG

In this post I cover if AI will cite, or not cite, sources in the search results, and the potential impact to publishers that rely on that traffic for revenue. i.e. Will clicks from the SERPs get pummeled like we think it might?

Last week, both Google and Bing announced AI chat in the search results following the incredible rise of OpenAI’s ChatGPT in the fall of 2022. And to be crystal clear, AI chat in Search is one of the most important, and monumental, innovations we have ever seen in the industry (especially with how it can potentially impact the entire web ecosystem). On the one hand, it’s amazing to live through this change. On the other hand, it’s been extremely scary for site owners, publishers, SEOs, and basically anyone that publishes content on the web. With both Google and Bing adding AI chat to the search results, there could be a massive drop in referring traffic from both engines. And that traffic is the lifeblood of many websites and publishers.

The rise of OpenAI and ChatGPT was so fast, and so powerful, that Google issued a “Code Red” internally to address the situation. Even Google founders Larry Page and Sergey Brin were called back in to review Google’s AI projects. And now that “code red” has triggered thousands of more “code reds” at publishers as many try to wrap their heads around Search without attribution (which would mean a lack of, or decrease in, links to publishers from search engines).

Without precious visitors from Search (which often accounts for a majority of a site’s traffic), site owners could have a heck of a time surviving financially. For example, if you run advertising on your site, you need eyeballs to earn revenue. If you run an affiliate site, you need visitors to read your content and click through to e-commerce sites to convert. If you run a news site and are trying to gain subscribers, then you need people visiting your content so they can understand the value you are providing. So on and so forth.

For example, it’s not unusual to see Google organic dominate referring traffic sources for many publishers. And it’s also not unusual to see organic search dominate as a channel. See an example below:

This is why AI chat can be extremely scary for site owners. If AI can provide the answer, and not cite sources, then those visits go away. And even if the engines cite sources at the bottom of an answer (more on that soon), then sites will undoubtedly receive less traffic. So that’s still not a great situation. I have heard from many site owners, from some of the largest publishers on the web, since last week’s announcements, and many are nervous, frustrated, and scared about what’s coming. And it’s coming fast… really fast. ChatGPT launched in November of 2022 and here we are in February of 2023 talking about AI chat rolling out in the search results. The pace of AI change has been mind-blowing.  

In this post, I’ll cover more about AI chat launching in the search results, including Google Bard and Bing AI Chat, how AI will cite sources (or not), how some lesser-known search engines have been handling the AI chat situation for a while, I’ll cover real publisher search data so you can understand the true risk involved, and then I’ll end with the user adoption issue, and where I think this is ultimately going.

So grab a cup of coffee, a Red Bull, or your favorite cocktail, this is going to be a heck of a ride.

AI Chat Rising (or Exploding) and The Dilemma of Citing Sources.
Google was first up with its announcement of Bard on Monday, February 6. And after that blog post launched, with gifs and screenshots of Bard in action, there was a collective mind explosion for SEOs and publishers. The reason? Google’s AI chat was not citing sources in chat mode. i.e. It was not providing links to sites where Bard “learned” about the content, or where you can find more information about the topic. So, Bard was trained on a ton of data, including content from across the web, but it was simply providing answers to questions without citing those sources and driving users downstream.

Here is what Bard looked like in its experimental phase:

See my points from earlier about the revenue implications of losing traffic to AI chat. I was extremely vocal about this as soon as I saw Bard in action… I called it an act of war by Google against publishers. Harsh words? Yep, but I have helped too many sites and publishers over the years and fully understand the negative impact this could have on their businesses. It could be a huge blow to them financially.

In its current experimental form, I don't see any attribution or citations. No links, no clicks. That's an act of war against publishers IMO. Let's see how that goes… NeevAI's implementation is far better at this point. At least they cite sources with links to those sites. https://t.co/gXrEoCp9ld

— Glenn Gabe (@glenngabe) February 6, 2023

Now, Bard was actually providing a carousel of sites below its AI answers when Google showed an example of Bard in the search results. But, the carousel was below the AI answer with a “Read more” label. Note, with carousels, several of the listings aren’t even visible until you scroll through them. So this is better than Bard providing no citations, but it’s still not great for publishers in my opinion.

After the announcement, Google caught tons of flak for not linking to publishers, so I believe we could see citations when Bard rolls out over the next few weeks. Time will tell, but for now the situation is extremely scary for many site owners. It’s also worth noting that Google has not answered any questions about citing sources. And I mean literally nothing has come from anyone at Google about linking to publishers (which makes me think they were unprepared for the question). That’s also scary… so we’ll see where this ends up.

Next up, Bing AI Chat: ChatGPT in the SERPs
Then on Tuesday, Microsoft had a huge event in Seattle (which was kept secret until the night before). As expected, they announced AI chat in the Bing search results, and in my opinion, they did a fantastic job with the event, demos, and implementation. Bing really crushed Google with their presentation, the way AI chat is incorporated into the SERPs, and how it works with Edge (think Chrome for Microsoft). Remember, Microsoft has invested billions of dollars in OpenAI, the creator of ChatGPT, so it’s no surprise that their implementation of AI chat in the SERPs was slick and well thought out.

Yep, Bing AI Chat Contains Citations!
After Bard’s lack of links to publishers, everyone watching was eagerly waiting to see how Bing’s AI chat powered by ChatGPT would handle attribution. And low and behold, Bing’s AI chat cites sources. Yep, they are providing the sites where it “learned” about the answers it’s providing and where you can find more information based on the answers you are receiving from Bing Chat. You can hover over a segment of the answer, see the site where you can learn more about that topic, and yes, even link downstream to the page for more information. Precious search traffic remains!

And then Microsoft CEO Satya Nadella gave a great interview with Nilay Patel from the Verge and reiterated how important it is to Bing to drive traffic downstream to publishers. He explained search is about fair use and they fully intend to provide links to publishers from AI chat. It was amazing to hear this from him… You can watch the video below starting at 6:43:

He even said they won’t crawl publisher websites unless they can link downstream. That’s a bold statement in a new AI world, and publishers rejoiced. Well, sort of. I’ll cover why next.

Bing AI Chat citations are amazing, but Google is exponentially more important for publishers (at least as of now):
Again, hearing Satya Nadella’s comments about linking downstream to publishers was great, but the truth of the matter is that Bing just doesn’t drive a ton of traffic for most publishers. Google typically drives exponentially more traffic than Bing. I have helped many sites over the years and have seen the massive difference in referring traffic time and time again.

For example, here is a comparison of clicks and impressions data for a site (comparing Google and Bing). This data is directly from each search engine:

For the first site, Google drove 13.2M clicks over the past three months:

And the same site saw just 404K clicks from Bing. Note, that increase you see is due to a major algorithm update that Bing confirmed (where they used the new AI model on Bing’s core ranking engine):

The next site saw 3.8M clicks from Google over the past 3 months:

But only 52.9K clicks from Bing:

So again, it was great to hear Satya’s statements about fair use and linking to publishers, but let’s face it, we need Google to do the same. That’s unless Bing starts gaining many users based on AI chat hitting the scene. I’ll cover that next.

By the way, it’s definitely worth noting that both Google and Bing are behind some other newer and smaller engines that are moving at a rapid pace with AI chat. For example, Neeva, You.com, and Perplexity.ai all provide AI chat, and have for some time. The great news is that they all  provide links to publishers from the chat window (letting users link downstream to find more information from the sources where the content originated from).

For example, here is NeevaAI providing an answer with my own content cited. It’s a weird feeling seeing this, but now after Bard’s lack of citations, it’s better than no citations at all. :)

Field, I mean SERP, of Dreams: If you build it, will they come? The user adoption issue.
As mentioned earlier, Bing killed it with their announcement and outdid Google. That was super-clear, but questions remain about how people will react. And I mean real people that use search engines for many tasks throughout the day. In other words, Bing AI chat looks pretty amazing, but will users follow?

When the buzz dies down, and you’re at home or work needing to search for something, will you quickly visit Google like you always do, or will you shift and visit Bing instead? On that note, Bing announced that they had one million people sign up for early access to the Bing AI chat experience in the first 48 hours after Tuesday’s presentation (see below). That’s great, but it’s not billions. And out of the one million, how many are just interested in seeing how it works, but will still rely on Google for most of their searches?

We're humbled and energized by the number of people who want to test-drive the new AI-powered Bing! In 48 hours, more than 1 million people have joined the waitlist for our preview. If you would like to join, go to https://t.co/4sjVvMSfJg! pic.twitter.com/9F690OWRDm

— Yusuf Mehdi (@yusuf_i_mehdi) February 9, 2023

In addition, Bing’s mobile app skyrocketed up the charts based on people trying to gain early access to AI chat. It was one of the requirements for getting access to AI chat quicker than just being on the waitlist. But are people really using Bing’s mobile app or was it just a checkbox for gaining early access? So there was lots of buzz, but will that translate into more users? Nobody knows yet.

Bing also announced something interesting financially. They said for every 1% of share they gain, it equates to $2 billion in revenue. Here is a quote from Microsoft’s CFO for Windows and Search, Philip Ockenden, about the opportunity. Bret Kinsella covered this in his post about Google’s stock tanking after the Bard presentation on Wednesday, February 8th:

“For every one point of share gain in the search advertising market, it’s a $2 billion revenue opportunity for our advertising business.”

So, there’s a huge upside for Bing right now. Remember, Microsoft has many sources of revenue and Search is a relatively small one at this point. It’s profitable, but very small compared to Google. But that also gives them an advantage. They don’t have to worry as much about the search impact when testing big changes since they are driving massive revenue via other products and services. Google, on the other hand, earns a large percentage of revenue from Search. It’s much riskier to test big changes…

Back to usage, when asking my kids (18 and 16) if they would switch to Bing for AI chat, my daughter came back with a quick “maybe”. She said, “If it answers my questions quickly, I might give it a try…” My son on the other hand, said “hell no”. But after explaining more about Bing’s AI chat, he came around and said, “Well, I’d have to see it in action…” So there is hope for Bing. :)

Jarvis: Where the AI Search Juggernaut Is Ultimately Heading…
To wrap up this post, I wanted to quickly cover where I think AI Search is ultimately heading. And this shouldn’t be a surprise since we’ve seen it a thousand times in science fiction movies. I think it’s Jarvis from Iron Man.

If you’re not familiar with Jarvis, Tony Stark built an incredibly powerful personal AI assistant that he can call upon wherever he, is and for whatever he needs. Jarvis can answer any question and knows everything about Tony. It can answer any question thrown its way and can provide supporting information, or even take action on Tony’s behalf. See a quick video below:

So if you think about Jarvis for Search, then AI chat is the very beginning of that. Sure, some like Microsoft will vow to keep linking downstream to publishers, but isn’t that just a stopgap? Again, I have no idea how long it will take to get to the Jarvis stage of AI assistants, but in my opinion it’s coming. And at that point, I think we’ll all look back and laugh about having to enter keywords into a search engine, browsing endless search results pages, and pogosticking across sites trying to find an answer. But for now, I’d be much happier with AI chat providing links to my site (and to publishers across the web).

Summary: AI Chat in the search results has arrived, and publishers are watching closely.
AI chat has seen explosive growth since November of 2022 when ChatGPT launched, and we are all quickly going to learn how that impacts publishers. The great part is that the subsequent analysis will NOT be based on sample data and hypotheticals, but real searches by real people. And we’ll know fairly quickly if, and how, publishers can survive.

Since I help a lot of companies across verticals that rely on search traffic, I’ll be watching closely and analyzing the situation the best I can. Expect more posts from me soon on the impact of AI chat in Search based on real data. And believe me, I hope I can report that publishers are flourishing in this new AI world. Time will tell. Stay tuned.

GG