The goal of the study AI & Streaming: Chatbots for Content Discovery was to determine how effectively today’s generative AI can help users find streamed content. The authors examined whether chatbots can correctly identify the streaming platform, provide consistent answers, and deliver a direct link to the specific title being searched for. The research shows that, even as chatbots are increasingly integrated into operating systems and smart TVs, their ability to enhance content discoverability remains limited, especially for local European streaming services.
How did the tested chatbots perform?
The aggregate results of the study show that, for now, chatbots do not handle basic navigation particularly well. Only 68% of responses included the correct streaming platform where the requested title is available. Consistency fared even worse – only 57% of responses were the same across three repetitions of the same query. The biggest issue, however, was direct links to specific titles: only 31% of responses included the correct deep link.
AI & Streaming: Chatbots for Content Discovery, veed analyticDifferences between chatbots and regions
The study also highlighted fairly significant differences between individual chatbots and geographic regions. The researchers compared the ability of four chatbots (Claude, Gemini, ChatGPT, and Perplexity) to correctly identify the streaming platform and also provide a deep link to a specific video title. The data cover the countries surveyed. How did each chatbot perform?
The most consistent results in this area are achieved by Claude. Across all regions, it shows a relatively high success rate in correctly identifying the streaming service, with differences between countries being fairly minor. Moreover, it is one of the few chatbots where correct deep links appear in a significant proportion of cases. This is especially true for the USA, the UK, and Germany, where the share of correct deep-links exceeds 50%.
Gemini, on the other hand, is characterised by a very strong ability to recognise the correct streaming service, even achieving better results in France, Italy, and Spain than in the USA, the UK, and Germany for this metric. However, these good results are undermined by poor handling of deep links. Correct deep links appear only occasionally, and follow-up queries within the chatbot are often necessary.
ChatGPT ranks closer to the lower end of the evaluation. Its ability to identify the correct streaming service is limited, overall achieving less than half the success rate. Slightly better results were observed in the USA, the UK, and Germany. Performance with deep links is even weaker – correct direct links appear only sporadically in ChatGPT’s responses. According to the study, ChatGPT is more suitable as a tool for gaining basic orientation. It still has a long way to go before it can effectively simplify the path to content.
The worst-performing chatbot, however, is Perplexity. It correctly identifies less than half of streaming services, and when it comes to deep links, it almost never gets them right. While there are some regional differences in favour of the USA, the UK, and Germany, they do little to change the overall picture of very low-quality results. In this comparison, Perplexity proves to be a chatbot that, in its current form, is unable to serve as a reliable tool for content discovery.
AI & Streaming: Chatbots for Content Discovery, veed analyticsAI & Streaming: Chatbots for Content Discovery, veed analytics
The study’s results confirm that AI chatbots still have a long way to go when it comes to discovering streaming content. While some chatbots are already relatively good at identifying the correct streaming service, providing consistent and accurate deep links remains a major weakness across all AI solutions.
Local streaming services – the weakest point of the ecosystem
A key finding of the study is the significantly poorer performance of chatbots when dealing with local streaming services, especially in Europe.
It is precisely in this area that current AI chatbots encounter their greatest limitations. While they can at least partially identify content availability on global platforms, results for local streaming services are significantly weaker and much less consistent.
Claude and Gemini appear to be AI solutions with relatively decent ability to recognise local streaming services, at least when it comes to identifying the platform itself. Claude correctly identified 22 out of 27 tested streaming services, and Gemini 21 out of 27. All cases where identification failed concerned exclusively European local services. This means that even the highest-rated chatbots have fragmented and incomplete knowledge of the local European market.
The situation is much worse when it comes to deep links. Neither Claude nor Gemini were able to provide a correct deep link to a specific title for local streaming services. For Claude, the absence or inaccuracy of a deep link related to 14 out of 27 streaming services, and for Gemini, 12 out of 27. This means that even if the chatbot “knows” where the content can be found, it cannot directly guide the user to watch it.
Perplexity shows even more problematic results. This chatbot failed to find any correct result for more than half of the tested streaming services (16 out of 27), and once again, all unsuccessful cases concerned local services. Moreover, Perplexity does not appear at all in the top-right quadrant of the chart, the area where both conditions (correct identification of the streaming service and a correct direct link) would be met simultaneously.
ChatGPT, however, presents an interesting outcome. On one hand, it failed to provide any relevant results for 11 local streaming services, meaning it failed both in platform identification and in providing deep links. On the other hand, it was the only chatbot to achieve placement in the top-right quadrant of the chart for two local streaming services, the category where both tracked metrics are met. This suggests that ChatGPT’s performance regarding local streaming services is uneven and highly dependent on the specific service or market.
AI & Streaming: Chatbots for Content Discovery, veed analyticsAI & Streaming: Chatbots for Content Discovery, veed analytics
Overall, it is clear that local streaming services represent a systemic problem for AI chatbots. These are not isolated failures, but a structural weakness across all tested solutions. While the identification of global platforms is gradually improving, local streaming services often remain unrecognised, revealing a stark gap between global and local content distribution. This suggests that without deeper technical and data integration, local players are likely to remain at a disadvantage in content search via AI compared to global streaming services for the foreseeable future.
In summary
Chatbots are gradually beginning to replace traditional search engines and are being integrated directly into platforms such as Android TV (Gemini) or Samsung Tizen (Perplexity). However, the study shows that the current state represents more of an early stage of development. The results are inconsistent and insufficient, especially when it comes to local streaming services. According to the study’s authors, it will be necessary in the future to develop better technical, product, and business models that enable deeper integration of streaming services, greater consistency of responses, and higher-quality presentation of results.
Source: veed-analytics.com