When done correctly, personalized search leads to monetization opportunities for media and entertainment companies, providing a way for them to tap into potential sales.
As more and more consumers embrace digital media and entertainment, it’s therefore pivotal to understand what customers need from personalized search and how that impacts digital transformation initiatives for CIOs and CDOs.
Consumers want media and entertainment brands to treat them in a personalized way. They expect custom offers and recommendations of what to watch, read, play or listen to. But there are signs that customers’ expectations are outpacing their experiences.
Top of the list of complaints is that content search generates the wrong results. This is followed by gripes that results are in the wrong order.
Some chief marketing officers echo these concerns. They also question why content search doesn’t lead to better product sales. The reason? These are all symptoms of a poor-performing search platform, one that fails to display the correct results and product ads.
This emphasizes how much media and entertainment companies rely on their technology teams. It’s also why you need to create the best content search experiences.
How can quality engineering (QE) teams take a deeper dive in understanding and testing the search systems that cater to these growing consumer needs? Here are the questions that QE and product management teams should be asking in order to solve this conundrum.
“For the things we have to learn before we can do them, we learn by doing them.” – Aristotle.
To help identify where you can further expand your areas of testing focus, let’s take a look at a few key concepts.
Today’s big name media companies handle content from many sources. Making sure that this content shares the correct set of metadata is vital for its distribution and presentation.
This introduces the concept of data enrichment. Using machine learning and natural language processing techniques can help to further enrich the content or the metadata associated with the content. This enrichment of the data plays an important role in making sure that the content has the right set of vocabularies or standards used by the media and entertainment industry globally.
These taxonomies help to create the right solutions and also serve as requirements that can be asserted for best practices by QE teams. Data enrichment and presentation concepts can be further leveraged to present the right ads to customers when searches are recorded and analyzed for patterns.
Storing information structurally and serving it in the fastest way without adding technical and architectural debts all requirements for gaining competitive advantage that should be tested thoroughly – this applies to functional and non-functional requirements.
Understanding the different taxonomies and standards used in the data enrichment process helps to test the consistency of the metadata sitting in your respective search platforms.
A common problem for organizations in their digital journey is not having relevant and contextual data that drives business outcomes in their system development and test environments. Data relevance from a timeline perspective is also of the utmost importance. In addition, market trends, innovation, news and economic conditions change consumer buying patterns and challenge how content is presented, which in turn influences what consumers search for and which products they want to buy.
Take querying the keywords “Trump” or “Biden”, for example, and the different sets of results we’d get in 2022 versus 2019. This shows how heavily we rely on having the right set of contents in the test environment. More importantly, it demonstrates how crucial it is for us to design test cases using data that customers are most interested in. The ideal way to achieve this is to constantly redesign test stories in conjunction with product teams.
Keeping test environments in parity with live production environments can also help to uncover a variety of non-functional issues. For example, establishing a baseline and measuring the performance of search queries against your growing database of contents continuously can help to identify and prevent any bottlenecks with disk I/O operations. This can further help capacity planning and determining the production hardware needed to sustain the average load.
In addition, monetizing on obsolete data and buying patterns won’t lead to the correct results. Instead, product teams should understand trends in the marketplace and work with the technology and QE teams to plan new enhancement stories that seize the opportunity to capture the most revenue through targeted ads.
How important is it to understand indexing of contents or the metadata in your respective search clusters? How important is design in ensuring a seamless customer experience and gaining a competitive advantage?
The answers to these questions are important and the test strategy needs to drive assertion criteria that address these requirements. As mentioned above, content sets can vary and be grouped in many ways, for example, based on different media types like text, photos, videos, audios, graphics and interactives. In order to increase the speed of the presentation and ensure that the data being served is not erroneous or irrelevant to the search results, some best practices around content design and architecture must be implemented.
For example, the data can be grouped based on most frequently searched data versus less frequently searched data; by separating live data from archive data; or by ensuring indexing is segregating data on the basis of metadata requirements. These best practices and indexing rules help to make sure that the right content sets are indexed, or stored, at the right location. They also ensure content delivery to the right distribution and presentation platform(s).
Understanding content indexing plays an important role in validating that the contents are indexed or stored in the right location. This in turn may decide what customers see as part of their searches in the respective presentation platforms.
Having business and technical domain knowledge of content indexing plays a vital role in validating if the contents are indexed or stored in the right location. Not having a breadth of technical and business knowledge in this discipline can lead to search and presentation defects when features are deployed into production. This can erode customer confidence and loyalty.
The key takeaway in the world of content search and monetization is to maintain the steady quality of the content with enriched tags, vocabularies and taxonomies, and maintain and capitalize on the relevance of the returned set of search data.
With that in mind, here are some of the key function points to focus on when testing.
These validate that the returned result set matches the keyword(s) provided for the search.
Like keyword-based searches, these tests ensure that the result sets returned have the exact search phrase or sentence in the content.
This is about working with marketing teams to understand and document what type of product ads should be displayed based on customer queries and buying patterns, and then creating a test strategy that addresses these scenarios.
Validating the accuracy of all the metadata of the returned results ensures that the result set maintained its course as it went through the enrichment process of taxonomies and classification.
Understanding and validating the different retention policies set for your data is important; failing to do so may lead to legal issues. For example, consumer access to promotional content may need to expire after a certain number of days. Likewise, you should avoid infringing licensing agreements for images, while content restricted for domestic distribution only should not be visible to international consumers.
Validating that the right quantity of content sets is returned for a given search is key. For example, the content set returned for the search keyword “Super Bowl 2022” before and after the event will be different, as after the event we’re bound to get more results returned.
Another instance would be to validate that the data returned for a specific search criterion is consistent no matter how many times you issue the search or at what time of the year: for example, “Super Bowl 2022 Winner”.
From a monetization perspective, another example may be marketing the most popular ads to search consumers the day after the Super Bowl based on the ads’ target rating points. It’s important to understand business rules and ensure that your test strategy includes detailed assertions to meet those requirements.
This is another important feature that consumers use to further personalize their search. You should also give validating different content types, or sorts, and filtering criteria the utmost importance in your test planning activities.
This is a favorite feature that makes life easier by completing the name of a movie, for example, based on the first few characters you enter as you start typing. Understanding and testing the rules defined for the enrichment of autosuggest libraries and making sure they work in your search engines is a good test.
Like autosuggest, this feature is consumer friendly. It’s also widely used, often as a “Did you mean” feature. Including this in your test strategy maintains the quality of searches by keeping consumers engaged in your respective platforms.
As the contents grow in an enterprise search cluster, making sure that searches return results within a few milliseconds consistently is important. So, the non-functional testing aspect plays a vital role here.
Defining the key performance indicators and service-level agreements for an environment where the contents are continuously growing is important to monitor and govern the performance of the search.
Continuous and shift-left performance testing, built within the build pipelines and exercising a focused set of queries simulating a variety of searches, helps to set the performance benchmarks for each build. It also lets you catch any performance bottlenecks early in the development cycle.
Capacity planning helps to ensure that the given search infrastructure is provisioned to handle the average expected traffic on a day-to-day basis.
Load and spike testing make sure that your search environment can perform well under the expected load, twice the expected load, four times the expected load, and so on. Spike testing simulates an event-based burst in traffic to check that the underlying infrastructure can handle it and perform well.
Failover testing determines if your enterprise search platforms handle and perform well under critical failures of the search infrastructure.
It’s vital that your QE teams build comprehensive test strategies for content search that assure revenue for the business. They should also ensure that all systems are performing at their best, both functionally and non-functionally.
To achieve this, your engineers need to understand:
Get it right, and you’ll deliver the personalized search that your customers expect. Keep it accurate, relevant and seamless and your customers will keep coming back for more.
