Searching for relevance
Analyzing keyword searches can determine which searches were most successful in achieving the desired outcome, whether success is defined as finding specific information or improving conversion rates on e-commerce sites. Relevancy tuning helps users zero in on the content they want.
Muck Rack helps communications and public relations professionals find the right journalists and bloggers to pitch to promote their company’s news by searching any keyword, competitor, beat, outlet and media type. In addition, Muck Rack monitors articles about its clients’ companies so they can better respond to news by contacting the journalists creating content.
When the client company is looking for a journalist, effective and tailored search is essential to finding the right match. “We don’t believe in the mass blast,” says Rob Shapiro, director of product strategy at Muck Rack. “Instead of sending a pitch to 5,000 journalists, today’s PR professionals are much better off identifying the five people that have the right content knowledge, experience and audience, then working to build a real relationship with them.”
To achieve that level of granularity, Muck Rack needed not only a robust search engine, but also a software solution that would provide vigilant relevancy tuning to optimize search results for its customers. The company decided to deploy Quepid, an analytics tool from OpenSource Connections. The solution is run on top of Elasticsearch, an open source search tool from Elastic. Muck Rack uses Elasticsearch to search on information collected from journalists’ tweets and social media profiles, articles published online and content shared on social media.
Quepid produces a quantitative relevancy score that can be monitored over time. An initial baseline score is obtained from relevancy ratings by subject matter experts or analytics data. When changes are made in search algorithms that are used to locate journalists, Quepid uses the training set to calculate how far the new results are from the target relevancy level. Subsequent scores are calculated automatically and monitored by the search business to ensure search works right for their users and content.
“Our primary concern is understanding the impact of changes we make on a daily basis,” Shapiro says. “We did not want to inadvertently reduce relevancy, which meant we needed to be able to easily check the results before and after changes. For example, we tweaked our search algorithms to improve negations of certain keywords to prevent results that do not match the user’s intent.” This filters out false hits, such as a journalist who writes about banks (financial institutions) when the target is one who writes about riverbanks (environment).
Maintaining relevancy
The parent company of Muck Rack is Sawhorse Media, which hosts the Shorty Awards for social media influencers. “Our co-founders Gregory Galant and Lee Semel found that journalists were using social media and tools on the Internet more than other early adopters,” Shapiro says. “Although there have always been less specific places to find journalists who covered various topics, there has never been a searchable, constantly updated repository that integrates this many content sources and helps communications professionals identify the most relevant people creating the news.”
Many companies find themselves in a position of needing to change from one search engine to another, and Quepid can be used to ensure that relevancy is being maintained during the transition. “Quepid can track and monitor the new search environment to ensure confidence in the new setup,” says Karen Renshaw, a search relevancy consultant who has experience using Quepid. “Quepid is the only product that allows a query-by-query analysis to compare the old with the new.”
The main goal is to make sure that users see equally relevant results under the new search engine. “Search data can be aggregated to identify the ones that lead to the most profitable outcomes, and then tested on the new system,” Renshaw explains. “Relevancy scores can be recalculated instantly to ensure that the revenue of high-value searches is not disrupted.”
Even when the search environment is not undergoing change, frequent tuning is required, and analytics helps in that process. “Organizations do not always understand that maintaining a search environment is an ongoing process,” Renshaw says. “It is not a plug-and-play situation.” Considerable input involving human action may be required, and this should be guided with support from analytics that verify relevancy.
Quepid was initially developed as an in-house product for OpenSource Connections, which provides professional services in the area of search. After its customers saw the product in action, they wanted to purchase it for their own use. “Increasingly, any search engine starts with an acceptable level of search relevance,” says Doug Turnbull, search relevancy consultant for OpenSource Connections, “but we’re finding organizations increasingly need to customize results, ranking for their content, users and use cases. Google has trained users to rely on search—so controlling relevance ranking, it turns out, is vital to keeping users engaged.”
Achieving better search is not an IT problem, although it is often given to IT to solve. “Developers need the knowledge of SMEs, but traditionally there has not been a good way to share the input,” Turnbull says. “The solution is a test-driven relevancy dashboard used to apply a rigorous process for iterative improvements to search relevance. You need to define what ‘good’ looks like and then measure against that. Quepid provides transparency into search results, based on rigorous testing practices.”
Customer ID and access management
One of the hottest areas of information security is customer identity and access management, as companies strive to protect corporate assets and deliver personalized customer journeys. Gigya offers a SaaS platform that collects first-party data for visitors to websites, unifies it from multiple sources into a customer profile and segments it to maximize the chances of conversion.
“Gigya’s platform is the engine behind some of the world’s largest customer identity and access management systems,”’ says Ian Hametz, documentation team leader for Gigya. “We have developers all over the world who need to access our documentation, but we found that our existing search solution was not providing users with the relevant results they were expecting.” After checking out different options and testing available solutions, Gigya chose Swiftype, a hosted search solution.