Gaining competitive advantage from non-textual information

Automatic image descriptions

The alt text (alternative text) function is gaining traction across a variety of applications, particularly social media ones, for creating swift descriptions, which serve as metadata, of images. Alt text is applicable for what Kirpalani characterized as “much more simple objects—a static image—to begin with. You know, videos. Alt text is definitely a standard form of making sure that we can have some level of accessibility.” Accessible rich internet applications (aria) labels are another function that supports similar types of information about visual content. Both of these functions add to the metadata about non-textual content, making it more searchable.

Generative foundation models can also provide automatic descriptions of images at a level that exceeds that of these other functions. “When you think of, maybe, a visual image of a business process, the alt text of a business process is not very helpful,” Nivala admitted. However, generative models and other forms of machine learning can output a long-term version of an alt text description. This generation “actually describes the content of that process diagram,” Nivala affirmed.

Discoverability

The point of creating information models to describe textual and non-textual content, as well as implementing metadata descriptions and tags about this content, is to make it easily searchable. For visual information in particular, “The systems that store that information, as part of the knowledgebase of the organization, should be capable of making that content discoverable, perhaps by producing a textual description of the picture itself, or at least coming up with keywords or themes or topics to enable returning that image as a response to a user’s search query,” Nivala commented. Kirpalani indicated that the longer textual descriptions he referenced were directly searchable. Such an advantage is fortified in solutions in which visual components can be implemented in configurable dashboards for tracking business metrics such as sales goals, for instance.

“When you’re designing the dashboard, you can actually search for any content in the metadata descriptions as well, to bring that component back so you can now configure your own dashboard,” Kirpalani mentioned. Top solutions involve natural language search fortified by language models, so users can query even complex back-end systems, including knowledge graphs or data fabrics, as easily as they can ask a colleague a question. In this case, the language model is “aware of the ontology describing the data and the knowledge graph, and it turns the user’s question into a graph query, or SPARQL query,” Altair’s Martin said. Plus, because knowledge graphs are rife with non-textual and textual information that has been vetted by the enterprise, the model’s search results “give you a generation of facts that have been put into context,” Martin added.