The next wave of big data technology: distributed automation

The year 2017 established the fact that big data’s immediate future was bound to the assorted expressions of artificial intelligence, the Internet of Things and blockchain. The year 2018 ushers in a new era in which organizations deal with the ramifications of the singular point of commonality between those technologies: decentralization. Each additional distributed transaction on blockchain, digital personal assistant interaction with AI or edge computing application in the IoT only increases the need to control those resources in a uniform manner.

“There’s never one model, whether it’s centralized or decentralized, that’s going to dominate,” says Mark Hung, Gartner research VP. “I think it’s always going to be a heterogeneous mode of operation where you need both.”

The heterogeneity of the big data landscape is characterized not only by the distributed nature of its most prominent technologies, but also by their underlying architectures. Organizations are contending with on-premise, conventional cloud, edge computing and hybrid architectures compounded by the low-latency concerns of automation spawned from:

• AI—AI’s automation capabilities are apparent. Its dynamic algorithms expedite everything from facets of data science to fraud detection and market segmentation. It contributes to the decentralized movement by supplementing IoT analytics and implementing natural language capabilities in intelligent agents known as bots. In each of those cases, “what AI does is erode another barrier between humans and computing resources,” says Whit Andrews, Gartner VP distinguished analyst.
• The IoT—The speeds at which the IoT produces streaming data are ideal for automated processes, particularly when supplemented by AI’s advanced machine learning. Its decentralization is evinced in the distributed nature of its endpoints and its edge computing capabilities. A synopsis of Forrester’s “The Top 10 Technology Trends to Watch: 2018 to 2020” states: “The Internet of Things (IoT) shifts computing toward the edge. Firms in the vanguard of this trend will use edge computing to build distributed applications that close the gap between data, insight and action.”

• Blockchain—By definition blockchain is a distributed ledger system that complements AI’s automation via the speed at which its transactions occur. “One of the most immediate benefits of the shared ledger of blockchain is you eliminate information latency,” says Adeel Najmi, One Network SVP of products. “Anything happening in the value network that anyone needs to know and is permitted to know, you can know immediately.”

Because data’s overarching value frequently stems from the contextualized analysis of integrating data, the need to uniformly control distributed computing environments will likely persist. Marc Hung suggests, “You always want a centralized or single point of access or single point of view. But how that’s actually physically implemented, there’s a lot of flexibility.”

Artificial intelligent agents

Other than the practical utility of preparing data for consumption with cognitive computing, the most compelling use cases for AI’s future value are predicated on distributed deployments. According to Andrews, “We know that the most common uses of AI today are decision support and process improvement.” Advanced machine learning capabilities are ideal for analyzing unstructured text, image and machine-generated sources.

Quintessential examples include deploying AI to analyze pictures of vehicular accidents in asset insurance, medical images (such as MRIs) in healthcare and text analysis in financial services. That paradigm depends on AI’s analysis, which knowledge workers interpret. “That would be true for any environment in which we’ve gone from physical presence to expert captured imagery, to expert evaluated automated imagery, to a final outcome—seeing something that is mediated by an expert,” Andrews explains.

The expanding use of chatbots will also emphasize AI’s tendencies toward decentralization. The “Gartner Top 10 Strategic Technology Trends for 2018” states: “Chatbots will become the face of AI.” The trend impacts AI’s distributed automation in multiple ways. Interactive digital virtual assistants (encompassing multiple forms including mobile devices and smart speakers) use not only machine learning but also facets of speech recognition, natural language processing and what Andrews calls “natural language generation,” which is instrumental for “democratization that allows you to assemble a semi-complex syntactical structure.”

In addition to enhancing customer experience and broadening AI’s adoption in distributed settings, bots are also useful for internal applications. Use cases in which they help workers with automated processes such as search or other repetitive tasks “are happening in the ultimate natural expression, which is speech,” Andrews says. “Chatbots are important and exciting because they augment thinking in ways that previously have not been possible.”