The next wave of big data technology: distributed automation

The IoT’s edge

AI should play a formidable role in the IoT’s advanced analytics. Hung describes the convergence of the IoT and AI as a “virtuous cycle” in which the automation of the latter heightens the distributed advantages of the former. The immense scale of IoT data is ideal for the quantities required to train neural networks (including deep learning) for advanced analytics. Conversely, AI’s advanced analytics can potentially actualize the IoT’s promise of revolutionizing big data. “The hope is that deep learning and making use of the data coming from the IoT will somehow engender new applications that we haven’t thought of today that will be the killer app for IoT,” Hung says.

Moreover, AI could conceivably impact the IoT in its edge computing applications that process data at the cloud’s periphery. The temporal benefits of that approach are tantamount to the reduced network and bandwidth constraints they provide. Edge processing is pivotal for applications “that are latency sensitive where you cannot afford to go to the cloud and back, or there are data sovereignty issues where you need the data local,” Hung explains. “There are also data availability issues where you need the data right at the gateway.” By processing data at the cloud’s extremities, edge computing redoubles the IoT’s impact on decentralization.

Advancements in security and standardization are creating a similar effect. A longstanding IoT concern is endpoint security on devices that are “not physically protected, so that increases the attack surface for malicious behavior,” Hung says. However, security developments in hardware are addressing those issues. In particular, measures involving roots of trust (RoT) are gaining credence for their ability to verify the “identity for the endpoint,” Hung says. “This can be something like using a separate chip for a secure element. Or, it could be in the processor core itself like a microcontroller in the endpoint for a trusted execution environment (TEE).” Additionally, standards such as Wi-Fi, Bluetooth and low-power wide area network (LPWAN) options such as LoRa (lora-alliance.org) are normalizing wireless communications for the IoT. There are also protocols existent at the application layer standardizing the IoT to make it more attractive to users—garnering greater traction for distributed automation.

A single blockchain network

Blockchain’s distributed capabilities may exceed those of the IoT and AI. In addition to its traditional utility within finance via cryptocurrencies such as Bitcoin, Blockchain is suited for most transactions in which non-repudiation and low latency are desirable. Verticals in which it’s spurring adoption rates include insurance, advertising, healthcare, law and “the biggest one of all: government … all of the government transactions—social security, Medicare, all of these kinds of things,” predicts Charlie Silver, CEO of Algebraix Data.

Blockchain’s lack of a centralized authority is renowned for its security implications, which Silver terms “infinitely more secure than having data in a centralized cloud like Equifax. When it’s centralized, you can more easily hack large amounts of data. The whole idea of the blockchain is it’s decentralized so it lives in a peer-to-peer way on millions of computers around the globe.” Blockchain’s viability is increasing due to permissions enabling confidentiality, so not every chain participant is privy to all its data. Vendors are also implementing measures (most of which are proprietary) to rectify the technology’s scalability.

Blockchain’s ultimate advantage to the increasingly distributed big data milieu may lie in its ability to connect what amounts to numerous decentralized data sources. Thus, it furthers the propensity toward decentralized automation and does so with the added benefit of reducing silos. “The blockchain is a step in the evolution,” Silver notes. “If you think about the World Wide Web, even though there’s enormous silos, the Web eliminated a lot of silos. The blockchain is the next step in that evolution.” Blockchain’s capacity to diminish silos is multifold. For organizations with data in multiple environments such as the cloud, on premises and in hybrids, internal blockchains may be a solution for uniting data and attaining the single access view Hung mentioned.