How the convergence of HPC and AI is helping industries build intelligent, future-forward tech solutions

Innovative Artificial Intelligence (AI) applications have powered transformational solutions for big data challenges across industries like healthcare, logistics, education, scientific, engineering, finance, among others. The convergence of AI and High Performance Computing (HPC) has become apparent to reduce time-to-insight, and enable building intelligent and industry-relevant AI architectures, so as to optimise data-driven discoveries for high performance.  

To understand how companies are leveraging this convergence, Microsoft and YourStory co-presented a roundtable discussion titled ‘Big compute, big data challenges - Convergence of HPC and AI’ with Sachin Nagpal, Solution Sales Leader, Microsoft Azure; Ashes Ganguly, CTO, SigTuple; Dhruv Agrawal, Co-founder and COO, Shipsy; Kuldeep Singh Chauhan, CTO, Synapsica; Yasen Dimitrov, Chief Analytics Officer, Intelligence Node; and Vikranth Ramanolla, Co-founder and CTO, Dataweave.

Technology to handle data at scale

The convergence of HPC and AI is showing some incredible applications which range from global climate modelling, genomic sequencing, autonomous driving to many other industries. “This convergence is helping decode large, multi-dimensional data sets that can help identify fraud and credit card transactions, or help predict which patients are at risk for certain diseases,” said Microsoft Azure’s Sachin.

The main driver, according to him, of this convergence has been a persistent growth of data which has led technologies to handle the ever-increasing scale across industries like healthcare, sustainability, logistics, science, among others.

Making healthcare accessible

In countries like India, where access to trained healthcare professionals is limited, this convergence has made healthcare accessible to far-flung villages in the country.

“Our scanning device can be placed at any remote location, which uploads images on the cloud for a pathologist to review, produces microscopic images, which can then be reviewed by a pathologist on their desktop or laptop. It also offers automatic classification of cells, which helps them to create reports much faster and more efficiently with help from AI algorithms and analysis,” said Ashes of SigTuple, a startup that is working towards building intelligent screening solutions to aid blood and urine diagnosis through AI-powered analysis of visual medical data. He added that this in future will not only create change in clinical pathology, but also histopathology.

Besides remote diagnostics, this convergence is also helping healthcare professionals automate backend work while giving them the bandwidth to focus on accurate and faster report writing. For instance, Synapsica built an AI-powered platform to bring accuracy, transparency, and assist radiologists with AI generated biomarkers so that they could write objective, self-explanatory, and transparent reports.

“We saw the value of faster report generation during the recent COVID-19 surge in India. During the pandemic, radiologists could handle the 10X surge in report creation from the safety of their homes. This was very crucial, as India has only 10,000 radiologists for a population of 130 crore, which makes the ratio abysmal,” said Kuldeep. Their cloud platform also facilitates remote reporting, where all clinicians and stakeholders have access to each report and images, hosted on the cloud, automating mundane reporting and backend tasks, and creating quality reports at a much lesser time.

Data for faster decision making

Logistics is another segment that has leaned into AI and Big Data to tackle disruptions, fraud detection, and delivery scale. “Customer expectations with regards to deliveries are on a different level today. With faster and same-day expectations, the fundamental need is to make quicker decision making, which needs online machine learning, where a data model is constantly getting upgraded. Companies are constantly consuming streams of data coming from their workforce, regions where there is congestion, delivery optics of team members, among other such metrics,” shared Dhruv of Shipsy.

The convergence of data and AI has evolved to help take high-dimensional inputs into decision making to address demand fluctuations, augment and incentivise a workforce, factoring feedback, building fair pay out mechanisms, among other such capabilities today, he added.

Data for trend forecasting, accurate pricing in retail

The convergence of AI and data in retail has helped Intelligence Node, an e-commerce price intelligence platform, to build what they call a future success model, said Yasen. “Assortment Intelligence has multiple incorporations. While one is a linear side, the other is to redefine data models to analyse historical data, evaluate performance of metrics and predict trends, so that retailers can work on building inventories that will be relevant six months in advance,” he said.

With the use of different AI models and algorithmic techniques, the startup can predict what clusters or trends will be relevant in the future, through tracking market movements including stock outs, customer reviews, visibility of products, etc. Yasen gave an example of how the market exploded for masks, once the pandemic set in, which was a trend the model could predict based on market movements and AI-enabled data crunching.

Computing data with AI and ML algorithms has also helped build pricing and product catalogues in retail by converting unstructured and semi-structured data into structured information for Dataweave.

“We normalise data that is collected through various sources, use image-based algorithms to extract key attributes, and identify missing attributes through various AI and ML techniques. And all this helps us deliver accurate insights to our customers, so that they can make pricing and product decisions based on the data that we share,” explained Vikranth.

Technology, a key differentiator

“To perform insights across sectors for data crunching, data analytics, machine learning, deep learning algorithms, technology requires a scalable platform. This is where Microsoft Azure is helping partners with its cloud capabilities that are purpose-built to the full spectrum of HPC and AI workloads, including the most demanding use cases,” said Sachin.

Commenting on the innovations discussed in the round table in sectors like health care, logistics, retail, he added, “Today, technology is a key differentiator for different environments. It is no longer a support function.”

Watch the panel discussion here