Every minute, hour and day we are generating huge volumes of data, which means ever more sophisticated and powerful tools are required to analyse it if meaningful insights are to be delivered. One such tool is machine learning - but what is machine learning?
In order to more efficiently spot patterns in massive datasets, machine learning was developed to give computers the ability to learn without being explicitly programmed.
Today, it largely remains a human-supervised process, at least in the developmental stage. This consists of monitoring a computer’s progress as it works through a number of “observations” in a data set that has been arranged to help train the computer to spot patterns between attributes as quickly and efficiently as possible.
Once the computer has started to build a model to represent the patterns identified, the computer then goes through a looping process, seeking to develop a better model with each iteration.
The aim of this is to allow computers to learn for themselves, knowing when to anticipate fluctuations between variables - which then helps us forecast what may happen in the future.
When a computer model is trained on a specific data problem or relationship it allows data professionals to produce reliable decisions and results. This, in turn, leads to the discovery of insights that might have remained hidden without this new analytical technique.
If that sounds like science fiction, consider this: Every time you’ve bought something from an online shop and had recommendations based on your purchase – they are almost certainly based on machine learning.
Hundreds of thousands of purchases have been aggregated and analysed to spot correlations based on real users’ purchasing patterns, and then the most relevant patterns are presented back to you based on your browsing behaviour and what you added to the basket.
You may see these as “recommended for you” or “this was frequently bought with that”. Amazon and Ebay have been doing this for years, and more recently, Netflix has too.
That’s all well and good, but how can machine learning help the rest of us in business?
This is distinguished from other data science practices by the use of deep neural networks. This means that the data models pass through networks of nodes in a structure that mimics the human brain. Structures like this are able to adapt to the data they are processing, in order to execute in the most efficient manner.
Some emerging uses of these cutting edge techniques seem set to have profound impacts on how we live and interact with each other. For example, the imminent launch of commercially available real-time language translation requires a speed of analysis and processing that has never been available before.
Similar innovations have evolved in handwriting-to-text conversion. “Smartpads”, such as the Bamboo Spark, bridge the gap between technology and traditional note taking.
Other applications mimic human components of understanding; classify, recognise, detect and describe (according to SAS.com). This has now entered mainstream use with anti-spam measures on website contact forms - the software knows which squares contain images of cars, or street signs.
Huge leaps have been made in the healthcare industry too. At Szechwan People’s Hospital, China, computers have been “taught” how to spot the early sign of lung cancer in CT scans. This has come at a time where there is a shortage of trained radiologists to examine patients’ diagnostic results.
It’s important to understand - in case you were worried - that machine learning and deep learning takes care of the massive analysis of data that just cannot be physically done by a human being. The results of that analysis provide humans with information on which to base business decisions or healthcare treatments.
There are exceptions such as in fintech, where the buying and selling of shares on markets are transacted automatically, in nanoseconds. But the life and death decisions should, It can be argued, still rest with humans - for the time being.
In summary, there have been huge leaps in data analysis and science in the last couple of years. The future looks bright for the wider range of real world issues to which we can apply more and more sophisticated techniques and tackle previously impossible challenges.
This article is based on an original by Justin Price