Connect with us
big data transformation big data transformation

Picks of the week

Here’s How Big Data Transformation Will Sweep Information Technology

 9 min read / 

With an estimated 40 zettabytes (43 trillion gigabytes) of data to be created by 2020 (an increase of 300 times on the amount of data in circulation in 2005), at a rate of 2.5 quintillion bytes per day, data is a valuable resource. Slava Koltovich, CEO, EastBanc Technologies stated that, ‘In 2017, we’ll see data become more intelligent, more useable, and more relevant than ever,’ and such a prediction is not far off. The rise of AI and the Internet of Things (IoT) has pushed firms to utilise and deploy Big Data mechanisms to stay ahead. Such a paradigm shift has caused economic trends to form.

Market intelligence firm IDC, predicts that the global big data market is expected to exceed $200bn by 2020,[1] with Asia-Pacific firms reaping an expected $65bn in productivity benefits over less data-driven peers. Since ‘The majority of Asia Pacific organizations consider data as a strategic asset,’ says Qiao Li, Senior Market Analyst, IDC Asia/Pacific, 2020 could see Asia-Pacific emerging as the centre for Big Data. IDC also found that large enterprises with 500 or more employees will lead the investment charge in big data, accounting for about $154bn in revenue.

Driven by AI, IoT and Innovation

Future Ready Singapore predicts that the number of devices capable of IoT connection in Asia is to increase to 8.6 billion by 2020, up from 3.1 billion in 2015. IoT devices, from fitness trackers, smart TVs and smartphones, collect data in increasing volumes and firms are realising the potential that these collectors hold. What’s more, you do not need to be a data collector to take advantage of Big Data; partnering with collectors and using data that they either do not need or data that you can reuse is also a keen adaptability trait firms should seek to exploit. For example, a vitamin supplement manufacturer could seek to use Nike’s App data about users’ running habits, allowing them to target drink adverts, supplement placement and other product support through the use of habit data.

Thus, it can also be seen that companies such as Facebook, Tencent, Alibaba and Google are in a prime position to control innovation. These giants are the leaders in data collection and are at the forefront of using big data in their business.

Companies that can streamline their data collection and optimise their storage, from physical location to storage format (such as tagging data, allowing AI to quickly access the data) will outperform in the coming years. Simply collecting vast amounts of data is not the priority; collecting usable data and storing that data, accessibly, is what will set competitors apart. Not only will this require a restructure (Digi-driven restructure) but it will necessitate talent acquisition. As there is already a shortage of data scientists, specialists in AI and digital experts, smaller companies will find it difficult to acquire outside talent. Relying on open software, training employees and staying abreast of developments can mitigate this.

Optimising Workflows

Specifically, sectors such as law, rely on a lot of data processing and comparatively small decision-making. Whilst AI and Big Data are garnering interest in this sector, if one looks closer, the systems are in fact being used to speed up ‘routine’ legal work such as research and work that requires very little decision-making such as writing a will.

Thus, the systems are replacing legal services and turning low-level legal advice into a form of packaged, low-cost modules for areas such as wills, contracts, pre-nuptials and non-disclosure agreements for the benefit of consumers. This is due to the fact that more work is required until AI can effectively be utilised for delivering legal services to clients with minimal human involvement. Thus, Roy Russell, CEO of Ascertus Limited, concluded that,

“Until then, in 2017 and perhaps for a few more years yet, we will continue to see incremental innovative efforts to leverage the technology, but in the vein of commoditisation – similar to what we have seen in the last 12 months.”

Withal, as the advancements continue, one will see AI and Big Data become commoditised by companies such as Google and Microsoft, meaning that it will be more accessible and become easy for developers to analyse larger sets. One can see this occurring in the rise of ‘Chatbots.’ In 2016, the foundations were laid for these AI and in 2017 one will likely see them reach greater levels of integration. Similarly, Big Data will see commoditisation as it becomes an integral part of larger business operations, particularly Business to Consumer, due to the effect of companies fearing that they are missing out and thus implementing data strategies.

Highlight: China

Ultimately, Big Data is another tech race to observe and one such participant has three advantages. China, which recently set its sights on a 2030 AI industry domination, has a vast pool of engineers to write the software, with new data centres being set up, as well as an enormous base of 751 million internet users to test it on, and most importantly, a legal environment conducive to development.

China’s Guizhou province is seeking to be China’s “Big Data Valley” by partnering with Silicon Valley and using top talent to lead growth. This is part of China’s 2015 adoption of Big Data as a national aim, creating a  development strategy establishing a Pilot Zone in early 2016.

“Guizhou enjoys the advantages that the Big Data industry requires. It is one of the most suitable places for developing big data in China or in the world,”

stated Qin Rupei, vice-governor of Guizhou.

The province has begun developing 12 green data centres, which can support the installation of 158,000 servers and by 2020, those centres are expected to load 2 million servers, with an industrial chain value of more than $15bn.

Why is Guizhou important? Not only does it have favourable tax policies to lure enterprises, but the laws present an environment that is conducive to land use, logistics, lower electricity prices, financial services and personnel training, with tech companies like Alibaba, Baidu, Tencent and Huawei setting up R&D centres in the province. Furthermore, it shows that China’s government is strongly supporting the scheme, proving there is a powerhouse to reinforce the global ambition.

Data Access

As aforementioned, China’s largest advantage is data access. The laws regulating data not only protect this data by restricting outside access as data must remain in China, but also provide suitable room for research. The data itself is similar, utilising one language, one people, one political view and one culture, meaning that the data is highly specialised and very representative. Consequently, as big data is being used more for training AI, an AI can learn very quickly about Chinese users which is useful for manufacturing, advertising, recruitment etc. as the AI can learn very quickly about the Chinese user base.

“Data access has always been easier in China, but now people in government, organizations and companies have recognized the value of data,”

said Jiebo Luo, a computer science professor at the University of Rochester who has researched China.

Compared with Google’s struggle to access medical records in the UK, WeChat has an easier time accessing useful data.

Tencent’s messenger service has over 900million active users, using voice recognition software, sending messages and playing games as well as using companion apps, the data being created on one platform in one format dwarfs what Google and Facebook are capable of accomplishing. Fundamentally, Chris Nicholson, co-founder of Skymind Inc. poignantly sums up the Chinese advantage here,

‘The Chinese AI market is moving fast because people are willing to take risks and adopt new technology more quickly in a fast-growing economy… AI needs big data, and Chinese regulators are now on the side of making data accessible to accelerate AI.”

However, as Oren Etzioni, director of the Allen Institute for Artificial Intelligence, argues, data is not the only thing AI requires, talent, algorithms and funding are also important factors,

‘Sure, there might be data sets you could get access to in China that you couldn’t in the U.S… But that does not put them in a terrific position vis-a-vis AI. It’s still a question of the algorithm, the insights and the research.’

Other Movements

Recently, Toyota and Intel have committed to forming a Big Data alliance to spearhead self-driving car development. Cars are already creating a lot of data, from their onboard sensors, through engine temperature and tire pressure, to passenger usage and drive time, and self-driving cars can utilise this data. The alliance hopes to create a data ecosystem to improve the safety of the cars, maps with real-time data, and driving assistance based on data input from the cloud.

Across the sea, the Oil & Natural Gas Corp (ONGC) is making a bet on Big Data to optimise costs and increase efficiency in operations. Companies, like Shell, are pooling data across their operations, from machines to employees, and building predictive models and programmes to augment strategy and management. ONGC hopes to use big data, including geological, seismic, well log, drilling and output figures, to reduce time spent planning drills. For example, having access to data on a recent issue that has been seen before could help the firm solve issues faster,

“The problem in drilling or production we face today is not always new. There are chances of us having encountered a similar problem in a different field or a basin in the past. So if we have a ready reference to how we solved the problem then can help us move quickly now,”

said Ajay Kumar Dwivedi director-exploration at ONGC.

Before Brexit is conducted, the UK will seek to replicate the data protections of the EU’s data protection law, the General Data Protection Regulation (GDPR). The GDPR seeks to protect consumers and present them with more control over their data. The UK’s proposals seek to instigate a right to be forgotten, limiting a firm’s ability to store lifetime’s worth of data on any single user. As well as expanding the definition of personal data, to include among other things IP addresses, the law seeks to help businesses with data usage. The GDPR is expected to become law in May 2018 and the UK’s big data scene could change very soon afterwards.


The holy trinity of IoT, AI and Big Data continue to be at the forefront of the conversation on data techniques and disruption. With the rise of China’s homegrown talent and the access to unprecedented levels of data, Chinese giants are seeking to lead the industry. Other regions see key partnerships in either deploying big data or seeking to develop an ecosystem to perfect future research.

Have your say. Sign up now to become an Author!

Click to comment

Leave a Reply

Your email address will not be published. Required fields are marked *

Send this to a friend