The demand for Big Data is growing with companies realising the predictive potential this incredible technology holds and which can make them more competitive as they identify patterns often impossible for the human brain to conceive.
Despite the surge in positive sentiments regarding Big Data, there are some concerns regarding how much storage space it requires and the current capacity to meet the demand. There are also fears that Big Data, combined with psychological profiling, could be used to influence voters and directly affect how they cast their ballots.
Everything can be data – actions, behaviours, sentiments. It has always existed, but the technology of today allows this data to be stored in massive quantities for later analysis. The data sets are usually so large that they will need extraordinarily complex processing software to make sense of them.
Companies, governments and institutions are waking up to the use of Big Data to try, for example, to prevent diseases or help fight crime. But most of all, Big Data is being used to detect new business trends and make companies more competitive.
Walmart uses Big Data to try and improve its profits by looking at customer transactions. It crosses the items bought with the time and number of purchases. This data gives the company clear evidence of what needs to change and what is worth keeping in the business strategies.
Good For Business
With the growth of the digital world, the amount of digital information keeps increasing at an exponential rate. Google is betting heavily on the Big Data it collects from its users to provide a better service in the future. Its search engine collects and translates the number of clicks on any website to determine how much that site is worth.
There have also been a few companies trying to use Big Data to match people looking for a romantic partner. Websites like match.com and OkCupid use data generated by the users’ choice of preference to pair them with possible compatible matches.
Recently, Coca-Cola made headlines because it released two new flavours. The novelty surged after the customers chose their preferred non-existent flavours in a few selected “Freestyle” machines the company installed for testing. The devices collected the data from what the users preferred and started developing the two most wanted flavours.
But the potential of Big Data can also thread into political waters. An app called Ovia, specialised in helping women conceive by tracking their menstrual cycles, mapped which American women were more conservative depending on where they lived and whether they were trying to get pregnant or not.
Big Data is becoming a valuable commodity for companies. It can play a valuable role in establishing new development strategies or even serve as the basis for entire business models.
The analytical results of Big Data can affect business decisions and, in some cases, provide the certainty companies need in order to tackle new investments. According to a report by Forbes magazine, at least 60% of the companies that used Big Data in at least half of their marketing campaigns admitted that they exceeded their initial goals.
It is influencing more and more of the business processes within companies as it makes them more successful at identifying the specific areas which they need to improve.
Big Data sounds promising to companies, but as the technology grows, it pressures the IT aspects that make it secure. It swallows up the storage space currently available to the limit, and it also makes it harder to guarantee the safety of all that data while maintaining the privacy that some of the data may require.
In fact, market intelligence firm IDC estimates that the so-called unstructured data that comes from Twitter and Facebook posts and photos is going to grow from 9.3 zettabytes in 2015 to 44.1 zettabytes in 2020. The upside is that according to an estimation by Forrester, the total global Big Data management solutions market will grow an average of 12.8% a year from 2016 to 2021, making it possible to deal with that enormous growth.
But some of the existential challenges of Big Data go beyond technological drawbacks and enter the fields of ethics and morality. There have been examples of the use of Big Data returning some frightening results.
There was a police website that mapped police interventions in a city in the US and inadvertently ended up revealing that the only day of the week where officers wouldn’t “sweep” a street known for prostitution was the Wednesday. This strategy was a secret within the police department and was supposed to be kept secret, a secret that Big Data unveiled.
When a majority of the British electorate voted in favour of the United Kingdom exiting the European Union in the Brexit referendum on June 23, most of the media seemed perplexed. Similarly, the election of Donald Trump as President of the United States was baffling to many who had followed the polls religiously. One of the most widely-spread theories is that there is a big part of the population that is not reflected by social media and that lives outside of the major cities where under media scrutiny.
But there might be other explanations, one of which is the usage of big data to influence political margins. It is possible to combine psychological profiling of the electorate with the data they produce and create advertising content that explores the unconscious biases of niche voters on an individual level.
This strategy may be compared to the commercial subliminal messages that movie-goers may be exposed to during films, where they are often influenced into buying the products they see on the adverts. In the 1970s, these kinds of messages were banned.
Trump won the Republican primaries, using a smaller budget than Hillary Clinton, the Democratic Party candidate. Trump’s campaign resolved to design a unique digital strategy which took the masses by surprise. Trump’s team hired British data company Cambridge Analytica who worked alongside Donald Trump for the whole campaign and helped him figure out how to win the Presidency using Big Data.
Cambridge Analytica has, according to its CEO Alexander Nix, between 4000 and 5000 data points (or pieces of information) on every adult citizen in the United States. This includes age, gender, ethnicity, what press they read, what TV shows they watch, what food they eat and what cars they drive. It comes as no surprise that Trump happily bought their services.
So, by assuming that people who buy the same things have similar personalities, this Big Data company dug deep into the electorate and determined where it would be more cost-effective to advertise and which portion of the electorate Trump had the bigger chances to win. This way, the ads the campaign produced were tailored to particular kinds of voters and their preferences.
It resulted in 4000 different online ads that ended up being viewed 1.5 billion times. While Hillary Clinton chose to spend more than $133m of campaign money on ads, Trump spent none of his budget on advertisement until September 2016 when he began to discretely post his ads on Twitter.
A 2015 study conducted by IDG revealed that 39% of the respondents had no plans of introducing Big Data analysis in their companies in the following year. However, the survey shows that 36% of the companies have an increasing will to dedicate a bigger percentage of their IT budget to data-driven projects.
But scepticism is also on the rise when it comes to the implementation of Big Data. Jumping into Big Data might be a natural step for tech titans like Facebook or Amazon, but smaller firms might not necessarily need this kind of technology to help their business, especially considering the costs.
Irrespective of the success of Big Data in mainstream workplaces, it is already everywhere in everything humans do. With GPS technology, locations are now data. Even what people ‘like’ on Facebook is now data, quantified and prone to serious analysis.
This data collection and analysis will likely continue to be carried out and developed in the future. It may serve as a commodity in business, a source of information and a tool to fix problems.
Its development will likely be a reflection of society and how humans think, react and interact with each other. It is a fascinating technology, but companies and people shouldn’t let it blind them without taking into account the problems it has and the answers it doesn’t have.