Researcher John Mashey began using the phrase big data in 1997 to describe datasets that were too large for standard software tools. The term evolved by the early 2000s to encompass three core characteristics: volume, velocity, and variety.
Big data is characterized by volume which refers to the sheer quantity of generated and stored data often exceeding terabytes or petabytes. Velocity describes the speed at which data is created and processed sometimes occurring in real-time while variety captures diverse types of information ranging from structured numbers to unstructured text images and audio. A fourth characteristic called veracity was later added to address the reliability and quality of the data itself.
Teradata Corporation marketed its first parallel processing system the DBC 1012 in 1984. That same year hard disk drives held only 2.5 gigabytes of storage before systems storing one terabyte of data appeared in 1992.
IDC estimated there would be 163 zettabytes of data worldwide by 2025 after predicting growth from 4.4 zettabytes to 44 zettabytes between 2013 and 2020. Global spending on big data and business analytics solutions reached $215.7 billion in 2021 according to one estimate.
Privacy advocates worry about threats posed by increasing storage and integration of personally identifiable information which has allowed for the abolition of trust in almost every fundamental institution holding up society. Sarah Brayne notes that big data policing can reproduce existing societal inequalities by placing people under increased surveillance using mathematical algorithms.