Today Data has become the most valuable currency for any business to leverage their next action. If sales are the air a company breathes, then data is the blood running through its veins. However, most of you would be intrigued to know that Data existed before computers, technology, and the Internet crept into our day-to-day lives.

In this blog post, we’ll learn:

• Data Before Tech Advent
• Big Data and the Evolution of Search Engines
• Technology to Handle Big Data

Let’s get started

Data Before Tech Advent

It won't be hyperbole to say that Data existed even before money. Remember, the first official currency, cowrie shells, was used to buy and barter for goods such as food or clothing. Credit agencies were formed to log customers’ likelihood of paying back debt manually in books on the barter principle. Similarly, barkeeps kept tabs on their regulars to ensure they knew how much they were actually consuming. Data is older with more impact on human life than many of us could ever imagine.

Many of you might be intrigued and doubt if it is the same data we talk of now, and if data existed forever, why did it take the 21st century for data to become a Big deal?

The story dates back to when Data started getting more critical after the need for automation increased significantly due to two significant events -

First was when data was used to predict if any enemy could launch another attack on the same spot or it was safe enough to bring the fleet hidden away, or to keep a record of the soldiers injured, recovered, and predict the demand for fleet size.

The second event was when Hollerith Tabulating Machine used punched cards to automate data processing, reducing the time it took to complete a census from eight years to one and a half years. Data, in this case, was the details of American's that were part of the census, including their names, age, occupation, and more.

After facing challenging times, Hollerith sold his business to Charles Flint, who later partnered with Thomas Watson to found International Business Machines, popularly known as IBM. Thus, the original Hollerith designs for using punch cards to automatically count things led to many breakthroughs, which were the dawn of what we now think of as computing.

Big Data and the Evolution of Search Engines

Computers were designed to serve the data processing the world needed. Still, the question is, why did it take so long to coin the term “data science” when practically computers, phones, laptops primarily served the function of helping us process data and transactions.

The answer is that recorded data wasn't ever as Big as today, thanks to the internet!

Before the explosion of the internet in the 90s, companies only had to handle a small amount of transactional data. They were selective about what they would choose as the data storage was costly. This has since changed, and data storage has become increasingly cheap, down to fractions of a penny per gigabyte.

As more and more people gain access to the internet, it is believed that over 2.5 quintillion bytes of data are created every day. This data helped refine the internet searches that we do today with either a click or merely asking google what to find. However, back in time, you had to know the exact URL the www.whatever.com to find it. Else you'd only find irrelevant answers to your query.

A search engine finding web pages based on search terms requires sophisticated algorithms and processing a staggering number of requests. Big data has made possible the development of competent, accurate, and relevant online search engines. Now the question is, how are companies handling big data coming in from several data streams?

Technology to Handle Big Data

In 2003, Google released a paper on the Google File System. The idea was to store chunks of data in multiple locations, so triplicate, and then you could retrieve those bits by looking at kind of a directory of where all the data lived. Subsequently, Google released a framework for querying this data called Map Reduce. It indexed files for you to interact with that index.

Fast forward to 2007; Yahoo had two clusters of this system which later changed from Google's naming to Apache Hadoop. Several companies jumped on the bandwagon to add new tools to interact with this data. Facebook created Apache Hive that allowed users to run SQL queries on these new big data platforms. The analysts working on the traditional databases could now work in this new and scalable environment with unique data architecture. However, this isn’t the point yet, where data has done its job in refining technology.

Technology never settles on anything but innovation.

Several tools are open source and are shared to constantly improve your accessibility and ability as a data professional to use these tools quickly and efficiently and make a difference from the knowledge and information from the data.

Technology doesn't cease to improve, nor should you. Continue this journey of knowing all things data. Head over to freethedataacademy.com/yt to see our entire catalog and sign up for a seven-day free trial, so you can start learning today to elevate your career tomorrow.