Big Data Analytics

Data generation via electronic means had grown exponentially in the last few years.

Imagine: all the new Facebook users adding pictures and statuses online; all the smartphone apps that allow e-commerce purchases at anytime; all the calls made without a landline; availability of video capturing devices that propagate and receive wider usage; satellite tracking systems that allow the recording of data recordings of various objects and devices.

These datasets have grown partly out of availability of cheaper and more reliable data storage, much of it in the cloud, and they have these commonalities: 1. they are large in size. 2. they are in various forms, and do not necessarily fit nice and neatly into some columnized tables.

With that, let's introduce a new term here to capture the impression that could've made upon you with this concept: a petabyte. You'd remember the 'byte' from way back (the 8-bits that is). 1000 bytes make a Kb; 1000 Kbs make a Mb, or, 1 million bytes. Then 1000 Mb becomes a Gb, and 1000 Gb is a Terabyte (TB). Now we have a TB being a million Mbs. Then, a petabyte (PB) is a thousand times a TB, which is a million Gb.

That's a lot of data, and that's great news for talented DBAs, we have a lot to work with. You may or may not be surprised to know that PB is actually a suitable unit measure for describing the volume of data that transfers or transacts through many systems on a daily basis.

One idea that attempts to capture this Big Data is the Aoache Hadoop. It is an open source framework that attempts to analyze Big Data. Others include IBM mainframe or supercomputers, with the government and some major tech firms heading certain initiatives in its research and development.

Outside of reliance on the speed and capacity of large computers, i.e. hardware capacities, Big Data is likely to utilize indexing and parallel processing on the software side. The exact forms are still to emerge so we will wait and see.