What’s Big Data?

Two facts why Big Data is not a fad and will revolutionize how we do business and how it affects our daily lives.

1.     More data has been created in the past two years than in the entire previous history of the human race.

2.     Data is growing faster than ever before and by the year 2020, about 1.7 megabytes of new information will be created every second for every human being on the planet.

What really does Big Data mean? Is it only a matter of amount of data?

Back in 2001, Doug Laney of Gartner introduced the 3V’s concept: Volume, Velocity and Variety.

Volume - Volume refers to the amount of data generated. The growth of social medias and e-commerce has created huge amount of data in terms of transactional information and likely more than we can handle and analyze.

Velocity - Velocity is the speed of data being generated. Let’s look at how news are reported. Yesterday’s news is recent as far as newspapers are concerned but nowadays’ News are reported as soon as it’s unfolding. Data are being generated on real-time level with the advent of new technologies such as RFID, activity trackers, social media posts and so on.

Variety - Data comes in different formats from structured numeric and text data that fits in to traditional databases to unstructured data such as emails, videos, tweets, photos and so on.

Why Lean Six Sigma is still Relevant in Big Data?

When performing Big Data Analysis, using the Six Sigma framework can indeed be vital.

Defining the problem and the goal of your analysis before anything else is very vital before performing big data analysis.

You still need to think about how to measure your current condition and at times you may need to validate your measurement system.

Understanding the type of data you have, how you’re going to analyze your data as well as how to interpret the results of your analysis and finally how do you come up with solutions or actions with respect to the identified problem -  I think these are still requirement regardless if you’re dealing with “small” data or big data.

Let’s not also forget, moving forward with your improvements, what type of controls you will need to put in place to guarantee the improvement remains.

It’s not a coincidence that two large companies who are in the forefront of Big Data already have a strong and successful implementation of Six Sigma culture (IBM and GE). Big Data is here and most companies will operate with Big Data in the near future. Leveraging on strengths of both Six Sigma Methodology and Big Data Analysis will go a long way in a successful implementation of big data strategy.