Big data - What is it really about - In laymen’s terms
I hope everybody on the platform noticed the massive shift in the trending pages, not only do we have much more diversity of content at the top of the trending pages, we also have a much wider spread of posts receiving rewards. I have just checked the pages and the top 10 pages all have at least $100 pay-out amounts. If you did the same thing last week, there was a definite drop in reward payments from page 3. There were massive payments, and then a drop to next to nothing.
This is massive as this means that either the whales are spreading their weight more efficiently, or there are more dolphins and orca's which is created n an ongoing basis, and thereor the voting power is starting to be spread. This will ultimately be the best thing that can happen on Steemit. More evenly voting power.
Big data
I am not a specialist on the subject, but I have been in IT for many years, and I am blessed to understand a fairly diverse technology set. I have also been told that I have the ability to translate Information Technology into "English". What I hope to do with this post, is to open up the understanding of the the Big data world to the non-technology minds within the community. I will also use an example to explain this in laymen's terms.
What is Big Data?
It is exactly what it says, a large set of data. But most people don’t really get what Big mean. Even some of the more junior developers in the industry gets this wrong. I am speaking about BIG, no ... large, no wait massive amounts of data, no I think what we must call it is enormous amounts of data. I do not think there are specific words which actually show the size of the data we are talking about here. The data sets are so large, that it is even too big to store in a database. Sure some Big Data projects is based on data that is stored, but let me show you an example what I am talking about. There are many new ways which technologists is adapting to this technology and creating new ways to interrogate this type of data, but it will be too technical.
Big Data Example
So you are driving on your way to the office and you get a call from you energy service provider. The lady from the call centre says: @jacor,Did you not perhaps forgot to turn off your stove before you left home for work today. Now how on earth will she know this. It is all about analysis and trends. We are already have a very integrated "network of things", but the matter of the fact is that when energy service providers start monitoring the use of electricity in each households, business, factory or manufacturing plant, they will start seeing these type of trends.
How will they know that you forgot your stove on? They have been building a trend of your usage over a long period of time. They have a dashboard, which highlights specific things, when something is out of the ordinary. This spike is out of the ordinary in this case, as for the past 101 days, you have only used a specific amount of energy when it comes to the use of your stove. Today you are using the stove much longer than usually, so something might be wrong. If we go further into IOT, they will know you are leaving for the office at a specific time, again a feature of connectivity on the IOT. Your vehicle is connected to the grid and they can see that you are not at home, your wife just dropped of the kids at school, because they have just logged into the international school database. Aha 'Big brother is waching us' .
At the end of the day, the energy provider, cannot store all the information about every household's energy forever, but they can store the trends and 'spikes' and monitor the live feed of energy flow and report on that. For me that is big data!
(I am planning to write a about IOT - Internet of Things- in a follow-up post, depending if this post is accepted well in our community)
What will big data be used for?
There are a multitude of applications for this new technology, and will be too technical to start explaining Hadoop and the likes to the specific audience I am targeting with this post.
What I can show you in graph form, is what direction the utlisation of big data is being used for by business today as we speak.
Why this post?
I am thinking of writing a series of posts to explain some terms, which a lot of people just take for granted and beleive it is clear that everobody just know what it is. These terms are used in the industry on an ongoing basis. If there are people in the community that will be interested in understanding more of this. Please leave comment. If there are specific things you want to learn more about, please leave a comment.
I would love to hear some comments of what each of you think will be a good apllication for this technology set.
Happy Steeming!
Please follow me on my blog @jacor if you enjoy my topics and content.
@jacor features authors to promote a diversity of content and new authors. All STEEM Dollars for these posts go to the featured author.
If you are looking to be featured as an author, please contact me via email - [email protected]
[
] Hello @jacor
Thank you. I am one that need Information Technology into "English".
My question: I hear the terms Whales, Dolphins etc. How many categories are there? What are the numbers or % in each? What are the trends here?
Regards
The Viking from Norway
Hi @ramta, great to hear from you. Good question ... You can see the distribution and categories by selecting clicking here On this page they are not called Whales or Dolphins, etc. This is the name that the community decided on when it started. There are many ways to interpret, but in my view:
I hope this helps...
I might be wring but that is how I classify them for myself.
[
] Thank you @jacor
It became perfectly clear to me.
[
]
Regards
The Viking from Norway (Large Dolphin)
We can use the siacoin for that. It is a good way
Very interesting post, I'm still amazed at the extraordinary nature of steem; the future will be inspired by innovation.
I think big data mining can be a scary matter. From the perspective of the consumer I am monitored for my activities and spending pattern. This is an intrusion of my privacy. Especially with the convenience of technology in assisting with data collection, it makes this whole setup even more frightening. Not sure about others or technology set used but I am really concern with privacy as that is my basic human rights.
Agreed, but I am afraid there is not much we can do about it. Agree with you on the basic human rights. However, even if you look at something like the blockchain, where all the data is public, it is just a matter of time before somebody will use some sort of scraping tool to get data about each of us to use in marketing going forward. One of thse things I guess!
I suppose it is all about the underlying motive. In fields like education, farming and health care, as well as science, being able to get access to, mine and interpret big data, people's lives are being vastly improved every day. But yes, there are always those that wish to exploit something for themselves. It is a balance, but we can not afford to stop moving forward. We just need to be cautious. I am often reminded of those that used to say the car will never catch on as it is to slow, noisy, and dangerous .... they will never get one. See where we are today
I think it is a challenge to find the balance between sharing and being authentic online and not giving away personal details that can be used against us. A lot of people do things like fill in these cute surveys to see what type of character they are or something equally silly. They don't realize that these are just scraping personal data. There's something going around these days about the first 7 jobs you had. When you open certain online accounts, you have to answer a security question. What type of questions are these? Your first school, your mother's maiden name, your first pet or teacher or car or job. If some hacker or identity thief gets your password to your account, it may not be too hard for them to find the answer to your security question in some other data set. At least, that's my fear and why I stay away from those types of things. Most people have no clue how big the data sets are and how easily info from one set can be connected to another. Sometimes this can be a good thing. It's the potential for nefarious use that worries me.
You make it almost sound like a good thing that this development led to traffic accidents being among the most likely causes of death for human beings without anyone shrugging a shoulder about it or considering it, in a silent moment of objective reflection, outright perverse to value the machine more than the human.
Thanks for the perspective @jacor. So much data that it can't be stored - yikes!
Brilliant stuff @jacor. I enjoyed this. I think we to often presume people know what these buzz words mean, so glad someone takes the time to write about this in simple terms
Thank you @robertm . Great to hear from you! I hope to build a community of followers from a non-technology background as well. Always good to add value to the community.
I think that is where the world is moving, where most of us are tech users, but only a few of us the ones under the hood .... so to speak. Good luck with that.
The future will depend on how well humans manage to tame the beast.
The data will be scoured by bots, their filters pre-programmed and genetically adaptive, responsive to personalized reward system defined by the user, now confronted with the tasty creem.
The question is which service the community will reward more: the freedom of rough speech or the predigested narrative so well known from all other media. The right interface would filter these attempts at attemption-grabbing out and make them invisible, and visualize automated analytics to highlight conflicts of financial interest.