Big Data, and how it concerns you.

As an engineer in my field of study, I wanted to bring some attention to a term/concept known as "big data". In my industry, this is a fast growing market of information, and sadly, every one of us plays a role with it, just as it plays a role with us. I'm breaking this up into sections for easy reading, and as always, I'll provide my sources of information at the bottom.

What is it?
My definition of "big data" is a loose term to describe large amounts of data that can be used to create/analyze patterns. For example, weather data is big data, because it is always changing by the second. Population is also big data, just as traffic flow, pixels in an image, and even your mental state are.

What is good about it?
The good news with big data is that it can come in especially useful when you need to bridge the gaps between two completely different industries.

For example, consider a future where an earthquake warning will be sounded off before the earthquake actually happens, because the seismograph data collected from previous years indicated a pattern that the earthquake.

If you want a recent example, I found one from Intel that makes note of a city in China that is analyzing data from an area of increased traffic to better catch offenders. Normally, due to the amount of traffic, violations were not being accurately captured. However, now that all of this data can be processed easily, they can alert the authorities to the vehicle's color, driving patterns, and past history of driving.

In this case, you are taking driving data, and using it to track down a violation. Case

Or, in Hitachi's case, use smartphone data to collect information about busy roadways to help with urban planning and helping to reduce CO2 by providing better planned or shorter commutes. Case

The Dark Side

Well, here is the part that concerns me - How many times have you logged into ATS, and found that several of the ads you see are for similar products that you searched for elsewhere? Guess what, this is big data at it's worst - it's target: You.

In all fairness, it seems harmless, right? Ads that are customized to something you already like, coupons for products that you already buy. Youtube clips similar to other Youtube clips that you want. In reality, this is not good for you, because you are becoming a big data entry point.

Way back in the day, companies would stop producing something if there was no desire to produce said product. They would take inventory of what was sold, and what had not sold, and then if the demand was low enough, remove the item from the store shelves in that location.

How do you fit into the equation? Companies realized that they could get a "reading" of what people liked, and what they didn't like by capturing more data about a person. For example, when you look at a product online, that website will retain information about what you like (look at Amazon - if you log in, look at an item, and then continue looking, you'll notice the "similar items" reflects items that are just like what you originally looked at).

Again, it's still harmless, right? Let's take it a step further: How about trading the ability to track you more, for discounts on selected products or services? For example, Kroger offers a reward card that allows you to get some of their food for a discount. In return, they now can determine which stores you visit the most, and what products you buy.

Here's the scary part: There is another evolution in Big Data that has been hinted at in several seminars I've attended (wish I could upload them, as it's definitely ATS fodder). Companies are realizing that this data is worth something to other markets. I feel that it's only a matter of time before this is considered a form of currency (For example, you can get internet for free, but in order to do so you must view ads every couple minutes to show where your interest is; you can have this shirt, but only if you give our company some of your interests in a 1-minute survey; in order to receive medical help, you need to fill out our survey to determine if you are the type of patient we want in our facilities).

So, what does everyone think about this? Is this something "good" for mankind, or a step in the wrong direction? Personally, from my standpoint, this doesn't help the majority of us. As I showed above, big data can help us just as much as it can hurt. I'm curious to hear what you guys think! (And, open for taking tips to improve or add onto the discussion - as I think this is my first time down the conspiracy rabbit hole).

References:
www.forbes.com...
www.forbes.com...
www.mastersindatascience.org...
www.informationweek.com...
www.consumerreports.org...
grait-dm.gatech.edu...

Also, found this as an interesting read - Some of you might like it.
www.whitehouse.gov...

-fossilera

(And I apologize for the long read - tried to cram about a textbook-worth of research into a single post).

Drives me insane.
Look at a item on the internet, find it on Facebook immediately.
Unremovable super tracking cookies are BS as well.

Just wanted to add 3 odd observations I have noticed with Google maps and nav through Verizon on android.

1.Complete redirects of route, miles out of the way to go past a road with a mall on it.

The two that really make me suspicious:

2.Complete redirects miles out of the way to toll roads.
3.Complete redirects miles out of the way, off of main highways to police checkpoints while crossing the country.

The last one was absolutely insane. I will use a stand alone gps from now on for travel

a reply to: fossilera

I must admit to seeing a bright light on the 'dark side' because if my wife is scanning the clothing sites whjat she was looking at shows up on the ATS and I was just checking it out as well as the cards we hold - bless her.

For me its about an invasion of privacy. Yeh which I have openly admitted to violating but who gave me the opportunity when innocently sitting at the screne because I couldn't sleep?

Sure you are right in that we arfe getting useful data for many things that are a benefit but as Mandroid 7 says there is a nasty and dishonest side to how the institutions are using this data which I object to because its spying basically. A right I never voted for in elections - spy abroad but get out of my face.

Another aggro which you might not have in the USA is the interupting and irritating cold phoning because they have tapped into what you own from some source that has sold your information. I am sick to death of the Far East calling and trying to sell me or ask me personal questions and especially when they think they have the right to survey my household. Again the wife - bless her doesn't suffer intrusions and I usually hand the phone to her as my man-talk isn'[t always polite when I am saying exactly what i think.

Theres the rumors of the Amazon Data warehouse in Kentucky, a monster warehouse of nothing but servers.. and Ali Baba is 5 times the size of amazon. Im not sure where the ali baba data monster is stored but Ill bet you my last dollar that site is earthquacked or whatever in the next 2 years

a reply to: fossilera

Great job making sense of a complicated subect, I really enjoyed reading your OP, and I think about some of the same questions that you have posed often

So, what does everyone think about this? Is this something "good" for mankind, or a step in the wrong direction?

The short of it is that I think that we will either allow this technology to master us, and create whole new meanings out of the term "caste system", or we will master it, and it will usher in a new era of human communication self-expression and we'll solve a bunch of our problems.

Dunno which.

I can envision subscribing to private bartering systems that go underground, via Torrents, pick producers, suppliers, satisfy contracts, and completely eliminate advertising and profit sapping middlemen. A good home for bitcoin, as well.

As with any other force, the context of good or bad is subjective. Governments and large corporations routinely work with big data and it is an exponentially increasing reality of this growing interconnected world. As with other forces like fire and electricity it does take some time and trial to master.

The main danger I see with big data is those who get lost in trying to understand it.

Let's say there was a way of analysing an internet user's browsing habits to determine their likelihood of carrying out a violent or otherwise criminal act, imminently, or at some point in the future. Let's also say that it was 100% accurate. Would we want such a tool at the disposal of our law and order system, or not?

I like a lot of the responses - And although I'm not hitting everyone above in this post, I did read the comments and may still yet refer back to them.

@Bybyots - I can agree with all that you've said.

@Mandroid7 - I'm going to check out the google maps, now that you mentioned it. Seems to me the last time I went down to Ohio, it took me well out of my way to take the toll road down there - maybe it just picks the quickest/most popular route first, but I want to look into it now.

@Speckle - Do you have a link or youtube for the rumor with Amazon? I haven't heard about that, and would like to read more on it.

@Shiloh7 - We do get cold calling here in the USA; The only reason I didn't include it is although the callers do know when you buy some products, I haven't found any articles or documents saying that companies sell this data to them.

@IvanAstikov - Such a tool would be both a blessing and a curse. Blessing in that like you said, it could be used to help track down people that might be planning something. But, to my eyes, even if such a tool was used all the time for "good" purposes, it's also possible that someone like an ATS'er investigating something of a similar nature (think school killings, terrorists, 9/11, etc) could also be targeted.

@Everyone - Right now, I feel that this concept is probably going to help in the immediate future, which is why I pulled several recent examples of how this is helping (because the original was already huge, I couldn't include the medical examples). But in the long run, somewhere this process will become corrupt, as all things eventually do.

-foss

a reply to: Mandroid7

I drive for work and I've noticed that with the mapping on my phone and Ipad.It used to be if I didn't want to take a toll road I could drive past the onramp and mapping would give me an alternate route. Now it just keeps trying to redirect me back to the toll road. Also wiping my phone of preloaded google features I don't want is a chore now.