You are viewing a single comment's thread from:
RE: Will the Rise of Digital Media Forgery be the End of Trust?
Another reason why I think we need an exocortex. I do see the same problem you see. I don't see any top down solution to it.
Another reason why I think we need an exocortex. I do see the same problem you see. I don't see any top down solution to it.
Are you talking a Brain Computer Interface? Whoa, and I was just worried about computers, cars, and toasters being hacked. What malicious things could someone do if they could hack a BCI? Mind boggling.
As for fake communications, the problem has existed for centuries. We can easily add certs, hashes, ID/auth to digital communications for validation. The tech and tools for trust are possible, but will they be integrated and used?
When I refer to exocortex I am not referring to BCI. BCI is BCI. Exocortex does not require BCI because BCI is merely a communication interface and the interface in my opinion isn't the mind. I also don't think the brain itself is the mind, so to have BCI merely connects the brain to the computer but the what matters in my opinion is what the computer can do. So we could use a website like this here, a smartphone, or anything, as the interface technology I expect will evolve over time.
The concepts are on my blog. I think the biggest issue is the lack of wisdom and I think we can use some of these decentralized technologies to provide what I call a wisdom engine. A wisdom engine is a function of an exocortex. An exocortex in my opinion has to exist because a human brain is biologically limited and not suited to scale with the complexity of our society.
Details are in these blog posts for you to review and comment on:
Isn't the risk of such capabilities is that we become heavily dependent upon them? There is an axiom in cybersecurity: The value of technology capabilities is proportional to accompanying increases in risks.
i.e. if an exocortex is created with all the tremendous capabilities, we will likely become heavily dependent upon it. Therefore attacks which compromise, deny, and expose it for malicious manipulation become more attractive to malevolent parties.
In my latest post I identified the problem we face as societal complexity explosion. This in mathematics would be combinatorial complexity where the more hyper connected we become to each other the more complicated our interactions, social lives, also become. The technology evolving I compared to the analogy of adding new chess pieces and squares to the chess board which increases the complexity of our interactions with every additional piece and square.
So when you say there is a risk that we will become heavily dependent on an exocortex I think it's already too late. Society already is complex beyond the point which any particular human brain can grasp. Can any of us besides the best lawyers make the claim that we even know every federal law on the books (excluding state laws)? Even if we know these laws can we then claim we truly have an understanding of these laws including the probability of aggressive enforcement? Without that ability we cannot properly assess risk.
Let's move beyond the law, how about terms of service agreements? How many of us have the attention span to read and understand these agreements?
Let's move beyond legal. Let's move to social, how many of us actually have the social capacity to try to meet the expectations of 5000 friends? People have 5000 friends on Facebook whom they barely know yet each one of them gets to see their timeline and judge their every word.
So do you see my point here? It's already too late. We live in a society where most people don't even memorize their own phone number or other people's phone numbers anymore. We live in the society where people ask their smart phone for directions rather than applying their own spatial intelligence. We live in the society where people use spell check. Most importantly we live in the society of judgment, of personal responsibility, of mass incarceration, of public shaming, all which require exceptional levels of wisdom to avoid. All of the risks a person faces in terms of risk of negative socioeconomic outcomes are rising as the technology improves.
The technology puts increasing restraint on behavior, on speech, on even the ability to think. I don't really have the ability to reverse these trends but an exocortex is about surviving the changing trends. The purpose is to allow the individual in society to build their capacity for resilience and to allow for them to pursue wisdom in the face of obstacles known and unknown.
You are very correct that these exocortex technology will be attacked which is why I'm very concerned about the fact that all the current attempts to build them seem to be around centralized companies where the data is controlled by the company. If Facebook is going to offer for example intelligent agents and similar technology then do we have to worry about all of it being compromised if Facebook is compromised? The idea I have is to create security through diversity. To decentralize and generalize the technology so that users do not have to rely on any particular company to provide an excortex functionality. This would make it more like running your own email server vs paying a company to host your email vs going with Gmail. You can do any of these or all three, so the attacker cannot simply hack your Gmail account or get a job at Google and leak.
I don't have all the solutions of course so feel free to review and leave commentary. If you have a way to make it more secure I'm open minded. Also if you have a way where we can avoid depending on these technologies heavily I'm also open minded. I personally don't see any other way to scale myself up with societal complexity (more laws, more rules, more norms, more relationships, more information to process and evaluate), as my brain capacity is finite.
I agree it is already 'too late' for some aspects. Heck, I don't remember anyone's phone number anymore, but we still must manage the risk of this being used against us. Pushing forward without a plan or mitigating controls seems hazardous and short-sighted.
So, how do we identify the optimal risks and put compensating controls in place to mitigate as we continue down this path?
It's being used against us right now. That is one of the topics I'm concerned about. Actually put another way, our cognitive limitations are exploited by social engineers, hackers, scammers, and bad actors.
If the risk emerges from "wise" hackers exploiting cognitive limitations then the only mitigation I've been able to find is to figure out how to reduce that attack surface. For example social engineers, or psy ops using memes, they know very well that the human brain relies on stereotypes as a short cut to reduce cognitive load. They may not know the neuroscience behind this but they intuitively exploit it and if it's a foreign government behind the information operations then for sure they know what they are doing.
The problem is there is no viable immune system for this. I'm basically required to be ignorant and susceiptible to disinformation, to make decisions with cognitive biases, to make decisions with attention scarcity, and businesses thrive on this. I will not name the businesses but I'm sure you can think of some businesses which use marketing in such a way that profit can be extracted by exploiting these cognitive limitations.
In my opinion to do a risk assessment together, globally, collaboratively, requires building the platform in the first place. Else we will have to rely on small groups of experts, many of whom may be compromised, to make top down solutions which we may not have the chance to give our input or discuss. I would suggest we use decentralized technologies to specifically produce wisdom building tools so that we can do collaborative risk assessing, collaboratively design the very platform itself using the platform we are designing. This in the engineering sense would be a self optimizing design, and even Steem to some degree has some of these capabilities although Steem lacks the capacity for wisdom building, as it does not include a shared knowledge base, or have the symbolic reasoning AI, which in my opinion will be necessary to boost decision making. Steem's interative design improvement capacity relies on high technical expertise which results in ultimately less collaborative centralized decision making.
So I'm definitely not someone in favor of pushing forward without a plan, on the engineering level we require a specification, whether it be formal or informal. I do not think we can plan for everything though which is why the design has to be iterative, adaptable, capable of being upgraded over time as we learn more, become more wise, etc. So a wisdom engine or whatever we could call it, as it produces results we could use those results to in theory improve the wisdom engine continuously.
In my references I include two examples of designs which are self optimizing logic. Theoretically speaking I agree with the approach of using the platform itself to continuously improve on the design of the platform provided that such a platform can be built. The idea being that upon initial construction we don't really know any answers to the tough questions such as the ones you ask, and so we must use the platform to ask the tough questions in the appropriate way so as to receive the answers. Changes can go into effect if agreement by the agreed upon mechanism can be reached. Most important of all, public sentiment is critical in my opinion to decisions which effect stakeholders, so in terms of putting controls in place we would again need this to be a data driven process where we have some idea what the values of people are, and factor this into the safety decisions somehow. Thanks for asking tough questions because that is exactly what is needed to build something great and keep it beneficial.
References
One more thing to consider. It may be the 'wise' attackers who develop inroads to exploitation, but it then will be disseminated to less technical, basically anyone, to leverage. That's when the problem scales unimaginably.
This is one of the problems with AI and cybersecurity that I work on. Adoption of AI could be a turning point for efficiency and effectiveness across just about all facets of mankind, but if those same tools we embrace are leveraged for malice, then we have created a tool for our demise. How do we proceed where we gain from the benefits and yet still mitigate the risks to an acceptable level.
Well it already is the case that anyone can set up bots so that's not hard. The offense always seems to have the advantage there. The benefit of AI is that it allows for automated detection. I think it's not possible to create a tool which cannot be abused by an attacker. The point is right now only the attackers have the bots which means we get all the negative use cases without any benefits to the every day user.
I suppose you could make a case that if the every day user had access then it requires slightly less technical sophistication but even if we look at Steem and make the same argument? If we release the bots to everyone does it really make the situation worse? I don't think it would. I think it would allow everyone to contribute to pro-social bots.
An argument can always be made from the top down that keeping people poor and ignorant reduces unknown security risks. This as a defensive strategy I think is a bad one because it assumes there can be perfect defense against unknown attackers in an information context. I prefer the strategy of building for resilience with the expectation that there is no way to predict or defend against all kinds of attacks (it's cat and mouse). But we can build the sort of network where recovery from any attack is quicker.
If we have a network, with a shared knowledge base, and bots, which is building wisdom for us all, compounding on itself, then this would also include the area of cybersecurity. This also would include answer the questions on how to recover from attack or how to mitigate the risks which inevitably come from abuse of information. The reputation system for example if it is to be designed will have to be built from the current state of the art in terms of knowledge.
Just one example of a pro-social use case. If we all have bots then we can simply tell our bots to filter information and automate shopping. This would exclude a lot of the attacks (not all). The reputation economy would mean every bot which is anti-social or evil or harmful gets down voted, a bad score, etc.
Sybil attacks and botnets will still exist but the idea is that the most effective and popular bots will be the bots which people appreciate (in theory). Also in theory you can set up the bots so that for instance Alice's bots have the reputation of Alice as a verified bot and if the bot does anything bad it damages Alice's reputation. Unverified bots or bots with no reputation would be the ones to be concerned about.
Pseudo anonymity would mean I do not need to know who is behind the bot. I just need to know via cryptography that someone with a good reputation in the community is behind it. When everyone has their bots then the good and useful bots gain the advantage. When it remains a mysterious technical thing, well then I still may have access to these bots due to my technical abilities but the problem is the people who will need them the most will have to in essence pay the bots run by technical people (just like bots on Steem now).
I see no stopping markets from forming, and bots from proliferating. I think the benefits far outweigh the risks if the wisdom aspects I mention can be implemented. It might be true that bad actors can get enhanced wisdom from it but it means you and I also will get these enhancements. I don't think the bad actors will have as much of an advantage until of course they do what they always do and find new attacks. Unavoidable.
It is technically impossible to create a new technology which can vastly improve the condition of the world without any risk of it being turned against us. The Internet, social media, even the printing press, can be turned into a weapon. In fact, all of them have been weaponized in some fashion, but I'd rather have it too so I can have a level playing field.
To clarify what I mean, if we look at for example Cyc and LucidAI then you know what I mean by shared knowledge base:
When I refer to AI, I am referring to common sense computing and symbolic AI (not neural nets or AGI).
When you have a larger and larger knowledge base then building new systems becomes easier. So the best thing you would be able to do with this sort of platform is sell your knowledge to it in exchange for tokens. This is the knowledge economy which builds the shared knowledge base similar to Wikipedia but with payments for whomever contributes to it. Once there is a very large knowledge base then the questions can be asked and we can use the platform itself to help answer.
So I agree with you there is a threat of autonomous weaponized attacks but I do not think in particular the symbolic AI is necessarily good or bad. If we were to for example in the very birth of such a platform make it so that ethics and cybersecurity knowledge is populating the knowledge base first then we can focus on building out the prosocial benefits. Perhaps we can build state of the art cybersecurity solutions.
Acceptable level of risk is yet to be determined. I think this is where public sentiment would come in. I think if there is wisdom then when we query the network there may in fact be ways to reduce risk such as resource constraints (whoever supplies the resources can cut off the flow of resources). Any sort of AI even if it's bots, will require computation resources. So if it's truly evil then we could simply agree to shut off any runaway AI which consumes resources as a mitigation. This of course would have to be something everyone agrees to do which is again a question of public sentiment but I do not see technical problems just political.
Finally, some of the very questions you ask are being asked at the LucidAI Ethics panel. They are building what we are talking about now but it's in the closed centralized manner:
I think the issue goes beyond just trust. The problems I refer to just in case you don't want to read each blog post are:
All of these problems in my blog I show trace back to neuroscience. The biological limits of the brain mean that wisdom (and morality) doesn't scale. So having greater transparency without wisdom and without capacity to be moral is likely a disaster. I also mention that the idea that we should create a sort of small town atmosphere also can't scale if the human brain is only capable of managing 150 relationships, so the metaphor of small town on something like Steem can't work because a community traditionally is 150 people not 1 million people or 10,000 people like we see online.
Fake sockpuppets, fake communications, etc, these are side effect problems that result from the biological limits. Because I don't have the ability to examine everyone to determine who I can trust then it's just a flood of information coming at me. It will only get worse until we develop an ability to filter and an exocortex (the concept I describe) can help with this.
Some of the technical requirements, for an exocortex in the most simplistic form Alice just needs to have bots. These bots are agents, and Alice would simply ask questions, delegate tasks to these bots. These bots form a network, and could seek answers, services, or solutions from other bots or from humans. These bots can be built on Steemit even now so that part is not technically sophisticated.
Alice also would require access to a shared knowledge base, as well as having her personal knowledge base. This way a knowledge economy can form. This part requires linked data, rdf, reasoner, etc. The Cyc project is the closest thing to it, but for the most part this is the most technically chllenging part. The ability to actually create knowledge, to structure the data in a way which symbolic style (logic based) AI can reason over it.
Alice requires privacy, the ability to control her data, her knowledge base, etc. This privacy component may be possible through homomorphic encryption but in my blog I just called it advanced cryptography. Theoretically it is possible but performance is still very slow until CPU power improves.
All in all I think it can be built but the funding currently isn't there. More focus is going into small picture projects with some exceptions. Enigma is trying to solve the privacy part by building the homomorphic encryption I speak of and they have pretty good funding.