RE: Will the Rise of Digital Media Forgery be the End of Trust?
It's being used against us right now. That is one of the topics I'm concerned about. Actually put another way, our cognitive limitations are exploited by social engineers, hackers, scammers, and bad actors.
So, how do we identify the optimal risks and put compensating controls in place to mitigate as we continue down this path?
If the risk emerges from "wise" hackers exploiting cognitive limitations then the only mitigation I've been able to find is to figure out how to reduce that attack surface. For example social engineers, or psy ops using memes, they know very well that the human brain relies on stereotypes as a short cut to reduce cognitive load. They may not know the neuroscience behind this but they intuitively exploit it and if it's a foreign government behind the information operations then for sure they know what they are doing.
The problem is there is no viable immune system for this. I'm basically required to be ignorant and susceiptible to disinformation, to make decisions with cognitive biases, to make decisions with attention scarcity, and businesses thrive on this. I will not name the businesses but I'm sure you can think of some businesses which use marketing in such a way that profit can be extracted by exploiting these cognitive limitations.
In my opinion to do a risk assessment together, globally, collaboratively, requires building the platform in the first place. Else we will have to rely on small groups of experts, many of whom may be compromised, to make top down solutions which we may not have the chance to give our input or discuss. I would suggest we use decentralized technologies to specifically produce wisdom building tools so that we can do collaborative risk assessing, collaboratively design the very platform itself using the platform we are designing. This in the engineering sense would be a self optimizing design, and even Steem to some degree has some of these capabilities although Steem lacks the capacity for wisdom building, as it does not include a shared knowledge base, or have the symbolic reasoning AI, which in my opinion will be necessary to boost decision making. Steem's interative design improvement capacity relies on high technical expertise which results in ultimately less collaborative centralized decision making.
So I'm definitely not someone in favor of pushing forward without a plan, on the engineering level we require a specification, whether it be formal or informal. I do not think we can plan for everything though which is why the design has to be iterative, adaptable, capable of being upgraded over time as we learn more, become more wise, etc. So a wisdom engine or whatever we could call it, as it produces results we could use those results to in theory improve the wisdom engine continuously.
In my references I include two examples of designs which are self optimizing logic. Theoretically speaking I agree with the approach of using the platform itself to continuously improve on the design of the platform provided that such a platform can be built. The idea being that upon initial construction we don't really know any answers to the tough questions such as the ones you ask, and so we must use the platform to ask the tough questions in the appropriate way so as to receive the answers. Changes can go into effect if agreement by the agreed upon mechanism can be reached. Most important of all, public sentiment is critical in my opinion to decisions which effect stakeholders, so in terms of putting controls in place we would again need this to be a data driven process where we have some idea what the values of people are, and factor this into the safety decisions somehow. Thanks for asking tough questions because that is exactly what is needed to build something great and keep it beneficial.
One more thing to consider. It may be the 'wise' attackers who develop inroads to exploitation, but it then will be disseminated to less technical, basically anyone, to leverage. That's when the problem scales unimaginably.
This is one of the problems with AI and cybersecurity that I work on. Adoption of AI could be a turning point for efficiency and effectiveness across just about all facets of mankind, but if those same tools we embrace are leveraged for malice, then we have created a tool for our demise. How do we proceed where we gain from the benefits and yet still mitigate the risks to an acceptable level.
Well it already is the case that anyone can set up bots so that's not hard. The offense always seems to have the advantage there. The benefit of AI is that it allows for automated detection. I think it's not possible to create a tool which cannot be abused by an attacker. The point is right now only the attackers have the bots which means we get all the negative use cases without any benefits to the every day user.
I suppose you could make a case that if the every day user had access then it requires slightly less technical sophistication but even if we look at Steem and make the same argument? If we release the bots to everyone does it really make the situation worse? I don't think it would. I think it would allow everyone to contribute to pro-social bots.
An argument can always be made from the top down that keeping people poor and ignorant reduces unknown security risks. This as a defensive strategy I think is a bad one because it assumes there can be perfect defense against unknown attackers in an information context. I prefer the strategy of building for resilience with the expectation that there is no way to predict or defend against all kinds of attacks (it's cat and mouse). But we can build the sort of network where recovery from any attack is quicker.
If we have a network, with a shared knowledge base, and bots, which is building wisdom for us all, compounding on itself, then this would also include the area of cybersecurity. This also would include answer the questions on how to recover from attack or how to mitigate the risks which inevitably come from abuse of information. The reputation system for example if it is to be designed will have to be built from the current state of the art in terms of knowledge.
Just one example of a pro-social use case. If we all have bots then we can simply tell our bots to filter information and automate shopping. This would exclude a lot of the attacks (not all). The reputation economy would mean every bot which is anti-social or evil or harmful gets down voted, a bad score, etc.
Sybil attacks and botnets will still exist but the idea is that the most effective and popular bots will be the bots which people appreciate (in theory). Also in theory you can set up the bots so that for instance Alice's bots have the reputation of Alice as a verified bot and if the bot does anything bad it damages Alice's reputation. Unverified bots or bots with no reputation would be the ones to be concerned about.
Pseudo anonymity would mean I do not need to know who is behind the bot. I just need to know via cryptography that someone with a good reputation in the community is behind it. When everyone has their bots then the good and useful bots gain the advantage. When it remains a mysterious technical thing, well then I still may have access to these bots due to my technical abilities but the problem is the people who will need them the most will have to in essence pay the bots run by technical people (just like bots on Steem now).
I see no stopping markets from forming, and bots from proliferating. I think the benefits far outweigh the risks if the wisdom aspects I mention can be implemented. It might be true that bad actors can get enhanced wisdom from it but it means you and I also will get these enhancements. I don't think the bad actors will have as much of an advantage until of course they do what they always do and find new attacks. Unavoidable.
It is technically impossible to create a new technology which can vastly improve the condition of the world without any risk of it being turned against us. The Internet, social media, even the printing press, can be turned into a weapon. In fact, all of them have been weaponized in some fashion, but I'd rather have it too so I can have a level playing field.
To clarify what I mean, if we look at for example Cyc and LucidAI then you know what I mean by shared knowledge base:
When I refer to AI, I am referring to common sense computing and symbolic AI (not neural nets or AGI).
When you have a larger and larger knowledge base then building new systems becomes easier. So the best thing you would be able to do with this sort of platform is sell your knowledge to it in exchange for tokens. This is the knowledge economy which builds the shared knowledge base similar to Wikipedia but with payments for whomever contributes to it. Once there is a very large knowledge base then the questions can be asked and we can use the platform itself to help answer.
So I agree with you there is a threat of autonomous weaponized attacks but I do not think in particular the symbolic AI is necessarily good or bad. If we were to for example in the very birth of such a platform make it so that ethics and cybersecurity knowledge is populating the knowledge base first then we can focus on building out the prosocial benefits. Perhaps we can build state of the art cybersecurity solutions.
Acceptable level of risk is yet to be determined. I think this is where public sentiment would come in. I think if there is wisdom then when we query the network there may in fact be ways to reduce risk such as resource constraints (whoever supplies the resources can cut off the flow of resources). Any sort of AI even if it's bots, will require computation resources. So if it's truly evil then we could simply agree to shut off any runaway AI which consumes resources as a mitigation. This of course would have to be something everyone agrees to do which is again a question of public sentiment but I do not see technical problems just political.
Finally, some of the very questions you ask are being asked at the LucidAI Ethics panel. They are building what we are talking about now but it's in the closed centralized manner:
Thanks for the video! I am also on an AI and Ethics board, looking for more insights across the academic and professional community.
The reason I take some of the stances I take on these issues is because I think the old model of discussing ethics is elitist and too many people are left out of the conversation. For example, I'm not on the ethics board. Is the ethics board open for anyone to join or do they select only certain people from a whitelist?
For this reason I prefer a more collaborative approach to developing AI, along with a completely open means of discussing the ethical concerns. On ethics there is always difference of opinion but a main area is do we want to maintain social stability above all else or do we value something else more? Disruption is unavoidable with AI, but often I see the concerns focused on the impact AI could have on jobs, or on common lifestyles people are accustomed to, but not a lot of vision or thought into the opportunities. There are always opportunities and risks to consider.