CrowdOracles - using the power of the crowd to solve the software engineering problems
What is an oracle?
An oracle is a mechanism design which facilitates a bridge between the outside world and the blockchain (smart contract). A smart contract cannot access anything outside of itself without an oracle to act as a bridge. The same could be said about Bitshares where a data feed requires an oracle to capture it and put it up.
One of the current premier examples of an oracle on Ethereum would be Oracalize. The paper shows that you can use an oracle to automate the bug testing process while leveraging the crowd.
In this paper we investigate whether it is possible to exploit CrowdSourcing to solve the oracle problem: We produce tasks asking users to evaluate CrowdOracles - assertions that reflect the current behavior of the program. If the crowd determines that an assertion does not match the behavior described in the code documentation, then a bug has been found.
The oracle problem in software engineering
Software testing is one of the reasons software engineering is so expensive. It's not enough to just generate code, but the testing process must take place. There is no known way to fully automate this testing process and one of the reasons this is the case is because of the oracle problem. CrowdOracles is an attempt to resolve this problem by allowing fully automated software testing (in theory). In practice it is not so simple to represent and test the correctness of assertions about the behavior of software, and in general while CrowdOracles does offer a way forward, it does not solve the problem completely.
How can we use the crowd to facilitate solving software engineering problems?
The main findings of our study are that the crowd can
perform well at verifying CrowdOracles, but the perfor-
mance strongly depends on the qualification of the crowd
and the clarity and complexity of the underlying tests and
documentation. Interestingly, our experiments also showed
that once the crowd has identified a wrong assertion, it can
also fix it.
It has been found in the CrowdOracles paper that software engineering tasks can be crowdsourced. Not only is that possible but it is also possible to crowdsource software formal verification and this is critical to understand. In Tauchain for example the current plan will be a similar sort of crowdsourced verification process but slightly more ambitious. In the academic examples it has been shown that you can use the crowd to do formal verification, as well as to handle other software engineering tasks, and this is something to definitely explore going forward for decentralization of development.
Conclusion
Crowdmining is in my opinion going to be a very useful method for using mechanism design to solve security problems. Formal verification, software engineering, and many other tough problems can perhaps be reduced to a series of tasks done by the crowd. In fact, I would expect that this is the direction we will go because it will reduce costs of development and also speed up development. Oracles are tremendously useful as well if we are going to make AI truly useful in a blockchain setting.
References
Amsterdamer, Y., Grossman, Y., Milo, T., & Senellart, P. (2013, June). Crowd mining. In Proceedings of the 2013 ACM SIGMOD International Conference on Management of Data (pp. 241-252). ACM.
Pastore, F., Mariani, L., & Fraser, G. (2013, March). Crowdoracles: Can the crowd solve the oracle problem?. In Software Testing, Verification and Validation (ICST), 2013 IEEE Sixth International Conference on (pp. 342-351). IEEE.
I think this is a wonderful post! Thanks for sharing! :) Upvoted & followed!! :)