RE: Towards a reputation system suitable for SteemSTEM
Thanks a lot for this comment. I am answering first the last sentence: yes question and experimentation is good, which is why I shared this post. Any comment trying to help me designing something meaningful (or just thinking about what could be meaningful) is more than welcome! :)
Now the rest.
I am not sure what your final usage of this.
Me neither. For the moment, let's say we are about to release a SteemSTEM app and it could be cool to add a reputation metric, just for the sake of it. It is funny and that's it (and hopefully it will incentive community members to engage more). No other usage than this, at least for now.
A few days ago heimindanger posted about the Dlive case and a comment he made against them. Anyone who upvoted that comment were then blacklisted for votes at the suggestion of one of the Dlive team (redjepi). This seems to be a risk of sorts here too where the personal relationships between poster and curator will effect decisions heavily in both the positive and negative, regardless of content quality. Does anyone who questions STEM or give any negative view then suffer at the hands of vindictive curators?
We have a team of 10-15 curators obeying to curation rules that are public and that have been designed with the goals of the community in mind. Having someone blacklisted only means this person has plagiarized or had an abusive behavior. Second chances are also possible so that a blacklsit state is not necessarily permanent. In addition, the blacklist is monitored independently by our crew of mentors that are not involved in curation.
Therefore, I can imagine that we are protected against any personal vendetta (this would also mean that one person should break the ball of all the curators at the same time).
Even if a post itself is brilliant and attracts high engagement from the STEM community, it isn't factored in unless it gets a vote and, that size of vote is dictated by the curation team itself.
This is true. We need to find the post (we are human and mistakes/misses are of course possible) and vote it according to our standards. The latter are however very well defined (we have only four vote levels). Moreover, double curation is required so that at least two curators must agree on the voting strength.
But yes, there is a bias, as behind any reputation metric. And we have chosen a bias that sounds reasonable to us.
I think the curators/management should be included in the calculations for transparency purposes also as they factor into the voting and quality of STEM posts.
Since by default, the team got slightly stronger upvotes as a reward for their work, this will bias the ranking. For this reason, I have excluded them when I ran the code. However, anyone can download the code and run it with a different list of removals. Somehow, we don't really hide anything but just try to make things cleaner.
The top 30 most reputed SteemSTEM authors of all time, out of 2662 authors
15 deathbatter 4.466
This person who I found interesting got support from their first post by the looks and all stem posts but hasn't posted at all in 2 months. They have only been registered for 4. Shouldn't reputation also come with some length of track record? Perhaps the degradation takes this into account but they are still number 15 of all time and they managed that in the first 2 of their 4 months with the last 2 being absent.
I agree. I however have no idea on how to improve that for now. Please see the answer to @alexander.alexis comment where I try to brainstorm to cook up something. Including the length of the track record is definitely a good idea, but I have no idea on how to do that in practice for now. As soon as the idea is defined, programming will be a piece of cake :D
Thanks, that answers a lot of my questions.
Keep an eye on circles forming also because it can happen somewhat unconsciously through organic relationships forming. If this happens too much then it gets increasingly difficult to make space for new authors coming in perhaps.
Bias in a reputation system is inherent in reputation so that isn't my concern and if it works for your purposes, that is what matters.
What about a reviver function? So as a point degrades, it can be boosted up again by a vote on a later post.
For example.
1 point degrades to 0.5 over 1 month and a vote will bring it back 50% so 0.75.
1 point degrades to 0.2 over 2 months and a vote will bring it back 50% so 0.3
and of course 0 points are dead. It could factor in strength of vote as to how much it will revive and of course time. Any point that degrades and gets revived can't get revived past its starting 1 value. so if it has only degraded to 0.8 over a week, a new vote will return it to 1.
EDIT: I should add that there should be some legacy value to points too for that track record so after some point, they are in the bank so to speak.
A delayed follow up... ;)
Yes we try, which is why we have the curators, sub-community curators and mentors that all act independently and somehow self controlling each other. The mentors being directly responsible for guiding the new authors (who however ask for it), this gives always an open door to anyone.
Moreover, we have a complaints/questions room on the discord so that anyone who would feel rejected for abusive reasons can say a word. Again, this room is managed by another independent group.
Even with all of this, it is true that 100% safety does not exist. We do our best, but are only humans at the end of the day.
Finally, thanks for the tip concerning the reviver function. This is a good idea. This however requires to rewrite entirely the script... which means, I won't do it in half a minute but this (or something along these lines) will be done more likely in the next version :)
I have been staring at the screen waiting....
The reason I brought it up is that STEM is a pretty strong community and it is only natural that group dynamics circle the wagons over time and become somewhat tribal.
I will be staring at the screen waiting.... :P
Please don't stare your screen too much. I am really busy at the moment and the update may be in a couple of weeks.
In any case, thanks for your concerns. They are very valid and even if today this is not much of a problem because of the current size of the community, this may become relevant with the expected expansion of SteemSTEM. We however have time to think about how to put in place a system that will allow us to avoid this (at least as much as possible). I keep your name on the list of people who could help and discuss when we will design something new. ^^