You are viewing a single comment's thread from:
RE: Steem Sincerity - Update and Community Involvement
in the meantime... @cuddlekitten's AI classification as a spammer has risen in ranks (she's now classified as the top-8 spammer on steemit - which is ridiculous!)
and I think I am already feeling the repercussions as @abusereports has started targeting her for auto-flagging! Though, that might be purely coincidental of course.
P.S.: Thanks for mentioning my kitten here, it's a tough day and I appreciate the moral support I get from reading this a lot!
@cuddlekitten doesn't seem to be classified as a spammer now for whatever reason. The classifier hasn't yet changed though.
Yeah, her score sincerity score changed from 1.0 spammer to 0.82 human after she wrote a single post about the issue... at first I thought you had manually changed something, but then I realised it's probably due to the significantly different appearance of the account data (a new post, before there were only comments - a ton of flags, before there were zero).
I've also been able to get her off the auto-flagging by abusereports - I do not know if there was any relation between the two situations though.
I've just conducted an experiment with the other cuddle-bot @steemkitten achieving the same result by making a single "wordy" post under the account: [see here]
Cool, but unfortunately publicising such feats of reclassification makes it more likely I'll need to change the algorithm as spammers learn from you. It's a battle to stay ahead of their game you see.
I guess that's exactly what I am trying to do... convince you to improve that algorithm. I feel it's still very unreliable as easily illustrated with my little experiment... so apart from actual spammer having it easy to trick your algorithm... non-spammers also easily get labelled wrongly...
so yeah... improve that algorithm... heh
I will certainly try and do that. I guess I can't expect too much support from people who are sailing so close to the wind ;)
I am giving all the support that I can.
You must excuse if I do sound a bit salty, but the top-8 spammer classification has brought a lot of negative attention to an otherwise positively received project of mine.
But in all of this, I didn't take any offense and as a matter of fact i am encouraging users to try steem plus and help provide human feedback
If there's anything else I could do to help, I will.
I'll say it again. I explicitly support the steem sincerity project. I believe it is a great addition to the steem ecosystem.
It's just that I experienced a negative backlash from an (imho) false classification of my bots and as such I try to provide feedback and hope to illustrate the urgency of improving this asap because it is already being publicly used.
Hi @andybets
I was hoping you can help me. Possibly something in a setting I need to change. Thanks in advance for any help!
I posted the below in a different article
"I just watched a really cool video by @exyle where he talks about the SteemPlus Chrome plugin. After learning about the new features I downloaded it straightaway. To my disappointment I am marked as a Spammer rather than a human. I have never spammed anyone so I was wondering how this could be and is there anyway I can change this mark?
Thanks
Dave @exploringirish"
You've replied to my comment here, so @andybets might not neccessarily see this comment of yours unless he has some sort of notification service he checks.
For now there's little that can be done as far as I understand the system. The AI gives you the score based on a machine learning algorithm and "comparing" your account data to what it "knows". This can be very misleading and equally inaccurate but an AI's algorithm is automated to a high degree and it's rather impractical to modify it's method of classification to a specific individual account.
If your account gets added to the training data as a "human", the AI might learn to distinguish your "data pattern" from what it currently knows as a "spammy" pattern.
P.S.: i just checked your detailed sincerity-API data and you're already scored as 57% human, either your account data changed for your benefit or the machine has learned since then.
Apologies for the incorrect classification scores. I have added you account to out system as an example of a human content creator, so it learns from this and improves in future.
Unfortunately, the SteemPlus feedback is proving problematic for various reasons. For example many people see a foreign language post, and report it as spam, or see bots and report them as spam. Basically many people are just using the classification to apart their own values, rather than actually considering whether the content is bad for the platform. So one reason for the delay in improvement is because any classifier however well designed is only as good as the data it is trained upon.
I appreciate your honesty anyway, and am working hard on this difficult problem.
Yeah, that seems to be a pitfall for "human-trained-AIs" ever so often... and then they turn into trolls themselves :(
Maybe you could ask for some "targeted" training evaluation of specific accounts via discord or so?! I guess some sort of human vetting of the resulting training set (and resulting AI behavior) cannot be foregone... Then again... maybe the "flat" account data just doesn't contain enough pointers for a reliable AI decision?!?
If I may make a suggestion, for the top-500, wouldn't it make sense to preselect a hierarchy based on comment/post-count, possibly filtered by incoming vote-diversity (to push voting-farm-spam to the top) and THEN classifying the spammer-probablity with the AI?!
just an idea... right now the leading criteria for the top 500 is the AI probability followed by comment-count... and that's awkwardly not including those real spam-heroes
P.S.: if that's not the case yet, maybe adding some of the really severe cases like a-0-0 with 25.000 comments to the training data as a top match for 1.0 spammer can help?!