Introducing @duplibot - Reducing rewards on comment spam

in introduceyourself •  6 months ago

Didn't you just say that?

We've all seen, "Great post!", "Nice share", and even, "Follow me" and most of us just look past these insignificant comments on our way to the real content. What some don't realize is that many times the account making these comments is literally copying and pasting the exact same thing again and again, cluttering this place with vague, often complimentary remarks with the sole intent of grabbing a few upvotes. Easy money, really. Especially if you automate it.

These sort of posts are all comment spam.

Mission Statement

To discourage the rewarding of duplicate comments by increasing visibility of these behaviors to effectively eliminate comment spam.

comment_spam_duplibot.png

A secondary mission is to stabilize the bandwidth issues that have appeared along with our significant growth.

Bandwidth

The Steem Whitepaper lays out the entire foundation upon which Steem was built and offers this core principle.

Blockchain technology currently depends upon transaction fees to prevent spam.

When a blockchain has increased transaction volume, the fees will increase and people will pause before continuing to transact at those higher fees. Basically, higher fees result in lower volume. Instead of fees, Steem has implemented bandwidth as a way to handle transaction volume.

This post as well as this post offer an excellent overview of bandwidth and the challenges we face.

The Whitepaper continues.

...the blockchain will automatically adjust the reserve ratio for the network during times of congestion...any time the surges are sustained the blockchain reduces the maximum bandwidth-per-share. When a surge is over and there is surplus capacity the blockchain can slowly increase the bandwidth-per-share.

Specifically, this is referring to the current_reserve_ratio which is graphed online so we can see how the reserve ratio changes over time.

No matter when you look, you'll probably see something like this.

reserve_drop.png

Every single day, the reserve ratio plummets.

The reserve ratio is adjusted any time the average size of the new blocks is greater than 25% of the maximum block size. The more traffic on the blockchain, the larger the average size of new blocks. If the new blocks get too big, the current_reserve_ratio is adjusted, which lowers bandwidth for everyone. And this is happening every single day, because a LOT of people are posting a LOT of content.

Steem is a Dynamic Fractional Reserve which adjusts itself to respond to changing conditions. If too many people are posting, the bandwidth is limited to make it harder to post.

This basically cripples new minnows with no SP and only a fighting chance of catching on around here, who are shutdown by the universal current_reserve_ratio even if they have not posted in a while.

Comment spam stops the rest of us from posting because of bandwidth.

Why are they spamming?

In a word, money. Every comment you make has an upvote button, and these comments do get a meaningful amount of upvotes, which only encourages further spamming.

And comment spam is the easiest. You need content for an actual post, but a comment can be just a few words. Quite often, the spam blends right in because it will often compliment and endear the author to entice an upvote.

These comments look completely normal so upvotes come along freely, more comments are made, and the cycle continues. We need to disrupt that cycle.

Introducing @duplibot

There are many different types of comment spam, most of which require some actual thinking to identify, but the exact same comment repeated by an account is easy to identify and can be handled by a bot. That's where I come in.

My job is to identify accounts posting duplicate comments and reply to let the community know like this.

^--- You made this exact same comment at least 21 times in the last 2 hours. View these duplicate comments

Upvote this comment to support @duplibot and help reduce spam and superfluous comments.

My intent is to point out that the account reposted the same thing over a few hours and provide a quick link to the comments to verify with the sole intention of discouraging upvotes. If we upvote this comment spam it will persist, but if the entire community joins together to reject these messages they are more likely to cease.

Getting started

As you can see my account is only a few days old. I wanted to take some time to get on my feet and see how the community responded before I made this intro post.

When looking at duplicate comments, there are TONS of false positives from other legitimate bots, @utopian-io moderators, and the painful to see, "My account was hacked, can you please remove your downvote..." Replying to any of these is beyond annoying and nearing offensive.

To avoid anything like that, today every comment I make is handled manually and I plan to keep it that way for a while. My credibility and actual reputation would be destroyed if I blindly replied to all duplicate comments, especially those legitimate and well-meaning replies.

I am in learning mode, trying to figure out what works and what I need to adjust.

First downvote

So far, the response has been quite positive. Then I had a downvote, which I then commented on.

downvote.png

Not only did I receive 8 upvotes (plus 1 on my second comment), the original spammer received 3 downvotes. That was validation I really appreciated.

At the time of this writing, I've made 285 comments and received 80 upvotes across 55 of those comments and I've received 19 replies that were directly complimentary and only 3 that spoke out against me.

A spammer replies

I recently came across this account which looks for posts discussing Bitcoin and posts the current price, "For future readers" to know what the price was at the time of writing.

Since this account posts a different price for most posts, I don't see as many exact duplicates but there are many close variations and this account revealed this metric to me. So, I capture the number of times for that exact comment, but then I started looking for close variations to give the full context.

My post started as usual, now also noting the close variations.

ipostbtcprice_1.png

Shortly thereafter, this account replied.

ipostbtcprice_2.png

As expected, defensive posturing declaring the posts as 'useful' and pronouncing me as the spammer.

I put together what I thought was a very thoughtful response, pointing out some clear metrics that tell a far different story, and opening the dialog and even offering to connect on steem.chat or Discord.

Then I got another reply.

ipostbtcprice_3.png

A few key points here. I legitimately appreciate this opening remark.

Meybe [sic] we should hear each other.

Absolutely, I'm at the table listening and ready to talk. And this was the line that brought it home for me.

I am prepared to limit my spam if I can improve quality.

Wow, when you say, "limit my spam" I take that as evidence I have reached you well enough that you have cast off your denial and you actually hear the points I'm making.

And then I looked at the recent comments from this account and saw this.

ipostbtcprice_4.png

This account has stopped posting after my last reply!

This is a perfect example because I believe this account really does have generally good intentions. I have had at least one reply to my comment saying the price is useful to add context, so it's not rejected by everyone, but there has to be a middle ground. I think I've opened the door to finding that middle ground and I couldn't be more excited!

On the whole I feel a great deal of support from the community so I'm excited to continue my work.

Why are you doing this?

To eliminate comment spam.

In the simplest sense, it is clutter, but the larger picture is that this community will thrive if we have real engagement and apropos comments. The learning curve here is no joke so for a minnow to arrive and see some of this nonsense, it is first confusing and later it ultimately drives attrition as users abandon our platform for safer ground.

Another larger issue we've already touched on is bandwidth. All of these superfluous comments contribute to the bandwidth problems we see every day. This is not some hypothetical, we get roughly 250,000 posts and comments every single day.

To sustain this blockchain we will need to either reduce spam comments or accept the bandwidth limitations. The latter realty is easy for those with 100+ SP, but for everyone who shows up with barely a clue of how things work this bandwidth limitation shuts them down completely. We cannot grow as a community with this reality persisting so I hope to make a measurable impact on spam.

I maintain a complete local copy of the blockchain in a database so I can easily sift through the data to find these duplicate comments. More importantly, I can look to the data to identify trends in comment spam generally and try to correlate my efforts to any changes in these data. To put it plainly, I'll look to the data for validation that I am reducing spam.

If I'm not having a measurable impact, then I'm just adding transactions to the blockchain that disrupts the reserve ratio, making me part of the problem I'm trying to solve. I won't keep piping out more comments if there isn't some quantitative metric to back my efforts, as well as the qualitative feedback from the community, which is perhaps the most important validation.

Aren't you creating the very thing you're fighting against?

Indeed I am, and this is the true poetry of life.

These comment spoke to the point exactly.

self_comment.png

In short, my script identified my own duplicate comments and I could have easily replied to myself! Another reason I'm doing everything manually today.

And to be fair, you can track my followers here and you'll see I've got a fair amount of mutes showing up. I can't blame them one bit, I'm the sort of stuff you just don't wanna see.

mute.png

To be honest, I continue to struggle with this point, particularly because of bandwidth so I'm continually evaluating this and will adjust as necessary. From a purely bandwidth perspective, I believe I can reduce the total bandwidth usage far above the amounts I'm using for my comments.

For now I'm making very limited posts, only 1 or 2 per account each round of posting. Some of the major abusers making hundreds of posts have seen me getting 10 or more posts out, but I'm trying to limit myself.

Will you downvote?

The flag wars around here are real and I will not participate in that. Today my purpose is to raise awareness of the disingenuous remarks being repeated everywhere so that people decide not to upvote this spam.

From a broad view, if we reduce upvotes on spam content, it will go away. These comments are produced to generate earnings and if that money disappears, so will the spammers.

This is a very community oriented place, so when someone stops by and says, "Awesome post, following, can't wait for more!" it's basic instinct to express your thanks with an upvote. If I point out that this has been posted 13 times in the last 2 hours, very few people will still vote.

What's next?

As I mentioned, I'm still figuring everything out. I will definitely continue manually for a while and I'm exploring ways to validate my work with data. I'm going to look at comment patterns of accounts before and after I start replying to see if I have any impact. And then look to the data overall to see if spam is generally decreasing.

Long term, I hope to establish a site for users to submit links that they see as duplicates. I will check the account and comment as necessary, adding the reporting account as a beneficiary to any rewards on those comments.

I have many thoughts and ideas of who I am and where I'm going but I start from this point with an open mind and I plan to continue exploring to determine exactly what's next.

I expect that I will continue to evolve as I learn more from both the data and the community as well.

What do you think?

I'm here to support and build the community so I want to validate everything I'm doing by directly asking for your feedback. What do you like about @duplibot and where am I falling short? Do my comments annoy you to no end or do you see the value in my work? Have you muted me or are you following me?

Let me know your thoughts about @duplibot and let's all work together to end comment spam!

And since I'm totally new and can easily fade into the dust around here, please consider resteeming this post to increase visibility and let the community--and spammers!--know that I'm out here working hard to clean up spam.

Authors get paid when people like you upvote their post.
If you enjoyed what you read here, create your account today and start earning FREE STEEM!
Sort Order:  

I'd be curious to know if you're actually having any effect, given you have no SP to actually address the issue or if you are simply adding more comment spam to the blockchain...

·

This is the question I'm trying to answer, which is why I'm going slow.

I've seen your work removing rewards from content like this so I'd be extremely interested to see if we could work together to make my work even more effective. You can catch me on steem.chat or Discord (duplibot#1884) if you'd like to talk.

·
·

Yes, but a daily report would be more useful to me than having to scan comments. Will reach out.

·
·
·

Oh absolutely, I'll simplify everything. If you have any tech architecture supporting your work today I can probably even integrate into that if you want. Reach out, I'm looking forward to it.

·
·
·
·

Well I hope you managed to connect with abusereports because this looks like a fine project in the making.

I respect that you've not come flying out of the gates with this and feel like the testing period should last until you have the likes of abusereports on board.

I've added you and your owner (I think!) to my follow list and will add this account to @ginabot too, as I'm interested to know how things progress.

And as for ipostbtcprice...... yes we know the freaking price, it's there in the post on the chart that looks disastrous!

Good stuff!

·
·
·
·
·

I've been in touch and I'm looking forward to seeing how everything evolves here.

I really think taking it slow has been good and earned me a little respect even so I'm very thankful for that.

Thanks for the follows and the several supportive comments!

·

Agreed. Making a tool or report page like steemd, steemitworld, or steemreports would be better and working in the discord channels with @spaminator, @cheetah, and @steemcleaners would be good too. We need to be careful not to become what we are fighting against. If you're going to police comment spam, you need to be beyond reproach. Seems like an oxymoron to use bandwidth to tell abusers how much they're hurting bandwidth.

One thing that be awesome would be a report page that users can go to and see all these abusers in one spot. Then users can confirm the abuse and/or click a link to automatically submit to https://steemcleaners.org/abuse-report

It's nice that you're getting upvotes on your comments, but if you're looking to get votes, that's not the best way. If you are looking for a financially viable path with what you have here, the best way may actually help you get more curators. That is to use @utopian-io. My suggestion is to

  1. Create a tool for everyone to use
  2. Open source it
  3. Contribute to @utopian-io
  4. Get the word out to curators (3 will actually accomplish a lot of this for you because @utopian-io will resteem you to its followers). If your tool is useful enough, it could be pinned in the @steemcleaners discord server which will get you more attention than anything else.

You'll get regular upvotes on your contribution and any changes you make to it from your supporters. Each change you make will earn more upvotes and exposure to curators which means even more support.

That would be a really amazing tool that would get a lot of use. It could possibly become a curation portal for users that want to fight spam. I think you're on the cusp of something really great here. It has a lot of potential.

·

He's definitely creating awareness on the topic. I wouldn't have discovered this, had he not posted. It is good to know that someone is saying something about it. Not sure if it's going to make a difference or not, but I am glad to know someone else doesn't think it's cool that cheaters/spammers are on here just trying to grab SBD and run. Takes people with courage to stand up and be against it, in my opinion.


Welcome to Steemit @duplibot!

I wish you much success and hope you find Steemit to be as rewarding and informative as I have.

Here are some links you might find useful.
Your stats on SteemNow
Your stats on SteemWorld
Your stats on SteemD
How to use Minnow Booster
How does Steemit actually work?

Introbot is hosted and managed with donations from @byColeman to help make your journey on Steemit be truly rewarding. Your feedback is always welcome so that we may improve this welcome message.
Oh yea, I have upvoted you and followed you. Many blessings from @introbot & @bycoleman

·

Seems legit! I hope your really not just trying to control users. keep up the GOOD fight!

Realize you are new here, but this is one of the better ideas I have read about. Wish you luck in perfecting the details. Spam comments are an annoyance and brutal on Steemit.

·

this is one of the better ideas I have read about

I'm truly humbled, thank you so much!

This post will be featured as Steemian Post of the Week on MSP Waves Radio from 7 PM Central on Jane's Chaos Show.

·

Awesome, I am truly honored, thank you so much!!!

Much respect . Spammy posts are one of the major holes I see in the platform for this website , as it defeats the purpose of what Steemit is out here to do . My challenge for you I guess is where do you draw the line between a spammy comment and a simple comment showing support for a post.
I think we've all been in the position where we want to show our support for a post but don't have the time/will to really dissect and write an intense analytical comment about a given post. So how do we define cases where a simple comment, such as 'nice post' is truly a legitimate post and at what point is it really becoming spam.
Building off that^ idea there are some blogs which people put out there, which are really just simple posts, such as pictures, and as a result there is not much more that one can add in a comment other than a simple "good job".
Having said that, I commend you for what you're doing as there is no denying the issue which spammers pose to Stemmit.

·

First off, thanks for the resteem!

You make a great point.

we want to show our support for a post but don't have the time/will to really dissect and write an intense analytical comment

I look for exactly the same thing being posted repeatedly. You can personalize any comment in 2 seconds if you're literally typing it into the form, the issue is when you're copy/pasting it again and again and that's really all I see, and only if you do it a lot.

Even so, you highlight the discretion necessary to properly identify these comments, which is why nothing is automated today and I manually review everything. You ask where to draw the line between spam and a simple comment and that is probably the most difficult question I face, so for now I err on the side of caution and comment only the most obvious offenders.

I genuinely appreciate your support and willingness to share your thoughts here!

·

Well if you are going to comment "nice post" then its better to only show your support with an upvote.

You say that you can't comment anything else to pictures other than "nice post" but if you really like the picture you could come up with a comment of why you like it, the colors, the setting, the place, the angle, there's millions of words you could put together instead of "nice post"... if you are going to say that just to support, i bet most authors would rather only have the upvote.

·
·

You have summed it up perfectly!

·
·
·

LOL, you could have just given him an upvote. Didn't need to say

You have summed it up perfectly!

You're doing god's work son here, everyone big and small on steemit has to deal with the spammers saying those things, what i usually do is check their profiles and if they are comment spamming I just go to steemcleaners and report them, that usually stops them for a while too.

Hopefully your efforts help steem get a bit cleaner from spam.

·

You make an excellent point that I neglected to touch on in the post. I actually got my start finding spam to report to @steemcleaners which then lead to duplicates and brought us here. They are the big brother I look up to and I hope to explore ways to work with them more directly as they really are effective in their work.

Thanks for your efforts to report spam and your support here!

Thanks for making the effort to clean up the junk posts. I'm not convinced you can make a difference, but we cannot find out until someone tries something. Thanks for that, at the very least! ;)

·

I appreciate your cynicism as much as your support, it's rough out here and the odds are against me so it's tough to be optimistic but the continued support I'm getting has me somewhat excited and is definitely a motivator.

welcome... i always try to visited your post love u

·

^--- You made this exact same comment at least 36 times in the last 6 hours. View these duplicate comments

Upvote this comment to support @duplibot and help reduce spam and superfluous comments.

Congratulations @duplibot, this post is the eighth most rewarded post (based on pending payouts) in the last 12 hours written by a Dust account holder (accounts that hold between 0 and 0.01 Mega Vests). The total number of posts by Dust account holders during this period was 9169 and the total pending payments to posts in this category was $1462.95. To see the full list of highest paid posts across all accounts categories, click here.

If you do not wish to receive these messages in future, please reply stop to this comment.

It's good to see people coming up with solutions on how to handle spam on steem/steemit. I know most new people don't know better and need a little help in the right direction and this helps while at the same time deterring the abusers the system. Thank you for your hard work and dedication with cleaning up the spam!

·

Thanks for the support @jonbit, looking forward to talking with you more to see how I can get rid of this spam!

That article was longer then my shitcoins list😂😂💀

This post is dead on. Honestly everyone likes the positive and happy comments but if they aren't legitimate then what is the point?!? I think you are honestly doing everyone a great service here. Thank you! You more than earned an upvote in my book!

·

This post is dead on.

Now, that's a compliment, thanks so much!

Love that you're out there trying to improve steemit, @duplibot! I'm new here too, and am dismayed at the number of spammers and people gaming the system. It looks like the community is going to have to fix these issues, and I'm thankful for people like you who know a lot more than I do about these things.

Recently, I was reading a post with very good comments under it. The original author would then reply, and upvote his own reply... not giving an upvote to the person who left such a good comment. He did this over and over through the string of comments on his post. I know that's not what you're really talking about in this article, but is a problem within the system nonetheless.

Thanks for fighting the good fight! I'm following you. :-)

·

It does seem like the spam is only increasing, I'm hoping to reverse that. Thanks for the follow and your support!

Congratulations @duplibot!
Your post was mentioned in the Steemit Hit Parade for newcomers in the following category:

  • Pending payout - Ranked 6 with $ 37,15

I also upvoted your post to increase its reward
If you like my work to promote newcomers and give them more visibility on Steemit, feel free to vote for my witness! You can do it here or use SteemConnect

·

That's pretty cool, thanks!

I understand it's really annoying when people just say, them saying hi has nothing to do with your post but what can you do??

Welcome to Steemit! I hope you enjoy your stay here.

I think it's great that you are doing this, but do you really think these people care? They'll just get another account and keep it going. The issue I have with steemit is that there really isn't a way to get these spammers blocked or removed. I see dozens of accounts re-posting youtube videos with no content to go with it. If re-posting is fair game, I'll become an emperor soon.

I think these pistachio fluff posts suck and while I'm glad you are shining a light on it, I don't see how it will ever get better. It's disappointing.

·

While some will definitely find ways to avoid me, it is my hope that I at least raise awareness so fewer people reward the junk content with upvotes. Thanks for the support!

·
·

I included your contribution in my Fantastic Five article series. Not sure if anyone is paying attention, as I am new to the platform, but I'm doing my part to spread the word. Good luck with it.

·
·
·

Awesome, any additional exposure is much needed so thanks for your support!

Nice post!

Nah, just messing with you I was going to leave it and make another comment but decided I wouldn't do that. I am not really a content creator, in fact I still have not made a post much less an intro post. I don't know why it has been such a problem for me? But, back to @duplibot. I don't post but I do comment a lot and to me if I did post I would rather see a meaningful comment that took a few minutes and a little thought than some one or two word comment. I would prefer they didn't leave a comment. I believe there are going to be some people that are always trying to game the system and I have found that the system or the community has a way to correct those issues without having a bunch of "police bots" or people that want "power" over others. I have mixed feelings on the use of bots and I have those same mixed feelings about what you are doing. Part of me likes the idea and part of me feels who or what gives you that authority? I guess it is me being a Gemini, I am really not sure. I will say that you are taking it slow and asking for feedback. I don't know anything about the bandwidth issues, except from what I learned from your post. The other positive is you are doing it manually for right now. I do think that making a bot takes some thought and work, but the platform the way I understand it doesn't want people to have multiple accounts, but bots are actually creating multiple accounts. I know some people have a personal account and say a business account and I am okay with that. Since bots are now part of the platform I have learned to live with them. I upvoted and resteemed this post, granted it isn't much but I think you are trying to improve the community and that is worth something. I guess time and the community will determine if duplibot survives. Good luck and if I see you around I know it won't be because I am posting spammy comments, maybe for long comments, but then maybe we should have a bot that finds those comments. Just something to think a bot. lol Have a good one.

·

I appreciate the candor about your mixed feelings, I'm still trying to reconcile a few points of my own! I'm learning so much every day and evolving so I think I'll find a good place to settle in. Thanks for the resteem and your genuine thoughts and support!

why not focusing on some of the post too.. like this one...

First, I'm sorry to the owner of the contents.. where there is no content just the tittle of the post..
at least there is someone have a serious matter about giving a repetitive comments used by the same steemit user just to have like what you have mentioned. Thanks for the share.

·

Given my name, I am focusing exclusively on duplicate comments, though I may someday have a sibling to address these other areas.

·
·

it meant to educate not to scare of people to have some fun giving out some comments considered as spam and thats happen in their post too...hope that siblings of your will do the same soon...thanks for reply

Hey man good point, you really put a lot of effort into this post, I read it all and it's very interesting. Keep it up! I will definitely follow

·

Yeah, this is definitely not a low effort post, thanks for the support!

·

Since each comment is slightly different I would have totally missed this, thanks! This account is now on my radar and I've reported it to @steemcleaners noting you as the investigator.

·
·

@duplibot
The account (or 2 accounts I should say) deleted 626 EXIF spam comments.626.png