Update: Communities/Hivemind

in #steem5 years ago (edited)

Hello Steemians! This is @roadscape, member of Team Steemit, and developer behind hivemind - the backend service which will soon introduce next-generation APIs and Communities for Steem. Now that it’s nearing completion, I would like to share a little information about how it works and what it will do.

More Than Communities

Hivemind (or “hive”) will power Communities on Steem, but that’s only one important use case. It’s designed to be a developer-friendly layer on top of the Steem blockchain that will enable any developer to access a database containing all the social information stored on the Steem blockchain without having to rely on minimalistic core blockchain APIs. As with all the software we release, hive is fully open source. It is our goal that everyone developing applications powered by Steem will want to take advantage of hive, so that a consistent user experience can be maintained across the growing ecosystem of Steem applications.

Hive is Developer Friendly

Currently, most Steem-powered applications rely exclusively on steemd full nodes as the backend for information like trending lists, account profile information, reputation scores, comments, follows, and so on. This model has certain strengths which have gotten us quite far, but developers have outgrown it and need a more powerful solution. Relying on steemd nodes prevents frontend developers from iterating rapidly, as (1) all the APIs are written in C++, and (2) any (and all) code changes must go through a rigorous review process because they are close to the critical consensus layer. Thus, it’s quite clear the concerns of application developers (flexibility/simplicity) and blockchain developers (security/stability) need to be separated.

Much of the information that app developers want to leverage is not consensus-critical; that is, it either does not belong in a blockchain node or it doesn’t need to be accessed directly from one. Instead, we can store all the relevant information in a traditional database and keep it in sync with the blockchain, while adding new custom layers of logic and features. This is exactly what hive does; and because it’s written in Python, developers can modify its behavior and APIs with ease... and experiment to their heart’s content.

Ensuring High Fidelity

Despite Hive being “non-consensus” it is important that the information it serves is accurate and up to date. The main challenge here is fork resolution; witness nodes handle this automatically with their in-memory undo database, but it’s not so easy for an outside observer. We could avoid forks entirely by following the last irreversible block, but this would result in a delay of around 45 seconds. Instead, hive (by default) trails behind by just 6 seconds, allowing us to avoid the most common case of harmless microforks. These are trivial for witness nodes but would require extra mechanisms to properly handle externally. In the rare case hive encounters a fork more than 2 blocks deep, we’ve implemented a basic recovery routine. Developers who wish to live on the wild side have the option to run hive with a 0-second delay; those who would rather play it ultra-safe can specify a delay of up to 5 minutes (100 blocks).

Hive is Specialized

Hive focuses on posts, relationships, social actions, custom operations, and derived states.

  • discussions: by blog, trending, hot, created, etc.
  • communities: mod roles/actions, members, feeds (in 1.5; spec)
  • accounts: normalized profile data, reputation
  • feeds: follows/unfollows and resteems/unresteems (yes, unresteems!)

Hive does not track most blockchain operations. For anything to do with wallets, orders, escrow, keys, recovery, or account history, you’ll want to query SBDS or steemd.

Hive is Creativity

Hive can be leveraged to create new features. Here are some that will be possible with hive:

  • reactions, bookmarks
  • comment on resteems
  • indexing of custom profile data
  • reorganizing of old posts (categorize, filter, hide/show)
  • voting/polls (democratic or burn/send to vote)
  • modlists: (e.g. spam, abuse, bad taste)
  • crowdsourced metadata
  • mentions indexing
  • full-text search
  • rich statistics
  • follow lists
  • bot tracking
  • mini-games
  • community bots
  • and much more

It has a lot of potential I’m excited to see leveraged. That’s why I have dedicated so much time to making sure it works great: because hive will lay the foundation for a much richer and more social experience on the Steem blockchain, whether you’re using steemit.com or any other site which chooses to take advantage of this new open source layer.

Development Status: ALPHA

Hive 1.0: Core

The 1.0 release contains a MVP API which replicates all the relevant bits of get_state and get_discussions_* needed to support condenser. This is really quite boring, but it’s crucial to get steemit.com running against hive so it can be tested for load, reliability, and compatibility. The immediate goal is to take over certain steemd responsibilities while establishing the foundation for next-gen APIs.

The core indexer and API service are nearing completion. Because this is critical infrastructure, we need to ensure reliability and consistency. Currently, work is focused on launching the production environment and tuning performance. Then we begin integration and API testing. Once stable, condenser (powering steemit.com) will be switched over to the new hive endpoints; if everything is integrated properly, users will not notice a thing. It’s akin to upgrading an airplane engine mid-flight while making sure there are no hiccups and all controls work as usual.

Currently, we’re looking at the results of our first production reindex. As you can see in the chart below, we’ve had two large surges of activity on our blockchain. Recent blocks take up to 0.03 seconds to process. This may not sound like a lot, but it adds up.

avg total ms per block per query, during initial sync

If there exists a blockchain with anywhere near this quantity of rich and open social data, I’d be surprised (but do let me know). For those keeping score: Steem witnesses have produced over 20,900,000 blocks containing more than 285,000,000 transactions so far!

Hive’s full potential will not be unlocked until the new, modern APIs are finalized. We’ll post another update once we’re ready for beta testers and API feedback.

Hive 1.5: Communities

With the hard part out of the way, the fun begins with hive 1.5. Communities bring a whole new form of interaction to the Steem blockchain, and we are excited to see the ecosystem evolve once users can collaborate easily with like-minded individuals, curate topics of shared interest, and truly organize. We anticipate that the Steem blockchain will become the place to be to discuss everything and anything under the sun.

We know that everyone is as excited as us for Communities to finally launch. We still have some work to do before they are 100% ready, but we are very close to having 1.0 complete. We will continue to keep you up to date with our progress.

Stay tuned!

@roadscape, Team Steemit


Thank you for this great update @roadscape! I can't tell you how excited I am for all of this to be available and how appreciative I am of the hard work you're doing to make it happen!

Please let me know if there is anything I can do to help.

It’s my intention that hive creates the infrastructure which helps ALL contribute much more effectively. It’s amazing to see how far we’ve come with such primitive tools. But it’s time for a major upgrade.

Hive core gives developers a flexible layer which allows them to create more powerful tools and interfaces.

Hive communities give everyone the ability to ORGANIZE around a set of values, and the power to do so effectively. It also introduces a new dynamic of influence which does not have anything to do with SP. Minnows can mute whales. Unlike other platforms the moderation log will be fully transparent.

Such a system will be far more effective than anything we have today for connecting creators and curators. Curators will be attracted to communities which are well-organized: focused, high-quality, low-noise.

Easier to curate good content = easier to get paid for good content. Curators influence the flow of rewards, but their tools are crude. Now the power will be in your hands.

I truly look forward to the future and the coming technology you describe. The current environment just doesn't end up being sufficient to attract and maintain committment from excellent content creators in each niche field. There are so many users and groups out there who are dedicated to keeping the quality of content on steemit as good as it possibly can be. Attracting top talent requires the face of the platform they arrive to see to be an extremely appealing one. One in which they can see themselves having success and building a strong following.

Better connecting creators and curators will (hopefully) go a long way to aiding in the building and development the of this environment.

I am, excited for what you all have and continue to develop. It sounds like exactly what this platform needs to make the jump to the next level.

The fact that you wrote these four words in bold characters, gives us hope.

Ask any of the (established) creators and curators in this network what's the most important thing to them in the process of content production and evaluation, and I bet at least 90% will reply: quality.

You can either build a blockchain where value is defined through speculation or a blockchain where value is backed by a product in top quality. An outstanding product is also the no. 1 driver to a stable user retention.

Looking forward!

That's the short of it ;)

I can only join my voice to those of all Steemians focusing on quality content on top of anything else. I am so happy to read your words and the fact that content still matters for the people up there.

I am as well very excited to try all new functionalities as soon as they will appear!

@roadscape, this all looks good and sounds promising, and.... well, I'm with @surfermarly in saying that "connecting creators and curators" sounds great!

The reason that makes me happy is that it has often seemed like almost everything that comes from STINC is about developers and apps and SMTs... which is all good and fine, but there's a large population of us who live down here at what you might call the "retail level" of Steemit.

Our concern isn't what we can develop but how we can interact with the site, with the content, and with each other. The biggest challenge we face is that content discovery is difficult and often clumsy. We have few fundamental social content tools to make the curation process easier. Even on Farcebook and twatter, you can (for example) sort the people you follow into "groups" from which you can create custom feeds.

I feel hopeful that hivemind/communities will allow for better categorization of content... and NOT ONLY for those who speak fluent C++. php or whatever technical underpinnings run this place. We need tools on Steemit that work along the lines of "explain this to me like I'm a five-year old."

Yes, it IS amazing how far we've come with "primitive tools!" (Thank God I still remember how to speak HTML!) and kudos for that. On the other hand, to gain ANY sort of mass market following, this needs to become a slickly coded, user-friendly, intuitive user interface.

I hope that's at least PART of what's on the way!

Our concern isn't what we can develop but how we can interact with the site, with the content, and with each other. The biggest challenge we face is that content discovery is difficult and often clumsy.

I couldn't have said this any better @denmarkguy!

What you call 'retail level' is the majority of users :-)

I've done all my blog posts in HTML, too, since the editor would've never allowed me to professionally design my articles. That's completely fine - for you and me and a couple of more people. But it's a barrier for the masses.

The average user needs usability, guidance and tools to easily bring their content on the blockchain. They also don't want to search content, they want it to be offered.

But hey, we've been giving the same advices for months. In Germand I'd say now:

Let's nail jelly to the wall.



Curators are the future of community building and if this happens according to your comment, then steem will not only be the first of it's kind, it will be the best. The thought of the ability of any empowered curator to reward good content gives inexplicable joy.

Am sure a lot of people are looking forward to this, thank you! Quality is still an integral part of this platform, this is great news.

Great to read about the quality focus and progress in general @roadscape. I was owning and moderating communities in other social networks that paid in the past and it was a game changer - positive as well as negative - hence i was so curious from the first announcement to hear more details on how that will work from the tech side and group / community structure - ownership / moderators - separate accounts to be created or not. Think I annoyed @andrarchy a lot with bombarding questions long ago. I can't wait to get that live soon but i learned to be patient - keep them coming!

It is so awesome to hear that the encouragement of quality content is part of the focus of these upcoming additions. I love that this will make it easier for curators to find and encourage the creation of engaging content.

Now this here is something that gets you excited and hopeful. Creators and Curators getting Connected is the way communities should function, and no other way.
When influence comes based on how well we are and not what amount of SP we wield, the system will get the respects it deserves and the users the reward.

Thanks @roadscape for the time put into this project.

that's indeed the way to go. Communities should take the responsibility to curate within their own communities thus strengthening the bond within. So looking forward to this to launch !

@roadscape I guess the community has spoken. :) Thank you for all the great work and the hours spent and the care you've put on this.

It’s amazing to see how far we’ve come with such primitive tools. I felt the same when I first came across steem around nine months ago. Though there were many tools created by the community they were not easily accessible to the new entrants on Steem blockchain. People generally join the platform thinking it is like any other social media/blogging platforms. But when they see that there is a learning curve that they were not aware of earlier, many people hang their boots. So it is important to make sure that these tools reach the hands of the right people who are in need of them.

From my experience in running a welcome bot @steemladder I have realised that just directing the people to the Steem Blue Paper has helped them a lot and they are better aware of the platform, difference between Steem/Steemit and how to navigate the platform. Once the dev tools get better we will have more people from community addressing these issues and onboarding of new users should become much smoother. All the best.

On a side note we are currently working on a platform that will enable running publications like Medium currently does. People can start a publication by creating a separate account for the publication. Then users can create posts as a part of that publication(the posts will be published by publication steem account) and the rewards will be shared with authors based on the pre-defined agreement. Do you think it is better to go ahead with the development or is it better to wait for the hive to be released. Also would love if you can give some suggestions so that it will be easier for us switch from the current endpoints to hive based API endpoints. If we have some insights we can prepared :P

The first version of the hive API will offer the same endpoints, so if you wanted to start now it's an easy switch. However the v2 APIs will be more powerful; they will return a lot less data but the data that is returned will be much more useful.

Thanks for the clarification @roadscape. All the best for the release.

"Hive communities give everyone the ability to ORGANIZE around a set of values, and the power to do so effectively. It also introduces a new dynamic of influence which does not have anything to do with SP. Minnows can mute whales. " Can you elaborate more?

Communities allow groups to form and work together to curate content they care about, to their standards. We of course have this to some extent already, except it's not possible to "claim" a tag as yours in any official capacity, nor is there the ability to brand your community on-site, or to appoint moderators. Currently SP is the only moderation system; hive communities introduces more powerful options and the ability to OWN your collective space.

Wubalubadub dub...... Rick

were goin to bendigo

Yeah i feel in the same way. All depends on thinking that you can do every think and this guy is going forward.

thanks for creating a skeleton that makes steemit a better platform, i hope you guys might wanna solve the problem in traffic network, because sometimes i'm worried about it when it happens. :)

:/ I love that communities are coming to steemit and some of the features sound good too, but I would rather see communities integrated as tenants on the steemit platform.

I would like more support for community driven tools. I see support for bots, but really? Bots? There's so much more that has been made. This feels more like a monolith than a microservice. It would be nice if there were hooks that allowed users to link applications by tenant.

I'd like to see identity management instead of normalized profile data.

I dunno. I like the features, but I feel like I'm shopping at goodwill instead of macy's. Like I'm being teased by what this could be in another year.

Here's the thing: any API tool that could be used to make something interesting can also be used by a bot. That's how we got all the bots we have. Any tool which can be used for insight and interaction can be used for automation.

If you want to remove the ability to automate, you have to remove the ability to interact. You don't want that. That is exactly the opposite of what you want.

Personally, my disappointment with this is that it is intended to only cover some of the social network side of the database and not, explicitly, the rest of the available public information on the blockchain which is specifically about wallets and transactions and money. That's going to be a real problem for people who are currently doing things like research into the interaction between the social network and fiscal/financial transactions on the blockchain.

And before you go and look at some of my recent work, yes, that means some forms of automated detection of bot-like behavior in order to make it easier for people to filter out those accounts.

I'm curious as to what the API for content-access is going to look like for building new discovery systems for content off the top of it. It looks like Hive will be a great boon for that kind of research. I'm looking forward to that part and of the fact that it should be easily accessible in Python.

I'm a little discontent that I will continue to have to go through a third party database provider to continue to have access to some of the most important data on the blockchain. That is more than a little annoying.

It is, however, good to see any kind of real development going on. Finally.

Hive has a tight focus on social and ignores financial only because they are two different use cases with quite different performance/integrity requirements. You may be interested in SBDS which is concerned with all raw operations and account histories.

Hi @roadscape I was wondering do you know why steemd is showing duplicate data entry for all actions on the blockchain?

You got a 2.44% upvote from @emperorofnaps courtesy of @uknowjpbitcoin!

Want to promote your posts too? Send 0.05+ SBD or STEEM to @emperorofnaps to receive a share of a full upvote every 2.4 hours...Then go relax and take a nap!

I'm not really opposed to bots or automation. Bots are IMHO intended for convenience. I don't want to make this political though. There's lots of software out there, and for better or worse, it comes down to people.

I just know that because bots are automated that they can only give superficial interactions with data, users, etc... Basically, it's software that's...not a high priority for a lot of people. We have a large set of data and potential to build really meaningful software from it.

1000% agree with you though. I think we all were hoping for more and we get treated like kids in a toys-r-us.

People oversimplify the issue.

The question is not "bots or not bots." The question is "what are you doing?"

In particular, because of the near criminally under-featured nature of Steemit as a social media platform, bots are almost a necessity to function within the context of its operations. I literally would not have stayed on this platform without Ginabot, which does a basic set of functions that you should expect a social network to do for itself. Mentions, keyword tracking, and notifications.

Ginabot is a bot.

But then we have automated voting trains and content harvesters and auto-reply systems and general spammers and mutual voting circles and all of the other horrible things which make living on the steem blockchain a generally miserable existence for a creator. All of the things that subvert the stated intent of Steemit as a platform (albeit exactly in line with the mechanisms of reward as designed).

We really need to start focusing on software behaviors that have negative repercussions of one sort or another, and be willing to talk about the trade-offs that come along with those activities – but that's not how discussion happens here. Either it's a crusade or everything is hunky-dory; there's no room for anything in between.

Given that Toys-R-Us has just declared bankruptcy, and a well-earned one at that, your illustration is both amusing and pointed.

We can actually have quite deep interactions with data and users in an automated way. In a real sense, that's exactly what Communities are; software constructs which provide content direction for a self-selected group of individuals. But that's a useful function, and one that is directly connected to the nature of the content in question.

A lot of the system as it stands is orthogonal to the issue of content. That's a real problem. It provides a lot of opportunity for analysis and observation of human behavior in a very synthetic environment, but as the basis for a social network which incentivizes behaviors which are prosocial, it's a real issue.

But now I'm reminded of all the Geoffrey the Giraffe fursuits out there just waiting to be repurposed by furry cosplayers and I'm kind of excited.

I really must quote you here:
a) "the near criminally under-featured nature of Steemit" and
b) "all of the other horrible things which make living on the steem blockchain a generally miserable existence for a creator"
c) All of the things that subvert the stated intent of Steemit as a platform (albeit exactly in line with the mechanisms of reward as designed)
d) Communities are; software constructs which provide content direction for a self-selected group of individuals

Very Deep, thanks.

I am excited to see the communities ready!

That'a what I'm wondering too.. on where that data lives.
It appears that it will NOT be part of the blockchain.

Much of the information that app developers want to leverage is not consensus-critical; that is, it either does not belong in a blockchain node or it doesn’t need to be accessed directly from one. Instead, we can store all the relevant information in a traditional database and keep it in sync with the blockchain, while adding new custom layers of logic and features.

How can they boast that they have a superior blockchian when their solutions is to add layers where the bulk of the data will live. Databases which ARE NOT decentralized and worse, the features which separate a blockchain from a regular system WON'T be part of that layer...

I don't see this as a good thing.
Fragmentation DOES NOT make a system superior.

Hive processes only the data on the blockchain, nothing proprietary. To "talk" to hive you write to the blockchain.

What does this mean then:

Hive does not track most blockchain operations. For anything to do with wallets, orders, escrow, keys, recovery, or account history, you’ll want to query SBDS or steemd.


Instead, we can store all the relevant information in a traditional database and keep it in sync with the blockchain.

That's what I refer to. How does this equal decentralized.
I understand what HIVE will be. I want to know what the entire system will look like once all these layers are implemented?
If it's all fragmented how can it be considered secure and blockchain based if more and more of the data is going to exist on regular systems?

What does this mean then

It just means hive specializes in certain types of data for specific applications, and providing convenient APIs that wouldn't make sense as part of the steemd consensus node.

I want to know what the entire system will look like once all these layers are implemented?
If it's all fragmented how can it be considered secure and blockchain based if more and more of the data is going to exist on regular systems?

There's really not all that many layers here, and it makes for a more agile environment. If there was one daemon that did everything, nobody would be able to run it. So now we can have more lightweight engines talking to each other. You could run steemd+hive yourself and it will be less painful than a full steemd node while offering more power. Or you could run hive and sync off a public steemd node.

Because more users can run more lightweight services, I see this as increasing security and anti-fragility. The blockchain remains the source of truth, I don't see the issue with storing copies of blockchain data on "regular systems" for easy access.

Good to hear. Looking forward to what all this will look like and if it can alleviate some of the current issues by helping to build the real community this place could be. Hopefully it will encourage more engagement and manual curation again.

Some beautiful, long clamored for features in here. The way you describe it, it almost sounds like everything with steemit.com will turn out...just fine.


Hopefully the steemit core of quality content producing people can hold themselves together that long against the barrage of negativity that seems to be choking up my feed lately. (Anti bots, anti whale, anti bernie, anti self-vote, etc).

Just...gotta...hang...in there.

Thanks for taking time out of your busy schedule to update us muddy folks in the trenches with this progress report.

Though, some of that negativity is warranted, and a bit funny to hear you, with your username of all people, saying it, heh.

I mean, on this platform, it's up to us to fix what we find to be an issue. That starts with identifying it and convincing others of it. I think that the bot situation is literally killing bots. But it won't get better if we ignore it. It's only gotten worse month over month this year.

And while the stuff mentioned in this post is great, it doesn't undo all the other stuff that needs fixing. That negativity is not actually negativity for its own sake, but a call to action.

This literally made me more excited about the platform! Thanks @roadscape for being an amazing and inspiring person. Thanks for clearing doubts! :)

This is hopefully going to make and filtering a LOT easier! I'm not a developer, so I can't directly build on it or review it, but I'm VERY excited to see what the community is going to do with these powerful new tools!

The high level support for a better user experience is something i can relate to and i'm excited to see it coming for Steemit! Currently i'm using steemit.com most of the time but there is a natural urge for things like unresteem.. if you think "i made a mistake, please undo!" and i think there are many other things which developers can now provide more easily to handle the big data created by millions of Steemians soon. Maybe it will support also things like:

  • finding appropriate user profiles to follow (interests, topics, behaviour) as recommendations "it looks like user x has similar interests like you. Visit his blog now!"
  • to get a better access to something like "channels" topic-related
  • reputation profiles which reveal user behaviour for better understanding
  • gamification with level-up (rep, rewards,..) achievements integrated in one sleak UI like Steemitboard
  • recommendations for topics, posts of interest (filter bubble)

..to make Steemit to a more natural extension of daily life.

Cool suggestions!

I've been waiting for Communities for more than a year now. For me, it's an essential piece of the social networking puzzle that'll make Steem usable. I've been following progress on GitHub. Look forward to Hive 1.5. Thanks for all the hard work over the many months.

I'm also quite intrigued by the stuff in 2.0+ - gamification, rep score replacement etc. Hope to see those in 2019.

Unless I'm mistaken, it seems like community names will be directly tied to Steem accounts. Sadly, a lot of high value, universally popular community names are owned by account name squatters. It'd be a shame to have stuff like "science", "movies", "gaming", or "france" run by random abusers. Do we have a solution for this?

👍communities are absolutely essential and needed ASAP. As I mentioned in another comment, it’s amazing to see how far we’ve come with such primitive tools. Can only imagine where the next generation might take us..

Wrt names, it's a concern but there's a lot of plain words on other platforms which are reserved or closed off. A lot of the best content is niche to some extent, with names reflecting that. If it's truly an issue we can implement a 2nd layer namespace.

I would be very interested in knowing this too. If accounts are the only way to form a community, how will we also interact? Does that mean one account for a community and one account for say, posting and commenting? Anyway, I appreciate you @liberosist asking the question.

Let a MILLION community account names bloom?
I currently buy from blocktrades..


nearing completion

4 months later, are you? How's it going, very rapidly, I suppose?

this is awesome, looking forward. any ballpark ETA? Are we talking this week, this month, 6 months? what's "very close"?

Thank you @steemitblog team and @roadscape for the exciting update! We appreciate the transparency and can't wait for hivemind 1 & 1.5. Thank you for developing new tools to empower Steem growth and communities! Steem on :)

ps... looking forward to unresteeming... and even more so for reunresteeming hopefully in the near future ;)

We anticipate that the Steem blockchain will become the place to be to discuss everything and anything under the sun

no one like to walk in the dark and no one is comfortable when somebody else is trying to still his data without his knowledge ..... and i hope that someday we will be able to be free

Some actual "use" cases of hivemind in production mode would be nice to see... who is using it, and what they think of it.

Demonstrating it app-wise will help people see it working and get them even more excited.

Which projects do you know of, that are currently planning on using hivemind?

Many devs are interested in evaluating hive; we will put out a call for testers after internal QA is complete. Hive should be good enough to become a standard.

Awesome work! Can't wait!

Well, this is nearly good news. I am very close to hitting the upvote button!
I still need to do some work, but soon I will do so.
Keep up the awesome job - especially in marketing the blockchain!

Thanks for your updates. we love steemit a lot.

I'm no techie by a long shot, but where does this leave SMT's? I'm not sure how it all ties in together.

They are in parallel development, and can eventually be used together

Excellent update! Thank you so much.

It’s akin to upgrading an airplane engine mid-flight while making sure there are no hiccups and all controls work as usual.

That's a perfect description to help people understand the complexities involved here. Thank you.

Communications like this, though they take time and effort away form doing the actual work of coding, are critically important to the community of investors, authors, witnesses, and participants who support all the underlying value which makes this worth doing in the first place. Thank you for demonstrating how much you value us by spending your limited, valuable time on this communication.

Indeed, that was a great comparison! I hope this plane will keep on flying without any shuddered :D

So Luke, you tell me please, about the last issue Dan raised on STEEM I am aware of?
Someone owns 51% of the STEEM chain, is that true?


Can you clarify what you mean by “owns the STEEM chain”? Steemit, inc does own quite a bit of STEEM tokens and they aren’t currently voting for any witnesses though, technically, nothing prevents them from doing so (other than economic rational self-interest).

This has been discussed at length by witnesses so I wouldn’t call it the elephant in the room. Some ideas thrown around include making their accounts non-voting accounts and adjusting the savings system to have a variable lock-up time so their tokens could be locked up via smart contract. There’s talk of them using their holdings to support SMT launches, but that’s speculation at this point. The main reason for the funds (as far as I know) was to pay for the onboarding faucet, but now that we have delegation (and the possibility of some form of account mining in the future), it’s a little less clear what the funds are for.

Clarifying what those are used for should be priority one if you ask me.

There are two features I would desperately like to be able to access:

  1. Average post payout for a user

  2. Post priority

2 is moot because it doesn't exist yet. I was thinking users need another sliding bar when creating posts so they can give their own posts a priority. That way you can flood the feed and people have a way to sort through it.

This comes just as Reddit is ramPing up censorship! Perfect timing. I think steem should dominate the market for providing coin specific delayed discussion on crypto (telegram has real time discussion) . Just get it out ASAP! Should have been out last year.

This is really exciting1 I can only imagine how much work you have put into this.

Some great things for the average user mentioned here:

  • bookmarks
  • caregorizing of old posts and filters
  • mentions indexing
  • rich statistics

polls and comments on resteemed could be interesting too!

Question : what are follow lists? Will we be able to make categories of users from our list of follows, for example filter out our favorite users to check their posts or filter out users who post certain kinds of material that we organize into our own lists? This would be the king of all updates to how we interact on steemit!

We need to make Steemian understand the importance of having a strong communities, inspite of all differences we all need to support Steem, blockchain technology and crypto to grow, and accepted by the world as a changes toward a better life and future. I think Steemit should also consider about Steemians that are "far away" from investors abd can not afford to invest, we need to keep on the track that Steemit is social media that are not for capitalist, but social character. I hope this program will lead us back to this character as 1ned always said in the beginning.

Hopefully that's how Steem chart looks like after this is implemented.

That's an impressive wave @yesaye time to get the surf board out 😊

I hope you don't mean "surf board" as an innuendo.

I don't think they meant that. I think it was in reference to what the graph currently looks like in that image? I don't know what it's showing; I can't see images at all.

Your work is just amazing @roadscape. and i assume that this is gonna be equally good with steemd.If only everyone contributed as much as you do for the growth and further development of the platform.Communities is what will help users to find their identities.I can't really think of something better right now
Cheers brother

Sounds really great, but I just had a look on the GitHub repo of hivemind and there hasn’t been a single commit for more than two weeks.

There seem to be a lot of talented developers in the STEEM community, so why don’t you try to move development more in the open?

Development is all open. I'm currently assisting with a few other projects while ops works on getting hive launched in production.

Thanks for everything you're doing @roadscape! I'm really looking forward to using Hivemind, Communities will be the best thing to ever happen on the Steem blockchain and projects like this add value to the platform.

Stay awesome!

This is absolutely the best news thus far on the platform. Lately we’ve been waiting for a light of hope, this is exactly what i was waiting for. Thank you so much!!

Thanks for this update @roadscape. I believe that Hivemind will pave the way for Steemit to become a major community platform. With the right features, Steemit communities could eventually not only replace Reddit, but even become a replacement for Facebook groups (which are the only reason that I still have a Facebook account!).

We at @travelfeed are desperately awaiting these features as they will make our curation job so much easier as well as help to grow the travel community on Steemit! I would love to see some features that would make steemit.chat/discord/skype groups like our Steemit Travellers Discord obsolete and bring many of the features that Discord groups are currently used for to the Steem Blockchain (e.g. a community calender for organising meetups and community posts managed by multiple mods e.g. for collecting contests relevant to the community), eventually even an integrated community chat (maybe outside the blockchain since I am not sure how a chat could work on a Blockchain with 3 seconds between blocks).

Is there any ETA for Hivemind 1.5 (I know you said in an earlier post that you won't set dates, but roughly maybe?)? We are thinking about developing a steem-python bot for @travelfeed to make our curation work easier, but Hivemind would make most if not all of it obsolete, so there is no point in putting a lot of time into developing a bot if Hivemind will be available soon!

Yes, I'm excited for hivemind to strengthen the community's ability to organize. I hope it will make old tools obsolete so devs can focus on building even more powerful ones. I can say we'll provide another update when 1.0 is ready for testers (soon as we finish the initial test) with an outlook for 1.5.

I'm very excited to see if this can help @teamgirlpowa and @feminism reach their goals !! Thanks for your gard work. I need to find a developer to help me with the hive hehe

Thanks for a terrific update! Very informative and exciting.

This is great to hear, can't wait to see them and hopefully this release will increase the value of Steem. I am actually confident that the intrinsic value of Steem will increase with this one!

I just habe a question, how is performance improvements of the STeem blockchain going? I mean full nodes require more thn 256 GB of RAM right now, if I understand correctly, and that is a lot!

@andrarchy recently mentioned on a podcast interview that they plan to do a post about this topic soon.

I have listened to that podcast, it was on steem witness forum on msp waves. It still cocernes me and I have seen a lot of witnesses worried about this so this is why I was asking. I want to hear that this is addressed and it is a priority, as many people here think.

Thank you for takimg the time to answer my comment, I hope we will hear more updates from you guys soon!

cant wait for hivemidn to kick in

Congrats! These kind of posts are like fresh air for the majority of the followers of this project. And we need a lot of fresh air these days!

If I can contribute in some way, I'd be happy to do it!

Glad you liked it!

I'm really excited about the communities feature, but am also looking forward to the possibly faster development times that will come along with this. I know you said that it's the boring bit, but I think anyone really interested in the future of Steem will see this not as boring, but an exciting change that will enable more rapid development on the platform.

Thanks for the update and the great work guys!

Yes, absolutely -- maybe boring on the surface but huge in terms of 2nd order effects.

This is really exciting! I can only imagine how much work you have put into this.

Great news and some truly needed advances are now possible. Resteemed.
Particularly looking forward to see these options being implemented:

"reactions, bookmarks
comment on resteems
indexing of custom profile data
reorganizing of old posts (categorize, filter, hide/show)
voting/polls (democratic or burn/send to vote)"

Great to hear. Anticipating what this will look like and in the event that it can reduce a portion of the present issues by building the genuine group this place could be.

Thanks; for the update; A lot of people have been waiting to hear from the team I m excited to see what communities will bring? We need ways for more people to get attention for good

Terrific update.
One question though
Will it run Minecraft?

Not sure about Minecraft, but Doom for sure

Thanks for the update, looking forward to this!

this is brilliant. The list of features rocks. Glad to see analytics in there

I believe communities to be core in Steemit, unfortunately all of these communities gather off steemit at the moment. This is a welcome and anticipated change and I will be there to champion it.

I wonder in the more part will it include direct messaging???

Direct/group messaging itself isn't well-suited for blockchain storage but it could definitely be a steemd side protocol!

if we want really communities to be formed on steemit then someone should look at creating this side protocol. Thank you for the update @roadscape

@acidyo are you here? I will searching you!

I'm looking forward to all the amazing stuff we as a community will build with this...

thanks for the infomation ,very good

Thank you for the update and will look forward how this will change the aspect of communities in Steem!

I'm glad to see this post, very useful!

I'd love to see this implemented in the near future eventho I've only just joined steemit . I'd like to see the community transition once we get those API and go into beta.

  • It’s designed to be a developer-friendly layer on top of the Steem blockchain that will enable any developer to access a database containing all the social information stored on the Steem blockchain without having to rely on minimalistic core blockchain APIs.

How does this affect the 'Decentralized' aspect of STEEM? If the information is in a database is it not then 'Centralized'? This is probably a dumb question, but I am not a Code Geek.

Anyone who wants to can run their own local instance of Hivemind and have their own copy of the data.

How will this work with things not stored on the blockchain such as bookmarks?

It will likely save them on the blockchain as some type of custom JSON

Hive only reads from the blockchain, and simply provides a convenient way to query the same underlying data.

Interesting stuff, thanks for getting word out.

To listen to the audio version of this article click on the play image.

Brought to you by @tts. If you find it useful please consider upvote this reply.

This was a really exciting post :)

Excelente post, creo que todos deseamos estar dentro de esa colmena de conocimientos y aprendizajes continuos gracias por compartir.

Where can I RTFM?

And will this all be json? Please tell me it will be json.

Great [email protected] we all are very appreciating your time and work. Maybe you could also tell us something about the support you have gotten from Steemit inc.? Please also report us what could be done better in terms of developer support, communication with the steemit community and the overall experience you had to develop this amazing tool in full flight!!!

Wow this is such great news! 3rd party apps are definately going to benefit and make the steem experience a richer one over all! Exciting times, glad I’m here to see all these things unfold 😬

mil gracias por esta información, te sigo

your work just mindblowing

That is awesome news, thank you for the update on hivemind!

Jesus this looks amazing

But always when new things come out i am so scared of testing them.
I am scared of being scammed or phished.

If you could get some big people to use the system i will use it too :)

Good luck !

Commenting on resteems and organizing old posts - THANK YOU! 😃

Your Post Has Been Featured on @Resteemable!
Feature any Steemit post using resteemit.com!
How It Works:
1. Take Any Steemit URL
2. Erase https://
3. Type re
Get Featured Instantly & Featured Posts are voted every 2.4hrs
Join the Curation Team Here | Vote Resteemable for Witness

I am an impatient MOFO!

Awesome. This is a developer's dream come true :)

Eager for the upcoming of this.

If matches with my plans, I am going to buy some more memory for the test node and give it a go again... this time on a non-NAS server.

Honestly, how do you manage to build all of this.
I'm no coder and from my point of view this looks super hard thing to do.
I have more chances to go out on a date with Adriana Lima than learning how to do this what you do

Excellent update @roadscape! Pleased to discover this type of proposals from the community. Greetings from Venezuela and continue the work so great that you do

I'm really happy that this is happening in steemit platform. I really belive in this ideas and projects. Hope for the best!

Good work :)

Where are the tags / categories hidden in the announcement above?

Will API calls to filter posts by tag still hide past posts like they do now? Or will ALL posts that use a tag searched for be returned?

Thank you very much for your reply.

Hive currently stores up to the first 5 tags per post, but right now the API only queries the first. There is no limit on age of posts. We can improve tag support, but I think it's more important to focus on communities for organization. Tags are better suited for a search engine (probably a separate service, but could be integrated with hive).

Thank you very much for your reply. Understood - hive is for communities :)

Besides, my question wasn't clear enough.

What I am saying is that NOW API calls for any 'tags' queries don't return all the posts with the tags, queried by. The results don't include every post with specific tag. Is this by design?

Are you referring to already paid-out posts? If so, this is by design; steemd nodes keep all these stats in memory, so it's too expensive to keep all old posts around for tag lookup. If you are seeing pending posts not showing up in API results then it sounds like a bug!

I am certain that not all old posts show up when you filter by certain tag. I suspected that this is by design and now you confirmed it. I don't like it but understand the logic behind it.

I am not certain about the posts which are yet to be paid out. I will check it. If I find out that some posts, younger than seven days, are not displayed in the list, I will let you know.

Thanks for your reply.

Lucene.net, anyone?

@blackpimple another steem blockchain project to look into. Dev-friendly, community-oriented

reorganizing of old posts (categorize, filter, hide/show)

Now that is what I'm excited about, finaly some structure, hopefully steemit.com will implement that.

Finally something happening! And an informative update about it!!! My prayers have been answered! :P

you write plenty comments to get $0.1 hmmmm... this is serious...thanks for the update

great share. thank you so much

Congratulations on this achievement! I can imagine how many late nights and peppermints you’ve burned through to implement this. I can’t wait to see the adoption.

Can't wait to see how the community features work. Will stay tuned for sure. Thanks for the update

black mirron dizisi gibi

happy to see this.

Looks like you have been working very hard. Looks like a lot of work. It is needed desperately hers so hopefully it will be ready soon. No pressure lol.

Totally excited about this update! Looking forward to it!

Nek followed by

Glad to read this! I can't wait anymore :)

This here is an update that is as explanatory as it is educative. Looking forward to see how we can use these new features to grow the Steem network.

Thank you for adressing our future steps. It means a lot to be included into a project and many more, but most important, how we decide to build everything as a team, as a real "society".

I have this problem around my steemd, keeps showing me twice and even three times every activity log, is that happening right now to others, or is it something that can be fixed on mine?

Again, thank you for taking the time to explain even more, and to let us be a part of the team. Cheers!

this is damn fine! looking forward

Interesting! I resteemed this post. Thank you.