So my buddy @btcmillionaire is super excited about the upcoming release of @Zappl, and earlier today he linked me a youtube livestream where one of the creators gave a quick run-through of the upcoming social app that's being built on the Steem blockchain.

As the demo unfolded however, a really interesting question arose around "app filtering" - specifically, how apps that post to, and read from, the same content database will maintain their own "content ecosystem" (my phrase), to prevent content created in one app from polluting the content viewed in another.

And with more apps in the works, it's a really important question, which I thought was worth having a discussion about

Quick Disclaimer: I've got a tech/compsci background but have not yet taken the time to sit down and really understand how Steem or blockchains in general work at a technical level. In writing this post I have done some reading around the proposed methods for app filtering, and taken a quick look at how this works at a technical level, but I may have some major lack of understanding that makes all my assumptions wrong. Hopefully I've got it right, but if I haven't then I'm quite confident you friendly Steemians will set me straight.

Challenge: How do we stop Steemit posts showing up in Zappl and vise-versa?

It was about 20 minutes into the demo that I heard the presenter raise the issue of filtering content within Zappl, and at this point I began to realise that because of the way apps store their content in the blockchain, whereby apps are essentially saving all of their data in the same database, there's an inherent challenge in ensuring that content from one app doesn't spill over into the other.

First of all, let's have a look at what @Thedegensloth had to say:

t=19m55s
(in response to user question)
"Umm, Zappl is more of a standalone... right now why you see the content on the site is because we're pulling data from the database because it's the only data that we have to use. But, after launch we will only be using Zappl data, so if it's posted on Zappl it'll show on Zappl and it'll post on every other site that pulls the database, but if you post on Steemit it will not show on Zappl"

And then a bit later he discusses it more.

t=21m38s
"Q: Have I had a conversation with Steemit team around how they will be filtering out zaps on their side."
"A: I'm not sure yet. I have mentioned it to a couple of developers but as for the steemit team, the front end developers, uhh, not yet.
I will have to have a chat with them of how... I just wanted to see how it would go on launch, and if it became a problem, well then now we know which... what we have to do, that app filtering is gonna have to be a thing.
But I feel that app filtering is going to have to be a thing, no matter what, because the more apps that we want to come on the network, well they're gonna want to post a lot too, and it's gonna just flood. So, sites like Steemit that are, I guess, more... long-form friendly, are gonna get just bombarded by short-form posts."

So there's two key take-aways here:

1. Zappl will filter content so that only content posted from Zappl will appear in Zappl. And according to this comment from the team at Steepshot, they'll also be doing this
2. Steemit.com does not currently filter content. If it's posted on the blockchain, it shows up on Steemit - this will include posts from other apps posting to the blockchain.

The first one makes a lot of sense, but the second one seems a little bit odd. After all, if we're going to keep bringing new apps onto the Steem blockchain, then Steemit.com is going to be flooded as those apps gain popularity, and will eventually become unusable.

But it starts to be a little less odd once you realise what Zappl and Steepshot are doing at the technical level to implement this app filtering.

So how does it work at the technical level?

Without going full nerd, every post and comment on the Steem blockchain is accompanied by a set of JSON metadata which gives you a little bit of extra information about the post itself. And, while at a technical level you could stick whatever parameters you want into this metadata set, in practice its usage is dictated by an open standard called Steem Script, put forward by @fabien in this post back in Nov 2016.

For comments, the Standard defines the metadata parameters with the following example:

{
    "app": "steemit/1.23",
    "format": "html",
    "tags": ["steemit", "steem"],
    "users": ["ned", "dan"],
    "images": ["https://img.busy6.com/@ned"],
    "videos": ["https://www.youtube.com/watch?v=rkQ7b-u8_6g", "https://www.youtube.com/watch?v=H399YZ0pv0o"],
    "status": "archived",
    "canonical": "http://blog.steem.io/steem/@ned/the-first-phase-of-the-steem-faq-and-wikee-consolidation-of-knowledge"
}

Now, I believe what @Zappl and @Steepshot will be doing to filter content for their apps is focusing on that "app" parameter. My understanding is that if the app parameter for a post/comment doesn't contain "zappl" or "steepshot", it won't appear in the relevant app. Seems pretty straight forward and logical, really.

But there's a bit of a problem here, in that the Steem Script standard doesn't actually have any usage guidelines for how the parameters in the comment JSON metadata should be used, and so while Zappl/Steepshot are looking to use the "app" parameter to filter content, its historic use has been exclusively for identifying the user agent that was used to post to the Steem blockchain.

To reiterate - the parameter these apps are most likely to use use to filter content tells which app the content came from, not which app the content is for. And that's sort of an important difference.

Why does this matter?

There are three key scenarios where I believe this sort of matters:

1. When developing 3rd party apps to interact with other apps' ecosystems

Starting with what is possibly the least impactful scenario, if the app parameter is used to identify the agent that is posting to the Steem blockchain, and 3rd party app creators adhere to this convention, then 3rd party apps will never be able to post into the ecosystem of apps like zappl and steepshot.

In otherwords, you will only ever be able to post to zappl from zappl.

Makes sense, right? Well.. not really. When I post an image on Instagram, I usually hit that share button and get it to send out to Facebook as well. In fact, many attribute this image sharing as one of the primary reasons Instagram became so big in the first place.

In fact, in this scenario, if the actual real Instagram app wanted to add a "Zappl" option to its sharing, it wouldn't be able to - because, if they followed convention, they would identify themselves as "instagram" in the blockchain's app parameter, and their posts would be filtered out of the Zappl ecosystem.

2. It creates a silo'd content ecosystem

The second concern, and it's very similar to the first, is that when we start filtering content in this way, we're beginning to create a silo'd ecosystem where content will only ever be able to appear in one app.

Now there are very solid reasons why Zappl would not want photos posted from Steepshot to Steepshot for Steepshot users showing up in the Zappl feed, but there's also lots of use cases where it makes sense for content to appear in multiple apps.

For example, outside of Steem it's common for people to post the same blog post on LinkedIn and then post it again on Medium. Particularly when we're attaching value to upvotes and sharing, and particularly when your followers across multiple apps are likely to be the same, it makes sense for this to be a single post to the Steem blockchain, rather than two posts.

However, if we're filtering based on the "app" parameter, a single-value field representing the app the content was posted from, then you'd need to post the same article twice, from two different apps, to make sure you reached the followers that tend to use one app over the other.

3. Alternative clients - the challenge of filtering Steemit.com

Lastly, we need to talk about what happens when you post to Steemit using a third party client like eSteem, whose "app" field looks something like this: "esteem/1.4.6".

It's not an issue right now, because Steemit just shows us whatever gets posted to the blockchain. But, if we get put in a position where we need to start filtering out micro-content (eg, a tweet, or a photo with a caption), then filtering on the "app" field to only include posts from "steemit" is going to make all the esteem content disappear.

So sure, Steemit.com can just filter on "steemit" and "esteem", problem solved.

But what if I want to create my own Steemit client, Rossteem? Do I have to convince the Steemit.com devs to add me to their filter? And the esteem developers too? What a pain!

Solution: We add an "ecosystem" parameter to the Steem Script Standard

The better solution seems, to me, to amend the Steem Script standard to include an "ecosystem" multi-value parameter which allows you to specify which app content ecosystem your post should appear in, with usage guidelines that state 'Steemit' will be the default ecosystem where no tag is supplied.

By doing it this way:

• All existing content will be preserved as is in the Steem ecosystem, without any need to modify the Steem blockchain
• Future apps can specify their own ecosystem attributes when posting from that app, and even allow sharing into other ecosystems
• 3rd party apps can add multiple ecosystems within their posting options
• Apps built on the Steem blockchain will have a standard convention for filtering content - at the moment they all could do it a different way. (eg, one could filter on "app", one could ignore the standard and use its own parameter fields)

I might also suggest a further optional parameter along the lines of "appkey" - if a post to a content ecosystem doesn't feature an approved public key, the post doesn't get included in the native app. This not only adds some security around the app filtering (currently, someone could fake posting from your app) but it also would give you the ability to approve third party apps to post into your ecosystem.

I have posted this suggestion as an issue to the github page for Steem Script.

Did I get it right? What do you think?

As mentioned at the start of this post, this is all based on my admittedly limited understanding of how the Steem blockchain works, and how I believe the upcoming apps plan to filter content within their ecosystems. Hopefully there's no fundamental misunderstandings that negate everything I've had to say - I actually feel like I learnt a fair bit about how it all fits together during the process of writing this post :)

Would be keen to hear what people think of my proposed solution, if they have any of their own, and whether they can offer any additional insight into the challenges of app filtering as we begin to see more and more apps come online.

Either way, it seems to me that now is the time that we need to do something. If we don't come up with a solution soon, app makers are going to have to find their own solutions, and by the time a standard approach is decided upon, we're going to have to reverse engineer something or otherwise find a way to integrate posts made by apps before the standard was agreed upon..