Announcing SteemMsg - Encrypted Private Messages on the Steem Blockchain

11 months ago

Screen Shot 2016-11-18 at 9.20.19 PM.png

Note- this is a working Alpha product for testing.

I'm very excited to announce the alpha release of our newest development - SteemMsg.

https://www.steempower.org/steemmsg

SteemMsg is a fully encrypted private messaging system built on top of the Steem blockcain. The current 'Memo Field' requires you to send a small amount of funds to send someone a message. SteemMsg does not require that, nor do we require your owner or active key.

Anyone can send you messages, but unless you login to https://www.steempower.org/steemmsg, it will not spam your Steemit.com inbox or show on your Steemd.com page.

I am monitoring my inbox now, send a message to 'charlieshrem' :)

Screen Shot 2016-11-18 at 9.19.05 PM.png

Process flow - sending

When the send_private_message cap (see our API documentation to find out more about our capabilities/caps system) is invoked the web app does the following:

  1. Basic sanity checks (user actually exists, cap is valid etc)
  2. Lookup the profile info for the user the message is being sent to and grab their public posting key
  3. Derive the shared secret and encrypt the message content
  4. Pass the encrypted message content and the public key along with the sending user's private key to a backend process
  5. Forget the posting key - we never store your posting key on disk for obvious security reasons, it lives in RAM only while a request is served

The backend process is connected to either via the loopback interface or via an encrypted SSL connection, upon receiving the request it does the following:

  1. More basic sanity checks and authentication
  2. Construct the custom_json operation and the surrounding transaction
  3. Sign the transaction and then lose the private key
  4. Pass the serialized transaction off to a steemd node for broadcast
  5. Return to the web app confirming everything was sent correctly OR (hopefully not) with an error message

After broadcast, our server also caches the transactions for every user - this cache will be used for running AJAX updates soon too.
No sensitive data is in the cache - it's simply our way to locate messages quickly without crawling the whole blockchain.

Screen Shot 2016-11-18 at 9.20.02 PM.png

Process flow - reading

Due to the cache mentioned above this is fairly simple, we grab data from the cache to build your inbox and outbox, organize it into conversations
and sort by the time sent. While building the response to send to your browser we use your private key to decrypt them and then forget it.

Remember, our system only has access to your private key when your browser makes a request - the caps URL itself has the key in an encrypted format.

Shared secret deriviation

Here I must give credit to @xeroc for his highly useful python-steemlib. Anything on SteemPower that requires signing transactions relies on this library to do the serialisation and signing.

For SteemMsg in particular there's another function used from python-steemlib: the memo encoding and decoding. Although intended for use in transaction memos, this still works fine for encrypting arbitrary messages between 2 users with public/private keypairs.

Essentially the code in python-steemlib offers a useful function that takes a private key and a public key and creates a shared secret suitable for use in AES. It also provides facilities for serialisation and deserialisation of the message.

Using this shared secret, we encrypt the message content and store it in the msg field, read below for the full process.

Screen Shot 2016-11-18 at 9.20.34 PM.png

How SteemMsg works [Technical] - Written by @garethnelsonuk

First of all, the most important thing to note is that we do NOT use the existing send_private_message functionality in cli_wallet. I investigated this approach and found that it required using people's active keys and SteemAccess currently only supports the posting key - this is by design, we do not want your active key and you should not want to give it to us.

Instead we use a custom JSON operation, and if you look at my or charlie's page on steemd.com you'll find a few from testing.

The custom_json operation is quite cool for our purposes as it allows inserting arbitrary JSON into the blockchain and only requires your posting key. Aside from this, JSON fits neatly into the development workflow as it's far simpler to manually manipulate as needed.

In the custom JSON operation we send just 2 fields: "to" and "msg". While it should be self-explanatory what these fields are for, let's look more closely at the contents of the "msg" field.

Coming soon - AJAX and notifications and bears oh my

Well this doesn't need a lot of explanation: we've got a nice async notification service with websockets support that will be used to send
typing notifications and such and to update messages in realtime.

Sadly we do not have a lot of bears - it appears that it is not in fact possible to serialise actual bears into JSON objects - perhaps msgpack?

Being serious, what comes after the notifications is further integration with SteemDeck - another product launching very soon.

Please feel free to make feature requests to either myself or Charlie and thanks for reading.


Help keep SteemPower running! Voting for us as witness pays for the development of apps and tools for Steem.

Vote for us as a witness the following way:

https://steemit.com/~witnesses click the arrow next to "charlieshrem"

Authors get paid when people like you upvote their post.
If you enjoyed what you read here, create your account today and start earning FREE STEEM!
Sort Order:  trending

our system only has access to your private key when your browser makes a request - the caps URL itself has the key in an encrypted format.

I don't like if my browser need to send out my private key to a server (no matter "encrypted" or not). Why not sign transactions on the client side?

·

I was saying to Charlie that he should mention this in the post as someone would definitely bring it up.
Put simply: you're right, there's no need to have to encrypt the message serverside and it should be in the browser.

The only reason it isn't that way already is due to complexity of implementing it in javascript - but that's coming.

For now you can either trust us (and remember, it's only your posting key used) or simply not use the service until browserside crypto is in place.

Not that you can't PM someone with any other third-party application anyway, but this subverts the transparency of the Steemit blockchain

·

How does it subvert?

Quiero saber algo, esto afectara en algo a Steemit Chat?
"I want to know something, will this affect something to Steemit Chat?"

·

No creo. Quizas lo que mas afecte sea que menos personas usen el steemit.chat si mas personas deciden usar este nuevo que se esta proponiendo, pero no es un reemplazo y las aplicaciones segun entiendo son diferentes. Esta mensajeria se supone esta montada en la misma cadena que corre la plataforma Steemit, en Steem. Pero el steemit.chat segun entiendo es una aplicacion aparte.

Super tight, thanks!

Nice, can't wait for this feature.

This is amazing !

This. Is. FANTASTIC. I've mentioned the need for direct messaging on the Steem blockchain several times over the last few months, and I am supremely pleased to see folks with the technical know-how to implement it developing such a system.

Hopefully you guys or others can find a way to integrate this system into a UI, either here at Steemit.com or the others that I know are forthcoming. Keep up the good work!

Boom Bip!

this is fan-flipping-tastic!

Cool. Thanks.... image

Hi Charlie. That is a really cool Feature! I have been waiting for a way to send messages to other users. I have sent you a test message. I hope it arrived.

·
·
·

I think it went better this time. I used my public posting key the first time but I needed to use my private posting key.

Make sure to put the @ sign as I forgot :(
I think it matters ? I'm trying to figure out if the end user got my message?
Also, I sent the message to @roelandp did you get it ?

·

You actually shouldn't put the @ sign, sorry if that isn't clear

Thanks for the info ? This is great !!! I signed up for your news !

I'm glad to see Steemit moving into more areas of social media in order to combat online censorship. I'm happy to see how much enthusiasm is in this community, we are going to win the battle against censorship.

Thanks. Good job!

need to check this out

Great Job!!! I am very excited to try this out!

I'll check it out Charlie! Thank you!

Very nice! Any hopes to see it interfaced to the maim Steemit platform at some point (to avoid having to monitor different websites)?

This post has been linked to from another place on Steem.

Learn more about and upvote to support linkback bot v0.5. Flag this comment if you don't want the bot to continue posting linkbacks for your posts.

Built by @ontofractal

I am not so sure about this approach. Although the message itself is encrypted you can still see who the messages are being sent to. It would be cool to have the names of the recipient and the sender encrypted. That way it would be a true private message.

·

Yes, I agree and we can easily do this! We're working on it for the beta.

·
·

It isn't easy! The only way to hide receiver is for receiver to scan all messages. Sender cannot be hidden due to bandwidth enforcement.

·
·
·

Yeh gareth was explaining to me it would it would make it impossible to automatically reconstruct our database from the blockchain though unless we encrypt the "to" field using our own key. He said "it'll take a bit of time to engineer into a proper solution, but the basic trick is to allow each user to store an arbitrary block of data in our database, encrypted - and with deduplication"