EOS.IO DAWN 2.0 Released & Development Update

in #eos6 years ago (edited)

dawn_2.0_steemit_post.jpg

EOS.IO Dawn 2.0 has been released along with a public test network which will be maintained by the block.one team. This release provides an Alpha implementation of most of the remaining features described in our Roadmap for Fall 2017 and originally targeted for completion by December 21, 2017. As published in our Roadmap, “Phase 2 - Minimal Viable Test Network” was to demonstrate the following by Fall 2017:

  • P2P Network Code
  • Wasm Sanitation & CPU Sandboxing
  • Resource Usage Tracking / Rate Limiting
  • Genesis Import Testing
  • Inter-Blockchain Communication

At this time, we have achieved the initial implementation of most of these features; however, due to parallel development paths our implementation of Inter-Blockchain Communication exists on a separate branch that will not be used for the initial test network.

Those interested in performance-testing our EOS.IO Dawn 2.0 release will find that all blockchain and network code necessary to launch and operate a private network can be found in our Github repository. Our internal testing shows we can sustain several thousand transfers per second and 1 second blocks using our single-threaded implementation on average hardware. That said, there are known attack vectors for which we have unimplemented solutions. For example, compilation of new contracts for the first time can take up to 34ms, which if exploited could cause the network to fragment at transaction rates over 30 TPS.

Our solution for this issue is to place limits on the frequency with which contract code can be updated, as well as a time delay between when code is updated and when transactions using that new code can be processed. This time delay will be on the order of 60 seconds to allow all block producers time to compile/cache the optimized x86 instructions from the web assembly.

Because of these outstanding attack vectors, performance testing will remain a task for private test networks, but feature testing can now be performed on a public test network which we are artificially limiting to 30 TPS to ensure uptime and access.

Over the next 6 months, we will be continuously testing and debugging the network to improve stability and performance.

New Features in Dawn 2.0

Genesis Import Testing

We have implemented a snapshot tool that will import initial state based upon the EOS ERC-20 token distribution on the Ethereum network. Our test network will only include balances which registered a valid EOS public key. About 20% of ERC-20 tokens have been properly registered to an EOS public key. Our snapshot tool also implements a fallback tool for all unregistered ERC-20 tokens held by an Ethereum account for which we can recover a public key from signed ethereum transactions. This covers 99% of all EOS ERC-20 tokens, but will require importing your Ethereum private key into your EOS.IO wallet.

For security purposes, our test network will not ask users to import their Ethereum private keys recovered via the fallback process. If your EOS private key is compromised while testing, you can always register a new key on the Ethereum network.

Token Faucets

We have also implemented a “faucet” facility to allow testing of the network by those who do not hold tokens or have not yet registered a valid EOS public key.

Resource Usage & Rate Limiting

We have implemented basic rate limiting and resource usage tracking. This tracks bandwidth, database storage, and computational usage. At this time there are some known bugs with our rate limiting algorithm, but nothing that should interfere with testing and developing of applications.

We know that many people have been asking for more information about how rate limiting will operate, who will be billed, and how they can lease their staked tokens for income.

Bandwidth

All transactions consume some of the maximum network bandwidth as configured by the block producers. All accounts whose authority is required for the transaction will have their 3-day average bandwidth incremented based upon the size of the transaction. Bandwidth will require the authorizing account (not the contract) to have staked tokens or to be delegated staked tokens by the application provider.

Computational Bandwidth

All transactions consume some computation. Computation can be executed in parallel, so it can be viewed as a multi-lane highway with each lane having different congestion. Each scope (lane) will have its own independent rate limit and a user will be billed O(S^2) for the number of simultaneous scopes (lanes) requested and rate limited based upon the most congested scope.

Database Storage

EOS.IO contracts have access to an in-memory database where they can store application state. The contract is billed based upon the total data they store plus a constant overhead factor for each independent database entry. This in-memory database is independent and separate from the EOS.IO Storage protocol for decentralized bulk hosting and storage.

P2P Network Code

We have a basic implementation of mesh network code that is being demonstrated by our public test network. Block.one is operating 21 independent servers each with one of the initial producers configured.

EOS Dawn 3.0

EOS Dawn 3.0 will re-introduce horizontal scaling of single chains and infinite scaling via secure inter-blockchain communication. With these two features there will be no limit to what can be built on blockchain technology, nor for how decentralized the network of blockchains can become.

Infinite Scaling and Infinite Decentralization

The holy grail of blockchain technology is to enable secure communication between two independent blockchains without requiring both blockchains to validate everything on the other blockchain. This requires making one blockchain a light-client of another blockchain.

Light clients authenticate transactions using only the block headers and merkle proofs. EOS.IO will be the first proof-of-stake protocol with support for light client validation. More importantly, it will be the only one capable of generating proof-of-completeness. This means it will be possible to prove you have received all relevant prior messages from another chain in order without having a waiting/challenge period.

Whereas traditionally light clients have to process all block headers, EOS.IO will enable light clients that only have to process block headers when producers change or when a new message is required from a more recent block. This will enable efficient infrequent communication between chains along with frequent communication. In the worst case, the overhead of two blockchains communicating every 500 ms will be about 2 transactions per second above the total number of messages sent.

Under this model, the communication will be secured so long as at least ⅓ of producers are honest. Furthermore, if even one producer is corrupt they can be automatically punished if they sign any message that could potentially corrupt a light client (aka foreign blockchain).

Lastly, the round-trip time for communicating to another blockchain depends upon the latency until irreversibility of each chain. An EOS.IO based chains will be able to send a message to a foreign EOS.IO chain and get a cryptographically verified response in under 3 seconds.

This level of interchain communication and security enables the creation of two-way pegs between chains with very low latency. While the two-way peg is the most obvious example, any business-to-business communication can be performed using this same method.

Public / Private Communication

With interchain communication it will be possible for private blockchains to have secure two-way communication with public blockchains. This enables all kinds of blockchain applications which are not well suited to the public nature of traditional blockchains. For example, someone could create the Swiss-Bank of blockchains that is super secret to everyone but the bank owners and the individuals.

Development Progress

In order to deliver our public test network, we divided our development into two parallel paths so that we could refactor significant portions of our code for readability, performance, and inter-blockchain communication. This refactoring work has been occurring in the eos-noon branch.

In past updates we indicated our intention to focus on shared-memory architectures so that developers could easily perform synchronous read-access and atomic transactions with other contracts. The consequence of this approach was a loss of horizontal scaling beyond a single high end machine.

With EOS Dawn 3.0 we will be restoring the ability to do multi-machine horizontal scaling by use of up to 65,000 different regions. All regions will share the same accounts and contract code, but have independent in memory databases. Contracts within one region must use asynchronous transactions to communicate with their counterparts in other regions. With this architecture a single block producer could be implemented as a cluster.

Working Integration with Apple’s Secure Enclave

In our last update we announced our intention to support the same elliptic curve used by Apple, Android, and many smart cards. Our eos-noon branch now includes a fully functional proof-of-concept where messages are signed and verified using Touch ID (and also Face ID) on the latest MacBook Pro’s. Similar code also works on native iPhone applications. This means that EOS.IO based mobile applications will be among the most secure blockchain wallets known.

Furthermore, the eos-noon branch has now integrated this support for multiple signature types which means it is possible to use secure enclave to sign transactions which will be validated on eos-noon.

500 ms Block Confirmation

On our eos-noon branch we have implemented a number of changes to the underlying DPOS framework to support 500 ms blocks (2 blocks every second). This change will dramatically increase the responsiveness of decentralized applications. To achieve this we have introduced some changes in how block scheduling occurs.

The same producer will now produce 12 blocks in a row before handing off to the next producer. This solves the single biggest bottleneck on block production which is producer-to-producer handoff. Under the new structure unexpected latency may cause a few blocks to be missed every time there is a hand off, but between handoffs there should be very fast confirmation. We will be experimenting with different hand-off periods. The longer the handoff period the fewer missed blocks during normal operation, but the longer the outage will be if a single node goes down. With 500ms and hand off every 12 blocks, the “down time” is no worse than when a single producer misses a single block on Steem and BitShares. In this event it could take 6 seconds for first confirmation.

Removing Runner Up Producers

Inter-blockchain communication requires light clients to keep track of all blocks where the set of active producers changes. The “runner up producer list” causes a new producer to be added or removed every minute which forces light clients to process at least one block header per minute, if not more. In order to reduce the frequency of producer set changes we have changed block scheduling to only include the top 21 producers. We are considering offering some kind of stand-by pay for the runner ups, but they will not actually be tasked with producing blocks.

One Second Irreversibility

Every block producer will sign every block which will enable a block to be marked irreversible as soon as ⅔+ producers have signed it. Producers are only allowed to sign one block header per block height. This means that in the event of a fork producers cannot sign blocks at the same height on both forks. Any such a signature will be cryptographic proof of misbehavior of a producer which can be dealt with by a number of methods including automatic loss of producer position, potential loss of bond, and potentially liability for damages under arbitration.

Unlike other protocols which gather ⅔+ signatures before the next block can be produced, EOS DPOS does optimistic pipelining that allows the blockchain to advance in “pending state” while the signatures are gathered. These additional signatures occur outside the blockchain and can be pruned after a block becomes irreversible under traditional DPOS rules of Steem or BitShares.

Under this model, it is possible to achieve byzantine fault tolerance because it is impossible for any block to receive ⅔+ signatures without cryptographic evidence of the byzantine nodes.

Removing Producer Schedule Shuffling

In order to minimize the number of missed blocks during producer handoff, it is desirable to minimize the latency between consecutive producers. If a producer in New York is scheduled to follow a producer in China it may take 250ms to receive a block under normal conditions (50% of block interval) and potentially much longer if there is network congestion. A producer in New York and Texas on the other hand would only have 50ms of latency (10% of block interval). This means there is a significantly lower probability of missing blocks during a handoff from New York to Texas than from New York to China.

If we schedule block production such that it rotates from New York, to Texas, to California, to Hawaii, to Japan, China, India, Israel, Italy, England, Iceland, and back to New York then there is never a hand off of more than 50 to 100ms. However if the order is randomized then the average hand off will be significantly higher.

Producer shuffling was introduced to minimize the potential of one producer to pick on a subsequent producer. This risk was in a world where producers were presumed to be potentially malicious, but in the world of highly vetted, public, producers with high quality data centers it no longer makes sense. There is a constitution and expected level of behavior along with a process for resolving disputes in the event one producer intentionally harms his neighbor.

Under EOS the producers will vote on the production rotation order in a way that minimizes average latency and minimizes total missed blocks due to Internet network congestion.

Known Issues

There are a number of known issues with EOS Dawn 2.0 and it is expected for there to be significant instability with this early release. The purpose of this release is to demonstrate a basic capability and our team will be ironing out bugs and improving stability and performance over the the next 6 months.

In order to support stability of the test network, we have disabled producer voting.

Conclusion

We would like to thank our development team for working around the clock and around the globe to build and deliver EOS Dawn 2.0, an alpha version of what will become the most robust, highest performance, most decentralized application platform available. We are executing according to our published roadmap and delivering more features and capabilities than originally planned. We look forward to 2018 and are confident that all features will be complete and bugs resolved by the time the EOS token distribution is concluded.

Disclaimer
block.one is a software company and is producing the EOS.IO software as free, open source software. This software may enable those who deploy it to launch a blockchain or decentralized applications with the features described above. block.one will not be launching a public blockchain based on the EOS.IO software. It will be the sole responsibility of third parties and the community and those who wish to become block producers to implement the features and/or provide the services described above as they see fit. block.one does not guarantee that anyone will implement such features or provide such services or that the EOS.IO software will be adopted and deployed in any way.

All statements in this document, other than statements of historical facts, including any statements regarding block.one’s business strategy, plans, prospects, developments and objectives are forward looking statements. These statements are only predictions and reflect block.one’s current beliefs and expectations with respect to future events and are based on assumptions and are subject to risk, uncertainties and change at any time. We operate in a rapidly changing environment. New risks emerge from time to time. Given these risks and uncertainties, you are cautioned not to rely on these forward-looking statements. Actual results, performance or events may differ materially from those contained in the forward-looking statements. Some of the factors that could cause actual results, performance or events to differ materially from the forward-looking statements contained herein include, without limitation: market volatility; continued availability of capital, financing and personnel; product acceptance; the commercial success of any new products or technologies; competition; government regulation and laws; and general economic, market or business conditions. Any forward-looking statement made by block.one speaks only as of the date on which it is made and block.one is under no obligation to, and expressly disclaims any obligation to, update or alter its forward-looking statements, whether as a result of new information, subsequent events or otherwise.

Sort:  

@ned if you want to flag this post please leave a comment with your reasons.

If you do not have arguments then it is abuse of power.

Seconded. @ned please provide an explanation. Otherwise, this casts a poor light onto yourself.

Seems bitter.

I'd love to hear the lame explanation he has as well. I don't expect him to say a word about it. There's already a ton of people on here calling him out for his childish behavior.

What kind of power you mean?

steem power

Hi, Steemit community! Please check out my steem t-shirt, I was working for a long time on it!
front.jpg
https://teespring.com/steem-to-the-moon

Removing Runner Up Producers
Inter-blockchain communication requires light clients to keep track of all blocks where the set of active producers changes. The “runner up producer list” causes a new producer to be added or removed every minute which forces light clients to process at least one block header per minute, if not more. In order to reduce the frequency of producer set changes we have changed block scheduling to only include the top 21 producers. We are considering offering some kind of stand-by pay for the runner ups, but they will not actually be tasked with producing blocks.

I would like to hear more about this. One of the benefits of rotating the 21st BP is to provide incentive for other BPs to continue to support the blockchain. If we remove this, we could see a scenario where only 21 BPs exist in total. I would prefer there to be 100 BPs waiting in queue for the chance to get the 21st block. Very interested to hear about the stand-by pay and what the community will want. As it stands today I would be pro stand-by pay.

standby pay could even require them to sign every block, just not getting to produce blocks.

I like this idea. Seems workable and would incentivize multiple BPs to continue to sign blocks and be a part of the network.

So in other-words... it's far superior to Ethereum even in its Alpha phase?

I read the whole thing, and got a major boner... I'm so excited!

That's what I didn't understand with the old system. Who would have a high powered system on standby ready to be a block producer and earn nothing whilst not being nominated. Now, are you thinking of having a higher inflation rate of EOS tokens, or pay the 21 producers less, to cover the cost of the runner ups?

Producers were always planned to have less than 5% total, most of that 5% was intended for community benefit contracts.

@dan Would the EOS token itself be inflated up to 5% a year as well ? I understand that new blockchains adopting the EOSIO software could have inflation. Thanks !

@ned why are you downvoting this?

I'd like to hear the answer to this as well. A downvote from you @ned on this topic feels like investment advice (i.e. sell all my STEEM and buy another 10k of EOS).

@ned your silence is deafening. Are you purposefully not coming back on this to fuel speculation?

speculation ? Are you blind ? He down-voted the post.

@ned is obviously not aware that half of the Steem infrastructure can just fuck off from Steem just because of this childish and completely non-professional move he made publicly.

I read this info in my mail this morning, I wonder why I don't get that from Steem Inc.?

it amounts to attempting censorship for your own gain @ned. appaling.

Thanks for these news.

And thanks to the EOS team for delivering :)

I read all this in a hearth beat and I can't wait for the real deal to be implemented. What's 6 months!?

Hopefully EOS gets added to Poof for crypto payments https://www.poof.io

Is this r/bitcoin ?!?!

Why is it saying that this post was hidden "due to low ratings"? I'm seeing 100+ upvotes as of 3pm ET in the USA. Is this post being censored by Steemit?

@ned has beef with dan and was being a child

Very childish. No reason to downvote something to try to hide it. Just makes it obvious who's an adult and who's not.

If a creator can censor a decentralized platform what was the point of it in the first place?

On blockchain level, there is no censorship. It's only there at app level.
Also, Steemit is only one app on Steem among many others.

Thank you for posting @eosio.

Are Steemians looking forward to this development....indeed they are.

Appreciate the updates.

Thank you to all those working around the clock.

Wishing you all the best. Cheers.

NB; The greek word......'iaspas'...pronouned E..OS..pas means diamond....a translucent....transparent precious stone.

Excuse moi, Maybe this is of topic but I bought my EOS shares at my BTS account do I have to registrate them somewhere?

Yes. You need to send them to an Etherium Wallet. Then Register the address with eos.io

Thanks for your comment..

only thing i can sign up for is mail-news...

To people which do not know what EOS is, @tokenika recently had very nice talk about that: [VIDEO] EOS at Warsaw Block on November 30th + survey results


Selection_736.png

Thank you @noisy.

Coin Marketplace

STEEM 0.29
TRX 0.12
JST 0.035
BTC 65090.19
ETH 3379.11
USDT 1.00
SBD 4.55