Bitcoin blockchain got so big, I ran out of disk space... Crazy reptilian solution
I have a much larger partition that can hold all the files and more, so I started a cp -R
That copies recursively all the files using the built in operating system (debian VPS)
So far, nothing that crazy. But due to the number of GB and files it was taking a while, so I decided to start an iguana sync from scratch. I havent done a BTC sync from scratch for a few months as it is not exactly a fast process, even on my fast VPS that has a 1gbps connection that can sustain 100MB/sec transfers.
I have slowed down the iguana processing a bit as most users dont have such fast connections so it is sustaining about 30MB/sec or 200mbits/sec which is within the possibility for a home connection. But it used to be 100MB/sec and faster during peaks, so a bit strange why it is so slow.
as I write this, the copy just finished and using the same disk for the copy and parallel sync is really not a good idea, but still the iguana has made decent progress:
BTC.RT0 u.0+c.0 b.0 v.0 (0+1253/2000 1st.170).s0 to 176 N[213] h.425653 r.344503 c.340000 s.344520 d.1 E.169 maxB.6 peers.21/64 Q.(3070 0) (L.425653 212:1653) M.425652 000000000000000001ec532cf96c58ffeb6a5886f232c9c5078f870b47a3bfe2 ledger.00000000 bQ.2499 0:29:54 stuck.0 max.43
169 bundles are already saved, out of 212, so around 340K blocks after 30 minutes. I just realized that I ran the minimal config version with max 64 peers, which is why it is so slow. For best speed better to have max peers at 512 and let it connect and optimize 128+ of them.
I was wondering why the bandwidth was so much lower than before. Anyway, looks like it still needs more time to finish and I dont want to restart it with a better config as it is already at 171 bundles. i will post a comment with the final completion stats. Keep in mind it should be going 50% to 100% faster if I ran it using the fast config.
James
BTC.RT0 u.0+c.0 b.0 v.0 (0+1998/2000 1st.200).s0 to 212 N[213] h.425657 r.409359 c.404000 s.409360 d.5 E.197 maxB.6 peers.21/64 Q.(1659 0) (L.425657 212:1657) M.425656 000000000000000004f08bd2a7a3e1bd45f7795b7907bf0407640fdb3a526764 ledger.00000000 bQ.2467 1:00:12 stuck.12 max.43
Looks like it will take 90 minutes or more in the slow config.
Nice article. Thanks
finished saving all the bundle files at 1hr 15 min, now it generates the spends files, which are invariant once created. and balance files, which change each time a new bundle file is created
BTC.RT0 u.0+c.0 b.0 v.0 (0+1658/1658 1st.212).s0 to 212 N[213] h.425658 r.425658 c.424000 s.425658 d.0 E.212 maxB.6 peers.21/64 Q.(0 0) (L.425658 212:1658) M.425657 00000000000000000502e1b069f0fb89d26bee78e76935a377335e310322c5c5 ledger.00000000 bQ.2452 1:15:17 stuck.0 max.43
5 minutes for the bundle spends files
3 minutes for bundle balance files
3 minutes for balance summary files
3 minutes for fastfind tables
At this point, it starts verifying all the bundles and creates a ledger file for all addresses, including p2sh addresses and brute force validates that against the calculated listunspent returned data. Without SSD this is a painfully slow process. But 90 minutes to the validation stage I guess is not so bad
BTC.RT0 u.212+c.212 b.212 v.212 (0+1664/1664 1st.212).s0 to 212 N[213] h.425664 r.425664 c.424000 s.425664 d.0 E.212 maxB.6 peers.21/64 Q.(0 0) (L.425664 212:1664) M.425663 00000000000000000007bfb5da523f2666cda7ea6171cb36861d25ae822c680e ledger.5f6f054b8d33a9aa bQ.2453 2:06:03 stuck.0 max.43
A bit over 2 hours to start validating the ledger file: validating DB/BTC/utxoaddrs.424000 HIST BALANCE aaa9338d4b056f5f45e2e6734d1812dd886ff3dcc9011303aac49b77732cfdf8 15799989.79530316 errs 0
but this took a long time even on the SSD and things are now in a state where iguana can run to the chaintip, it is just set to bruteforce validate the ledger file when it is created. It is interested to note that the full bitcoin ledger is 300MB when it is distilled down to its essence (rmd160 + balance).
on SSD server, got to ledger validation in 1hr 20 min
Hi @jl777, I know you had some questions for Sunny King. He's going to be in live chat tomorrow (August 19th) on Fuzzy's Beyond Bitcoin mumble.
(I have a post on my steemit blog that details more about it - hope you can make it)
The reason I wrote this here.. is that I agree with you, the Bitcoin chain takes forever to sync, and is pretty cpu intensive too. I gave up using a regular bitcoin client. It's just too huge of a chain!
I also saw you had some peercointalk questions for Sunny before -- now is your opportunity. :)