Building a cheap and second hand FP64 Monster -- Gridcoin

Hi all,

Have you ever wanted to build a very capable computer to support science that is on the other hand side quite cheap to buy and re uses hardware that has been on the market for a long time ? There is a good opportunity out there for exactly doing that and additionally you can even earn gridcoins for that computation in exchange.
In this series of articles i will exactly do that, from the basics and the shopping list, the building and installing on to the crunching :)


But lets start with the basics first.

The Basics

The key to all this is called FP64. GPUs are generally very good at doing math, the precision within that mathematics can be set by the needs of the programmer. FP64 hereby specifies double precision, FP32 being the single and FP16 half precision.
Why is this important ?

Game development is more about FP32 and nowadays also a lot about FP16 calculations, therefore the designers of graphic cards (for the consumer market) are more into optimizing this than the FP64 power.
The outcome of this can be seen in the following table:

(GFlops means Giga Floating Point Operations per Second = 10^9 Flops, the more the better )

Graphic card	Chipset	FP64 GFlops	FP32 GFlops	Year
Radeon 6970	Cayman XT	675	2703	2010
Radeon 7870 Ghz Edtion	Pitcairn XT	160	2560	2012
Radeon 7870 XT	Tahiti LE	749	2995	2012
Radeon 7950	Tahiti PRO2/PRO	761	3046	2012
Radeon 7970	Tahiti XT	947	3788	2012
Radeon 7970 Ghz Edition	Tahiti XT2	1075	4300	2012
Radeon 7990	New Zealand	1894	8200	2013
Radeon R9 270x	Pitcairn XT	160	2560	2013
Radeon R9 390	Hawaii Pro	640	5120	2015
Radeon R9 Fury X	Fiji XT	537	8601	2015
Radeon RX 480	Ellesmere XT	323	5161	2016
RX Vega 56	Vega 10	659	10544	2017
RX Vega 64	Vega 10	792	12665	2017
Nvidia GTX1050ti	GP107-400-A1	66	2123	2016
Nvidia GTX1060	GP106-400-A1	137	4372	2016
Nvidia GTX1070	GP104-200-A1	202	6463	2016
Nvidia GTX1080ti	GP102-350-K1-A1	354	11340	2017

There are some very remarkable things to be seen here. First of old Radeon Cards like the 7970 or even the 7870 XT are superior to even the GTX 1080ti in terms of FP64. Even the 6970 with the Cayman XT Chip is almost doubling the FP64 performance of a 1080ti. What also can be seen that in terms of performance increase over the years there was a lot movement, just not in FP64. The AMD cards are no better, the brand new Vega 64 is inferior to an old 7970.

What does that mean ? Are those cards really faster ?
Well there is no short answer to that, but in this specific field the answer is clearly yes. In a lot of other topics like memory bandwith, FP32 calculations, in short in all relevant things for gaming the answer is clearly no.

Ok but for what do we need this FP64 power ?

This is where the fun starts, there are scientific projects that require this double precision calculations. Milkyway@home for instance is such a project. This project uses the FP64 calculations to create a highly accurate three dimensional model of the Milky Way galaxy.

The shopping List

This all got me thinking and i came up with a shopping list for a 4 GPU crunching machine (should yield somewhere >150 mag easily).

This is just an idea, for sure the hardware can differ if you want to build such a thing for yourself. I took the Asus Board because i already have a few of them running and they are solid as a rock :) and on the other hand brings enough PCIe ports to connect the GPUs to. Iam not yet fixed in what GPUs i want to have, 7970 would be fine but 7950 or 7870XT would also do. Only in the PSU i will make no compromise, i want to have a stable new one, i wont trust in something used.

I will try Ebay and other sources of old hardware, lets see what i can get there. Maybe there is also a price drop with the new Vega Cards out, altough i dont really believe in that, there are just so few shipped ...

Shopping

I will start the shopping now, once it is concluded iam going to follow this up with the next chapter :)

---

If you want to know more about Gridcoin please take a look here:

Gridcoin: Gridcoin Website

If you want to know more about FP64 crunching in specific, please look into the posts from @vortac. There are quite a few and they are brilliant.

Example on Vortacs posts for GPUs: GPU Mining 6

---

Content credit:
Gridcoin Header, @joshoeah