The network is already slow enough as is, losing miners will just exacerbate that and cause a runaway train effect. BCH price will rise and more miners will flee which will cause slower transactions which will lower BTC price which will cause more miners to flee, so on and so forth. Meanwhile BCH will chug along with 8 times the transactions per second.