Minecolonies & Core-Rework-Refactoring-Madness #2

Hey everyone,

Do you remember the last time OCD got me hard here and I had to rework significant parts of our code?
It's worse this time!

But let me start from the beginning.

These days I went to our official server (which you can get with this mod pack: here) and I noticed a lot of lag, so I started to run some tools to find out which might be causing this.
First I found the typical problems, too many entities like fishes, and because of certain mods, doors and trapdoors were using significant parts of the server performance.
But, besides that general overhead lag, we also had terrible lag spikes now and then.
Therefore, I ran a lag spike profiler as well.
Within a few minutes, it told me that there is a certain overhead where we store our colonies to NBT.
I investigate this and I found that with every single change within each of our colonies, we write ALL the colonies to NBT (to hard disk). And who knows IO also knows that this is an obvious bottleneck.
Therefore, I knew I had to do the following two things:

a) Changes to the colony write the colony dirty and after a few seconds, it writes the colony manager dirty so it can write it to disk (No need to do that every second, on average every 5 seconds works perfectly).

At a ton of places:

b) Cache the stream of the colonies which didn't have any changes so we don't have to write those to disk as well (Our main bottleneck is CPU, not RAM).

Which at the same time allowed me to decrease the size of the code of the colony saving code:

Besides that, I had analogous to the previous refactoring I extracted the package management of the colony to a dedicated package manager.

This way I was able to reduce the size of colony.java to around 1100 (including the additional caching) which is now a perfect file size to work with.

But these are very significant changes in the colony code and that's why I also write some unit tests to make sure things are still in place after this update.

Therefore, I tested if the colony tags are correctly updated after the colony changed:

Besides that I am planning to do the following:

In the future, I want to decentralize the colonies from the colony manager and store them independently to disk.
This way we'll only have to store the colonies to disk which really changed and we'll be able to recover independent colonies of users more easily as well.

I hope you liked my next refactor madness, I'm sure another day it will hit me again and it won't stop until I refactored enough and posted about it.

---

Posted on Utopian.io - Rewarding Open Source Contributors

---