Server Boots into Black Screen after Upgrading to Ubuntu 18.04

Recently I upgraded a few servers from Ubuntu 16.04 LTS to the latest version of 18.04 LTS. However, I ran into a strange issue that the system will always boot into black screen immediately after the grub menu screen. I spent quite some time digging around, and finally solved this display issue caused by (spoiler alert) deprecated hardware and missing drivers.

The Black Screen

After booting up the system, the furthest stage I could get was the grub menu, which is the screen that let me choose if I want to boot the normal Ubuntu or boot into recovery mode. Soon after the selection screen, all I could get from the display was pitch black, no matter how many times I tried.

Clearly it wasn’t a hardware failure, because it worked fine with just 16.04 prior to the upgrade. The OS installation should also be okay, since I could boot up normally all the way up to grab menu selection screen. In addition, I tried SSH into the server when it was up and running with black display. It worked. The server behave normally from a remote shell’s point of view. So in conclusion, there must be something wrong with the graphic card and it’s most likely a software issue.

Find out the Hardware

These servers were already there before I joined the lab. So I know practically nothing about their configurations. Had I known I would lose visual to the servers once I upgrade them, I would definitely take a note about their hardware specification before I begin the process. But how should I expect things like this would happen?

Luckily, I could still connect to the servers over network. By SSH into the server, I was able to get a better sense on why the problem happened. I checked the graphic card on the server and got the surprising result:

$ lspci | grep VGA
0b:00.0 VGA compatible controller: Matrox Electronics Systems Ltd. G200eR2 (rev 01)

I don’t know too much about Matrox and their graphic cards. Some Googling around lands me on their legacy driver download page and it looks like their driver support stops at 2006. :(

I don’t want to waste too much time looking for up-to-date drivers any more because it is clear at this point that legacy hardware and outdated drivers are the root cause for the black screen.

Solution — Hot Fix

The black screen upon boot issue is well addressed in this Ask Ubuntu post.

Upon booting up, you have to stop the default grub menu selection from happening. When you see the grub menu selection screen, you can press “up”/”down” arrow to stop the timer (buy some time to read instruction carefully), or directly press “e” to edit grub menu entry.

Once you are in the grub menu editing screen, you can change the line starting with linux. This line specifies location for linux kernel image and options for booting up the kernel. I added nomodeset right after the ro flag. So the line looks like

linux /boot/vmlinuz-xxx-generic <blah_blah_blah> ro nomodeset <other_flags>

Remember the name of the grub menu entry you just edited, because you need to select the right one the next time you select from the grub menu. Now after you made the modification, press Ctrl+X or F10 to reboot. After the reboot and inside the grub menu selection screen, select the entry you just inserted nomodeset into. Now the system should boot up normally and eventually you get a visual on the screen asking you to login to the system! Although the display may or may not look a little bit funky due to display driver mismatch.

Solution — Permanent Fix

The quick fix would set nomodeset flag for the corresponding menu entry once (note: not sure if the flag would still be there when you reboot) and let you in. The original post ask you to install the right video driver to solve the problem once for all. However, what if the graphic card is so out-of-date and not supported anymore? I don't want to change the menu entry each time I want to have a visual from the server, although this might work just fine because these servers are going to machine room and we could SSH into them.

In order to get modified grub menu the default option each time the system boot up, I looked into ways to change grub menu in a persistent manner. The solution is also pretty simple. We need to edit the file at /etc/grub.d/10_linux. There is a function linux_entry() which looks like this

# /etc/grub.d/10_linux
linux_entry() 
{
os="$1"
version="$2"
type="$3"
args="$4"
...
}

If you look further down the code, the variable ${args} is used as boot options for linux kernel, and it's positioned right after the ro flag. Therefore, we can simply add another option flag in this variable to enable nomodeset persistently. The new file should look like this:

# /etc/grub.d/10_linux
linux_entry() 
{
os="$1"
version="$2"
type="$3"
args="$4"
args=”nomodeset $args”
...
}

After saving the changes, run

$ sudo update-grub

Then you should have an up-to-date modified version of grub menu! Try reboot the system and see if it can boot into a nomodeset version of kernel by default. It works for me.

Resources & References

Below are some links I found super useful while I was fixing this issue. It turns out this is a more common problem than I had thought. However, I want to share steps I took just in case someone else is also looking for setting the nomodeset a persistent solution instead of installing the video drivers.

(Note: some of them are linked in the post already)