Much to my surprise the requirements to have a proper favicon have been extensively expanded and its no longer easy to get one installed. I tried doing it manually, but trying to meet the requirements of all browsers and platforms is really quite a mess for something so ‘trivial’.
The easiest way to do it is to use the favicon checker and generator at RealFaviconGenerator.net and their handy WordPress plugin.
I used their favicon checker and got all pink(bad) results. But with the plugin, everything went blue(good).