That's it. I've set:
LC_ALL="de_DE@euro"
LANG="de_DE@euro"
in one of my configuration files which are loaded at start-up (it's '/etc/env.d/02locale' for Gentoo) and I get all the Umlauts (e.g. ä, ö, ü, ß).
Now I just need to check which of the various LC_* variables is responsible for this behaviour as LC_ALL overrides all individual ones.
Thanks for pointing me into the right direction!