First off: I got a new graphics card since my old one died without any warnings, got the rx580 with 8GB of VRAM and tried to undervolt it.
Decreased the CoreVoltage from 1150 to 1050mV, after applying and 2 minutes stable play almost all graphics disappeared: Check this for reference, leaving me with a x-ray-type-vison. Quickly returned to stock settings, but problem stayed. After restart of game and again trying to undervolt everything was stable and the bug didn't appear again.
Also, just rerun the game and the monitoring tool in the top right corner shows the same stats as in the video, so the memory didn't lose anything I assume, all clocks stayed where they should be. VRAM is functioning correctly, tested with other games.