This is to be in line with requirements from HLSL/DX12 and WebGPU,
both of which do not support this. It's probably better to get
started removing our usage of them now :-)
Light locations had to be padded properly, that ended up being the ultimate fix. Leading to this bug though I ended up patching a lot of stuff that was mysteriously missing from the Vulkan backend like specialization constants and
