Where are you running du -sh *? (I.e. what directory, are you definitely scanning the whole file system?) I’m sure it’s obvious, but can never hurt to check!

What does du -sh / show? (Generally, the * glob pattern in the shell will not match hidden dot-files, so is it possible they are being excluded?)

But then for that you have distrobox, which is great. If that’s not enough, running another OS is also trivial, so that downside really is only ‘kinda’, as you say!