The CAPS LOCK initial setting is in the computer hardware BIOS. That's controlled BEFORE any OS is booted.

I think there are tiny applications that can toggle the state, but I've never needed to...