They don't load the whole world at once. The initial load does load in the immediate area around the character but also sets up all of the AI routines and game scripts and initialises the game state. The world is still streamed in on the fly, and given that PS5 has about 100x the data throughput of the PS4 you shouldn't see any hitches on that version.
The best thing was a modder on the PC version cutting GTA Online load times by 70% because Rockstar couldn't code their game properly, although that only applied to the PC's online mode.