4Gb is too low you're probably swapping out to disk from time to time. Whether to upgrade or not depends on the spec of your whole rig.
As an example during beta a tech dev was posting on here about how to get a good experience. He reckoned at the time that PS2 needed 6Gb to work smoothly which in practise means 8Gb installed. Of course that's not to say that it won't work with 4Gb.
Are you running a 64bit OS? I ask because 4Gb machines usually didn't and if your OS is 32bit then although you may have 4Gb installed you're actually only using about 3Gb.
Sethy is also correct, this game relies heavily on a fast CPU, if you have a slow CPU you'll get poor performance.
__________________
|