In Windows, I have been using and upgrading pgAdmin 4 since version 3.1. Now, have just upgraded to 4.4.3 version.
What I do is just download the installer (see here) and run it. It will overwrite previous version. The server connection setups I had in previous versions are all kept when starting the new version as well.