Yes, I find this frustrating too. There is a code way to try and fix it which can be run at the python window or added as a macro - see instructions at this answer:
import os
from qgis.core import QgsProject
from PyQt5.QtCore import QSettings
prjhomepath = QgsProject.instance().homePath()
if os.path.exists(prjhomepath):
QSettings().setValue('/UI/lastFileNameWidgetDir',prjhomepath)
QSettings().setValue('/UI/lastRasterFileDir',prjhomepath)
QSettings().setValue('/UI/lastVectorFileFilterDir',prjhomepath)
QSettings().setValue('/Processing/LastInputPath',prjhomepath)
QSettings().setValue('/Processing/LastOutputPath',prjhomepath)```
The code changes various folder locations stored in your QGIS.ini file to your projects home folder. There may be other folders to change - take a look at QGIS3.ini in a txt editor.
