When you move labels manually their positions are stored in an auxiliary file coming along with your project (this file has a .qgd extension)
If you delete this file (or move its location i suppose...) QGIS won't find the information.
In your layers panel right-click on the layer, choose Properties and then
Auxiliary Storage : You should see how this whole information is stored.
If you click on these two red icon (for x and y position - those highlighted in your screen capture) you should find a reference to the attributes found in this auxilliary file.
The Auxiliary Storage tab gives some additionnal functionnalities
like exporting this information enabling you to use these stored label
coordinates in an another project ...
That should be the main reason you lose your labels ...