If you were using ArcGIS, you use the spatial adjustment toolbar. However, QGIS offers a similar functionality through plugins.
Go to the plugin manager and install 'Affine Transformations'
Also install 'Vector Bender'
Since you already have the control points, you should be able to transform your data back to the original "projection". Load your control points into the plugin and perform the transformation. I haven't tried this yet so you may have to play with the control points. The affine transformations plugin uses constants to perform the transformation. Since you already have control points, you'll want to use the vector bender to perform a 'rubber sheet'.
Vector bender is not in the plugin repository, so you'll need to install it manually by adding it to C:\Users.qgis2\python\plugins\VectorBender.
Second option (may only work for rasters, haven't tested with vectors) - In the georeferencer, you can save your control points to a file. You can also load control points from a file.
Here is what you need to do:
Save the cpg file. Change the extension to .csv. Open the file in excel. Swap the values in the to/from columns. Save. Change the extension back to .points.
Once you have the control points set up, you can transform back and forth using the control points and your preferred transformation method.