You're really close with your second attempt. What you need to do is create the bounding box for each set of features, then connect the points from one box to the other. That creates the control vectors for the AffineWarper.
Here's my workspace:
Ignore the contents of the bookmark. That's just me setting up some data. Anyway, I create the bounding box for a raster feature and a set of roads. Then I chop them into single points.
The next step is to create vectors from them. I give each point an ID with a Counter (each Counter transformer must have a different counter name) so that we have points 1,2,3,4,5 for each bounding box. Then I create an ID number so we know which points come from which set of data.
Then I sort the data. I sort by counter ID, and ID number. Now I have BoundingBox1-pt1, BoundingBox2-pt1, BB1-pt2, BB2-pt2, BB1-pt3, etc in the correct order.
The LineBuilder builds lines, breaking them each time the Counter changes.
Finally we have our control vectors which are passed into the AffineWarper along with the original data.
If the results don't get warped, then the control vectors are probably pointing in the wrong direction. The simple fix there is to change the direction of sorting in the Sorter (ascending instead of descending, or vice versa).
You can download my workspace and data from Dropbox at: https://www.dropbox.com/s/bo8qpvn3p7f91xm/AffineWarping.fmwt?dl=1 (sorry I just realized I built it in FME2019. If you need an earlier version, let me know).