Here you can find an article concerning the OverviewMap Control.
I'll cite the relevant paragraph:
[If] you'd like to change the criteria by which the overview map decides when to update itself, you can override the isSuitableOverview() method. To do this, first construct the control, then modify any properties/methods, then add the control to your map:
var ovControl = new OpenLayers.Control.OverviewMap();
ovControl.isSuitableOverview = function() {
return false;
};
map.addControl(ovControl);
This will cause the overview map to update itself with each move of the main map - so the extent rectangle will always stay centered. If the same function always returns true, then the overview map will never zoom or recenter itself.
So, your problem should be solved with the following code:
var ovControl = new OpenLayers.Control.OverviewMap();
ovControl.isSuitableOverview = function() {
return true;
};
map.addControl(ovControl);