If you want to interactivelly change your points coordinates, one by one, I believe you can do what you want by using the Numerical Vertex Edit Plugin. I can install it in Plugins > Fetch Python Plugins and search it by its name.
After installing you will notice a new icon on you digitalizing toolbar like this
. To make it active you must be editing a layer.
Click the plugin button, then click on the desired vertex (in your case the point feature). A dialog show the current coordinates of the vertex, and you can edit them.

Doing this to too many points can be quite boring.
UPDATE to QGIS 3.10
In more recent versions of QGIS, there are more options to edit a point or a vertex coordinate.
Using the vertex editor tool
- While editing, click the vertex tool
2.then right-click the feature you want to edit
- The vertex editor opens, you can change the coordinates of the vertexes

Using the advanced digitizing panel
If your layer is in a planimetric SRC, you can use the advanced digitizing panel.
- While editing, open the advanced digitizing panel (View > Panels > advanced digitizing panel)
- Enable advanced digitizing
- Click the vertex tool
- Click the vertex you want to move
- In the advanced digitizing panel, fill in the x and y coordinates and lock them. You can use the X and Y keys as shortcuts to fill the values and the Enter key to lock the values
- click the map canvas near the pink cross and the feature map and the node should move to the desired coordinates
