Numpy is installed by default in QGIS (Standalone version, OSGeo4W version).
For rasterio, there are many problems. I know that rasterio is powerful and easy to use (I use it) but why rasterio in a QGIS plugin while you have all the functions of PyQGIS ? (it was designed for the Python shell or for scripts, not to be used with QGIS/PyQGIS, as Fiona for the shapefiles)
In Linux or Mac OS X (Unix systems), QGIS uses the standard Python installation (independent of QGIS) and if is easy to install or upgrade a Python module (there are also many compilers installed by default)
In Windows, Python is not installed by default, and QGIS installs his own version. To install pure Python modules (without C, C++ libraries) you have solutions (QGIS Standalone and the Python Modules, Installing Python setuptools into OSGeo4W Python or Installing PySAL for OSGeo4W)
But for modules that needs to be compiled (no compiler by default), it is not easy. You can try to copy/paste your installed module in the QGIS Python site_folder but there are problems.
Rasterio need to be compiled against an installed GDAL library and if you install it from Christoph Gohlke Unofficial Windows Binaries for Python Extension Packages, for example, I am not sure that the version of GDAL used by QGIS is the same (for the references to the dll libraries)
If you use exclusively numpy and rasterio, why not an independent script ?