A workaround could be using R scripts in QGIS.
For your specific purpose use this script:
##LayerA = raster
##LayerB = raster
##LayerC = raster
##LayerD = raster
##output = output raster
library(raster)
calc(stack(LayerA, LayerB, LayerC,LayerD), fun = mean, na.rm = T)
And you'll be able to compute NDVI mean taking account No Data
pixels (masked pixels).
Requirements:
Rasters should have the same extent and pixel resolution. You can solve this before with GDAL, SAGA or GRASS tools or add a normalization step in R script.
Notes:
If you stack layers before, you can define only one input file and work with several layers (from 2 to hundreds of layers or even more, depending on the amount of memory required) with minor changes (deleting the others inputs)