As I understand, here is your setup:
- A: raster
- B: raster, same size as A
- nodata_b: nodata value for B; 0 in this instance
Your goal is to create raster C, which is A masked by valid B.
C = A where (B != nodata_b) else nodata_a
The question now concerns what are valid possibilities for nodata_a. This is something that you don't give, so there are a few options. Remember that rasters are required to have a value for each pixel. The nodata tag is just there to help interpret such values within the raster array as having a special property (nodata). You need to determine what nodata_a should be.
If nodata_a is allowed to be 0, this is pretty trivial.
gdal_calc.py -A rasterA.tif -B rasterB.tif --outfile=rasterC.tif \
--calc ="A*(B!=0)" --NoDataValue=0
Here, B!=0 is translated into 0 for False and 1 for True on a pixel-by-pixel basis. So 1*value = value and 0*value = 0.
If nodata_a needs to be a specific value, things become a little more tricky. Let's say 0 is a meaningful value in raster A. Setting nodata_a to 0 would mean that meaningful data is absent in raster C. Maybe there are no meaningful negative values in A (and the datatype of the raster permits negative values). So if nodata_a==-999.
gdal_calc.py -A rasterA.tif -B rasterB.tif --outfile=rasterC.tif \
--calc ="A*(B!=0) + ((B==0)*-999)" --NoDataValue=-999
Here, there are two parts to the calculation. On the left is the same calculation as above -- A where B!=0 else 0. On the right, the result is either 0 for valid pixels or -999 for invalid. Summing the left and right values will result in the correct values of A where valid and -999 where invalid.
You could of course substitute any value you want for -999, as long as that value is guaranteed to not be present in the valid portion of A.