What finally worked for @RobotCaleb was the following:

1. Discard the alpha channel:

    gdal_translate -mask4 stadium3857.tif stadium3857_RGB.tif
   

2. Compute a new alpha channel from NoData values:

    gdalwarp -dstalpha stadium3857_RGB.tif stadium3857_RGBA.tif

edit: apparently gdal2tiles does not need an alpha channel in the input dataset but also converts NoData values into transparency, so step 2 is not necessary

What finally worked for @RobotCaleb was the following:

1. Apply mask to alpha (band 4):

    gdal_translate -mask 4 stadium3857.tif stadium3857_RGB.tif
   

2. Compute a new alpha channel from NoData values:

    gdalwarp -dstalpha stadium3857_RGB.tif stadium3857_RGBA.tif

edit: apparently gdal2tiles does not need an alpha channel in the input dataset but also converts NoData values into transparency, so step 2 is not necessary

What finally worked for @RobotCaleb was the following:

1. Discard the alpha channel:

    gdal_translate -mask4 stadium3857.tif stadium3857_RGB.tif
   

2. Compute a new alpha channel from NoData values:

    gdalwarp -dstalpha stadium3857_RGB.tif stadium3857_RGBA.tif
   

What finally worked for @RobotCaleb was the following:

1. Discard the alpha channel:

    gdal_translate -mask4 stadium3857.tif stadium3857_RGB.tif
   

2. Compute a new alpha channel from NoData values:

    gdalwarp -dstalpha stadium3857_RGB.tif stadium3857_RGBA.tif

edit: apparently gdal2tiles does not need an alpha channel in the input dataset but also converts NoData values into transparency, so step 2 is not necessary