I've managed to break the ability to use Notebooks in ArcGIS Pro by installing the deep learning modules in a cloned environment.
These are the dependencies:
conda install -c esri -c fastai -c pytorch arcgis=1.8.2 scikit-image=0.15.0 pillow=6.2.2 libtiff=4.0.10 fastai=1.0.60 pytorch=1.4.0 torchvision=0.5.0 --no-pin
The cloned environment works fine in an IDE, but notebooks refuse to open in Pro. I'm assuming one of the dependencies is the problem, is there a way to re-activate notebooks in Pro?