I have both Python 2.7 with ArcGIS 10.5 Desktop and Python 3.6 with ArcGIS Pro. A few weeks back, I tried to install Anaconda (w/ Conda, Spyder, Jupyter, etc..) and it somehow overwrote my system settings for Python 2.7 (as well as pip). I noticed that I could no longer use pip or import stuff in 2.7. I uninstalled Anaconda and the updated version of pip (it somehow updated the pip version) and everything in 2.7 worked again. Any suggestions on reinstalling Anaconda without wrecking my Python 2.7 development environment again?
You could try installing through the Python Package Manger in ArcGIS Pro. Go to Project, Python, and Add Packages. You will find anaconda there as an option.
An alternative option would be to use the conda package manager to get the packages of your choice. ArcGIS Pro 1.3 and greater ship with conda installed. This way you won't be mixing your pip and conda packages, and the conda installs will only be associated with Pro. You will find that some of the packages you mentioned will require the others as dependencies, and you will probably get all the packages you want rather quickly, even without the full anaconda distribution. I use the packages you mentioned, downloaded using conda for 3.6 with Pro, and it didn't affect my Desktop license or Python 2.7.
Here is another useful link that talks about conda as it relates to ArcGIS Pro.
I remember having issues with pip, but haven't had any issues since I started managing my packages with conda. Hope that helps.