python 3.5.6 export PATH=/root/anaconda3/bin:$PATH python -c "import cntk; print(cntk.__version__)" 新的名字：conda-cntk-pass cntk2.7 theano caffe2 直接使用 python 3.6.9 import caffe2 source activate tensorflow 1.11.0 新的名字 docker commit ba9743bcfc7d gpu-tensflow-1.11:1.11.0 keras export PATH=/root/anaconda3/bin:$PATH conda env list source activate keras python3.5 tensorflow 1.11.0 keras 2.2.2 nvidia-docker run -it --rm pytorch-gpu:1.1.0 /bin/bash pytorch 直接使用 [root@191ddd30d4ae /]# python Python 3.6.9 |Anaconda, Inc.| (default, Jul 30 2019, 19:07:31) [GCC 7.3.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import torch >>> print(torch.__version__) 1.1.0 >>> print(torch.cuda.device_count()) >>> print(torch.cuda.is_available()) ValueError: You are trying to use the old GPU back-end. It was removed from Theano. Use device=cuda* now. See https://github.com/Theano/Theano/wiki/Converting-to-the-new-gpu-back-end%28gpuarray%29 for more information ————————— vim ~/.bashrc 2：添加如下命令： export THEANO_FLAGS='mode=FAST_RUN,device=cpu,floatX=float32' 3：使修改的theano设置生效： source ~/.bashrc 4：编辑theano对于gpu的配置文件： vim ~/.theanorc 5：添加内容如下： [global] device = cuda floatX=float32 [nvcc] flags=--machine=64 [lib] cnmem=100 gpu-theano-in-use:1.0.4 python2.7 source activate theano python test.py >>> import theano /root/anaconda3/envs/theano/lib/python2.7/site-packages/theano/gpuarray/dnn.py:184: UserWarning: Your cuDNN version is more recent than Theano. If you encounter problems, try updating Theano or downgrading cuDNN to a version >= v5 and <= v7. warnings.warn("Your cuDNN version is more recent than " Using cuDNN version 7603 on context None Mapped name None to device cuda: GeForce GTX 960M (0000:01:00.0) >>> theano.__version__ u'1.0.4' https://www.jianshu.com/p/4cc75a79dce9 Linux下安装miniconda 在官网下载miniconda3 执行：bash Miniconda3-latest-Linux-x86_64.sh 　之后跟随提示步骤，安装过程中可以自动添加路径到配置文件，也可以之后进行配置。在这期间输入 yes　 no　　　（在这里我是之后配置的所以执行３） 将其添加到大环境变量中去 －vim ~/.bashrc －export PATH=~/anaconda3/bin:$PATH －source ~/.bashrc 创建虚拟环境并安装theano (主要参考官网教程http://deeplearning.net/software/theano/install_ubuntu.html) 基于python2.7创建一个名为theano的环境： conda create --name theano python=2.7 进入虚拟环境： source activate theano －使用conda安装：conda install numpy scipy mkl pip install parameterized conda install theano pygpu －使用pip安装：pip install Theano Install and configure the GPU drivers (这一步我没有尝试，因为本来就安装好了) 配置theanoGPU环境 vim ~/.theanorc 在空白文件中添加 [global] floatX = float32 device = gpu3 [lib] cnmem = 0.6 意味着有百分之60的显存分给当前终端 也可以不用5，直接在运行的时候使用命令：THEANO_FLAGS='device=cuda,floatX=float32' 默认为cuda0） test.py 文件： from theano import function, config, shared, tensor import numpy import time vlen = 10 * 30 * 768 # 10 x #cores x # threads per core iters = 1000 rng = numpy.random.RandomState(22) x = shared(numpy.asarray(rng.rand(vlen), config.floatX)) f = function([], tensor.exp(x)) print(f.maker.fgraph.toposort()) t0 = time.time() for i in range(iters): r = f() t1 = time.time() print("Looping %d times took %f seconds" % (iters, t1 - t0)) print("Result is %s" % (r,)) if numpy.any([isinstance(x.op, tensor.Elemwise) and ('Gpu' not in type(x.op).__name__) for x in f.maker.fgraph.toposort()]): print('Used the cpu') else: print('Used the gpu') caffe2 https://blog.csdn.net/qq_35451572/article/details/79428167 cmake \ -DCUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda-9.0 \ -DCUDNN_ROOT_DIR=/usr/local/cuda # To check if Caffe2 build was successful python -c 'from caffe2.python import core' 2>/dev/null && echo "Success" || echo "Failure" # To check if Caffe2 GPU build was successful # This must print a number > 0 in order to use Detectron python -c 'from caffe2.python import workspace; print(workspace.NumCudaDevices())' https://blog.csdn.net/Yan_Joy/article/details/70241319 https://www.nvidia.com/en-gb/data-center/gpu-accelerated-applications/caffe2/ https://blog.csdn.net/qq_35451572/article/details/79428167 https://blog.csdn.net/qq_16525279/article/details/79724728 https://blog.csdn.net/y_f_raquelle/article/details/83278953 https://www.cnblogs.com/nanzhao/p/9596844.html conda create -n xx --clone nn(已经存在的虚拟环境) tensorflow conda env list source activate tensorflow pip install tensorflow==1.11.0 python import tensorflow as tf 和 tf.__version__ 1.11.0 keras pip install tensorflow==1.11.0 pip install keras==2.2.2 conda env list source activate keras import keras 2.2.2 print(keras.__version__) import tensorflow as tf tf.__version__ 1.11.0 pytorch https://pytorch.org/get-started/locally/ 安装 pip3 install torch==1.2.0+cpu torchvision==0.4.0+cpu -f https://download.pytorch.org/whl/torch_stable.html 不行 conda install pytorch torchvision cpuonly -c pytorch -n pytorch import torch print(torch.__version__) print(torch.cuda.device_count()) print(torch.cuda.is_available()) 1.2.0 pip install https://cntk.ai/PythonWheel/CPU-Only/cntk-2.7.post1-cp35-cp35m-linux_x86_64.whl /root/anaconda3/bin/conda env list source activate cntk-py35 需要添加变量 python 3.5.6 export PATH=/root/anaconda3/bin:$PATH python -c "import cntk; print(cntk.__version__)" 新的名字：conda-cntk-pass cntk2.7 theano caffe2 直接使用 python 3.6.9 import caffe2 conda create -n caffe2 python=3.6 conda activate caffe2 conda install pytorch-nightly-cpu -c pytorch -n caffe2 python -c 'from caffe2.python import core' 2>/dev/null && echo "Success" || echo "Failure" pip install protobuf pip install future source activate tensorflow 1.11.0 新的名字 docker commit ba9743bcfc7d gpu-tensflow-1.11:1.11.0 keras export PATH=/root/anaconda3/bin:$PATH conda env list source activate keras python3.5 tensorflow 1.11.0 keras 2.2.2 nvidia-docker run -it --rm pytorch-gpu:1.1.0 /bin/bash pytorch 直接使用 conda install pytorch torchvision cudatoolkit=9.0 -c pytorch conda install pytorch torchvision -c pytorch -n pytorch [root@191ddd30d4ae /]# python Python 3.6.9 |Anaconda, Inc.| (default, Jul 30 2019, 19:07:31) [GCC 7.3.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import torch >>> print(torch.__version__) 1.1.0 >>> print(torch.cuda.device_count()) >>> print(torch.cuda.is_available()) ValueError: You are trying to use the old GPU back-end. It was removed from Theano. Use device=cuda* now. See https://github.com/Theano/Theano/wiki/Converting-to-the-new-gpu-back-end%28gpuarray%29 for more information ————————— vim ~/.bashrc 2：添加如下命令： export THEANO_FLAGS='mode=FAST_RUN,device=cpu,floatX=float32' 3：使修改的theano设置生效： source ~/.bashrc 4：编辑theano对于gpu的配置文件： vim ~/.theanorc 5：添加内容如下： [global] device = cuda floatX=float32 [nvcc] flags=--machine=64 [lib] cnmem=100 gpu-theano-in-use:1.0.4 python2.7 source activate theano python test.py >>> import theano /root/anaconda3/envs/theano/lib/python2.7/site-packages/theano/gpuarray/dnn.py:184: UserWarning: Your cuDNN version is more recent than Theano. If you encounter problems, try updating Theano or downgrading cuDNN to a version >= v5 and <= v7. warnings.warn("Your cuDNN version is more recent than " Using cuDNN version 7603 on context None Mapped name None to device cuda: GeForce GTX 960M (0000:01:00.0) >>> theano.__version__ u'1.0.4' https://www.jianshu.com/p/4cc75a79dce9 Linux下安装miniconda 在官网下载miniconda3 执行：bash Miniconda3-latest-Linux-x86_64.sh 　之后跟随提示步骤，安装过程中可以自动添加路径到配置文件，也可以之后进行配置。在这期间输入 yes　 no　　　（在这里我是之后配置的所以执行３） 将其添加到大环境变量中去 －vim ~/.bashrc －export PATH=~/anaconda3/bin:$PATH －source ~/.bashrc 创建虚拟环境并安装theano (主要参考官网教程http://deeplearning.net/software/theano/install_ubuntu.html) 基于python2.7创建一个名为theano的环境： conda create --name theano python=2.7 进入虚拟环境： source activate theano －使用conda安装：conda install numpy scipy mkl pip install parameterized conda install theano pygpu －使用pip安装：pip install Theano Install and configure the GPU drivers (这一步我没有尝试，因为本来就安装好了) 配置theanoGPU环境 vim ~/.theanorc 在空白文件中添加 [global] floatX = float32 device = gpu3 [lib] cnmem = 0.6 意味着有百分之60的显存分给当前终端 也可以不用5，直接在运行的时候使用命令：THEANO_FLAGS='device=cuda,floatX=float32' （默认为cuda0） test.py 文件： from theano import function, config, shared, tensor import numpy import time vlen = 10 * 30 * 768 # 10 x #cores x # threads per core iters = 1000 rng = numpy.random.RandomState(22) x = shared(numpy.asarray(rng.rand(vlen), config.floatX)) f = function([], tensor.exp(x)) print(f.maker.fgraph.toposort()) t0 = time.time() for i in range(iters): r = f() t1 = time.time() print("Looping %d times took %f seconds" % (iters, t1 - t0)) print("Result is %s" % (r,)) if numpy.any([isinstance(x.op, tensor.Elemwise) and ('Gpu' not in type(x.op).__name__) for x in f.maker.fgraph.toposort()]): print('Used the cpu') else: print('Used the gpu') # To check if Caffe2 build was successful python -c 'from caffe2.python import core' 2>/dev/null && echo "Success" || echo "Failure" # To check if Caffe2 GPU build was successful # This must print a number > 0 in order to use Detectron python -c 'from caffe2.python import workspace; print(workspace.NumCudaDevices())' https://blog.csdn.net/Yan_Joy/article/details/70241319 https://www.nvidia.com/en-gb/data-center/gpu-accelerated-applications/caffe2/ https://blog.csdn.net/qq_35451572/article/details/79428167 https://blog.csdn.net/qq_16525279/article/details/79724728 https://blog.csdn.net/y_f_raquelle/article/details/83278953 https://www.cnblogs.com/nanzhao/p/9596844.html Install NLTK: run pip install --user -U nltk Install Numpy (optional): run pip install --user -U numpy Test installation: run python then type import nltk joblib (>= 0.11) If you already have a working installation of numpy and scipy, the easiest way to install scikit-learn is using pip pip install -U scikit-learn or conda: conda install scikit-learn conda create -n caffe_gpu -c defaults python=3.6 caffe-gpu conda create -n caffe -c defaults python=3.6 caffe import caffe python -c "import caffe; print dir(caffe)" https://blog.csdn.net/weixin_37251044/article/details/79763858 一、编译Caffe、PyCaffe URL : https://github.com/BVLC/caffe.git 1.下载Caffe git clone https://github.com/BVLC/caffe.git cd caffe 注意：如果想在anaconda下使用，就先 source activate caffe_env 然后在这个环境下安装 利用anaconda2随意切换proto的版本，多proto并存，protobuf，libprotobuf 2.编译caffe 用cmake默认配置： [注意]：一般需要修改config文件。 进入caffe根目录 mkdir build cd build cmake .. make all -j8 make install make runtest -j8 3.安装pycaffe需要的依赖包，并编译pycaffe cd ../python conda install cython scikit-image ipython h5py nose pandas protobuf pyyaml jupyter for req in $(cat requirements.txt); do pip install $req; done cd ../build make pycaffe -j8 4.添加pycaffe的环境变量 终端输入如下指令： vim ~/.bashrc 在最后一行添加caffe的python路径(到达vim最后一行快捷键：Shift+G)： export PYTHONPATH=/path/to/caffe/python:$PYTHONPATH 注意： /path/to/caffe是下载的Caffe的根目录，例如我的路径为：/home/Jack-Cui/caffe-master/python Source环境变量，在终端执行如下命令： source ~/.bashrc 注意： Source完环境变量，会退出testcaffe这个conda环境，再次使用命令进入即可。 执行如下命令： python -c "import caffe; print dir(caffe)" fatal error: pyconfig.h: No such file or directory 如果使用的是系统的python路径，解决方法如下： make clean export CPLUS_INCLUDE_PATH=/usr/include/python2.7 make all -j8 如果使用的是anaconda Python，路径如下： export CPLUS_INCLUDE_PATH=/home/gpf/anaconda3/include/python3.6m http://blog.csdn.net/GPFYCF521/article/details/80387869 cd /usr/local/src/caffe-master/ 2 ll 3 make pycaffe 4 find / -name "Python.h" 5 export CPLUS_INCLUDE_PATH=/usr/local/src/Python-3.6.4/Include/Python.h:$CPLUS_INCLUDE_PATH 6 make clean 7 make pycaffe 8 export CPLUS_INCLUDE_PATH=/usr/local/src/Python-3.6.4/Include/:$CPLUS_INCLUDE_PATH 9 make clean 10 make pycaffe 11 export CPLUS_INCLUDE_PATH= 12 export CPLUS_INCLUDE_PATH=/usr/local/src/Python-3.6.4/Include/:$CPLUS_INCLUDE_PATH 13 make clean 14 make pycaffe 15 find / -name "pyconfig.h" 16 yum install python-devel.x86_64 17 make clean 18 make pycaffe 19 find python3.6 20 locate python3.6 21 make clean 22 export CPLUS_INCLUDE_PATH=/usr/include/python2.7 23 export CPLUS_INCLUDE_PATH= 24 export CPLUS_INCLUDE_PATH=/root/anaconda3/include/python3.5m 25 make all 26 find / -name "pycaffe" 27 history 装的是python3.6，项目中用到boost相关代码，编译时找不到pyconfig.h。看了一下/usr/include/python3.6和/usr/include/python3.6m，都只有一个pyconfig-64.h文件。 网上查了一圈，找了各种方法都搞不定，其中一种方法可以安装一堆.h进/usr/include/python2.7，3.6文件夹中还是没有。方法如下： 1. 可以先查看一下含python-devel的包     yum search python | grep python-devel 2. 64位安装python-devel.x86_64，32位安装python-devel.i686，我这里安装:     sudo yum install python-devel.x86_64 受此启发，输入命令查找3.6版本相关的python包 yum search python | grep python36 发现下面这个应该是我们想要的 python36u-devel.x86_64 : Libraries and header files needed for Python yum install python36u-devel.x86_64 conda create -n caffe_gpu -c defaults python=3.5 caffe-gpu conda create -n caffe -c defaults python=3.5 caffe cd ../python conda install cython scikit-image ipython h5py nose pandas protobuf pyyaml jupyter for req in $(cat requirements.txt); do pip install $req; done cd ../build make pycaffe -j8 4.添加pycaffe的环境变量 终端输入如下指令： vim ~/.bashrc 在最后一行添加caffe的python路径(到达vim最后一行快捷键：Shift+G)： export PYTHONPATH=/path/to/caffe/python:$PYTHONPATH 注意： /path/to/caffe是下载的Caffe的根目录，例如我的路径为：/home/Jack-Cui/caffe-master/python Source环境变量，在终端执行如下命令： source ~/.bashrc 注意： Source完环境变量，会退出testcaffe这个conda环境，再次使用命令进入即可。 执行如下命令： python -c "import caffe; print dir(caffe)" 输出结果如下： 注意： 如果创建了conda环境，每次想要使用caffe，需要先进入这个创建的conda环境。 export PATH=/root/anaconda3/bin:$PATH conda create -n caffe -c defaults python=3.5 conda install caffe-gpu conda install tensorflow-gpu==1.11.0 conda create --name tensorflow python=3.5 source activate tensorflow source deactivate conda remove -n tensorflow --all import tensorflow as tf 和 tf.__version__ 您正在使用GPU版本。您可以列出可用的tensorflow设备 from tensorflow.python.client import device_lib print(device_lib.list_local_devices()) conda 安装pytorch conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/free/ conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/pytorch/ 添加清华源 命令行中直接使用以下命令 conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/free/ conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/ conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/msys2/ # 设置搜索时显示通道地址 conda config --set show_channel_urls yes ———————————————————————————————————————————————————————————————————————————————— 设置搜索时显示通道地址 | conda config --set show_channel_urls yes conda GPU的命令如图所示： conda install pytorch torchvision -c pytorch conda CPU的命令如图所示： conda install pytorch-cpu -c pytorch pip3 install torchvision pytorch-gpu conda install pytorch torchvision cudatoolkit=9.0 -c pytorch import torch print(torch.__version__) print(torch.cuda.device_count()) print(torch.cuda.is_available()) --------------------------------------------------------------------------------| conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/free/ conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/ conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge/ conda config --set show_channel_urls yes 查看已经添加的channels conda config --get channels 已添加的channel在哪里查看 vim ~/.condarc conda search gatk 安装完成后，可以用“which 软件名”来查看该软件安装的位置： which gatk 如需要安装特定的版本: conda install 软件名=版本号 conda install gatk=3.7 查看已安装软件: conda list 更新指定软件: conda update gatk 卸载指定软件: conda remove gatk https://blog.csdn.net/Jonms/article/details/79550512 ubuntu1604 cuda -cudnn 接着，运行下面的命令安装anaconda $ sh Anaconda3-5.1.0-Linux-x86_64.sh anaconda的安装很简单，这里就不多描述。 CNTK需要你的系统安装有OpenMPI。在Ubuntu中可以通过以下命令安装 $ sudo apt install openmpi-bin 然后，创建名为cntk-py35的虚拟环境 $ conda create --name cntk-py35 python=3.5 numpy scipy h5py jupyter 激活cntk虚拟环境 $ source activate cntk-py35 关闭cntk虚拟环境 $ source deactivate 激活虚拟环境后，用pip安装CNTK(GPU)即可 $ pip install https://cntk.ai/PythonWheel/GPU/cntk-2.4-cp35-cp35m-linux_x86_64.whl 测试CNTK是否安装成功并输出CNTK版本 $ python -c "import cntk; print(cntk.__version__)" pip install https://cntk.ai/PythonWheel/CPU-Only/cntk-2.7.post1-cp35-cp35m-linux_x86_64.whl python -c "import cntk; print(cntk.__version__)" ImportError: No module named 'cntk._cntk_py' ImportError: libpython3.5m.so.1.0: cannot open shared object file: No such file or directory find / -name "libpython3.5m.so.1.0" 找到路径 使用conda安装的 /root/anaconda3/envs/cntk-py35/lib/ 加入环境变量 #cd /etc/ld.so.conf.d #vim python3.conf 将编译后的python/lib地址加入conf文件 #ldconfig 容器环境变量会丢失，使用dockerfile重新赋值。 export PATH=/root/anaconda3/bin:$PATH 上面的链接库配置 pip https://cntk.ai/PythonWheel/CPU-Only/cntk-2.7.post1-cp36-cp36m-linux_x86_64.whl theano apt-get install python-numpy python-scipy python-dev python-pip python-nose g++ libopenblas-dev pip install Theano NumPy (~30s): python -c "import numpy; numpy.test()" SciPy (~1m): python -c "import scipy; scipy.test()" Theano (~30m): python -c "import theano; theano.test()" 已安装cuda export PATH=/usr/local/cuda-5.5/bin:$PATH export LD_LIBRARY_PATH=/usr/local/cuda-5.5/lib64:$LD_LIBRARY_PATH # to test nvidia-docker run -it caffe2ai/caffe2:latest python -m caffe2.python.operator_test.relu_op_test # to interact nvidia-docker run -it caffe2ai/caffe2:latest /bin/bash python -c 'from caffe2.python import core' 2>/dev/null && echo "Success" || echo "Failure" #返回Success就OK python2 -c 'from caffe2.python import workspace; print(workspace.NumCudaDevices())' #返回1就OK #进入python输入 from caffe2.python import workspace ModuleNotFoundError: No module named 'google' pip install protobuf ModuleNotFoundError: No module named 'past' pip install future 安装后检测 python -c 'from caffe2.python import core' 2>/dev/null && echo "Success" || echo "Failure" gpu检测 python -m caffe2.python.operator_test.relu_op_test Python2.7和Python3.6下都可以，不过只是cpu版本，只限于Mac和Ubuntu平台下： conda install -c caffe2 caffe2 参考网址： https://blog.csdn.net/qq_35451572/article/details/79428167 https://blog.csdn.net/Yan_Joy/article/details/70241319 https://blog.csdn.net/zmm__/article/details/90285887 https://blog.csdn.net/u013842516/article/details/80604409