先概括一下深度學習的幾大流行的框架:Pylearn2, Theano, Caffe, Torch, Cuda-covnet,Deeplarning4j等。
Caffe的網站上提供了安裝說明。由於其依賴的庫比較多,通常安裝過程會出現許多問題,在不同的機器和操作系統上可能遇到不同的問題。安裝時可以根據網站上提供的說明步驟進行,遇到有問題時用Google搜索一下基本都能找到。本文記錄了筆者在Mac上安裝遇到的問題和解決辦法。系統版本:OS X 10.9.5。
1.1 安裝CUDA。推薦7.0以上版本,6.*版本也可以。我安裝的是最新版CUDA 7.5。
1.2 安裝BLAS。這裡我使用了OpenBLAS。推薦使用brew安裝:brew install openblas
1.3 安裝Boost。
通過brew install boost默認安裝版本為1.60。但建議使用1.59。因為1.60編譯後可能會出現問題。
$ brew search boost boost homebrew/versions/boost-python159 ✔ boost-bcp homebrew/versions/boost149 boost-build homebrew/versions/boost150 boost-python homebrew/versions/boost155 homebrew/science/boost-compute homebrew/versions/boost159 ✔ Caskroom/cask/iboostup Caskroom/cask/turbo-boost-switcher Caskroom/cask/pivotalbooster $ brew install –build-from-source homebrew/versions/boost159
安裝好後可以後在/usr/local/opt/boost159下看到該庫。Caffe中把某些依賴庫所在的文件夾名字限定為boost,可以將/usr/local/opt/boost159復制粘貼產生備份,將備份改名為/usr/local/opt/boost。
1.4 安裝CuDNN。下載cuDNN v5.0版本。解壓後將include和bin文件夾中的內容分別復制到/usr/lcoal 下面的/include和/bin中。
1.5 使用brew install 分別安裝 protobuf, glog, gflags, hdf5, snappy, leveldb, szip, lmdb等。
如果使用python, protobuf安裝命令為
$ brew install --build-from-source --with-python -vd protobuf<code> </code><code></code>
1.6 (可選)OpenCV, 我使用2.4.6版本。
1.7 (可選)Python 版本:2.7。
需要安裝numpy。推薦使用Anaconda,裡面包含了一個python版本2.7.11並且包含了大多數所需要的庫,包括hdf5、numpy等。Anaconda默認安裝在$(HOME)/anaconda目錄下。
還需要安裝python-boost。與boost類似的方法,推薦1.59版本。
1.8 (可選)Matlab 版本 2015a
2.1 下載Caffe後在caffe-master文件夾下,以Makefile.config.example為模板根據第一步中的安裝情況,建立配置文件Makefile.config,內容如下:
## Refer to http://caffe.berkeleyvision.org/installation.html # Contributions simplifying and improving our build system are welcome! # cuDNN acceleration switch (uncomment to build with cuDNN). USE_CUDNN := 1 # CPU-only switch (uncomment to build without GPU support). # CPU_ONLY := 1 # uncomment to disable IO dependencies and corresponding data layers # USE_OPENCV := 0 # USE_LEVELDB := 0 # USE_LMDB := 0 # uncomment to allow MDB_NOLOCK when reading LMDB files (only if necessary) # You should not set this flag if you will be reading LMDBs with any # possibility of simultaneous read and write ALLOW_LMDB_NOLOCK := 1 # Uncomment if you're using OpenCV 3 OPENCV_VERSION := 2.4 # To customize your choice of compiler, uncomment and set the following. # N.B. the default for Linux is g++ and the default for OSX is clang++ CUSTOM_CXX := clang++ # CUDA directory contains bin/ and lib/ directories that we need. CUDA_DIR := /usr/local/cuda # On Ubuntu 14.04, if cuda tools are installed via # "sudo apt-get install nvidia-cuda-toolkit" then use this instead: # CUDA_DIR := /usr # CUDA architecture setting: going with all of them. # For CUDA < 6.0, comment the *_50 lines for compatibility. CUDA_ARCH := -gencode arch=compute_20,code=sm_20 \ -gencode arch=compute_20,code=sm_21 \ -gencode arch=compute_30,code=sm_30 \ -gencode arch=compute_35,code=sm_35 \ -gencode arch=compute_50,code=sm_50 \ -gencode arch=compute_50,code=compute_50 # BLAS choice: # atlas for ATLAS (default) # mkl for MKL # open for OpenBlas BLAS := open # Custom (MKL/ATLAS/OpenBLAS) include and lib directories. # Leave commented to accept the defaults for your choice of BLAS # (which should work)! # BLAS_INCLUDE := /usr/local/opt/openblas/include # BLAS_LIB := /usr/local/opt/openblas/lib # Homebrew puts openblas in a directory that is not on the standard search path BLAS_INCLUDE := $(shell brew --prefix openblas)/include BLAS_LIB := $(shell brew --prefix openblas)/lib # This is required only if you will compile the matlab interface. # MATLAB directory should contain the mex binary in /bin. # MATLAB_DIR := /usr/local MATLAB_DIR := /Applications/MATLAB_R2015a.app # NOTE: this is required only if you will compile the python interface. # We need to be able to find Python.h and numpy/arrayobject.h. #PYTHON_INCLUDE := /usr/include/python2.7 \ /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/core/include # Anaconda Python distribution is quite popular. Include path: # Verify anaconda location, sometimes it's in root. ANACONDA_HOME := $(HOME)/anaconda PYTHON_INCLUDE := $(ANACONDA_HOME)/include \ $(ANACONDA_HOME)/include/python2.7 \ $(ANACONDA_HOME)/lib/python2.7/site-packages/numpy/core/include \ # Uncomment to use Python 3 (default is Python 2) # PYTHON_LIBRARIES := boost_python3 python3.5m # PYTHON_INCLUDE := /usr/include/python3.5m \ # /usr/lib/python3.5/dist-packages/numpy/core/include # We need to be able to find libpythonX.X.so or .dylib. #PYTHON_LIB := /usr/lib PYTHON_LIB := $(ANACONDA_HOME)/lib # Homebrew installs numpy in a non standard path (keg only) # PYTHON_INCLUDE += $(dir $(shell python -c 'import numpy.core; #print(numpy.core.__file__)'))/include # PYTHON_LIB += $(shell brew --prefix numpy)/lib #PYTHON_LIB +=/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/core/lib PYTHON_LIB +=$(ANACONDA_HOME)/lib/python2.7/site-packages/numpy/core/lib # Uncomment to support layers written in Python (will link against Python libs) WITH_PYTHON_LAYER := 1 # Whatever else you find you need goes here. INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/Cellar/boost159/1.59.0/include LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/cuda/lib /usr/local/Cellar/boost159/1.59.0/lib /usr/local/opt/boost-python159/lib # If Homebrew is installed at a non standard location (for example your home directory) and you use it for general dependencies INCLUDE_DIRS += $(shell brew --prefix)/include LIBRARY_DIRS += $(shell brew --prefix)/lib # Uncomment to use `pkg-config` to specify OpenCV library paths. # (Usually not necessary -- OpenCV libraries are normally installed in one of the above $LIBRARY_DIRS.) # USE_PKG_CONFIG := 1 BUILD_DIR := build DISTRIBUTE_DIR := distribute # Uncomment for debugging. Does not work on OSX due to https://github.com/BVLC/caffe/issues/171 # DEBUG := 1 # The ID of the GPU that 'make runtest' will use to run unit tests. TEST_GPUID := 0 # enable pretty build (comment to see full commands) Q ?= @
2.2,命令行進入caffe-master文件夾下,運行:
$ make all
出現問題:
PROTOC src/caffe/proto/caffe.proto make: protoc: No such file or directory
解決辦法: 需要用brew建立protobuf的鏈接。為此,運行
$ brew link protobuf
如果運行上述命令又出問題,比如brew的權限問題:permission denied for /usr/local。需要設置一下權限,更新一下brew, 為此,運行
$ sudo chown -R $USER:admin /usr/local $ cd /usr/local $ git reset --hard origin/master $ brew update
上述問題可以得到解決。
2.3 上一步通過後,運行
$ make test
這一步沒問題。將build_release/lib下的所有文件復制到/usr/local/lib
$ cp -a .build_release/lib/. /usr/local/lib/
再運行
$ make runtest
報錯:
.build_release/tools/caffe dyld: Library not loaded: @rpath/libcudart.7.5.dylib Referenced from: /Developer/caffe/.build_release/tools/caffe Reason: image not found
為此需要設置一下環境變量DYLD_FALLBACK_LIBRARY_PATH
$ export DYLD_FALLBACK_LIBRARY_PATH=/usr/local/cuda/lib:/usr/local/lib:$(HOME)/<span >anaconda/lib</span>
再運行make runtest,一切順利。
2.4 如果使用python,再運行
<pre name="code" class="html">$ make pycaffe $ make pytest
詳細步驟見:http://caffe.berkeleyvision.org/gathered/examples/mnist.html
3.1,下載mnist數據。在caffe-master目錄下運行
$ ./data/mnist/get_mnist.sh
3.2,建立訓練數據和測試數據,運行
$ ./examples/mnist/create_mnist.sh
出現以下錯誤,說convert_mnist_data.bin找不到:
Creating lmdb... ./examples/mnist/create_mnist.sh: line 16: build/examples/mnist/convert_mnist_data.bin: No such file or directory ./examples/mnist/create_mnist.sh: line 18: build/examples/mnist/convert_mnist_data.bin: No such file or directory Done.
解決辦法:搜索convert_mnist_data.bin發現該文件位於./distribute/bin目錄下,因此在在./examples/mnist/create_mnist.sh文件中將BUILD的值改為distribute/bin即可。
3.3,訓練和測試,運行:
$ ./examples/mnist/create_mnist.sh
如果出現和上面類似的錯誤,說caffe找不到 (caffe.bin位於./distribute/bin目錄下或者build/tools下),檢查create_mnist.sh的內容,保證caffe.bin的路徑正確
./distribute/bin/caffe.bin train--solver=examples/mnist/lenet_solver.prototxt
然後就能看到運行結果了。
詳見:http://nbviewer.jupyter.org/github/BVLC/caffe/blob/master/examples/00-classification.ipynb
該例子用Caffe中已經訓練好的模型(基於Alexnet的結構)對圖像進行分類。並且可以顯示不同層中訓練得到的特征。
本文永久更新鏈接地址:
http://xxxxxx/Linuxjc/1141044.html TechArticle