Bluemix虛擬機Docker使用direct-lvm存儲方式運行容器

最近在Bluemix的虛擬機上玩docker，但是發現默認安裝使用的是loop-lvm的模式做後端存儲，這個肯對對後面的實驗會造成影響，而且docker官方也不建議在生產環境下使用loop-lvm，下圖為docker官方給出的存儲方案優劣對比：

所以今天開題分享一下如何更改docker的devicemapper存儲方式為direct-lvm。

docker 最先是跑在ubuntu和debian上的,使用aufs存儲器. 由於docker越來越流行,許多公司希望在RHEL上使用,但是上游內核中沒有包括aufs,所以rhel不能使用aufs. 最終開發者們開發了一個新的後端存儲引擎devicemapper,基於已有的Device Mapper技術,並且使docker支持可插拔,現在全世界有很多真實案例在生產環境使用devicemapper。Device Mapper是Linux系統中基於內核的高級卷管理技術框架。Docker的devicemapper存儲驅動就是基於該框架的精簡置備和快照功能來實現鏡像和容器的管理。

devicemapper是Red Hat Enterprise Linux下Docker Engine的默認存儲驅動，它有兩種配置模式:loop-lvm和direct-lvm，loop-lvm是默認的模式，但如果是在生產環境的部署Docker，官方不推薦使用該模式。我們使用docker info命令可以看到以下警告:

 

WARNING: Usage of loopback devices is strongly discouraged for production use. Either use `–storage-opt dm.thinpooldev` or use `–storage-opt dm.no_warn_on_loop_devices=true` to suppress this warning.

 

direct-lvm是Docker推薦的生產環境的推薦模式，他使用塊設備來構建精簡池來存放鏡像和容器的數據。

本文的操作系統是Centos7.2，使用docker版本為1.12，devicemapper版本為device-mapper-1.02.107-5.el7.x86_64。

1.停止Docker服務

首先運行docker info 查看當前系統的docker配置：

 

# root at bastion.shanker in ~ [15:03:10]
# docker info
Containers: 6
 Running: 0
 Paused: 0
 Stopped: 6
Images: 40
Server Version: 1.10.3
Storage Driver: devicemapper
 Pool Name: docker-253:0-11668417-pool
 Pool Blocksize: 65.54 kB
 Base Device Size: 10.74 GB
 Backing Filesystem: xfs
 Data file: /dev/loop0
 Metadata file: /dev/loop1
 Data Space Used: 3.641 GB
 Data Space Total: 107.4 GB
 Data Space Available: 103.7 GB
 Metadata Space Used: 5.972 MB
 Metadata Space Total: 2.147 GB
 Metadata Space Available: 2.142 GB
 Udev Sync Supported: true
 Deferred Removal Enabled: true
 Deferred Deletion Enabled: true
 Deferred Deleted Device Count: 0
 Data loop file: /var/lib/docker/devicemapper/devicemapper/data
 WARNING: Usage of loopback devices is strongly discouraged for production use. Either use `--storage-opt dm.thinpooldev` or use `--storage-opt dm.no_warn_on_loop_devices=true` to suppress this warning.
 Metadata loop file: /var/lib/docker/devicemapper/devicemapper/metadata
 Library Version: 1.02.107-RHEL7 (2015-10-14)
Execution Driver: native-0.2
Logging Driver: journald
Plugins: 
 Volume: local
 Network: bridge null host
Kernel Version: 4.7.9-200.fc24.x86_64
Operating System: Fedora 24 (Twenty Four)
OSType: linux
Architecture: x86_64
Number of Docker Hooks: 2
CPUs: 8
Total Memory: 14.99 GiB
Name: bastion.shanker
ID: SWHZ:KSZ3:CQMS:W5HN:F33Z:HWU2:2GUE:OQII:7BSE:J62P:6EMX:RHL4
Username: shanker
Registry: https://index.docker.io/v1/
Registries: docker.io (secure)

 

 

發現Storage Driver是Devicemapper，Data File和Metadata File都是loop設備，下面我們將docker停掉：

 

#systemctl stop docker

 

2. 添加磁盤並創建thin-pool lv

 

查看新添加的磁盤

 

fdisk -l /dev/sdb

Disk /dev/sdb: 21.5 GB, 21474836480 bytes, 41943040 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

 

創建pv

 

# root at model.shanker in ~ [14:36:01]
# pvcreate /dev/sdb
  Physical volume "/dev/sdb" successfully created

 

創建vg

 

 

# root at model.shanker in ~ [14:37:49]
# vgcreate docker /dev/sdb
  Volume group "docker" successfully created

 

創建data lv

 

 

# root at model.shanker in ~ [14:41:00]
# lvcreate --wipesignatures y -n thinpool docker -l 95%VG 
  Logical volume "thinpool" created.

 

創建metadata lv

 

# root at model.shanker in ~ [14:41:06]
# lvcreate --wipesignatures y -n thinpoolmeta docker -l 1%VG
  Logical volume "thinpoolmeta" created.

 

注意作為meta的pool大小不能超過16GB！！！

 

將pool 轉換為thin-pool

將 thinpool lv 的 chunksize 改為 512KB,並且將前 4KB 字節清零。

 

 root at model.shanker in ~ [14:42:31]
# lvconvert -y --zero n -c 512k --thinpool docker/thinpool --poolmetadata docker/thinpoolmeta
  WARNING: Converting logical volume docker/thinpool and docker/thinpoolmeta to pool's data and metadata volumes.
  THIS WILL DESTROY CONTENT OF LOGICAL VOLUME (filesystem etc.)
  Converted docker/thinpool to thin pool.

 

創建一個thinpool的profile

 

 

# root at model.shanker in /etc/lvm/profile [14:45:42]
# vi docker-thinpool.profile

# root at model.shanker in /etc/lvm/profile [14:45:50]
# pwd
/etc/lvm/profile

# root at model.shanker in /etc/lvm/profile [14:45:53]
# cat docker-thinpool.profile 
activation {

thin_pool_autoextend_threshold=80

thin_pool_autoextend_percent=20

}

 

應用配置

 

 

# root at model.shanker in /etc/lvm/profile [14:45:56]
# lvchange --metadataprofile docker-thinpool docker/thinpool
  Logical volume "thinpool" changed.

 

注意： docker-thinpool 即剛才創建的 profile 文件名的前綴，不需要加.profile,而且要在