您现在的位置： Linux教程網 >> UnixLinux > >> Linux管理 >> Linux服務

Linux服務器文件刪除空間未釋放的問題，linux服務器

一、問題起源

在Linux系統中，通過rm刪除文件將會從文件系統的目錄結構上解除鏈接(unlink)，如果文件是被打開的（有一個進程正在使用），那麼進程將仍然可以讀取該文件磁盤空間也一直被占用

這樣就會導致我們明明刪除了文件，但是磁盤空間卻未被釋放

二、問題分析

1、首先獲得一個已經被刪除但是仍然被應用程序占用的文件列表

root@instance-zo0fkfng:~#  lsof|grep deleted 
init          1             root   10w      REG              253,1        283     155958 /var/log/upstart/network-manager.log.1 (deleted)
init          1             root   12w      REG              253,1        314     155934 /var/log/upstart/mountall.log.1 (deleted)
init          1             root   13w      REG              253,1        508     131203 /var/log/upstart/modemmanager.log.1 (deleted)
init          1             root   17w      REG              253,1        881     155967 /var/log/upstart/mysql.log.1 (deleted)
init          1             root   24w      REG              253,1         63     155949 /var/log/upstart/dbus.log.1 (deleted)
mysqld     1159            mysql    4u      REG              253,1          0        374 /tmp/ibkjeh3Z (deleted)
mysqld     1159            mysql    5u      REG              253,1          0       3458 /tmp/ibdbHetZ (deleted)
mysqld     1159            mysql    6u      REG              253,1          0      29466 /tmp/ibQSxcTY (deleted)
mysqld     1159            mysql    7u      REG              253,1          0        282 /tmp/ibxSgZNX (deleted)
mysqld     1159            mysql   11u      REG              253,1          0      29469 /tmp/ibXTzqnX (deleted)
mysqld     1159  1185      mysql    4u      REG              253,1          0        374 /tmp/ibkjeh3Z (deleted)
mysqld     1159 17965      mysql    4u      REG              253,1          0        374 /tmp/ibkjeh3Z (deleted)
mysqld     1159 17965      mysql    5u      REG              253,1          0       3458 /tmp/ibdbHetZ (deleted)
mysqld     1159 17965      mysql    6u      REG              253,1          0      29466 /tmp/ibQSxcTY (deleted)
mysqld     1159 17965      mysql    7u      REG              253,1          0        282 /tmp/ibxSgZNX (deleted)
mysqld     1159 17965      mysql   11u      REG              253,1          0      29469 /tmp/ibXTzqnX (deleted)
php5-fpm   3724         www-data    3ur     REG              253,1          0      29468 /tmp/.ZendSem.gfHFuR (deleted)
php5-fpm   3853         www-data    3ur     REG              253,1          0      29468 /tmp/.ZendSem.gfHFuR (deleted)
php5-fpm   7741             root    3u      REG              253,1          0      29468 /tmp/.ZendSem.gfHFuR (deleted)
php5-fpm   9900         www-data    3u      REG              253,1          0      29468 /tmp/.ZendSem.gfHFuR (deleted)
php5-fpm   9901         www-data    3u      REG              253,1          0      29468 /tmp/.ZendSem.gfHFuR (deleted)
php5-fpm   9930         www-data    3u      REG              253,1          0      29468 /tmp/.ZendSem.gfHFuR (deleted)
php5-fpm   9961         www-data    3u      REG              253,1          0      29468 /tmp/.ZendSem.gfHFuR (deleted)
php5-fpm   9965         www-data    3u      REG              253,1          0      29468 /tmp/.ZendSem.gfHFuR (deleted)
php5-fpm   9966         www-data    3u      REG              253,1          0      29468 /tmp/.ZendSem.gfHFuR (deleted)
php5-fpm  10126         www-data    3u      REG              253,1          0      29468 /tmp/.ZendSem.gfHFuR (deleted)
php5-fpm  10127         www-data    3u      REG              253,1          0      29468 /tmp/.ZendSem.gfHFuR (deleted)
php5-fpm  10128         www-data    3u      REG              253,1          0      29468 /tmp/.ZendSem.gfHFuR (deleted)
php5-fpm  10174         www-data    3u      REG              253,1          0      29468 /tmp/.ZendSem.gfHFuR (deleted)
php5-fpm  10196         www-data    3u      REG              253,1          0      29468 /tmp/.ZendSem.gfHFuR (deleted)
php5-fpm  10248         www-data    3u      REG              253,1          0      29468 /tmp/.ZendSem.gfHFuR (deleted)
php5-fpm  10316         www-data    3u      REG              253,1          0      29468 /tmp/.ZendSem.gfHFuR (deleted)
php5-fpm  10375         www-data    3u      REG              253,1          0      29468 /tmp/.ZendSem.gfHFuR (deleted)
php5-fpm  10396         www-data    3u      REG              253,1          0      29468 /tmp/.ZendSem.gfHFuR (deleted)
php5-fpm  10400         www-data    3u      REG              253,1          0      29468 /tmp/.ZendSem.gfHFuR (deleted)
php5-fpm  10401         www-data    3u      REG              253,1          0      29468 /tmp/.ZendSem.gfHFuR (deleted)
php5-fpm  10438         www-data    3u      REG              253,1          0      29468 /tmp/.ZendSem.gfHFuR (deleted)
php5-fpm  10479         www-data    3u      REG              253,1          0      29468 /tmp/.ZendSem.gfHFuR (deleted)
php5-fpm  10511         www-data    3u      REG              253,1          0      29468 /tmp/.ZendSem.gfHFuR (deleted)
php5-fpm  10531         www-data    3u      REG              253,1          0      29468 /tmp/.ZendSem.gfHFuR (deleted)
php5-fpm  11150         www-data    3u      REG              253,1          0      29468 /tmp/.ZendSem.gfHFuR (deleted)
php5-fpm  11638         www-data    3u      REG              253,1          0      29468 /tmp/.ZendSem.gfHFuR (deleted)
php5-fpm  11670         www-data    3u      REG              253,1          0      29468 /tmp/.ZendSem.gfHFuR (deleted)
php5-fpm  12210         www-data    3u      REG              253,1          0      29468 /tmp/.ZendSem.gfHFuR (deleted)
php5-fpm  14974         www-data    3u      REG              253,1          0      29468 /tmp/.ZendSem.gfHFuR (deleted)
php5-fpm  17101         www-data    3u      REG              253,1          0      29468 /tmp/.ZendSem.gfHFuR (deleted)
php5-fpm  17161         www-data    3u      REG              253,1          0      29468 /tmp/.ZendSem.gfHFuR (deleted)
php5-fpm  17261         www-data    3u      REG              253,1          0      29468 /tmp/.ZendSem.gfHFuR (deleted)
php5-fpm  17470         www-data    3u      REG              253,1          0      29468 /tmp/.ZendSem.gfHFuR (deleted)

從輸出結果可以看到哪些文件還被使用，未被釋放空間

或者

root@instance-zo0fkfng:~# find /proc/*/fd -ls | grep  '(deleted)'
 71253    0 lrwx------   1 root     root           64 Jun 10 22:09 /proc/1159/fd/4 -> /tmp/ibkjeh3Z\ (deleted)
 71254    0 lrwx------   1 root     root           64 Jun 10 22:09 /proc/1159/fd/5 -> /tmp/ibdbHetZ\ (deleted)
 71255    0 lrwx------   1 root     root           64 Jun 10 22:09 /proc/1159/fd/6 -> /tmp/ibQSxcTY\ (deleted)
 71256    0 lrwx------   1 root     root           64 Jun 10 22:09 /proc/1159/fd/7 -> /tmp/ibxSgZNX\ (deleted)
 71260    0 lrwx------   1 root     root           64 Jun 10 22:09 /proc/1159/fd/11 -> /tmp/ibXTzqnX\ (deleted)
221991074    0 lrwx------   1 root     root           64 Aug 27 19:43 /proc/18132/fd/3 -> /tmp/.ZendSem.hIpOFr\ (deleted)
221991101    0 lrwx------   1 www-data www-data       64 Aug 27 19:43 /proc/18135/fd/3 -> /tmp/.ZendSem.hIpOFr\ (deleted)
221991122    0 lrwx------   1 www-data www-data       64 Aug 27 19:43 /proc/18136/fd/3 -> /tmp/.ZendSem.hIpOFr\ (deleted)
221991147    0 lrwx------   1 www-data www-data       64 Aug 27 19:43 /proc/18137/fd/3 -> /tmp/.ZendSem.hIpOFr\ (deleted)
221991168    0 lrwx------   1 www-data www-data       64 Aug 27 19:43 /proc/18138/fd/3 -> /tmp/.ZendSem.hIpOFr\ (deleted)
221991189    0 lrwx------   1 www-data www-data       64 Aug 27 19:43 /proc/18139/fd/3 -> /tmp/.ZendSem.hIpOFr\ (deleted)
221991210    0 lrwx------   1 www-data www-data       64 Aug 27 19:43 /proc/18140/fd/3 -> /tmp/.ZendSem.hIpOFr\ (deleted)
221991231    0 lrwx------   1 www-data www-data       64 Aug 27 19:43 /proc/18141/fd/3 -> /tmp/.ZendSem.hIpOFr\ (deleted)
221991252    0 lrwx------   1 www-data www-data       64 Aug 27 19:43 /proc/18142/fd/3 -> /tmp/.ZendSem.hIpOFr\ (deleted)
221991273    0 lrwx------   1 www-data www-data       64 Aug 27 19:43 /proc/18143/fd/3 -> /tmp/.ZendSem.hIpOFr\ (deleted)
221991294    0 lrwx------   1 www-data www-data       64 Aug 27 19:43 /proc/18144/fd/3 -> /tmp/.ZendSem.hIpOFr\ (deleted)
221991315    0 lrwx------   1 www-data www-data       64 Aug 27 19:43 /proc/18147/fd/3 -> /tmp/.ZendSem.hIpOFr\ (deleted)
222001888    0 lrwx------   1 www-data www-data       64 Aug 27 19:44 /proc/18157/fd/3 -> /tmp/.ZendSem.hIpOFr\ (deleted)
222001909    0 lrwx------   1 www-data www-data       64 Aug 27 19:44 /proc/18161/fd/3 -> /tmp/.ZendSem.hIpOFr\ (deleted)
222001938    0 lrwx------   1 www-data www-data       64 Aug 27 19:44 /proc/18170/fd/3 -> /tmp/.ZendSem.hIpOFr\ (deleted)
222001959    0 lrwx------   1 www-data www-data       64 Aug 27 19:44 /proc/18178/fd/3 -> /tmp/.ZendSem.hIpOFr\ (deleted)
find: `/proc/18182/fd/5': No such file or directory
 55130    0 l-wx------   1 root     root           64 Jun 10 22:09 /proc/1/fd/10 -> /var/log/upstart/network-manager.log.1\ (deleted)
 55132    0 l-wx------   1 root     root           64 Jun 10 22:09 /proc/1/fd/12 -> /var/log/upstart/mountall.log.1\ (deleted)
 55133    0 l-wx------   1 root     root           64 Jun 10 22:09 /proc/1/fd/13 -> /var/log/upstart/modemmanager.log.1\ (deleted)
 55137    0 l-wx------   1 root     root           64 Jun 10 22:09 /proc/1/fd/17 -> /var/log/upstart/mysql.log.1\ (deleted)
 55142    0 l-wx------   1 root     root           64 Jun 10 22:09 /proc/1/fd/24 -> /var/log/upstart/dbus.log.1\ (deleted)
find: `/proc/self/fd/5': No such file or directory

2、如何讓進程釋放文件，進而釋放磁盤空間

kill掉相應的進程或者重啟該進程，讓os自動回收磁盤空間

當linux打開一個文件的時候,Linux內核會為每一個進程在/proc/ 『/proc/nnnn/fd/目錄（nnnn為pid）』建立一個以其pid為名的目錄用來保存進程的相關信息，而其子目錄fd保存的是該進程打開的所有文件的fd（fd：file descriptor）

kill進程是通過截斷proc文件系統中的文件可以強制要求系統回收分配給正在使用的的文件

root@instance-zo0fkfng:~# service php5-fpm restart
root@instance-zo0fkfng:~# lsof|grep deleted 
init          1             root   10w      REG              253,1        283     155958 /var/log/upstart/network-manager.log.1 (deleted)
init          1             root   12w      REG              253,1        314     155934 /var/log/upstart/mountall.log.1 (deleted)
init          1             root   13w      REG              253,1        508     131203 /var/log/upstart/modemmanager.log.1 (deleted)
init          1             root   17w      REG              253,1        881     155967 /var/log/upstart/mysql.log.1 (deleted)
init          1             root   24w      REG              253,1         63     155949 /var/log/upstart/dbus.log.1 (deleted)
mysqld     1159            mysql    4u      REG              253,1          0        374 /tmp/ibkjeh3Z (deleted)
mysqld     1159            mysql    5u      REG              253,1          0       3458 /tmp/ibdbHetZ (deleted)
mysqld     1159            mysql    6u      REG              253,1          0      29466 /tmp/ibQSxcTY (deleted)
mysqld     1159            mysql    7u      REG              253,1          0        282 /tmp/ibxSgZNX (deleted)
mysqld     1159            mysql   11u      REG              253,1          0      29469 /tmp/ibXTzqnX (deleted)
mysqld     1159  1185      mysql    4u      REG              253,1          0        374 /tmp/ibkjeh3Z (deleted)
mysqld     1159  1185      mysql    5u      REG              253,1          0       3458 /tmp/ibdbHetZ (deleted)
mysqld     1159  1185      mysql    6u      REG              253,1          0      29466 /tmp/ibQSxcTY (deleted)
mysqld     1159  1185      mysql    7u      REG              253,1          0        282 /tmp/ibxSgZNX (deleted)
mysqld     1159  1185      mysql   11u      REG              253,1          0      29469 /tmp/ibXTzqnX (deleted)
mysqld     1159  1186      mysql    4u      REG              253,1          0        374 /tmp/ibkjeh3Z (deleted)
mysqld     1159  1186      mysql    5u      REG              253,1          0       3458 /tmp/ibdbHetZ (deleted)
mysqld     1159 18149      mysql    4u      REG              253,1          0        374 /tmp/ibkjeh3Z (deleted)
mysqld     1159 18149      mysql    5u      REG              253,1          0       3458 /tmp/ibdbHetZ (deleted)
mysqld     1159 18149      mysql    6u      REG              253,1          0      29466 /tmp/ibQSxcTY (deleted)
mysqld     1159 18149      mysql    7u      REG              253,1          0        282 /tmp/ibxSgZNX (deleted)
mysqld     1159 18149      mysql   11u      REG              253,1          0      29469 /tmp/ibXTzqnX (deleted)
mysqld     1159 18150      mysql    4u      REG              253,1          0        374 /tmp/ibkjeh3Z (deleted)
mysqld     1159 18150      mysql    5u      REG              253,1          0       3458 /tmp/ibdbHetZ (deleted)
mysqld     1159 18150      mysql    6u      REG              253,1          0      29466 /tmp/ibQSxcTY (deleted)
mysqld     1159 18150      mysql    7u      REG              253,1          0        282 /tmp/ibxSgZNX (deleted)
mysqld     1159 18150      mysql   11u      REG              253,1          0      29469 /tmp/ibXTzqnX (deleted)
php5-fpm  18132             root    3u      REG              253,1          0      28407 /tmp/.ZendSem.hIpOFr (deleted)
php5-fpm  18135         www-data    3u      REG              253,1          0      28407 /tmp/.ZendSem.hIpOFr (deleted)
php5-fpm  18136         www-data    3u      REG              253,1          0      28407 /tmp/.ZendSem.hIpOFr (deleted)
php5-fpm  18137         www-data    3u      REG              253,1          0      28407 /tmp/.ZendSem.hIpOFr (deleted)
php5-fpm  18138         www-data    3u      REG              253,1          0      28407 /tmp/.ZendSem.hIpOFr (deleted)
php5-fpm  18139         www-data    3u      REG              253,1          0      28407 /tmp/.ZendSem.hIpOFr (deleted)
php5-fpm  18140         www-data    3u      REG              253,1          0      28407 /tmp/.ZendSem.hIpOFr (deleted)
php5-fpm  18141         www-data    3u      REG              253,1          0      28407 /tmp/.ZendSem.hIpOFr (deleted)
php5-fpm  18142         www-data    3u      REG              253,1          0      28407 /tmp/.ZendSem.hIpOFr (deleted)
php5-fpm  18143         www-data    3u      REG              253,1          0      28407 /tmp/.ZendSem.hIpOFr (deleted)
php5-fpm  18144         www-data    3u      REG              253,1          0      28407 /tmp/.ZendSem.hIpOFr (deleted)
php5-fpm  18147         www-data    3u      REG              253,1          0      28407 /tmp/.ZendSem.hIpOFr (deleted)

可以看到php5-fpm變少了

還有一種方式，清除文件

: > /path/to/the/file.log
: > "/proc/$pid/fd/$fd"

三、文件恢復

在說明問題之前，先介紹下一些文件的基本概念：

文件實際上是一個指向inode的鏈接, inode鏈接包含了文件的所有屬性, 比如權限和所有者, 數據塊地址(文件存儲在磁盤的這些數據塊中). 當你刪除(rm)一個文件, 實際刪除了指向inode的鏈接, 並沒有刪除inode的內容. 進程可能還在使用. 只有當inode的所有鏈接完全移去, 然後這些數據塊將可以寫入新的數據.
proc文件系統可以協助我們恢復數據. 每一個系統上的進程在/proc都有一個目錄和自己的名字，裡面包含了一個fd(文件描述符)子目錄(進程需要打開文件的所有鏈接). 如果從文件系統中刪除一個文件, 此處還有一個inode的引用：/proc/進程號/fd/文件描述符

你需要知道打開文件的進程號(pid)和文件描述符(fd). 這些都可以通過lsof工具方便獲得, lsof的意思是”list open files, 列出(進程)打開的文件”. 然後你將可以從/proc拷貝出需要恢復的數據.

1.創建一個測試文件並且備份下，方面後續驗證

touch testfile
cp testfile testfile.backup.2014

2.查看文件的相關信息

stat testfile

File: 'testfile'
Size: 343545 Blocks: 241 IO Block: 4096 regular file
Device: fd00h/64768d Inode: 361579 Links: 1
Access: (0664/-rw-rw-r–) Uid: ( 505/ zhaoke) Gid: ( 505/ zhaoke)
Access: 2014-11-09 15:00:38.000000000 +0800
Modify: 2014-11-09 15:00:34.000000000 +0800
Change: 2014-04-09 15:00:34.000000000 +0800

3.刪除文件

rm testfile

4.查看文件

ls -l testfile
ls: testfile: No such file or directory
stat testfile
stat: cannot stat 'testfile': No such file or directory

testfile文件刪除了，但不要終止仍在使用文件的進程，因為一旦終止, 文件將很難恢復.

現在我們開始找回數據之旅，先使用lsof命令查看下

lsof | grep testfile
tail 5317 root 4r REG 253,0 343545  361579  /root/testfile (deleted)

第一個縱行是進程的名稱(命令名), 第二縱行是進程號(PID), 第四縱行是文件描述符
現在你知道5317進程仍有打開文件, 文件描述符是4. 那我們開始從/proc裡面拷貝出數據.
你可能會考慮使用cp -a, 但實際上沒有作用, 你將拷貝的是一個指向被刪除文件的符號鏈接:

ls -l /proc/5317/fd/4
lr-x—— 1 root root 64  09 15:00 /proc/5317/fd/4 -> /root/testfile (deleted)

使用cp -a命令測試恢復

 cp -a /proc/5317/fd/4 testfile.backup

使用ls命令來查看

ls -l testfile.backup
lrwxrwxrwx 1 root root 29  09 15:02 testfile.backup -> /roor/testfile (deleted)

通過上面的命令我們發現，使用cp -a命令，其恢復的是一個指向被刪除文件的符號鏈接

使用file命令分別查看文件和文件描述符

1.查看文件

file testfile.backup
testfile.backup: broken symbolic link to '/root/testfile (deleted)'

2.查看文件描述符

file /proc/5317/fd/4

/proc/5317/fd/4: broken symbolic link to '/root/myfile (deleted)'

根據上面的file結果，可以使用cp拷貝出文件描述符數據到一個文件中，如下:

cp /proc/5317/fd/4 testfile.new

使用上面的命令恢復後，我們需要最終確認一下文件是否恢復，以及文件內容是否正確:

 ls -l testfile.new

然後把新舊的兩個文件對比

diff testfile.new myfile.backup

4、lsof的一些其他用法

1、查看端口號

root@localhost:~# lsof -i :80
COMMAND     PID     USER   FD   TYPE    DEVICE SIZE/OFF NODE NAME
AliYunDun  1996     root   11u  IPv4 117919702      0t0  TCP 124.46.185.34:42372->140.205.140.205:http (ESTABLISHED)
aegis_qua  2114     root   11u  IPv4     14868      0t0  TCP 124.46.185.34:43077->42.156.166.25:http (CLOSE_WAIT)
nginx     13706 www-data    5u  IPv4 140102705      0t0  TCP 124.46.185.34:http->140.205.127.64:41851 (ESTABLISHED)
nginx     13706 www-data   15u  IPv4   6150826      0t0  TCP *:http (LISTEN)
nginx     13706 www-data   35u  IPv4 140096235      0t0  TCP 124.46.185.34:http->198.11.137.21:47450 (ESTABLISHED)
nginx     13706 www-data   36u  IPv4 140095680      0t0  TCP 124.46.185.34:http->140.205.127.52:39887 (ESTABLISHED)
nginx     13706 www-data   39u  IPv4 140102712      0t0  TCP 124.46.185.34:http->198.11.137.5:37626 (ESTABLISHED)
nginx     13706 www-data   40u  IPv4 140102699      0t0  TCP 124.46.185.34:http->140.205.127.11:36374 (ESTABLISHED)
nginx     13706 www-data   41u  IPv4 140102759      0t0  TCP 124.46.185.34:http->101.200.101.217:47550 (ESTABLISHED)
nginx     13706 www-data   42u  IPv4 140102722      0t0  TCP 124.46.185.34:http->101.200.101.201:38595 (ESTABLISHED)
nginx     13706 www-data   43u  IPv4 140102840      0t0  TCP 124.46.185.34:http->140.205.127.58:45859 (ESTABLISHED)
nginx     13706 www-data   44u  IPv4 140095709      0t0  TCP 124.46.185.34:http->101.200.101.216:44828 (ESTABLISHED)
nginx     13706 www-data   46u  IPv4 140070070      0t0  TCP 124.46.185.34:http->140.205.127.18:24113 (ESTABLISHED)
nginx     13706 www-data   47u  IPv4 140103294      0t0  TCP 124.46.185.34:http->198.11.137.61:43262 (ESTABLISHED)
nginx     13706 www-data   48u  IPv4 140102761      0t0  TCP 124.46.185.34:http->101.200.101.220:31912 (ESTABLISHED)
nginx     13706 www-data   50u  IPv4 140103297      0t0  TCP 124.46.185.34:http->101.200.101.203:42752 (ESTABLISHED)
nginx     13706 www-data   52u  IPv4 140095824      0t0  TCP 124.46.185.34:http->198.11.137.14:62968 (ESTABLISHED)
nginx     13706 www-data   53u  IPv4 140102730      0t0  TCP 124.46.185.34:http->101.200.101.205:27518 (ESTABLISHED)
nginx     13706 www-data   54u  IPv4 140102822      0t0  TCP 124.46.185.34:http->140.205.127.51:58945 (ESTABLISHED)
nginx     13706 www-data   55u  IPv4 140103313      0t0  TCP 124.46.185.34:http->140.205.127.15:20785 (ESTABLISHED)
nginx     13706 www-data   56u  IPv4 140102812      0t0  TCP 124.46.185.34:http->140.205.31.212:1963 (ESTABLISHED)
nginx     13706 www-data   57u  IPv4 140102929      0t0  TCP 124.46.185.34:http->101.200.101.213:42452 (ESTABLISHED)
nginx     13706 www-data   58u  IPv4 140095749      0t0  TCP 124.46.185.34:http->101.200.101.208:50444 (ESTABLISHED)
nginx     13706 www-data   59u  IPv4 140102769      0t0  TCP 124.46.185.34:http->140.205.127.60:59679 (ESTABLISHED)
nginx     13706 www-data   60u  IPv4 140102814      0t0  TCP 124.46.185.34:http->140.205.127.2:25156 (ESTABLISHED)
nginx     13706 www-data   61u  IPv4 140102771      0t0  TCP 124.46.185.34:http->198.11.137.53:16935 (ESTABLISHED)
nginx     13706 www-data   62u  IPv4 140102845      0t0  TCP 124.46.185.34:http->198.11.137.2:50535 (ESTABLISHED)
nginx     13706 www-data   63u  IPv4 140102773      0t0  TCP 124.46.185.34:http->198.11.137.52:8521 (ESTABLISHED)
nginx     13706 www-data   64u  IPv4 140102856      0t0  TCP 124.46.185.34:http->140.205.127.23:55360 (ESTABLISHED)
nginx     13706 www-data   66u  IPv4 140075271      0t0  TCP 124.46.185.34:http->140.205.127.48:64726 (ESTABLISHED)
nginx     13706 www-data   67u  IPv4 140102937      0t0  TCP 124.46.185.34:http->101.200.101.204:10400 (ESTABLISHED)
nginx     13706 www-data   68u  IPv4 140102932      0t0  TCP 124.46.185.34:http->101.200.101.213:42455 (ESTABLISHED)
nginx     13706 www-data   69u  IPv4 140102941      0t0  TCP 124.46.185.34:http->140.205.127.24:31585 (ESTABLISHED)
nginx     13706 www-data   70u  IPv4 140102934      0t0  TCP 124.46.185.34:http->198.11.137.50:11837 (ESTABLISHED)
nginx     13706 www-data   71u  IPv4 140103309      0t0  TCP 124.46.185.34:http->140.205.127.15:20777 (ESTABLISHED)
nginx     13706 www-data   92u  IPv4 140070742      0t0  TCP 124.46.185.34:http->140.205.127.31:65012 (ESTABLISHED)
nginx     13707 www-data    5u  IPv4 140092233      0t0  TCP 124.46.185.34:http->140.205.127.60:57698 (ESTABLISHED)
nginx     13707 www-data   15u  IPv4   6150826      0t0  TCP *:http (LISTEN)
nginx     13707 www-data   17u  IPv4 140092036      0t0  TCP 124.46.185.34:http->140.205.127.51:54339 (ESTABLISHED)
nginx     13707 www-data   30u  IPv4 140092031      0t0  TCP 124.46.185.34:http->198.11.137.59:64831 (ESTABLISHED)
nginx     13707 www-data   35u  IPv4 140092214      0t0  TCP 124.46.185.34:http->140.205.31.213:40510 (ESTABLISHED)
nginx     13707 www-data   39u  IPv4 140092202      0t0  TCP 124.46.185.34:http->140.205.127.9:19592 (ESTABLISHED)
nginx     13707 www-data   44u  IPv4 140092205      0t0  TCP 124.46.185.34:http->140.205.127.7:24059 (ESTABLISHED)
nginx     13707 www-data   46u  IPv4 140103741      0t0  TCP 124.46.185.34:http->198.11.137.22:61619 (ESTABLISHED)
nginx     13707 www-data   51u  IPv4 140072557      0t0  TCP 124.46.185.34:http->140.205.127.50:51697 (ESTABLISHED)
nginx     13707 www-data   61u  IPv4 140056450      0t0  TCP 124.46.185.34:http->140.205.127.37:27131 (ESTABLISHED)
nginx     13707 www-data   69u  IPv4 140072632      0t0  TCP 124.46.185.34:http->140.205.127.13:20352 (ESTABLISHED)
nginx     13708 www-data   15u  IPv4   6150826      0t0  TCP *:http (LISTEN)

2、查看進程

root@localhost:~# lsof  -p  13721 
COMMAND   PID     USER   FD   TYPE             DEVICE  SIZE/OFF     NODE NAME
nginx   13721 www-data  cwd    DIR              253,1      4096        2 /
nginx   13721 www-data  rtd    DIR              253,1      4096        2 /
nginx   13721 www-data  txt    REG              253,1    843688  1195996 /usr/sbin/nginx
nginx   13721 www-data  mem    REG              253,1     48024   269175 /lib/x86_64-linux-gnu/libnss_files-2.15.so
nginx   13721 www-data  mem    REG              253,1     47680   262260 /lib/x86_64-linux-gnu/libnss_nis-2.15.so
nginx   13721 www-data  mem    REG              253,1     97248   269169 /lib/x86_64-linux-gnu/libnsl-2.15.so
nginx   13721 www-data  mem    REG              253,1     35680   269174 /lib/x86_64-linux-gnu/libnss_compat-2.15.so
nginx   13721 www-data  mem    REG              253,1   1807032   269167 /lib/x86_64-linux-gnu/libc-2.15.so
nginx   13721 www-data  mem    REG              253,1    215136  1186378 /usr/lib/libGeoIP.so.1.4.8
nginx   13721 www-data  mem    REG              253,1    270400  1181963 /usr/lib/x86_64-linux-gnu/libgd.so.2.0.0
nginx   13721 www-data  mem    REG              253,1     84928  1195989 /usr/lib/x86_64-linux-gnu/libexslt.so.0.8.15
nginx   13721 www-data  mem    REG              253,1     43288   269172 /lib/x86_64-linux-gnu/libcrypt-2.15.so
nginx   13721 www-data  mem    REG              253,1    135366   269170 /lib/x86_64-linux-gnu/libpthread-2.15.so
nginx   13721 www-data  mem    REG              253,1    149280   269173 /lib/x86_64-linux-gnu/ld-2.15.so
nginx   13721 www-data  DEL    REG                0,4            6150828 /dev/zero
nginx   13721 www-data    0u   CHR                1,3       0t0     1029 /dev/null
nginx   13721 www-data    1u   CHR                1,3       0t0     1029 /dev/null
nginx   13721 www-data    4w   REG              253,1     85704   918896 /var/log/nginx/access.log
nginx   13721 www-data   13u  unix 0xffff8807ff4ef500       0t0 59526717 socket
nginx   13721 www-data   14w   REG              253,1    438012   918916 /var/log/nginx/admin.access.log
nginx   13721 www-data   15u  IPv4            6150826       0t0      TCP *:http (LISTEN)
nginx   13721 www-data   16u  IPv4            6150827       0t0      TCP *:http-alt (LISTEN)
nginx   13721 www-data   19u  unix 0xffff8807ff4ee900       0t0 59526719 socket
nginx   13721 www-data   20u  IPv4          140132456       0t0      TCP 124.46.185.34:http->140.205.127.60:16099 (ESTABLISHED)
nginx   13721 www-data   21u  unix 0xffff8807ff4ebf00       0t0 59526721 socket
nginx   13721 www-data   22u  IPv4          140132818       0t0      TCP 124.46.185.34:http->140.205.127.17:33358 (ESTABLISHED)
nginx   13721 www-data   23u  unix 0xffff8807ff4ee600       0t0 59526723 socket
nginx   13721 www-data   24u  IPv4          140132826       0t0      TCP 124.46.185.34:http->198.11.137.59:25415 (ESTABLISHED)
nginx   13721 www-data   25u  unix 0xffff8807ff4e9200       0t0 59526725 socket
nginx   13721 www-data   26u  IPv4          140106470       0t0      TCP 124.46.185.34:http->101.200.101.204:37659 (ESTABLISHED)
nginx   13721 www-data   27u  IPv4          140106678       0t0      TCP 124.46.185.34:http->140.205.127.34:53569 (ESTABLISHED)
nginx   13721 www-data   30u  IPv4          140132831       0t0      TCP 124.46.185.34:http->140.205.127.44:8165 (ESTABLISHED)
nginx   13721 www-data   31u  IPv4          140132828       0t0      TCP 124.46.185.34:http->47.88.135.31:64913 (ESTABLISHED)
nginx   13721 www-data   33u  IPv4          140132871       0t0      TCP 124.46.185.34:http->140.205.31.220:57845 (ESTABLISHED)
nginx   13721 www-data   34u  IPv4          140132842       0t0      TCP 124.46.185.34:http->195.27.31.56:55498 (ESTABLISHED)
nginx   13721 www-data   35u  IPv4          140132679       0t0      TCP 124.46.185.34:http->140.205.127.26:62043 (ESTABLISHED)
nginx   13721 www-data   36u  IPv4          140132838       0t0      TCP 124.46.185.34:http->140.205.127.26:64949 (ESTABLISHED)
nginx   13721 www-data   37u  unix 0xffff8807ff4ef200       0t0 59526727 socket
nginx   13721 www-data   38u  IPv4          140132862       0t0      TCP 124.46.185.34:http->140.205.127.25:23082 (ESTABLISHED)
nginx   13721 www-data   39u  unix 0xffff8807ff4ea700       0t0 59526729 socket
nginx   13721 www-data   41u  unix 0xffff8807ff4ec200       0t0 59526731 socket
nginx   13721 www-data   42u  IPv4          140132845       0t0      TCP 124.46.185.34:http->101.200.101.216:55441 (ESTABLISHED)
nginx   13721 www-data   43u  unix 0xffff8807ff4eb900       0t0 59526733 socket
nginx   13721 www-data   44u  IPv4          140106547       0t0      TCP 124.46.185.34:http->140.205.31.211:60180 (ESTABLISHED)
nginx   13721 www-data   45u  unix 0xffff8807ff4e9b00       0t0 59526735 socket
nginx   13721 www-data   46u  IPv4          140132879       0t0      TCP 124.46.185.34:http->140.205.31.203:42821 (ESTABLISHED)
nginx   13721 www-data   47u  unix 0xffff8807ff4e8300       0t0 59526737 socket
nginx   13721 www-data   48u  IPv4          140132847       0t0      TCP 124.46.185.34:http->140.205.127.45:53830 (ESTABLISHED)
nginx   13721 www-data   49u  unix 0xffff8807ff4ee000       0t0 59526739 socket
nginx   13721 www-data   50u  IPv4          140077698       0t0      TCP 124.46.185.34:http->140.205.127.43:43558 (ESTABLISHED)
nginx   13721 www-data   51u  unix 0xffff8807ff4ed400       0t0 59526741 socket
nginx   13721 www-data   52u  IPv4          140132857       0t0      TCP 124.46.185.34:http->198.11.137.19:22613 (ESTABLISHED)
nginx   13721 www-data   53u  unix 0xffff8807ff4ece00       0t0 59526743 socket
nginx   13721 www-data   55u  unix 0xffff8807ff4e8000       0t0 59526745 socket
nginx   13721 www-data   56u  IPv4          140132853       0t0      TCP 124.46.185.34:http->140.205.31.212:23679 (ESTABLISHED)
nginx   13721 www-data   57u  IPv4          140132859       0t0      TCP 124.46.185.34:http->140.205.127.61:7242 (ESTABLISHED)
nginx   13721 www-data   58u  unix 0xffff8807ff4e9800       0t0 59526748 socket
nginx   13721 www-data   59u  IPv4          140132864       0t0      TCP 124.46.185.34:http->101.200.101.201:35787 (ESTABLISHED)
nginx   13721 www-data   70u  IPv4          140093317       0t0      TCP 124.46.185.34:http->140.205.127.34:37276 (ESTABLISHED)
nginx   13721 www-data   80u  0000                0,9         0     6874 anon_inode
nginx   13721 www-data   87u  IPv4          140105862       0t0      TCP 124.46.185.34:http->140.205.127.11:40316 (ESTABLISHED)
nginx   13721 www-data   88u  IPv4          140093346       0t0      TCP 124.46.185.34:http->140.205.127.36:29659 (ESTABLISHED)

3、lsof -c init

root@iZ25n2yx37sZ:~#  lsof -c init
COMMAND PID USER   FD   TYPE             DEVICE SIZE/OFF   NODE NAME
init      1 root  cwd    DIR              253,1     4096      2 /
init      1 root  rtd    DIR              253,1     4096      2 /
init      1 root  txt    REG              253,1   167192 393221 /sbin/init
init      1 root  mem    REG              253,1    48024 269175 /lib/x86_64-linux-gnu/libnss_files-2.15.so
init      1 root  mem    REG              253,1    47680 262260 /lib/x86_64-linux-gnu/libnss_nis-2.15.so
init      1 root  mem    REG              253,1    97248 269169 /lib/x86_64-linux-gnu/libnsl-2.15.so
init      1 root  mem    REG              253,1    35680 269174 /lib/x86_64-linux-gnu/libnss_compat-2.15.so
init      1 root  mem    REG              253,1  1807032 269167 /lib/x86_64-linux-gnu/libc-2.15.so
init      1 root  mem    REG              253,1    31752 262372 /lib/x86_64-linux-gnu/librt-2.15.so
init      1 root  mem    REG              253,1   135366 269170 /lib/x86_64-linux-gnu/libpthread-2.15.so
init      1 root  mem    REG              253,1   276392 262185 /lib/x86_64-linux-gnu/libdbus-1.so.3.5.8
init      1 root  mem    REG              253,1    38888 262171 /lib/x86_64-linux-gnu/libnih-dbus.so.1.0.0
init      1 root  mem    REG              253,1    96240 262174 /lib/x86_64-linux-gnu/libnih.so.1.0.0
init      1 root  mem    REG              253,1   149280 269173 /lib/x86_64-linux-gnu/ld-2.15.so
init      1 root    0u   CHR                1,3      0t0   1029 /dev/null
init      1 root    1u   CHR                1,3      0t0   1029 /dev/null
init      1 root    2u   CHR                1,3      0t0   1029 /dev/null
init      1 root    3r  FIFO                0,8      0t0   7012 pipe
init      1 root    4w  FIFO                0,8      0t0   7012 pipe
init      1 root    5r  0000                0,9        0   6874 anon_inode
init      1 root    6r  0000                0,9        0   6874 anon_inode
init      1 root    7u  unix 0xffff88080b948000      0t0   7013 socket
init      1 root    8u  unix 0xffff88080b918600      0t0  12351 socket
init      1 root    9u  unix 0xffff880807670c00      0t0   1238 socket
init      1 root   10u  unix 0xffff8808076a9500      0t0   7100 socket
init      1 root   11w   REG              253,1      633 919710 /var/log/upstart/mysql.log.1 (deleted)
init      1 root   23u   CHR                5,2      0t0   1111 /dev/ptmx

　　。。。。