Dalvik虛擬機調試監視器

CR：

    做測試調研的時候感覺不是很熟，就找了點資料來學習下DDMS、JDWP和調試器的東西。翻譯了點資料。
DDM:dalvik debug monitor，調試監視器，很生硬的名字，但是這樣可以表明它的作用，同時區別於Debugger調試器。它只是一個監視器，提供了UI，連接VM和Debugger，是server；
Debugger：調試器，這才是調試用的東西；
JDWP：傳輸協議，允許vm-DDM-debugger的數據傳輸模式。
vm：dalvikvm，是client，調試其實就是client-server模式

1、簡介
    能實時觀察虛擬機的狀態時非常有用的，對於Android，我們需要能監視通過USB或者無線連接的設備上的多個虛擬機（CR:每個app創建一個vm實例）。本文檔描述了一個連接到多個vm的調試監視器終端（CR：下面都叫DDM，dalvik debug monitor server）以及一個API，虛擬機和應用可以通過API向監視器傳遞信息。
    通過Dalvik調試監視器（DDM）可以監視的東西：
（1）線程狀態：追蹤線程創建、退出、忙/空閒狀態；
（2）所有的堆的狀態：對於heap bitmap display和片段分析很有用。
    亦可以用除了VM之外的東西來作為DDM的客戶端，但是這是次要目的。例子中包含了“logcat”信息和對虛擬內存使用及LOAD平均的系統監視。（CR：這句話不太明白）
    通過將信息展現在設備的UI上，DDM也可以在設備上運行。但是，它最初的目的是一個利用桌面工具和屏幕資源來提供顯示的工具。
    DDM用JAVA語言寫成，從而可移植，它通過桌面UI工具（SWT）來實現接口。

2、協議
    為了利用已有的接口，DDM協議建立在JDWP協議之上（CR：piggy-back類似於在卡車後面掛車的意思）（java debug wire protocol，通常運行在vm和調試器之間）。這樣，對於一個無DDM的客戶端來說，DDM作用其實就像一個調試器。
    JDWP協議和我們想要實現的功能很像，特別的：
（1）它允許vender-defined包，因此沒有必要改變JDWP細則；
（2）VM可能在任何時間傳出事件，這些事件不會導致調試器輸出一個響應，這意味著客戶端可以傳出數據並且立刻繼續工作而不需要等待事件響應；
（3）基本的協議是無狀態、異步的，調試器端的請求包包含了一串數字，而vm則包含在了響應包中，這就實現了同時同步會話，意味著DDM通信和調試器通信可以交錯進行。
    下面是我們使用JDWP的一些情形：
（1）VM只等待和調試器的一個連接，因此你不能同時連接監視器和調試器。為此，就要將調試器和監視器連起來然後傳輸數據（我們目前可以通過jdwpspy傳輸數據，盡管還需要管理我們需要的ID）。連接了設備之後，通過這種方式比現在的方法（猜測傳輸端口號）更加方便；
（2）當調試器連接後，VM運行會不一樣。它會更慢，任何傳向監視器或者調試器的對象都不會產生GC；對此，我們可以通過直到監測到無DDM通信再激活慢路徑（CR：slow path，不懂）。另外，我們還希望能產生“調試器已經斷開/連接”的信息，從而使VM釋放調試相關資源並且不斷開連接；
（3）無DDM的vm在DDM連接後不應該出錯，事實上三方vm並沒有對此保證（例如某種主流vm會立刻崩潰），但是老式的JanVM會拒絕“hello”包。

2.1、建立連接
    可以通過兩種基本方法：DDM連接vm，vm連接DDM。前者沒有後者“精確”，因為你必須掃描客戶端，但是它也有自己的優點。
    有三種有趣的情形：
（1）DDM開啟，然後boot USB連接的設備或者啟動模擬器；（DDM，設備）
（2）設備或模擬器在運行DDM之前已經在運行；（設備，DDM）
（3）DDM已經在運行了，當一個已經啟動的設備通過USB連接的時候。（DDM，USB連接運行中的設備）
    如果設備連接上了已經啟動的DDM,我們只要處理情形1；如果DDM在它開啟的時候掃描VM，我們只要處理情形2；沒有處理情形3，而它可能是隨著設備的運行最重要的一支。
    因此可以設計一個下拉菜單，它有兩個選項：掃描服務器、掃描設備。前者使得DDM掃描“本地主機”（localhost）上的vm，後者使得它查詢adb連接另一邊的vm。DDM會每隔幾秒就掃描vm，或者檢測已知vm端口序列（例如8000——8040），或者和設備上的某些數據庫交互。改變模式會導致已有的連接中斷。
    當DDM第一次啟動時，它會嘗試運行“adb usb”來確保adb可以運行（注意：確保DDM從可以在該路徑中運行adb的shell中啟動很重要）。如果失敗，和設備的交互很可能會等到adb守護進程運行才開始。

2.2、連接調試器
    當DDM在所有vm的JWDP端口運行時，通過DDM連接調試器很重要。每個要調試的vm會有一個等待DDM監聽的端口，這使得你可以將一個調試器和多個vm同時相連。
    更常見的是，程序員只想將一個調試器和一個vm相連。一個端口會被DDM監聽（例如8700），然後任何和它相連的東西會被“當前vm”連接（從UI中選擇）。這使得程序員可以關注單個應用，而不需要每次重啟設備時調整IDE設置以適應不同端口號，如果不是這樣，序號就會不停變動。

2.3、包格式
    信息以塊（chunk）的形式傳遞，每個塊開頭如下：
U4 type
U4 length
    然後包含了不同數量的特定類型（type）的數據。不識別的類型會導致客戶端的空響應，這在DDM會被安靜地忽略。（一般是返回ERROR，需要一個ERROR塊類型及DDM端的handler）
    同樣的塊類型可能在傳遞方向不同時有不同的含義，例如，同樣的數據類型可能被用在詢問和對應的響應上，為了清晰明了，特定的類型必須用在對應的功能上。
    上面的包格式對於JDWP框架有些冗余，JDWP框架有4字節的length和2字節的類型碼（“命令集”和“命令”，一組命令集用於“vendor-defined命令和擴展”）。
    使用塊的形式允許潛在傳遞（underlying transport）的獨立性，避免了和JDWP客戶端碼元的插入集成（intrusive integration），提供了在單個傳輸單元中發送多個塊的方法。（包含多個塊的包是可以設計的，這裡不討論）
    因為我們可能通過慢的USB連接發送數據，塊會被壓縮。壓縮塊也是塊類型，但是指明了被壓縮，壓縮後的長度緊跟其後，最後是原來的塊類型和未壓縮的長度。例如，zlib的壓縮算法而言，塊類型是ZLIB。
    基於JDWP模型，DDM到客戶端的包都會被確認，但從客戶端到DDM的包卻不會，JDWP錯誤碼碼域總是被設為“no error”，因此特定請求未響應的情形一定要被編碼到DDM信息中。
    所謂的U4，是一個無符號32bit的值，U1是無符號的8bit的值。數據是大印第安序列形式以匹配JDWP。===================================CUT==========================

Dalvik VM
Debug Monitor

Status:Draft (as of March 6, 2007)

[authors] Modified: Sat Apr 11 2009

Introduction

It's extremely useful to be able to monitor the live state of the VM. For Android, we need to monitor multiple VMs running on a device connected through USB or a wireless network connection. This document describes a debug monitor server that interacts with multiple VMs, and an API that VMs and applications can use to provide information to the monitor.

Some things we can monitor with the Dalvik Debug Monitor ("DDM"):

• Thread states. Track thread creation/exit, busy/idle status.
• Overall heap status, useful for a heap bitmap display or fragmentation analysis.

It is possible for something other than a VM to act as a DDM client, but that is a secondary goal. Examples include "logcat" log extraction and system monitors for virtual memory usage and load average.

It's also possible for the DDM server to be run on the device, with the information presented through the device UI. However, the initial goal is to provide a display tool that takes advantage of desktop tools and screen real estate.

This work is necessary because we are unable to use standard JVMTI-based tools with Dalvik. JVMTI relies on bytecode insertion, which is not currently possible because Dalvik doesn't support Java bytecode.

The DDM server is written in the Java programming language for portability. It uses a desktop UI toolkit (SWT) for its interface.

Protocol

To take advantage of existing infrastructure we are piggy-backing the DDM protocol on top of JDWP (the Java Debug Wire Protocol, normally spoken between a VM and a debugger). To a non-DDM client, the DDM server just looks like a debugger.

The JDWP protocol is very close to what we want to use. In particular:

• It explicitly allows for vendor-defined packets, so there is no need to "bend" the JDWP spec.
• Events may be posted from the VM at arbitrary points. Such events do not elicit a response from the debugger, meaning the client can post data and immediately resume work without worrying about the eventual response.
• The basic protocol is stateless and asynchronous. Request packets from the debugger side include a serial number, which the VM includes in the response packet. This allows multiple simultaneous conversations, which means the DDM traffic can be interleaved with debugger traffic.

There are a few issues with using JDWP for our purposes:

• The VM only expects one connection from a debugger, so you couldn't attach the monitor and a debugger at the same time. This will be worked around by connecting the debugger to the monitor and passing the traffic through. (We're already doing the pass-through with "jdwpspy"; requires some management of our request IDs though.) This should be more convenient than the current "guess the port number" system when we're attached to a device.
• The VM behaves differently when a debugger is attached. It will run more slowly, and any objects passed to the monitor or debugger are immune to GC. We can work around this by not enabling the slow path until non-DDM traffic is observed. We also want to have a "debugger has connected/disconnected" message that allows the VM to release debugger-related resources without dropping the net connection.
• Non-DDM VMs should not freak out when DDM connects. There are no guarantees here for 3rd-party VMs (e.g. a certain mainstream VM, which crashes instantly), but our older JamVM can be configured to reject the "hello" packet.

Connection Establishment

There are two basic approaches: have the server contact the VMs, and have the VMs contact the server. The former is less "precise" than the latter, because you have to scan for the clients, but it has some advantages.

There are three interesting scenarios:

1. The DDM server is started, then the USB-attached device is booted or the simulator is launched.
2. The device or simulator is already running when the DDM server is started.
3. The DDM server is running when an already-started device is attached to USB.

If we have the VMs connect to the DDM server on startup, we only handle case #1. If the DDM server scans for VMs when it starts, we only handle case #2. Neither handles case #3, which is probably the most important of the bunch as the device matures.

The plan is to have a drop-down menu with two entries, "scan workstation" and "scan device". The former causes the DDM server to search for VMs on "localhost", the latter causes it to search for VMs on the other side of an ADB connection. The DDM server will scan for VMs every few seconds, either checking a range of known VM ports (e.g. 8000-8040) or interacting with some sort of process database on the device. Changing modes causes all existing connections to be dropped.

When the DDM server first starts, it will try to execute "adb usb" to ensure that the ADB server is running. (Note it will be necessary to launch the DDM server from a shell with "adb" in the path.) If this fails, talking to the device will still be possible so long as the ADB daemon is already running.

Connecting a Debugger

With the DDM server sitting on the JDWP port of all VMs, it will be necessary to connect the debugger through the DDM server. Each VM being debugged will have a separate port being listened to by the DDM server, allowing you to connect a debugger to one or more VMs simultaneously.

In the common case, however, the developer will only want to debug a single VM. One port (say 8700) will be listened to by the DDM server, and anything connecting to it will be connected to the "current VM" (selected in the UI). This should allow developers to focus on a single application, which may otherwise shift around in the ordering, without having to adjust their IDE settings to a different port every time they restart the device.

Packet Format

Information is sent in chunks. Each chunk starts with:

u4   type
u4   length

and contains a variable amount of type-specific data. Unrecognized types cause an empty response from the client and are quietly ignored by the server. [Should probably return an error; need an "error" chunk type and a handler on the server side.]

The same chunk type may have different meanings when sent in different directions. For example, the same type may be used for both a query and a response to the query. For sanity the type must always be used in related transactions.

This is somewhat redundant with the JDWP framing, which includes a 4-byte length and a two-byte type code ("command set" and "command"; a range of command set values is designated for "vendor-defined commands and extensions"). Using the chunk format allows us to remain independent of the underlying transport, avoids intrusive integration with JDWP client code, and provides a way to send multiple chunks in a single transmission unit. [I'm taking the multi-chunk packets into account in the design, but do not plan to implement them unless the need arises.]

Because we may be sending data over a slow USB link, the chunks may be compressed. Compressed chunks are written as a chunk type that indicates the compression, followed by the compressed length, followed by the original chunk type and the uncompressed length. For zlib's deflate algorithm, the chunk type is "ZLIB".

Following the JDWP model, packets sent from the server to the client are always acknowledged, but packets sent from client to server never are. The JDWP error code field is always set to "no error"; failure responses from specific requests must be encoded into the DDM messages.

In what follows "u4" is an unsigned 32-bit value and "u1" is an unsigned 8-bit value. Values are written in big-endian order to match JDWP.