Skip to content
This repository was archived by the owner on May 6, 2020. It is now read-only.

Commit 346b6d9

Browse files
author
James O. D. Hunt
authored
Merge pull request #846 from jodh-intel/improve-doc-for-debug
Improve doc for debug
2 parents 4830421 + 56aa874 commit 346b6d9

3 files changed

Lines changed: 90 additions & 63 deletions

File tree

‎README.md‎

Lines changed: 35 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -13,8 +13,8 @@
1313
* [Quick start for developers](#quick-start-for-developers)
1414
* [Community](#community)
1515
* [Configuration](#configuration)
16+
* [Logging](#logging)
1617
* [Debugging](#debugging)
17-
* [Enabling debug for various components](#enabling-debug-for-various-components)
1818
* [Limitations](#limitations)
1919
* [Home Page](#home-page)
2020

@@ -79,26 +79,51 @@ To see details of your systems runtime environment (including the location of th
7979
$ cc-runtime cc-env
8080
```
8181

82-
## Debugging
82+
## Logging
8383

84-
The runtime provides `--log=` and `--log-format=` options. However, it can
85-
also be configured to log to the system log (syslog or `journald`) such that
84+
The runtime provides `--log=` and `--log-format=` options. However, you can
85+
also configure it to log to the system log (syslog or `journald`) such that
8686
all log data is sent to both the specified logfile and the system log. The
8787
latter is useful as it is independent of the lifecycle of each container.
8888

89-
To view runtime log output,
89+
To view runtime log output:
9090

9191
```bash
9292
$ sudo journalctl -t cc-runtime
9393
```
9494

95-
### Enabling debug for various components
95+
To view shim log output:
96+
97+
```bash
98+
$ sudo journalctl -t cc-shim
99+
```
100+
101+
To view proxy log output:
102+
103+
```bash
104+
$ sudo journalctl -t cc-proxy
105+
```
106+
107+
Note:
108+
109+
The proxy log entries also include output from the agent (`cc-agent`) and the
110+
hypervisor, which includes the guest kernel boot-time messages.
111+
112+
## Debugging
113+
114+
The runtime, the shim (`cc-shim`), the proxy (`cc-proxy`),
115+
and the hypervisor all have separate `enable_debug=` debug
116+
options in the [configuration file](#Configuration). All of these debug
117+
options are disabled by default. See the comments in the installed
118+
configuration file for further details.
119+
120+
If you want to enable debug for all components, assuming a standard configuration file path, run:
96121

97-
The runtime, the shim (`cc-shim`), and the hypervisor all have separate debug
98-
options in the [configuration file](#Configuration).
122+
```bash
123+
$ sudo sed -i -e 's/^#\(enable_debug\).*=.*$/\1 = true/g' /usr/share/defaults/clear-containers/configuration.toml
124+
```
99125

100-
The proxy (`cc-proxy`) has a command-line option to enable debug output. See
101-
the [proxy documentation](https://github.com/clearcontainers/proxy#debugging) for further details.
126+
See the [agent debug document](docs/debug-agent.md) and the [kernel debug document](docs/debug-kernel.md) for further details.
102127

103128
## Limitations
104129

‎docs/debug-agent.md‎

Lines changed: 43 additions & 38 deletions
Original file line numberDiff line numberDiff line change
@@ -4,44 +4,49 @@ Clear Containers runtime launch a special virtual machine when a new pod is
44
created. This VM uses a VM-agent to spawn processes on behalf of the
55
pod/container(s) running inside the VM.
66

7-
From Clear Containers 3.0-beta release a new agent is used,
8-
[Clear Containers Agent](https://github.com/clearcontainers/agent):
7+
The Clear Containers 3.0-beta release uses a new agent (`cc-agent`),
8+
https://github.com/clearcontainers/agent.
99

10-
The Clear Containers Agent relies heavily on libcontainer used by runc
11-
(standard program when running containers on bare metal).
10+
The Clear Containers agent relies heavily on [`libcontainer`](https://github.com/opencontainers/runc/tree/master/libcontainer)
11+
used by [`runc`](https://github.com/opencontainers/runc/), which is the
12+
standard program when running containers on bare metal.
1213

1314
To provide a debug log of any agent activity on a guest, the Clear Containers
14-
agent sends logs through a QEMU serial console that are collected by [cc-proxy](https://github.com/clearcontainers/proxy)
15-
and shown in its logs. By default, the Clear Containers agent logs are not collected by
16-
`cc-proxy` but can be enabled by adding the `-log debug` option to `cc-proxy`.
17-
18-
1- Add the `-log debug` option to `cc-proxy`
19-
20-
```
21-
mkdir -p /etc/systemd/system/cc-proxy.service.d/
22-
cat << EOT > /etc/systemd/system/cc-proxy.service.d/proxy-debug.conf
23-
[Service]
24-
ExecStart=
25-
ExecStart=/usr/libexec/clear-containers/cc-proxy -log debug
26-
EOT
27-
# Restart cc-proxy to provide debug logs.
28-
systemctl daemon-reload
29-
systemctl restart cc-proxy
30-
```
31-
2- Run a container to generate the agent debug logs
32-
33-
```
34-
sudo docker run -ti busybox true
35-
```
36-
3- Filter the agent debug logs from the `cc-proxy` logs
37-
The `cc-proxy` logs show the sources of its collated information. To only see
38-
the agent debug logs, filter `cc-proxy` logs by the QEMU serial console (the
39-
agent logs were sent through it). The QEMU serial console is represented by
40-
`source=qemu`.
41-
42-
```
43-
journalctl -u cc-proxy | grep source=qemu
44-
```
45-
46-
The debug log format is:
47-
DEBU[0019] *AGENT MESSAGE* source=qemu vm=*CONTAINER_ID*
15+
agent sends log data through a QEMU serial console, which is collected by
16+
[cc-proxy](https://github.com/clearcontainers/proxy) and shown in the proxy's
17+
logs.
18+
19+
By default, the Clear Containers agent logs are not collected by `cc-proxy`
20+
but can be enabled by enabling the proxy debug option:
21+
22+
1. Enable proxy debug
23+
24+
Set the `enable_debug=` option in the `[proxy.cc]` section to `true`, which
25+
assumes a standard configuration file path:
26+
27+
```
28+
$ sudo awk '{if (/^\[proxy\.cc\]/) {got=1}; if (got == 1 && /^#enable_debug/) {print "enable_debug = true"; got=0; next; } else {print}}' /usr/share/defaults/clear-containers/configuration.toml
29+
```
30+
31+
1. Run a container to generate the logs:
32+
33+
```
34+
$ sudo docker run -ti busybox true
35+
```
36+
37+
1. Filter the agent debug logs from the `cc-proxy` logs
38+
39+
The `cc-proxy` logs show the sources of its collated information. To only
40+
see the agent debug logs, filter `cc-proxy` logs by the QEMU serial console
41+
that sent the agent logs. The QEMU serial console is represented by
42+
`source=qemu`.
43+
44+
```
45+
$ sudo journalctl -t cc-proxy | grep source=qemu | egrep '\<cc-agent\>'
46+
```
47+
48+
To extract all logs entries for a particular container:
49+
50+
```
51+
$ sudo sudo journalctl -t cc-proxy | grep source=qemu | grep vm=CONTAINER_ID | egrep '\<cc-agent\>'
52+
```

‎docs/debug-kernel.md‎

Lines changed: 12 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -10,37 +10,34 @@ fails due to the kernel.
1010
To provide a debug log of the kernel boot, Clear Containers supports an option
1111
to enable kernel boot logs using the configuration file.
1212

13-
1. Enable debug in `configuration.toml`
13+
1. Enable kernel boot messages
14+
15+
Set the `enable_debug=` option in the `[proxy.cc]` section to `true`, which
16+
assumes a standard configuration file path:
1417

1518
```
16-
$ enable_debug = true
19+
$ sudo awk '{if (/^\[proxy\.cc\]/) {got=1}; if (got == 1 && /^#enable_debug/) {print "enable_debug = true"; got=0; next; } else {print}}' /usr/share/defaults/clear-containers/configuration.toml
1720
```
1821

19-
2. Run a container to generate the logs
22+
1. Run a container to generate the logs
2023

2124
```
2225
$ sudo docker run -ti busybox true
2326
```
2427

25-
3. Filter the kernel boot logs from the `cc-proxy` logs
28+
1. Filter the kernel boot logs from the `cc-proxy` logs
2629

2730
The `cc-proxy` logs show the sources of its collated information. To see
28-
the kernel boot logs, filter `cc-proxy` logs by the QEMU serial console.
31+
the kernel boot logs, filter `cc-proxy` logs by the QEMU serial console,
32+
excluding the agent messages.
2933
The QEMU serial console is represented by `source=qemu`.
3034

3135
```
32-
$ journalctl -u cc-proxy | grep source=qemu
36+
$ sudo journalctl -t cc-proxy | grep source=qemu | egrep -v '\<cc-agent\>'
3337
```
3438

35-
4. Obtain logs using standalone `cc-proxy`
36-
37-
In some cases it may be easier to run the `cc-proxy` standalone by first stopping
38-
the `cc-proxy` service and running it standalone to capture the logs.
39+
To extract all logs entries for a particular container:
3940

4041
```
41-
$ sudo systemctl stop cc-proxy
42-
$ sudo /usr/libexec/clear-containers/cc-proxy -log debug
42+
$ sudo sudo journalctl -t cc-proxy | grep source=qemu | grep vm=CONTAINER_ID | egrep -v '\<cc-agent\>'
4343
```
44-
45-
This will result in the `cc-proxy` log being printed on the terminal. This method
46-
can also be used to capture the logs in a specific `cc-proxy` file.

0 commit comments

Comments
 (0)