diff --git a/Images/login.png b/Images/login.png new file mode 100644 index 0000000..64bea4e Binary files /dev/null and b/Images/login.png differ diff --git a/Images/pouchrun.png b/Images/pouchrun.png new file mode 100644 index 0000000..72139ed Binary files /dev/null and b/Images/pouchrun.png differ diff --git a/Images/pouchrun2.png b/Images/pouchrun2.png new file mode 100644 index 0000000..cf45a69 Binary files /dev/null and b/Images/pouchrun2.png differ diff --git a/Images/vb.png b/Images/vb.png new file mode 100644 index 0000000..2c39b3b Binary files /dev/null and b/Images/vb.png differ diff --git a/Images/vb2.png b/Images/vb2.png new file mode 100644 index 0000000..0392153 Binary files /dev/null and b/Images/vb2.png differ diff --git a/Images/vb3.png b/Images/vb3.png new file mode 100644 index 0000000..e5d513c Binary files /dev/null and b/Images/vb3.png differ diff --git a/Images/vbdone.png b/Images/vbdone.png new file mode 100644 index 0000000..e2bef5d Binary files /dev/null and b/Images/vbdone.png differ diff --git a/Images/vdi.png b/Images/vdi.png new file mode 100644 index 0000000..1ef20fb Binary files /dev/null and b/Images/vdi.png differ diff --git "a/blog-cn/PouchContainer\344\270\216\346\231\256\347\275\227\347\261\263\344\277\256\346\226\257.md" "b/blog-cn/PouchContainer\344\270\216\346\231\256\347\275\227\347\261\263\344\277\256\346\226\257.md" new file mode 100644 index 0000000..6a81500 --- /dev/null +++ "b/blog-cn/PouchContainer\344\270\216\346\231\256\347\275\227\347\261\263\344\277\256\346\226\257.md" @@ -0,0 +1,76 @@ +# PouchContainer与Prometheus + +PouchContainer 通过[普罗米修斯](https://prometheus.io/)来支持各式各样的监视度量标准。 现在我们已经具备了基本的Go语言运行时和一些API延迟度量单位，我们计划未来在如下两个主要的方向增加新的机制: + +* 重要的pouchd监视度量 +* 在监视过程中的完整的API调用列表 + +## 如何添加新的衡量标准 + +在PouchContainer中我们倾向于使用普罗米修斯的 [度量和标签命名法则](https://prometheus.io/docs/practices/naming) 最佳实践. 所以当你需要添加新的度量时, 请务必遵循度量和标签命名法则。 + +我们使用普罗米修斯 [go-sdk](https://github.com/prometheus/client_golang) 来监视pouchd. 它支持计数器, 测量仪和总结性的测量类型. 如需获取更多信息, 请参阅 [度量类型](https://prometheus.io/docs/concepts/metric_types/). + +## How to use + +用户可以通过 `pouchd -l tcp://0.0.0.0:4243` 来在 `0.0.0.0:4243`启动pouchd监听, 然后发出`GET http://127.0.0.1:4243/metrics` 请求来获取普罗米修斯格式化的度量输出的完整列表 + +``` +# HELP go_gc_duration_seconds A summary of the GC invocation durations. +# TYPE go_gc_duration_seconds summary +go_gc_duration_seconds{quantile="0"} 0.000111176 +go_gc_duration_seconds{quantile="0.25"} 0.000198062 +go_gc_duration_seconds{quantile="0.5"} 0.000269599 +go_gc_duration_seconds{quantile="0.75"} 0.000474291 +go_gc_duration_seconds{quantile="1"} 0.002013351 +go_gc_duration_seconds_sum 0.021835193 +go_gc_duration_seconds_count 52 +# HELP go_goroutines Number of goroutines that currently exist. +# TYPE go_goroutines gauge +go_goroutines 22 +# HELP go_info Information about the Go environment. +# TYPE go_info gauge +go_info{version="go1.9"} 1 +... +# HELP http_request_size_bytes The HTTP request sizes in bytes. +# TYPE http_request_size_bytes summary +http_request_size_bytes{handler="prometheus",quantile="0.5"} NaN +http_request_size_bytes{handler="prometheus",quantile="0.9"} NaN +http_request_size_bytes{handler="prometheus",quantile="0.99"} NaN +http_request_size_bytes_sum{handler="prometheus"} 0 +http_request_size_bytes_count{handler="prometheus"} 0 +# HELP http_response_size_bytes The HTTP response sizes in bytes. +# TYPE http_response_size_bytes summary +http_response_size_bytes{handler="prometheus",quantile="0.5"} NaN +http_response_size_bytes{handler="prometheus",quantile="0.9"} NaN +http_response_size_bytes{handler="prometheus",quantile="0.99"} NaN +http_response_size_bytes_sum{handler="prometheus"} 0 +http_response_size_bytes_count{handler="prometheus"} 0 +# HELP pouch_image_pull_latency_microseconds Latency in microseconds to pull a image. +# TYPE pouch_image_pull_latency_microseconds summary +pouch_image_pull_latency_microseconds{image="docker.io/library/ubuntu:latest",quantile="0.5"} 3.7803132e+07 +pouch_image_pull_latency_microseconds{image="docker.io/library/ubuntu:latest",quantile="0.9"} 3.7803132e+07 +pouch_image_pull_latency_microseconds{image="docker.io/library/ubuntu:latest",quantile="0.99"} 3.7803132e+07 +pouch_image_pull_latency_microseconds_sum{image="docker.io/library/ubuntu:latest"} 3.7803132e+07 +pouch_image_pull_latency_microseconds_count{image="docker.io/library/ubuntu:latest"} 1 +# HELP process_cpu_seconds_total Total user and system CPU time spent in seconds. +# TYPE process_cpu_seconds_total counter +process_cpu_seconds_total 4.78 +# HELP process_max_fds Maximum number of open file descriptors. +# TYPE process_max_fds gauge +process_max_fds 1024 +# HELP process_open_fds Number of open file descriptors. +# TYPE process_open_fds gauge +process_open_fds 9 +# HELP process_resident_memory_bytes Resident memory size in bytes. +# TYPE process_resident_memory_bytes gauge +process_resident_memory_bytes 3.4521088e+07 +# HELP process_start_time_seconds Start time of the process since unix epoch in seconds. +# TYPE process_start_time_seconds gauge +process_start_time_seconds 1.51064406778e+09 +# HELP process_virtual_memory_bytes Virtual memory size in bytes. +# TYPE process_virtual_memory_bytes gauge +process_virtual_memory_bytes 4.91610112e+08 +``` + +然后我们可以设置一个新的Target来在普罗米修斯中爬取这个度量点的数据即可完工。 diff --git "a/blog-cn/PouchContainer\346\217\222\344\273\266\345\214\226.md" "b/blog-cn/PouchContainer\346\217\222\344\273\266\345\214\226.md" new file mode 100644 index 0000000..de7f169 --- /dev/null +++ "b/blog-cn/PouchContainer\346\217\222\344\273\266\345\214\226.md" @@ -0,0 +1,156 @@ +# PouchContainer with plugin + +为了运行在某些点引起的由用户提供的自定义代码，我们提供了了一个介绍golang 1.8的插入框架。此时在插入框架中我们能够让用户在文件点中加入自定义代码。 + + +* 守护进程开始之前 +* 守护进程结束前 +* 容器创建前 +* 容器开始前 +* 容器创建结束前 + +以上四点是由两个插入界面提供的，分别是DaemonPlugin和ContainerPlugin，定义如下： + + +``` +// DaemonPlugin defines in which place does pouchd support plugin +type DaemonPlugin interface { + // PreStartHook is invoked by pouchd before real start, in this hook user could start dfget proxy or other + // standalone process plugins + PreStartHook() error + + // PreStopHook is invoked by pouchd before daemon process exit, not a promise if daemon is killed, in this + // hook user could stop the process or plugin started by PreStartHook + PreStopHook() error +} + +// ContainerPlugin defines places where a plugin will be triggered in container lifecycle +type ContainerPlugin interface { + // PreCreate defines plugin point where receives an container create request, in this plugin point user + // could change the container create body passed-in by http request body + PreCreate(io.ReadCloser) (io.ReadCloser, error) + + // PreStart returns an array of priority and args which will pass to runc, the every priority + // used to sort the pre start array that pass to runc, network plugin hook always has priority value 0. + PreStart(interface{}) ([]int, [][]string, error) + + //NetworkGenericParams accepts the container id and env of this container and returns the priority of this endpoint + // and if this endpoint should enable resolver and a map which will be used as generic params to create endpoints of + // this container + PreCreateEndpoint(string, []string) (priority int, disableResolver bool, genericParam map[string]interface{}) +} + +``` + +这两个插入符号会被记为分享的目标文件中的‘DaemonPlugin’(后台运行插件)和‘ContainerPlugin’(容器插件），示例如下： + +``` +p, _ := plugin.Open("path_to_shared_object_file") +daemonPlugin, _ := p.Lookup("DaemonPlugin") +containerPlugin, _ := p.Lookup("ContainerPlugin") +``` + +## example + +定义两个只在对应点处打印某些日志在的插入符号 + +``` +package main + +import ( + "fmt" + "io" +) + +var ContainerPlugin ContPlugin + +type ContPlugin int + +var DaemonPlugin DPlugin + +type DPlugin int + +func (d DPlugin) PreStartHook() error { + fmt.Println("pre-start hook in daemon is called") + return nil +} + +func (d DPlugin) PreStopHook() error { + fmt.Println("pre-stop hook in daemon is called") + return nil +} + +func (c ContPlugin) PreCreate(in io.ReadCloser) (io.ReadCloser, error) { + fmt.Println("pre create method called") + return in, nil +} + +func (c ContPlugin) PreStart(interface{}) ([]int, [][]string, error) { + fmt.Println("pre start method called") + // make this pre-start hook run after network in container setup + return []int{-4}, [][]string{{"/usr/bin/touch", "touch", "/tmp/pre_start_hook"}}, nil +} + +func (c ContPlugin) PreCreateEndpoint(string, []string) (priority int, disableResolver bool, genericParam map[string]interface{}) { + fmt.Println("pre create endpoint") + return +} + +func main() { + fmt.Println(ContainerPlugin, DaemonPlugin) +} +``` + +然后建立某些命令行，如下所示： + +``` +go build -buildmode=plugin -ldflags "-pluginpath=plugins_$(date +%s)" -o hook_plugin.so +``` + +利用生成的分享目标文件，开始封装其中带有旗标的`--plugin=path_to_hook_plugin.so`，然后当你开始结束守护进程和创建容器的时候，在某些日志中会像如下这样： + + +``` +pre-start hook in daemon is called +pre create method called +pre-stop hook in daemon is called +``` + +当你开始一个容器时，config.json文件（位置在$home_dir/containerd/state/io.containerd.runtime.v1.linux/default/$container_id/config.json）会包含预开始外挂脚本，特别在上述代码中，例如： + +``` + "hooks": { + "prestart": [ + { + "args": [ + "libnetwork-setkey", + "f67df14e96fa4b94a6e386d0795bdd2703ca7b01713d48c9567203a37b05ae3d", + "8e3d8db7f72a66edee99d4db6ab911f8d618af057485731e9acf24b3668e25b6" + ], + "path": "/usr/local/bin/pouchd" + }, + { + "args": [ + "touch", + "/tmp/pre_start_hook" + ], + "path": "/usr/bin/touch" + } + ] + } +``` + +如果你准确的运用上诉代码，每一次你打开一个容器，在/tmp/pre_start_hook的文件会被触达。 + + +## usage + +* 在守护进程开始之前，你可以开始帮助进程，比如被pouchd以及生命周期跟pouchd一样的进程所需要的帮助进程，比如网络插件和网络代理。 + +* 在守护进程结束前，你可以优雅地结束帮助进程，但是这个事件并不一定会被触发，因为pouchd可能会被SIGKILL信号终止。 + +* 在容器创建前，你可以根据某些规则改变输入流，在某些公司他们会有一些编配编排系统，这些系统利用env去传递一些Pouch容器中的限制，然后你可以利用这些事件去转换env中的值转换到PouchContainer创建api中的ContainerConfig或HostConfig的属性。 + +* 在容器开始之前，你可以创建更多预开始钩子用于oci容器，其中在容器入口开始之前，你可以做一些具体的事情。优先级决定钩子的执行顺序。libnetwork钩子优先级为0，所以在容器设置中，如果钩子被期望在网络的容器之前设置，你需要设置优先级为一个大于0的数，反之亦然。 + +* 在容器创建停止之前，你可以返回这个停止点的优先级，如果这个停止点需要解析器和这个停止点的通用参数。 diff --git "a/blog-cn/\345\256\211\350\243\205\350\257\264\346\230\216.md" "b/blog-cn/\345\256\211\350\243\205\350\257\264\346\230\216.md" index 46658f8..2237e2a 100644 --- "a/blog-cn/\345\256\211\350\243\205\350\257\264\346\230\216.md" +++ "b/blog-cn/\345\256\211\350\243\205\350\257\264\346\230\216.md" @@ -1,232 +1,27 @@ 原文链接：[INSTALLATION.md](https://github.com/alibaba/pouch/blob/master/INSTALLATION.md) -# 快速入门 - -总共提供了两个快速入门，一个用于终端用户，另一个用于开发人员。 - -希望使用PouchContainer的终端用户，请阅读 [终端用户快速入门](#终端用户快速入门)以安装和探索PouchContainer。 - -希望开发PouchContainer的开发人员，请阅读[开发人员快速入门](#开发人员快速入门)以开始开发并参与项目！ - -##
终端用户快速入门
- -只需很少的步骤，您就可以在您的机器上自动安装PouchContainer。目前我们支持两种Linux发行版：Ubuntu和CentOS。 - -### Ubuntu - -要安装PouchContainer，您需要一个Ubuntu 16.04（Xenial LTS）在维护的版本。不支持存档和测试版本。 - -PouchContainer与Docker冲突，因此您必须在安装PouchContainer之前卸载Docker。 - -**准备工作** - -PouchContainer支持LXCFS以提供强隔离，因此您应首先安装LXCFS。默认情况下，LXCFS是被启用的。 - -``` bash -sudo apt-get install lxcfs -``` - -安装下列包以允许'apt'通过HTTPS使用仓库： - -``` bash -sudo apt-get install curl apt-transport-https ca-certificates software-properties-common -``` - -**1. 添加PouchContainer的官方GPG密钥** - -``` bash -curl -fsSL http://mirrors.aliyun.com/opsx/pouch/linux/debian/opsx@service.alibaba.com.gpg.key | sudo apt-key add - -``` - -通过搜索指纹的最后8个字符，验证您现在是否具有指纹 `F443 EDD0 4A58 7E8B F645 9C40 CF68 F84A BE2F 475F`的密钥。 - -``` bash -$ apt-key fingerprint BE2F475F -pub 4096R/BE2F475F 2018-02-28 - Key fingerprint = F443 EDD0 4A58 7E8B F645 9C40 CF68 F84A BE2F 475F -uid opsx-admin -``` - -**2. 建立PouchContainer仓库** - -在新主机上首次安装PouchContainer之前，您需要建立PouchContainer仓库。我们默认启用了`stabel` 仓库，因为您始终需要`stabel` 仓库。要添加 `test` 仓库，请在以下命令行中的单词 `stable` 之后添加单词 `test` 。在此之后，您可以从仓库安装和更新PouchContainer。 - -``` bash -sudo add-apt-repository "deb http://mirrors.aliyun.com/opsx/pouch/linux/debian/ pouch stable" -``` - -**3. 安装PouchContainer** - -安装最新版本的PouchContainer。 - -``` bash -# update the apt package index -sudo apt-get update -sudo apt-get install pouch -``` - -安装PouchContainer后，将创建 `pouch` 组，但该组中未添加任何用户。 - -**4. 启动PouchContainer** - -``` bash -sudo service pouch start -``` - -在此之后，您可以拉取一个镜像并运行PouchContainer容器。 - -### CentOS - -要安装PouchContainer，您需要一个CentOS 7在维护的版本。不支持存档和测试版本。 - -我们已将rpm包放到Aliyun镜像中，您可以使用PouchContainer仓库安装PouchContainer。如果您在一台新主机上第一次安装PouchContainer，则需要建立PouchContainer仓库。然后，您可以从仓库安装和更新PouchContainer。 - -**1. 安装yum-utils** - -安装所需的包。 yum-utils提供了yum-config-manager的实用工具。 - -``` bash -sudo yum install -y yum-utils -``` - -**2. 建立PouchContainer仓库** - -使用以下命令添加PouchContainer仓库。 - -``` bash -sudo yum-config-manager --add-repo http://mirrors.aliyun.com/opsx/opsx-centos7.repo -sudo yum update -``` - -注意：上述命令设置了 `stable` 仓库，您可以通过以下命令启用 `test` 仓库。 - -``` bash -sudo yum-config-manager --enable pouch-test -``` - -您可以通过运行 `yum-config-manager` 命令和 `--disable` 参数来禁用 `test` 仓库。要重新启用它，请使用 `--enable` 参数。使用以下命令可以禁用 `test` 仓库。 - -``` bash -sudo yum-config-manager --disable pouch-test -``` - -**3. 安装PouchContainer** - -运行以下命令以安装最新版本的PouchContainer。如果您是第一次在您的主机上安装PouchContainer，系统将提示您接受GPG密钥，并显示密钥的指纹。 - -``` bash -sudo yum install pouch -``` - -安装PouchContainer后，将创建 `pouch` 组，但该组中未添加任何用户。 - -**4. 启动PouchContainer** - -``` bash -sudo systemctl start pouch -``` - -在此之后，您可以拉取一个镜像并运行PouchContainer容器。 - -## 卸载pouch - -在Ubuntu上卸载 - -``` bash -sudo apt-get purge pouch -``` - -在CentOS上卸载 - -``` bash -sudo yum remove pouch -``` - -运行 `remove` 命令后，您主机上的镜像，容器，存储卷和自定义配置文件不会被自动删除。若要删除所有镜像，容器和存储卷，请执行以下命令： - -``` bash -sudo rm -rf /var/lib/pouch -``` - -##
开发人员快速入门
- -本指南提供了在裸机服务器或虚拟机上部署PouchContainer的步骤说明。作为开发人员，您需要通过源代码构建和测试PouchContainer二进制文件。要构建被称为"PouchContainer Daemon"的pouchd和被称为"PouchContainer CLI"的pouch，需要安装以下系统依赖项： - -* Linux Kernel 3.10+ -* Go 1.9.0+ -* containerd: 1.0.3 -* runc: 1.0.0-rc4 -* runv: 1.0.0 (option) - - -### 预安装 - -由于pouchd是一种容器引擎，而pouch是一个CLI工具，如果您希望通过pouch体验容器的管理能力，还需要几个额外的二进制文件： - -* [containerd](https://github.com/containerd/containerd): 行业标准的容器运行时环境； -* [runc](https://github.com/opencontainers/runc): 用于根据OCI规范生成和运行容器的CLI工具； -* [runv](https://github.com/hyperhq/runv): 基于监管服务的OCI运行时环境； - -以下是安装 `containerd` 和runc的shell脚本： - -``` shell -# install containerd -$ wget https://github.com/containerd/containerd/releases/download/v1.0.3/containerd-1.0.3.linux-amd64.tar.gz -$ tar -xzvf containerd-1.0.3.linux-amd64.tar.gz -C /usr/local -$ -# install runc -$ wget https://github.com/opencontainers/runc/releases/download/v1.0.0-rc4/runc.amd64 -P /usr/local/bin -$ chmod +x /usr/local/bin/runc.amd64 -$ mv /usr/local/bin/runc.amd64 /usr/local/bin/runc -``` - -### runV安装 - -如果您希望额外体验基于监管服务的虚拟化，您需要安装[runV](https://github.com/hyperhq/runv)。 - -有关使用runV体验PouchContainer的更多指南，包括runv安装，请参考[PouchContainer run with runv guide](docs/features/pouch_with_runV.md)。 - -### PouchContainer的构建和安装 - -安装完所有依赖后，您可以构建和安装PouchContainer Daemo和PouchContainer CLI。克隆仓库并检出任意您选择的分支（在以下示例中，检出的是主干分支）： - -``` shell -mkdir -p $GOPATH/src/github.com/alibaba/ -cd $GOPATH/src/github.com/alibaba/; git clone https://github.com/alibaba/pouch.git -cd pouch; git checkout master -``` - -名为 `build` 的Makefile target将编译当前工作目录中的pouch和pouchd二进制文件。或者您可以执行 `make install` 来构建二进制文件并将它们安装在目标目录中（默认情况下为 `/usr/local/bin` ）。 - -``` shell -make install -``` - -### 启动PouchContainer -安装了所有需要的二进制文件后，您可以通过以下方式启动pouchd： - -``` shell -$ pouchd -INFO[0000] starting containerd module=containerd revision=773c489c9c1b21a6d78b5c538cd395416ec50f88 version=v1.0.3 -INFO[0000] setting subreaper... module=containerd -INFO[0000] loading plugin "io.containerd.content.v1.content"... module=containerd type=io.containerd.content.v1 -INFO[0000] loading plugin "io.containerd.snapshotter.v1.btrfs"... module=containerd type=io.containerd.snapshotter.v1 -WARN[0000] failed to load plugin io.containerd.snapshotter.v1.btrfs error="path /var/lib/containerd/io.containerd.snapshotter.v1.btrfs must be a btrfs filesystem to be used with the btrfs snapshotter" module=containerd -INFO[0000] loading plugin "io.containerd.snapshotter.v1.overlayfs"... module=containerd type=io.containerd.snapshotter.v1 -INFO[0000] loading plugin "io.containerd.metadata.v1.bolt"... module=containerd type=io.containerd.metadata.v1 -WARN[0000] could not use snapshotter btrfs in metadata plugin error="path /var/lib/containerd/io.containerd.snapshotter.v1.btrfs must be a btrfs filesystem to be used with the btrfs snapshotter" module="containerd/io.containerd.metadata.v1.bolt" -INFO[0000] loading plugin "io.containerd.differ.v1.walking"... module=containerd type=io.containerd.differ.v1 -INFO[0000] loading plugin "io.containerd.grpc.v1.containers"... module=containerd type=io.containerd.grpc.v1 -``` - -在pouchd运行之后，您可以通过PouchContainer CLI与pouchd进行交互： - -```bash -$ pouch images -IMAGE ID IMAGE NAME SIZE -3e8fa85ddfef docker.io/library/busybox:latest 2699 -504cf109b492 docker.io/library/redis:alpine 2035 -``` - -## 反馈 - -我们希望本指南可以帮助您使用和运行PouchContainer。如果您有任何疑问，请随时通过[ISSUE](https://github.com/alibaba/pouch/issues/new)发送反馈。如果您希望在本指南中为PouchContainer做出贡献，请提交PR。 +# PouchContainer体验环境搭建与上手 +## 安装VirtualBox +- 打开[VirtualBox Downloads](https://www.virtualbox.org/wiki/Downloads) +- 根据平台选择对应的下载链接并安装VirtualBox + +## 下载Ubuntu/CentOS镜像 +- 从群共享中下载``UbuntuPouch.vdi`` 或 ``CentOS.vdi.zip`` 如选择后者则需解压。分别获得``UbuntuPouch.vdi`` 或 ``CentOS.vdi`` + +## 新建VirtualBox镜像 +- 打开新安装完成的VirtualBox![vb](https://raw.githubusercontent.com/Naville/blog/master/Images/vb.png), 点击左上角的``New``按钮. +- 在Name中给虚拟机起名，Type选择Linux, Version中如选择使用CentOS镜像则选择``Other Linux(64Bit)``, 如使用Ubuntu则选择``Ubuntu(64-bit)``。 如图所示: +![vb2](https://raw.githubusercontent.com/Naville/blog/master/Images/vb2.png) +- 给虚拟机分配内存，建议按照宿主机本身物理内存合理分配并至少分配1024MB以上。 +- 进入硬盘创建界面，选择``Use an existing virtual hard disk file``, 并点击途中打圈的位置选择第二部中解压的``.vdi``文件，这里使用Ubuntu为例![vb3](https://raw.githubusercontent.com/Naville/blog/master/Images/vb3.png) +- 创建完成，点击Start 启动虚拟机 ![vbdone](https://raw.githubusercontent.com/Naville/blog/master/Images/vbdone.png) + +## 登陆虚拟机并使用Pouch +启动虚拟机后界面如图所示: +![login](https://raw.githubusercontent.com/Naville/blog/master/Images/login.png) +输入登录用户名和密码。Ubuntu 用户名：pouch，密码：123456。Centos 镜像 用户名：root，密码：Ali88Baiji +下述命令中如为Ubuntu环境则需在开头加上``sudo ``并使用登录密码123456进行鉴权 +执行命令``systemctl start pouch``启动pouch服务 +执行``pouch run -t -d busybox sh``启动一个busybox基础容器, 输出如图所示: +![pouchrun](https://raw.githubusercontent.com/Naville/blog/master/Images/pouchrun.png) +执行``pouch exec -it {ID} sh``登入启动的容器，其中ID是上条命令输出的完整ID +中的前六位, 此处为ab9ad6, 如图所示: ![pouchrun2](https://raw.githubusercontent.com/Naville/blog/master/Images/pouchrun2.png) diff --git a/blog-en/Installation.md b/blog-en/Installation.md new file mode 100644 index 0000000..58ddf33 --- /dev/null +++ b/blog-en/Installation.md @@ -0,0 +1,27 @@ +# Getting Started with PouchContainer +## Installing VirtualBox +- Open [VirtualBox Downloads](https://www.virtualbox.org/wiki/Downloads) +- Choose the download link that matches your platform, then download and install VirtualBox. + +## Downloading Ubuntu/CentOS Image +- Download``UbuntuPouch.vdi`` or ``CentOS.vdi.zip`` from shared Files in DingTalk. If you pick the later one you'll need to unzip it as well. In either case you will have either``UbuntuPouch.vdi`` or ``CentOS.vdi`` + +## Creating new VirtualBox Image +- Open the brand new VirtualBox you've just installed ![vb](https://raw.githubusercontent.com/Naville/blog/master/Images/vb.png), then click the ``New`` button at top left. +- Give your virtual machine a name and type in in the ``Name`` section，for the ``Type`` section you should choose Linux. In ``Version``, if you are using the CentOS image you should pick``Other Linux(64Bit)``, otherwise if you are using the Ubuntu image then you should pick``Ubuntu(64-bit)``. As shown in the image: +![vb2](https://raw.githubusercontent.com/Naville/blog/master/Images/vb2.png) +- Allocate memory for the virtual machine, it's recommended for you to reasonably allocate RAM and you should allocate at least 1024MB. +- Enter HardDisk Creation Menu, choose ``Use an existing virtual hard disk file``, click the circled button then select the ``.vdi``file you created in step 1. Here we use Ubuntu as an example![vb3](https://raw.githubusercontent.com/Naville/blog/master/Images/vb3.png) +- You've finished creating VM, click ``Start`` to run the Virtual Machine![vbdone](https://raw.githubusercontent.com/Naville/blog/master/Images/vbdone.png) + +## Log into the Virtual Machine and use Pouch +After the Virtual Machine has booted, the interface looks like the following: +![login](https://raw.githubusercontent.com/Naville/blog/master/Images/login.png) +Enter your Username and Password. +Ubuntu Username：pouch, Password: 123456 +Centos Username:root, Password: Ali88Baiji +For the following command if you are using Ubuntu you'll need to add ``sudo `` prefixing the command then use the login password ``123456`` to authorize. +Execute command ``systemctl start pouch`` to start pouch service +Execute command ``pouch run -t -d busybox sh`` to start a basic busybox container, the output looks like the following: +![pouchrun](https://raw.githubusercontent.com/Naville/blog/master/Images/pouchrun.png) +Execute command``pouch exec -it {ID} sh``to log into the container, ``ID`` here is the first six digits of the output of the previous command, in the case it's ``ab9ad6``, as shown in the image: ![pouchrun2](https://raw.githubusercontent.com/Naville/blog/master/Images/pouchrun2.png) diff --git a/blog-en/PouchContainerIndustryQualityPractice.md b/blog-en/PouchContainerIndustryQualityPractice.md new file mode 100644 index 0000000..d61bf20 --- /dev/null +++ b/blog-en/PouchContainerIndustryQualityPractice.md @@ -0,0 +1,268 @@ +# 0.Prologue +As [PouchContainer](https://github.com/alibaba/pouch)'s feature continuously evolve and complete, this attracted many outside developers to join the development of this project. Since the coding style and habit of each contributor is different, the code reviewer's responsibility is not only focused on logic correctness and performance issues, but also coding style, because a unified coding style is the prerequisite of ensuring the code's ease of maintenance. Furthermore the coverage and stability of the test is also one of the key metrics the project is focusing. Let's think about it, in a project without regression testing, how could we ensure each code update won't affect present features. + +In this blog we will share the practices PouchContainer used in coding style guide and golang unit testing. + +# 1.Unified Coding Style +PouchContainer is a project built upon golang, in this project shell scripts will be used to complete some automatic process, like compiling and packaging. Furthermore, PouchContainer also contains many Markdown style documentation, it's user's entrance, its formatting correctness and spelling correctness is also something the project's focus point. The following content will introduce the tools used in PouchContainer's Coding Style Guide and their related usage scenario. + +## 1.1 Golinter - Unified Coding Style +Golang has a easy-to-use syntax design, plus the community has complete [CodeReview](https://github.com/golang/go/wiki/CodeReviewComments) guidelines from the very beginning, this makes most golang projects have the same coding style, and very unlikely to fall into some meaningless __religion__ arguments. On top of the community's existing foundation, PouchContainer defined some extra specific rules to constrain the developers in order to ensure the code's readability, for detail please read [Here](https://github.com/alibaba/pouch/blob/master/docs/contributions/code_styles.md#additional-style-rules) + +But enforcing coding style based on documentations only is hard to keep coding styles consistent, so golang, like other languages, officially provided basic toolchains like [golint](https://github.com/golang/lint), [gofmt](https://golang.org/cmd/gofmt)，[goimports](https://github.com/golang/tools/blob/master/cmd/goimports/doc.go) and [go vet](https://golang.org/cmd/vet),etc. This tools can check and unify coding styles prior to compiling, which in turn provides automation possibility for further operations like code review. Currently PouchContainer runs the aforementioned tools on CirclesCI __for each__ Pull Request. If the syntax check tools shows that something is wrong, the code reviewer has the rights to reject reviewing or even reject merging the code. + +Besides official tools, we can also choose third party tools provided by the community, for example [errcheck](https://github.com/kisielk/errcheck) checks if the developer has taken care of all the error returned by functions. However these tools doesn't have a unified output format, which makes it hard to combine the output of various tools. Fortunately someone from the opensource community provided a unified layer [gometalinter](https://github.com/alecthomas/gometalinter) which can combine all kinds of code check tools. The recommended toolset is: + +* [golint](https://github.com/golang/lint) - Google's (mostly stylistic) linter. +* [gofmt -s](https://golang.org/cmd/gofmt/) - Checks if the code is properly formatted and could not be further simplified. +* [goimports](https://godoc.org/golang.org/x/tools/cmd/goimports) - Checks missing or unreferenced package imports. +* [go vet](https://golang.org/cmd/vet/) - Reports potential errors that otherwise compile. +* [varcheck](https://github.com/opennota/check) - Find unused global variables and constants. +* [structcheck](https://github.com/opennota/check) - Find unused struct fields +* [errcheck](https://github.com/kisielk/errcheck) - Check that error return values are used. +* [misspell](https://github.com/client9/misspell) - Finds commonly misspelled English words. + +Each project can customize the ``gometalinter`` toolset based on their needs + +## 1.2 Shellcheck - Reducing potential issues of the shell script +While shellscript is powerful, it still needs syntax check to avoid some potential, unexpected errors. For example unused variables, while they doesn't affect the execution of the script, but its existence will become a burden for maintainers. + +```powershell +#!/usr/bin/env bash + +pouch_version=0.5.x + +dosomething() { + echo "do something" +} + +dosomething +``` + +PouchContainer will use [shellcheck](https://github.com/koalaman/shellcheck) to check shell scripts in the current project. Use the aforementioned code as a example, shellcheck check will results in a warning about unused variable.This tools could find potential issues of the shell script during code review stage and reduce the possibility of runtime error. + +```plain +In test.sh line 3: +pouch_version=0.5.x +^-- SC2034: pouch_version appears unused. Verify it or export it. +``` + +PouchContainer's current continuous integration will scan for all the ``.sh`` scripts and use shellcheck to check them, for details please read [Here](https://github.com/alibaba/pouch/blob/master/.circleci/config.yml#L21-L24) + +> NOTE: When shellcheck is being to strict, the project could avoid the check by adding comments, or globally turning off one of the checks. For detailed checker rules please read [here](https://github.com/koalaman/shellcheck/wiki) + +## 1.3 Markdownlint - Unified Documentation Layout + +As an open-source project, PouchContainer's documentation is as important as its code, because documentation is the best way for users to know about PouchContainer. The documentation is written in Markdown, its misspelling and misformating are both key focus points of the project. + +Just like code, merely using text constraints also results in misjudgement, so PouchContainer uses [markdownlint](https://github.com/markdownlint/markdownlint) and [misspell](https://github.com/client9/misspell) to check for formatting and spelling errors, these checks are as important as `golint` , and will be run in CircieCI for each Pull Request. The code reviewer has the rights to __reject__ reviewing or merging the code if something is wrong + +PouchContainer's current CI jobs will checks for all the formatting issues in Markdown documentations, as well as all spelling issues, for detailed configuration please check [here](https://github.com/alibaba/pouch/blob/master/.circleci/config.yml#L13-L20)。 + +> NOTE: When markdownlint is being to strict, the project could globally turning off one of the checks. For detailed checker rules please read [here](https://github.com/markdownlint/markdownlint/blob/master/docs/RULES.md)。 + +## 1.4 Summary + +The aforementioned contents are all style and guideline issues, PouchContainer automates all style and guideline checks and intergrate them into each code review to help reviews find potential issues. + +# 2. How to write the unit test of golang + +Unit test can be used to ensure the correctness of a single module. In the pyramid of test areas, the wider the unit test coverage is and the more comprehensive coverage is, the more it can reduce the debugging costs of integration testing and end-to-end testing. In a complex system, the longer the link processed by the task is, the higher the cost of the location problem will be, especially the problems caused by small modules. The following sections will share a summary of PouchContainer's preparation of golang unit test cases. + +## 2.1 Table-Driven Test - DRY + +To simplily understand of unit testing is to give a given input to a function to determine whether the expected output can be obtained. When the function being tested has a variety of input scenarios, we can organize our test cases in the form of Table-Driven, as code shown below. Table-Driven uses arrays to organize the test cases and validate the correctness of the function by loop execution. + +```go +// from https://golang.org/doc/code.html#Testing +package stringutil + +import "testing" + +func TestReverse(t *testing.T) { + cases := []struct { + in, want string + }{ + {"Hello, world", "dlrow ,olleH"}, + {"Hello, 世界", "界世 ,olleH"}, + {"", ""}, + } + for _, c := range cases { + got := Reverse(c.in) + if got != c.want { + t.Errorf("Reverse(%q) == %q, want %q", c.in, got, c.want) + } + } +} +``` + +To debugging and maintain of test cases easlily, we can add some auxiliary information to describe the current test. For example [reference](https://github.com/alibaba/pouch/blob/master/pkg/reference/parse_test.go#L54) want to test [punycode](https://en.wikipedia.org/wiki/ Punycode) input, if you do not add the word `punycode`, for code reviewers or project maintainers, they may not know the difference bewteen `xn--bcher-kva.tld/redis:3` and `docker.io/library/redis:3`. + + +```go +{ + name: "Normal", + input: "docker.io/library/nginx:alpine", + expected: taggedReference{ + Named: namedReference{"docker.io/library/nginx"}, + tag: "alpine", + }, + err: nil, +}, { + name: "Punycode", + input: "xn--bcher-kva.tld/redis:3", + expected: taggedReference{ + Named: namedReference{"xn--bcher-kva.tld/redis"}, + tag: "3", + }, + err: nil, +} +``` + +However, some functions are more complicated, and one input cannot be used as a complete test case. For example [TestTeeReader] (https://github.com/golang/go/blob/release-branch.go1.9/src/io/io_test.go#L284) , TeeReader read hello, world from buffer. After data being read, if you read it again, the expected behavior is meeting an end-of-file error. Such a test case needs to be done in a single case, which do not need to hard-bake the form of Table-Driven. + +In simple terms, if you test a function that needs to copy the most of the code, in theory, the test code can be extracted, and use Table-Driven to organize the test case. Don`t Repeat Yourself is our principle. + +> NOTE: The organization method of Table-Driven is recommended by the golang community. See [here](https://github.com/golang/go/wiki/TableDrivenTests) for details. + + +## 2.2 Mock - Simulating external dependencies + +Some dependence problems are often meet during the testing process. For example, the PouchContainer client requires an HTTP server, but this is too heavy for the unit, and this is a category of integration testing. So how to complete this part of the unit test? + +In the world of golang, the implementation of the interface belongs to [Duck Type](https://en.wikipedia.org/wiki/Duck_typing). An interface can have a variety of implementations, as long as the implementation conforms to the interface definition. If the external dependence are constrained by the interface, then the dependence behavior is simulated in the unit test. The following content will share two common test scene. + +### 2.2.1 RoundTripper + +Take the PouchContainer client test as an example. The PouchContainer client uses [http.Client](https://golang.org/pkg/net/http/#Client). The [RoundTripper](https://golang.org/pkg/net/http/#RoundTripper) interface is used in http.Client to perform an HTTP request, which allows developers to customize the logic for sending HTTP requests. This is also an important reason for golang to support the HTTP 2 protocol perfectly on the original basis. + +```plain +http.Client -> http.RoundTripper [http.DefaultTransport] +``` + +For the PouchContainer client, the test focus is mainly on whether the incoming destination address is correct, whether the incoming query is reasonable, and whether the result can be returned normally. So before testing, developers need to prepare the corresponding RoundTripper implementation, which is not responsible for the actual business logic, but only used to determine whether the input meets expectations. + +As the next code shown below, PouchContainer `newMockClient` accepts customized request processing logic. In the testing case of removing the mimirror, the developer determines in the customized logic whether the destination address and the HTTP Method are DELETE, then the functional test can be completed without starting the HTTP Server. + + +```go +// https://github.com/alibaba/pouch/blob/master/client/client_mock_test.go#L12-L22 +type transportFunc func(*http.Request) (*http.Response, error) + +func (transFunc transportFunc) RoundTrip(req *http.Request) (*http.Response, error) { + return transFunc(req) +} + +func newMockClient(handler func(*http.Request) (*http.Response, error)) *http.Client { + return &http.Client{ + Transport: transportFunc(handler), + } +} + +// https://github.com/alibaba/pouch/blob/master/client/image_remove_test.go +func TestImageRemove(t *testing.T) { + expectedURL := "/images/image_id" + + httpClient := newMockClient(func(req *http.Request) (*http.Response, error) { + if !strings.HasPrefix(req.URL.Path, expectedURL) { + return nil, fmt.Errorf("expected URL '%s', got '%s'", expectedURL, req.URL) + } + if req.Method != "DELETE" { + return nil, fmt.Errorf("expected DELETE method, got %s", req.Method) + } + + return &http.Response{ + StatusCode: http.StatusNoContent, + Body: ioutil.NopCloser(bytes.NewReader([]byte(""))), + }, nil + }) + + client := &APIClient{ + HTTPCli: httpClient, + } + + err := client.ImageRemove(context.Background(), "image_id", false) + if err != nil { + t.Fatal(err) + } +} +``` + +### 2.2.2 MockImageManager + +For dependencies between internal packages, for example, PouchContainer Image API Bridge depends on the PouchContainer Daemon ImageManager, in which the dependence behaviors are defined by interface. If we want to test the logic of Image Bridge, we don't have to start containerd, we just need to implement the corresponding Daemon ImageManager like RoundTripper. + + +```go +// https://github.com/alibaba/pouch/blob/master/apis/server/image_bridge_test.go +type mockImgePull struct { + mgr.ImageMgr + handler func(ctx context.Context, imageRef string, authConfig *types.AuthConfig, out io.Writer) error +} + +func (m *mockImgePull) PullImage(ctx context.Context, imageRef string, authConfig *types.AuthConfig, out io.Writer) error { + return m.handler(ctx, imageRef, authConfig, out) +} + +func Test_pullImage_without_tag(t *testing.T) { + var s Server + + s.ImageMgr = &mockImgePull{ + ImageMgr: &mgr.ImageManager{}, + handler: func(ctx context.Context, imageRef string, authConfig *types.AuthConfig, out io.Writer) error { + assert.Equal(t, "reg.abc.com/base/os:7.2", imageRef) + return nil + }, + } + req := &http.Request{ + Form: map[string][]string{"fromImage": {"reg.abc.com/base/os:7.2"}}, + Header: map[string][]string{}, + } + s.pullImage(context.Background(), nil, req) +} +``` + +### 2.2.3 Summary + +ImageManager and RoundTripper + +Besides the number of functions defined by the interface are different, the simulation ways are consistent. In general situlations, developers can manually defined a structure as fields by using of methods, shown in code below. + + +```go +type Do interface { + Add(x int, y int) int + Sub(x int, y int) int +} + +type mockDo struct { + addFunc func(x int, y int) int + subFunc func(x int, y int) int +} + +// Add implements Do.Add function. +type (m *mockDo) Add(x int, y int) int { + return m.addFunc(x, y) +} + +// Sub implements Do.Sub function. +type (m *mockDo) Sub(x int, y int) int { + return m.subFunc(x, y) +} +``` + +When the interface is large and complex, the manual method will impose a test burden on the developer, so the community provides automatic generation tools, such as [mockery](https://github.com/vektra/mockery), for easeing the burden on developers. + +## 2.3 Other methods + +Sometimes it depends on third-party services, for exapmle, the PouchContainer client, which is a typical case. The Duck Type described above can complete the test of this case. In addition, we can also complete the request processing by registering the http.Handler and starting mockHTTPServer. This testing is relatively heavy, it is recommended to consider the use of it when the Duck Type test is not possible, or put it into the integration test. + +> NOTE: Someone in the golang community has done [monkeypatch](https://github.com/bouk/monkey) by modifying the binary code. This tool is not recommended, we still recommend developers to design and write testable code. + +## 2.4 Summary + +PouchContainer integrates unit test cases into the code review phase, and reviewers can view the running of test cases at any time. + +# 3. Summary + +During the code revieW, code style checking, unit testing, and integration testing should be run through contin