diff --git "a/blog-cn/PouchContainer\345\256\211\350\243\205\346\225\231\347\250\213.md" "b/blog-cn/PouchContainer\345\256\211\350\243\205\346\225\231\347\250\213.md" new file mode 100644 index 0000000..4c9ee5f --- /dev/null +++ "b/blog-cn/PouchContainer\345\256\211\350\243\205\346\225\231\347\250\213.md" @@ -0,0 +1,253 @@ +# PouchContainer安装教程中文版 + +## 环境和前期准备 + + +- 宿主机系统:CentOS 7 / Ubuntu 16.04 +- 需要的软件:VirtualBox +- Virtualbox软件包获取方式: +使用阿里郎-管家-办公软件管理安装VirtualBox,默认版本为5.2.12。若没有阿里郎,请在钉盘上下载 + +Mac版本地址: +[Mac-下载地址](https://space.dingtalk.com/s/gwHOABma4QLOGlgkPQPaACBiMzk5ZWRjZTAyOGI0MTBkOGRkNTRjYzNkN2Q1NTFjOA) +   密码:p5Sb + +Windows版本地址: +[Windows-下载地址](https://space.dingtalk.com/s/gwHOABmLzwLOGlgkPQPaACBhNzNjYjI5NTYxMzQ0NmUwOWRmMTFlN2UzMTYxNDQ4Mw) +   密码:V7ms + +## 加载虚拟机和配置 + +### 虚拟机加载 +1. 在钉钉群中下载镜像,CentOS或Ubuntu都可以 +2. 打开Virtualbox,点击新建 +3. 虚拟机名称自起,类型选择`Linux`,版本选择`Linux 2.6/3.x/4.x(64-bit)` +4. 建议内存大小,默认`1024`即可 +5. 选择`使用已有的虚拟硬盘文件`,选择下载的虚拟机,点击创建 +6. 启动自己的虚拟机 + +### 虚拟机配置 + +#### CentOS +用户名:root + +密 码:Ali88Baiji + +尝试ping www.alibaba-inc.com,若能ping通,进入下一步 + +若不能,则要修改网卡信息,使用 `ip ad` 命令查看Virtualbox分配给虚拟机网卡的mac地址,如下图所示: + +![avatar](https://images2018.cnblogs.com/blog/761101/201807/761101-20180731012815511-1807884097.png) + +使用下面命令修改当前虚拟机的IP地址: + +``` +vi /etc/sysconfig/network-scripts/ifcfg-eth0 +``` +修改 `HWADDR=` 对应的值,修改为自己的查询结果(红框内的结果)。 + +#### Ubuntu +用户名:pouch + +密 码:123456 + +尝试ping www.alibaba-inc.com,若能ping通,进入下一步 + +## 安装PouchContainer +### CentOS +#### 1.安装所需的包yum-utils + +``` +sudo yum install -y yum-utils +``` +结果图: + +![avatar](https://images2018.cnblogs.com/blog/761101/201807/761101-20180731011533827-586132642.png) + + +#### 2. 添加PouchContainer库 + +``` +sudo yum-config-manager --add-repo http://mirrors.aliyun.com/opsx/opsx-centos7.repo +sudo yum update +``` +结果图: + +![avatar](https://images2018.cnblogs.com/blog/761101/201807/761101-20180731011449985-103277584.png) + + +#### 3. 安装PouchContainer + +``` +sudo yum install pouch +``` +结果图: + +![avatar](https://images2018.cnblogs.com/blog/761101/201807/761101-20180731011458334-832954266.png) + + +#### 4. 运行PouchContainer + +``` +//设置开机启动 +systemctl start pouch +//拉取远程镜像 +pouch pull busybox +``` +结果图: + +![avatar](https://images2018.cnblogs.com/blog/761101/201807/761101-20180731011509633-1996626722.png) + + +``` +//启动镜像,前6个字符为登陆需要的ID +pouch run -t -d busybox sh +``` +结果图: + +![avatar](https://images2018.cnblogs.com/blog/761101/201807/761101-20180731011519379-266427941.png) + + +``` +//登陆镜像,执行一些常见命令 +pouch exec -it {ID} sh +``` + +结果图: + +![avatar](https://images2018.cnblogs.com/blog/761101/201807/761101-20180731011527313-674343869.png) + +### Ubuntu + +#### 1.安装LXCFS + +``` +sudo apt-get install lxcfs +``` + +结果图: + +![avatar](https://images2018.cnblogs.com/blog/761101/201807/761101-20180731011541046-1004355148.png) + +#### 2.安装下列包以允许'apt'通过HTTPS使用仓库 + +``` +sudo apt-get install curl apt-transport-https ca-certificates software-properties-common +``` + +结果图: + +![avatar](https://images2018.cnblogs.com/blog/761101/201807/761101-20180731011649030-1905207872.png) + +#### 3.添加PouchContainer的官方GPG密钥 + +``` +curl -fsSL http://mirrors.aliyun.com/opsx/pouch/linux/debian/opsx@service.alibaba.com.gpg.key | sudo apt-key add - +``` + +结果图: + +![avatar](https://images2018.cnblogs.com/blog/761101/201807/761101-20180731011700640-614313500.jpg) + + +#### 4.通过搜索指纹的最后8个字符,验证您现在是否具有指纹`F443 EDD0 4A58 7E8B F645 9C40 CF68 F84A BE2F 475F`的密钥。 + +``` +apt-key fingerprint BE2F475F +``` + +结果图: + +![avatar](https://images2018.cnblogs.com/blog/761101/201807/761101-20180731011710188-616788657.jpg) + +#### 5.建立PouchContainer仓库 + +``` +sudo add-apt-repository "deb http://mirrors.aliyun.com/opsx/pouch/linux/debian/ pouch stable" +``` + +结果图: + +![avatar](https://images2018.cnblogs.com/blog/761101/201807/761101-20180731011718977-2015734432.jpg) + + +#### 6.安装PouchContainer + +``` +sudo apt-get update +sudo apt-get install pouch +``` + +结果图: + +![avatar](https://images2018.cnblogs.com/blog/761101/201807/761101-20180731011828447-1398873692.jpg) + +#### 7.启动PouchContainer(同CentOS环境) + +## shh远程登陆&文件共享 + +### shh远程登陆 + +#### 1.配置网卡为桥接模式 +virtualbox的NAT只支持虚拟机到宿主机的单向通信,即虚拟机可以ping通宿主机,但宿主机无法ping通虚拟机,故需将网络配置改为桥接模式 + +![avatar](https://cdn.nlark.com/lark/0/2018/png/123186/1532952006217-7fd4751c-f246-4008-8241-ae85af32ecf0.png) + +#### 2.安装ssh服务并开启,默认端口号是22 + +``` +apt-get install openssh-server +``` +![avatar](https://images2018.cnblogs.com/blog/761101/201807/761101-20180731094405012-1522115623.png) + +#### 3.查看ssh服务是否开启 + +![avatar](https://cdn.nlark.com/lark/0/2018/png/123186/1532952211218-fa86e7f1-3701-4cb9-bebd-c19bffed324c.png) + +#### 4.获取虚拟机IP + +![avatar](https://cdn.nlark.com/lark/0/2018/png/123186/1532952133466-32a66208-df68-47c7-9124-6d9dc8597a7f.png) + +#### 5.ssh登陆 + +``` +ssh 用户名@IP +``` +![avatar](https://cdn.nlark.com/lark/0/2018/png/123186/1532952276881-e3af58c4-7f23-4f40-81e1-b9a187ec9f5d.png) + + + + +### 文件共享 + +#### 1.配置共享文件夹 virtualbox -> 设置 -> 共享文件夹 + +![avatar](https://cdn.nlark.com/lark/0/2018/png/123186/1532952361541-99ba9094-7f9c-4a3b-ba7e-9822a2196ad7.png) + +#### 2.安装增强功能包: +如果要用VirtualBox自带的共享文件夹功能,必须先安装Guest Additions。 + +安装方法:置顶的菜单条->devices->Install Guest Additions + +![avatar](https://cdn.nlark.com/lark/0/2018/png/123186/1532952462469-cbd198d6-ac62-4565-ae4c-e76e0ebd8355.png) + +配置为第一IDE从通道 + +![avatar](https://cdn.nlark.com/lark/0/2018/png/123186/1532952473708-06dcb074-1358-48d5-bcb1-ddd590461790.png) + +#### 3.文件挂载 +如果遇到unknown filesystem type "vboxsf"问题,则执行下述命令即可: + +``` +apt-get update +apt-get install virtualbox-guest-utils +``` + +![avatar](https://cdn.nlark.com/lark/0/2018/png/123186/1532952524332-240c1604-9e57-464e-8132-fcecdc872906.png) + +然后重启虚拟机,重新挂载 + +![avatar](https://cdn.nlark.com/lark/0/2018/png/123186/1532951986751-fb248032-4172-4a34-87ac-270fef0f75da.png) + + + diff --git "a/blog-cn/\344\275\277\347\224\250TLS\344\277\235\346\212\244pouchd.md" "b/blog-cn/\344\275\277\347\224\250TLS\344\277\235\346\212\244pouchd.md" new file mode 100644 index 0000000..1ae72ed --- /dev/null +++ "b/blog-cn/\344\275\277\347\224\250TLS\344\277\235\346\212\244pouchd.md" @@ -0,0 +1,67 @@ +# 使用TLS保护pouchd + +当客户端仅通过同一个服务器连接pouchd时,只应该使用unix socket。这意味着需要通过参数 `-l unix:///var/run/pouchd.sock` 启动pouchd。如果pouchd通过远程服务器上的http客户端连接,它应该监听tcp端口,例如:`-l 0.0.0.0:4243`。在这种情况下,如果我们不进行tls保护,pouchd将忽略远程的身份并接受任何连接,这在生产环境中绝不安全。 + +为了验证客户端身份,应该创建CA来为pouchd和用户生成证书。 + +## 创建CA + +创建通用名称为pouch_test的CA,具体如下: + +```shell +openssl req -new -newkey rsa:2048 -days 3650 -nodes -x509 -subj "/C=CN/ST=ZheJiang/L=HangZhou/O=Company/OU=Department/CN=pouch_test" -keyout ca-key.pem -out ca.pem +``` + +应用上述指令后,当前目录下会出现两个文件,ca.pem和ca-key.pem。这两个文件就是CA,需要对它们保密。 + +## 创建证书来保护进程 + +为pouchd创建一个证书,它仅能用作为服务器证书。变量名为运行pouchd机器的主机名。 + +```shell +name=$HOSTNAME +mkdir -p ${name} +# 创建key +/usr/bin/openssl genrsa -out ${name}/key.pem 2048 +# 创建csr +/usr/bin/openssl req -subj "/C=CN/ST=ZheJiang/L=HangZhou/O=Company/OU=Department/CN=${name}" -new -key ${name}/key.pem -out ${name}/$name.csr +# 生成证书 +/usr/bin/openssl x509 -req -days 3650 -in ${name}/${name}.csr -CA ca.pem -CAkey ca-key.pem -CAcreateserial -out ${name}/cert.pem -extfile <(echo "extendedKeyUsage = serverAuth") +cp ca.pem ${name}/ca.pem +``` + +应用上述指令后,我们有了一个目录,其中包含了所有我们需要用来设置pouchd的tls保护的文件,以此来用类似如下的配置开始pouchd(用正确的值替换变量): + +```shell +--tlsverify --tlscacert=${name}/ca.pem --tlscert=${name}/cert.pem --tlskey=${name}/key.pem +``` + +当pouchd使用这些tls配置启动时,它监听的tcp地址只能由使用同一CA发布证书的客户端连接。 + +## 创建客户端证书 + +```shell +name=a_client +mkdir -p ${name} +# 创建key +/usr/bin/openssl genrsa -out ${name}/key.pem 2048 +# 创建csr +/usr/bin/openssl req -subj "/C=CN/ST=ZheJiang/L=HangZhou/O=Company/OU=Department/CN=${name}" -new -key ${name}/key.pem -out ${name}/$name.csr +# 生成证书 +/usr/bin/openssl x509 -req -days 3650 -in ${name}/${name}.csr -CA ca.pem -CAkey ca-key.pem -CAcreateserial -out ${name}/cert.pem -extfile <(echo "extendedKeyUsage = clientAuth") +cp ca.pem ${name}/ca.pem +``` + +应用上述指令后,我们有了一个目录,其中包含了所有我们需要用来设置pouchd的tls保护的文件,以此来用类似如下的配置来开始pouchd(用正确的值替换变量): + +```shell +--tlsverify --tlscacert=${name}/ca.pem --tlscert=${name}/cert.pem --tlskey=${name}/key.pem +``` + +然后我们有了一个目录,包含了所有我们需要的文件来使用PouchContainer客户端。例如:使用此证书作为身份证明来获取pouchd服务版本: + +``` +./pouch -H ${server_hostname}:4243 --tlsverify --tlscacert=${path}/ca.pem --tlscert=${path}/cert.pem --tlskey=${path}/key.pem version +``` + +当没有证书或未由同一CA发布的证书的客户端尝试连接到具有TLS保护的pouchd时,连接将被拒绝。 diff --git "a/blog-cn/\344\275\277\347\224\250runV\347\232\204PouchContainer.md" "b/blog-cn/\344\275\277\347\224\250runV\347\232\204PouchContainer.md" new file mode 100644 index 0000000..025ceef --- /dev/null +++ "b/blog-cn/\344\275\277\347\224\250runV\347\232\204PouchContainer.md" @@ -0,0 +1,137 @@ +# 使用runV的PouchContainer + +近来容器得到了迅速的发展。它为应用程序打包和资源利用改进提供了极大的便利。基于LXC的容器技术带来便利的同时,也损伤了相应的安全性。具体来说,容器共享一台机器的操作系统内核。一旦一个容器试图攻击内核,该主机的所有所有工作负载都将受到影响。 + +在一些高敏感性和高安全性要求的场景下,纯容器技术存在明显缺陷。更重要的是,在云时代,多租户是云用户的刚性要求。因此,强隔离必须得到保障。换句话讲,容器技术需要更多的安全加固。 + +[runV](https://github.com/hyperhq/runv) 是 [OCI](https://github.com/opencontainers/runtime-spec)基于管理程序的运行时。通过虚拟化以清晰边界隔离容器和主机的客户内核,runV可以轻松地帮助主机和容器提升安全性。虽然基于管理程序的容器为安全性带来了更多承诺,但它也因虚拟化技术失去了一些性能。且由于需完成更多工作(例如initrd.img加载、内核加载、系统进程启动),启动容器花费的时间随之增长。 + +## 架构 + +支持基于管理程序的OCI运行是PouchContainer的目标之一。PouchContainer允许用户决定要创建哪种容器。因此,通过统一输入PouchContainer的API,用户可以创建基于管理程序的容器和基于LXC的容器。使用以上两种运营商时,用户的应用程序可以根据需要灵活选择运行时。 +以下是PouchContainer支持runV和runC的架构: + +![pouch_with_runv_architecture](https://cdn.nlark.com/yuque/0/2018/png/153684/1532963146910-c872e457-7684-49be-9fd3-53ef6c12c421.png) + +## 安装预备 + +在安装之前,需要提醒一件重要的事情: **带有runV的PouchContainer只能在物理机上运行**. 目前尚不支持嵌套的VM。 此外,我们应该确保在[INSTALLATION.md](https://github.com/alibaba/pouch/blob/master/INSTALLATION.md)中描述的物理机上已经安装了 `containerd` and `pouchd` 。 + +确保已经完成上述工作。在体验基于管理程序的容器之前,还需要安装其他三个先决条件: + +* [QEMU](https://www.qemu.org): 通用机器模拟器和虚拟器。 +* [runv](https://github.com/hyperhq/runv): OCI兼容的二进制运行时。 +* [hyperstart](https://github.com/hyperhq/hyperstart): 一种基于管理程序容器的小型init服务。 + +### 安装QEMU + +需要[QEMU](https://www.qemu.org) 以运行VM。通过执行以下命令我们可以轻松地安装QEMU相关的工具。 + +在安装了Ubuntu OS的物理机上: + +``` +sudo apt-get install -y qemu qemu-kvm +``` + +在安装了RedHat系列OS的物理机上: + +``` +sudo yum install -y qemu qemu-kvm +``` + +### 安装runV + +[runv](https://github.com/hyperhq/runv) 不提供二进制包。我们需要通过源代码构建它。 + +首先,从GitHub克隆runV项目的v1.0.0版本: + +``` +mkdir -p $Home/go/src/github.com/hyperhq +cd $Home/go/src/github.com/hyperhq +git clone --branch v1.0.0 https://github.com/hyperhq/runv.git +export GOPATH=$HOME/go +``` + +其次,从源代码构建runV: + +``` +sudo apt-get install autotools-dev +sudo apt-get install automake +cd runv +./autogen.sh +./configure +sudo make +sudo make install +``` + +接着二进制的runV将位于您的路径中。 + +### 安装hyperstart + +[hyperstart](https://github.com/hyperhq/hyperstart) 为基于管理程序的容器提供init任务。我们同样需要从源代码版本v1.0.0来构建客内核和initrd.img: + +``` +cd $Home/go/src/github.com/hyperhq +git clone --branch v1.0.0 https://github.com/hyperhq/hyperstart.git +cd hyperstart +./autogen.sh +./configure +sudo make +``` + +在成功构建内核和initrd.img之后,我们应该将 `内核` and `initrd.img` 复制到runV将查找的默认目录下: + +``` +mkdir /var/lib/hyper/ +cp build/{kernel,hyper-initrd.img} /var/lib/hyper/ +``` + +## 启动基于管理程序的容器 + +在安装runV相关工具后,我们需要启动pouchd。然后我们可以通过命令行工具创建基于管理程序的`容器`。创建的容器具有与主机隔离的独立内核。 + +我们可以通过在命令创建中添加一个标志 `--运行时` 来创建基于管理程序的容器来列举容器。我们还可以使用 `pouch ps` 来列举运行时类型为`runv` 的基于管理程序的容器和基于runc的运行时类型为 `runc`的containerd . + +``` shell +$ pouch create --name hypervisor --runtime runv docker.io/library/busybox:latest +container ID: 95c8d52154515e58ab267f3c33ef74ff84c901ad77ab18ee6428a1ffac12400d, name: hypervisor +$ +$ pouch ps +Name ID Status Image Runtime +hypervisor 95c8d5 created docker.io/library/busybox:latest runv +4945c0 4945c0 stopped docker.io/library/busybox:latest runc +1dad17 1dad17 stopped docker.io/library/busybox:latest runv +fab7ef fab7ef created docker.io/library/busybox:latest runv +505571 505571 stopped docker.io/library/busybox:latest runc +``` + +更重要的是,我们能够检查物理主机和基于管理程序容器的不同内核,如下所示: + +``` shell +# one host physical machine +$ uname -a +Linux www 4.4.0-101-generic #124-Ubuntu SMP Fri Nov 10 18:29:59 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux +$ +# gen tunnel into hypervisor-based container +# check inside kernel +$ pouch start hypervisor -i +/ # uname -a +Linux 4.12.4-hyper #18 SMP Mon Sep 4 15:10:13 CST 2017 x86_64 GNU/Linux +``` + +事实证明,在以上实验中,物理主机的内核是4.4.0-101-generic,而基于管理程序的容器的内核是4.12.4-hyper。显然,它们在内核方面是互相隔离的。 + +## 运行原内核 + +runV(现在是kataconatiners)提供一种通用方法以提供基于OCI兼容映像的隔离Linux内核。坦白讲,由runV提供的在Guest OS中运行的Linux内核是非常先进和新颖的。但是如何在Guest OS中运行原的Libux内核仍然是业界面临的巨大挑战。 + +### 场景 + +使runV提供运行传统Linux内核的Guest OS(如Linux内核2.6.32)是非常合理的。对于整个行业,有太多负载在传统的Linux内核上运行。如果所有这些负载只能在原内核上运行(通常企业中的运营团队无法承担将旧Linux内核升级到4.12或4.14等新内核的风险),那么这样环境下的应用程序无法利用容器和Kubernetes等尖端技术的优势。事实上,毫无疑问,无法获得应用程序的发布速度在互联网时代将是致命的。 + +### 原内核支持 + +然而,PouchContainer提出了一种方法来容纳必须在原内核中运行的应用程序。这种解决方法仍在使用runV。但是这中方法需要做很多工作以将新的内核功能向后端移植到原内核中(此部分某种程度上不易开源)。一些演示如何在runV中提供的客户操作系统中运行Linuxneihe 2.6.32:[make legacy kernel in Guest OS](https://www.youtube.com/watch?v=1w5Ams2k-40). + +## 结论 +PouchContainer带来一种提供基于管理程序容器的普遍方式。通过PouchContainer,用户可以根据特定方案利用基于管理程序容器和基于LXC容器的优势。 diff --git a/blog-en/PouchContainer Engineering Quality Practice.md b/blog-en/PouchContainer Engineering Quality Practice.md new file mode 100644 index 0000000..76c1272 --- /dev/null +++ b/blog-en/PouchContainer Engineering Quality Practice.md @@ -0,0 +1,272 @@ +# 0. Introduction + +As the functionality of [PouchContainer](https://github.com/alibaba/pouch) continues to be iterated and refined, the project has grown in size, attracting a number of external developers to participate in the development of the project. Because each contributor's coding habits are different, the code reviewer's responsibility is not only to focus on logical correctness and performance issues, but also on code style, because a unified code specification is a prerequisite for maintaining project code maintainability. In addition to the unified project code style, the coverage and stability of test cases is also the focus of the project. In simple terms, in the absence of regression test cases, how to ensure that each code update does not affect existing features? + +This article shares PouchContainer's practices in code style specifications and golang unit test cases. + +# 1. Unified coding style specification + +PouchContainer is a project built by the golang language, which uses shell scripts to perform automated operations such as compiling and packaging. In addition to golang and shell scripts, PouchContainer also contains a large number of Markdown-style documents, which is the entry point for users to know and understand PouchContainer. Its standard layout and correct spelling are also the focus of the project. The following sections describe the tools and usage scenarios that PouchContainer uses in coding style specifications. + +## 1.1 Golinter - Unicode format + +Golang's grammar design is simple,and the community has a complete [CodeReview](https://github.com/golang/go/wiki/CodeReviewComments) guide from the beginning, so most of the golang projects have the same code style,and rarely fall into the unnecessary __religious__ dispute.On the basis of the community, PouchContainer also defines some specific rules to stipulate the developer, in order to ensure the readability of the code, the specific content can be read .[here](https://github.com/alibaba/pouch/blob/master/docs/contributions/code_styles.md#additional-style-rules)。 + +However, it is difficult to ensure that the project code style is consistent by relying on written agreements to make specifications. So golang, like other languages, provides the official toolchain, such as [golint](https://github.com/golang/lint) , [gofmt](https://golang.org/cmd/gofmt) , [goimports](https://github.com/golang/tools/blob/master/cmd/goimports/doc.go) , [go vet](https://golang.org/cmd/vet) and so on.These tools can check and unify the code style before compilation, providing automation for subsequent processes such as code review.Currently, PouchContainer runs the above code checking tool in CircleCI __every__ Pull Request submitted by the developer.If the inspection tool displays an exception, the code reviewer has the right to __refuse__ the review and may even reject the merge code. + +In addition to the official tools provided, we can also select third-party code inspection tools in the open source community, such as [errcheck](https://github.com/kisielk/errcheck) to check if the developer has processed the error returned by the function.However, these tools do not have a uniform output format, which makes it difficult to integrate the output of different tools. Fortunately, some people in the open source community have implemented this unified interface, [gometalinter](https://github.com/alecthomas/gometalinter),which can integrate various code checking tools. The recommended combination is: + +* [golint](https://github.com/golang/lint) - Google's (mostly stylistic) linter. +* [gofmt -s](https://golang.org/cmd/gofmt/) - Checks if the code is properly formatted and could not be further simplified. +* [goimports](https://godoc.org/golang.org/x/tools/cmd/goimports) - Checks missing or unreferenced package imports. +* [go vet](https://golang.org/cmd/vet/) - Reports potential errors that otherwise compile. +* [varcheck](https://github.com/opennota/check) - Find unused global variables and constants. +* [structcheck](https://github.com/opennota/check) - Find unused struct fields +* [errcheck](https://github.com/kisielk/errcheck) - Check that error return values are used. +* [misspell](https://github.com/client9/misspell) - Finds commonly misspelled English words. + +Each project can be customized according to your needs gometalinter package. + +## 1.2 Shellcheck - Reduce potential problems with shell scripts + +Although shell scripts are powerful, they still require syntax checking to avoid potential, unpredictable errors. For example, the definition of unused variables, although it does not affect the use of the script, but its existence will become a burden on the project maintainer. + +```powershell +#!/usr/bin/env bash + +pouch_version=0.5.x + +dosomething() { + echo "do something" +} + +dosomething +``` + + +PouchContainer will use [shellcheck](https://github.com/koalaman/shellcheck) to check the shell script in the current project. Taking the above code as an example, shellcheck detection will get a warning of unused variables. This tool can detect potential problems with shell scripts during the code review phase, reducing the chance of runtime errors. + +```plain +In test.sh line 3: +pouch_version=0.5.x +^-- SC2034: pouch_version appears unused. Verify it or export it. +``` + +PouchContainer's current continuous integration task scans the .sh scripts in the project and checks them one by one using shellcheck. See here for details. + +PouchContainer's current continuous integration task scans the `.sh` scripts in the project and checks them one by one using shellcheck.See [here] for details.(https://github.com/alibaba/pouch/blob/master/.circleci/config.yml#L21-L24)。 + + +> NOTE: When the shellcheck check is too strict, the project can be bypassed by a comment, or a check can be closed in the project. Specific inspection rules can be found [here](https://github.com/koalaman/shellcheck/wiki)。 + +## 1.3 Markdownlint - Unified Document Formatting + +PouchContainer is an open source project whose documentation is as important as the code, because documentation is the best way for users to understand PouchContainer. The document is written in the form of markdown, and its formatting and spelling errors are the focus of the project. + +As with the code, there is a text convention or a missed judgment, so PouchContainer use [markdownlint](https://github.com/markdownlint/markdownlint) and [misspell](https://github.com/client9/misspell) to check the document format and spelling errors.These checks have the same status as `golint` and will run in CircleCI every time the Pull Request is run. Once an exception occurs, the code reviewer has the right to __refuse__ to review or merge the code. + +PouchContainer's current continuous integration task checks the markdown document formatting in the project and also checks the spelling in all files. The configuration can be found [here](https://github.com/alibaba/pouch/blob/master/.circleci/config.yml#L13-L20). + +> NOTE: When the markdownlint requirement is too strict, the corresponding check can be closed in the project. Specific inspection items can be found [here](https://github.com/markdownlint/markdownlint/blob/master/docs/RULES.md). + +## 1.4 summary + +All of the above are style discipline issues, and PouchContainer automates code specification detection and integrates into each code review to help reviewers identify potential problems. + +# 2. How to write a unit test for golang + +Unit testing can be used to ensure the correctness of a single module. In the pyramid of test areas, the wider the unit test coverage and the more comprehensive coverage, the more it can reduce the debugging costs of integration testing and end-to-end testing. In a complex system, the longer the link processed by the task, the higher the cost of the location problem, especially the problems caused by small modules. The following sections share a summary of PouchContainer's preparation of golang unit test cases. + +## 2.1 Table-Driven Test - DRY + +A simple understanding of unit testing is to give a given input to a function to determine if the expected output can be obtained. When the function being tested has a variety of input scenarios, we can organize our test cases in the form of Table-Driven, as shown in the next code. Table-Driven uses arrays to organize test cases and validate the correctness of the function by loop execution. + +```go +// from https://golang.org/doc/code.html#Testing +package stringutil + +import "testing" + +func TestReverse(t *testing.T) { + cases := []struct { + in, want string + }{ + {"Hello, world", "dlrow ,olleH"}, + {"Hello, 世界", "界世 ,olleH"}, + {"", ""}, + } + for _, c := range cases { + got := Reverse(c.in) + if got != c.want { + t.Errorf("Reverse(%q) == %q, want %q", c.in, got, c.want) + } + } +} +``` + +To facilitate debugging and maintenance of test cases, we can add some auxiliary information to describe the current test. For example, if you want to test the input of punycode, if you don't include the word punycode, they may not know xn--bcher-kva.tld/redis:3 and docker.io/library for code reviewers or project maintainers. /redis: The difference between 3. +To facilitate debugging and maintenance of test cases, we can add some auxiliary information to describe the current test.For example, when [reference](https://github.com/alibaba/pouch/blob/master/pkg/reference/parse_test.go#L54) wants to test the input of [punycode](https://en.wikipedia.org/wiki/Punycode) ,if you don't include the word `punycode` ,the code reviewer or project maintainer may not know the differences between `xn--bcher-kva.tld/redis:3` and `docker.io/library/redis:3` . + +```go +{ + name: "Normal", + input: "docker.io/library/nginx:alpine", + expected: taggedReference{ + Named: namedReference{"docker.io/library/nginx"}, + tag: "alpine", + }, + err: nil, +}, { + name: "Punycode", + input: "xn--bcher-kva.tld/redis:3", + expected: taggedReference{ + Named: namedReference{"xn--bcher-kva.tld/redis"}, + tag: "3", + }, + err: nil, +} +``` + + +However, some functions are more complicated, and one input cannot be used as a complete test case. For example, [TestTeeReader](https://github.com/golang/go/blob/release-branch.go1.9/src/io/io_test.go#L284) , After TeeReader reads hello, world from buffer, the data has been read. If you read it again, the expected behavior is an end-of-file error. Such a test case needs to be done in a single case, without the need to hard-bake the form of Table-Driven. + +To put it simply, if you test a function that needs to copy most of the code, theoretically the test code can be extracted and used to organize test cases using Table-Driven.Don`t Repeat Yourself is our principle. + +> NOTE: The Table-Driven organization is recommended by the golang community, please check [here](https://github.com/golang/go/wiki/TableDrivenTests) for details. + +## 2.2 Mock - Simulating external dependencies + +There are often problems with dependencies during the testing process. For example, the PouchContainer client requires an HTTP server, but this is too heavy for the unit, and this is a category of integration testing. So how do you complete this part of the unit test? + +In the world of golang, the implementation of the interface belongs to [Duck Type](https://en.wikipedia.org/wiki/Duck_typing) An interface can have a variety of implementations, as long as the implementation conforms to the interface definition. If the external dependencies are constrained by the interface, then the dependency behavior is simulated in the unit test. The following content will share two common test scenarios. + +### 2.2.1 RoundTripper + +Take the PouchContainer client test as an example. The PouchContainer client uses [http.Client](https://golang.org/pkg/net/http/#Client).The http.Client uses the [RoundTripper](https://golang.org/pkg/net/http/#RoundTripper) nterface to perform an HTTP request, which allows developers to customize the logic for sending HTTP requests. This is also an important reason why golang can perfectly support the HTTP2 protocol on an original basis. + +```plain +http.Client -> http.RoundTripper [http.DefaultTransport] +``` + +For the PouchContainer client, the test focus is mainly on whether the incoming destination address is correct, whether the incoming query is reasonable, and whether the result can be returned normally. So before testing, developers need to prepare the corresponding RoundTripper implementation, which is not responsible for the actual business logic, it is only used to determine whether the input meets expectations. + + +As shown in the next code, PouchContainer `newMockClient` accepts custom request processing logic. In the use case of testing to remove the image, the developer determines in the custom logic whether the destination address and the HTTP Method are DELETE, so that the functional test can be completed without starting the HTTP Server. + +```go +// https://github.com/alibaba/pouch/blob/master/client/client_mock_test.go#L12-L22 +type transportFunc func(*http.Request) (*http.Response, error) + +func (transFunc transportFunc) RoundTrip(req *http.Request) (*http.Response, error) { + return transFunc(req) +} + +func newMockClient(handler func(*http.Request) (*http.Response, error)) *http.Client { + return &http.Client{ + Transport: transportFunc(handler), + } +} + +// https://github.com/alibaba/pouch/blob/master/client/image_remove_test.go +func TestImageRemove(t *testing.T) { + expectedURL := "/images/image_id" + + httpClient := newMockClient(func(req *http.Request) (*http.Response, error) { + if !strings.HasPrefix(req.URL.Path, expectedURL) { + return nil, fmt.Errorf("expected URL '%s', got '%s'", expectedURL, req.URL) + } + if req.Method != "DELETE" { + return nil, fmt.Errorf("expected DELETE method, got %s", req.Method) + } + + return &http.Response{ + StatusCode: http.StatusNoContent, + Body: ioutil.NopCloser(bytes.NewReader([]byte(""))), + }, nil + }) + + client := &APIClient{ + HTTPCli: httpClient, + } + + err := client.ImageRemove(context.Background(), "image_id", false) + if err != nil { + t.Fatal(err) + } +} +``` + +### 2.2.2 MockImageManager + +For dependencies between internal packages, such as the PouchContainer Image API Bridge depends on the PouchContainer Daemon ImageManager, and the dependency behavior is defined by interface. If we want to test the logic of Image Bridge, we don't have to start containerd , we just need to implement the corresponding Daemon ImageManager like RoundTripper. + +```go +// https://github.com/alibaba/pouch/blob/master/apis/server/image_bridge_test.go +type mockImgePull struct { + mgr.ImageMgr + handler func(ctx context.Context, imageRef string, authConfig *types.AuthConfig, out io.Writer) error +} + +func (m *mockImgePull) PullImage(ctx context.Context, imageRef string, authConfig *types.AuthConfig, out io.Writer) error { + return m.handler(ctx, imageRef, authConfig, out) +} + +func Test_pullImage_without_tag(t *testing.T) { + var s Server + + s.ImageMgr = &mockImgePull{ + ImageMgr: &mgr.ImageManager{}, + handler: func(ctx context.Context, imageRef string, authConfig *types.AuthConfig, out io.Writer) error { + assert.Equal(t, "reg.abc.com/base/os:7.2", imageRef) + return nil + }, + } + req := &http.Request{ + Form: map[string][]string{"fromImage": {"reg.abc.com/base/os:7.2"}}, + Header: map[string][]string{}, + } + s.pullImage(context.Background(), nil, req) +} +``` + +### 2.2.3 Summary + +ImageManager and RoundTripper are modeled in the same way except for the number of functions defined by the interface. In general, developers can manually define a structure that uses methods as fields, as shown in the next code. + +```go +type Do interface { + Add(x int, y int) int + Sub(x int, y int) int +} + +type mockDo struct { + addFunc func(x int, y int) int + subFunc func(x int, y int) int +} + +// Add implements Do.Add function. +type (m *mockDo) Add(x int, y int) int { + return m.addFunc(x, y) +} + +// Sub implements Do.Sub function. +type (m *mockDo) Sub(x int, y int) int { + return m.subFunc(x, y) +} +``` + +When the interface is large and complex, the manual method will impose a test burden on the developer, so the community provides automatically generated tools, such as [mockery](https://github.com/vektra/mockery),to ease the burden on the developer. + +## 2.3 Other methods +Sometimes it relies on third-party services, the PouchContainer client is a typical case. +The above describes Duck Type could complete the test of this case. In addition, we can also complete the request processing by registering the http.Handler and starting mockHTTPServer. This way of test is heavy, it is recommended to consider the use of the Duck Type test, or put it into the integration test. + +> NOTE: The golang community has done a [monkeypatch](https://github.com/bouk/monkey) by modifying the binary code. This tool is not recommended, we recommended that developers design and write testable code. + +## 2.4 Summary + +PouchContainer integrates unit test cases into the code review phase, and reviewers can view the running of test cases at any time. + +# 3. Summary + +In the code review phase, code style checking, unit testing, and integration testing should be run through continuous integration to help reviewers make accurate decisions. Currently, PouchContainer mainly performs code style checking and testing through TravisCI/CircleCI and [pouchrobot](https://github.com/pouchcontainer/pouchrobot). \ No newline at end of file diff --git a/blog-en/PouchContainer Installation Tutorial.md b/blog-en/PouchContainer Installation Tutorial.md new file mode 100644 index 0000000..38249f0 --- /dev/null +++ b/blog-en/PouchContainer Installation Tutorial.md @@ -0,0 +1,260 @@ + +# PouchContainer installation tutorial + +## Environment and preparation + + +- Host system: CentOS 7 / Ubuntu 16.04 +- Required software: VirtualBox +- How to get the Virtualbox package: +Use the Arirang-Housekeeper-Office software to manage the installation of VirtualBox. The default version is 5.2.22. If you don't have Arirang, please download it on the DingTalk plate. + +Mac version address: +[Mac-address](https://space.dingtalk.com/s/gwHOABma4QLOGlgkPQPaACBiMzk5ZWRjZTAyOGI0MTBkOGRkNTRjYzNkN2Q1NTFjOA) +   passwd: p5Sb + +Windows version address: +[Windows-address](https://space.dingtalk.com/s/gwHOABmLzwLOGlgkPQPaACBhNzNjYjI5NTYxMzQ0NmUwOWRmMTFlN2UzMTYxNDQ4Mw) +   passwd: V7ms + + +## Loading virtual machines and configurations + +### Virtual machine loading +1. Download the image in the DingTalk group, either CentOS or Ubuntu +2. Open Virtualbox and click `New` +3. Set the name of the virtual machine by yourself, the type is `Linux`, and the version is `Linux 2.6/3.x/4.x (64-bit)`. +4. Recommended memory size, the default `1024` +5. Select `Use existing virtual hard disk file`, select the downloaded virtual machine image, and click `Create`. +6. Start virtual machine + +### Virtual Machine Configuration + +#### CentOS +username:root + +passwd:Ali88Baiji + +Try to ping www.alibaba-inc.com. + +If success, go to the next step. + +If not, modify the NIC information and use the `ip ad` command to view the mac address assigned to the virtual machine NIC by Virtualbox, as shown in the following figure: + +![avatar](https://images2018.cnblogs.com/blog/761101/201807/761101-20180731012815511-1807884097.png) + + +Use the following command to modify the IP address of the current virtual machine: + +``` +vi /etc/sysconfig/network-scripts/ifcfg-eth0 +``` +Modify the value corresponding to `HWADDR=` and change it to your own query result (the result in the red box). +#### Ubuntu +username:pouch + +passwd:123456 + +Try to ping www.alibaba-inc.com. + +If success, go to the next step. + + + +## Install PouchContainer +### CentOS +#### 1. Install the required package yum-utils + +``` +sudo yum install -y yum-utils +``` +result: +![avatar](https://images2018.cnblogs.com/blog/761101/201807/761101-20180731011533827-586132642.png) + + +#### 2. add PouchContainer repo + +``` +sudo yum-config-manager --add-repo http://mirrors.aliyun.com/opsx/opsx-centos7.repo +sudo yum update +``` +result: + +![avatar](https://images2018.cnblogs.com/blog/761101/201807/761101-20180731011449985-103277584.png) + + +#### 3. install PouchContainer + +``` +sudo yum install pouch +``` +result: + +![avatar](https://images2018.cnblogs.com/blog/761101/201807/761101-20180731011458334-832954266.png) + + +#### 4. start PouchContainer + +``` +//Set boot +systemctl start pouch +//Pull remote mirror +pouch pull busybox +``` +result: + +![avatar](https://images2018.cnblogs.com/blog/761101/201807/761101-20180731011509633-1996626722.png) + + +``` +//Start the image, the first 6 characters are the ID required for login +pouch run -t -d busybox sh +``` +result: + +![avatar](https://images2018.cnblogs.com/blog/761101/201807/761101-20180731011519379-266427941.png) + + + +``` +//Log in to the image and execute some commands +pouch exec -it {ID} sh +``` + +result: + +![avatar](https://images2018.cnblogs.com/blog/761101/201807/761101-20180731011527313-674343869.png) + + +### Ubuntu + +#### 1.install LXCFS + +``` +sudo apt-get install lxcfs +``` + +result: + +![avatar](https://images2018.cnblogs.com/blog/761101/201807/761101-20180731011541046-1004355148.png) + +#### 2.Install packages to allow 'apt' to use a repository over HTTPS: + +``` +sudo apt-get install curl apt-transport-https ca-certificates software-properties-common +``` + +result: + +![avatar](https://images2018.cnblogs.com/blog/761101/201807/761101-20180731011649030-1905207872.png) + +#### 3.add PouchContainer's official GPG key + +``` +curl -fsSL http://mirrors.aliyun.com/opsx/pouch/linux/debian/opsx@service.alibaba.com.gpg.key | sudo apt-key add - +``` + +result: + +![avatar](https://images2018.cnblogs.com/blog/761101/201807/761101-20180731011700640-614313500.jpg) + + +#### 4.Verify that you now have the key with the fingerprint F443 EDD0 4A58 7E8B F645 9C40 CF68 F84A BE2F 475F, by searching for the last 8 characters of the fingerprint. + +``` +apt-key fingerprint BE2F475F +``` + +result: + +![avatar](https://images2018.cnblogs.com/blog/761101/201807/761101-20180731011710188-616788657.jpg) + +#### 5.add PouchContainer repo + +``` +sudo add-apt-repository "deb http://mirrors.aliyun.com/opsx/pouch/linux/debian/ pouch stable" +``` + +result: + +![avatar](https://images2018.cnblogs.com/blog/761101/201807/761101-20180731011718977-2015734432.jpg) + + +#### 6.install PouchContainer + +``` +sudo apt-get update +sudo apt-get install pouch +``` + +result: + +![avatar](https://images2018.cnblogs.com/blog/761101/201807/761101-20180731011828447-1398873692.jpg) + +#### 7.start PouchContainer(such as CentOS) + +## shh remote login & file sharing + +### shh remote login + +#### 1.Configure the NIC to be in bridge mode +The virtualbox NAT only supports one-way communication from the virtual machine to the host. That is, the virtual machine can ping the host, but the host cannot ping the virtual machine, so the network configuration needs to be changed to the bridge mode. + +![avatar](https://cdn.nlark.com/lark/0/2018/png/123186/1532952006217-7fd4751c-f246-4008-8241-ae85af32ecf0.png) + +#### 2.Install the ssh service and start it. The default port is 22 + +``` +apt-get install openssh-server +``` +![avatar](https://images2018.cnblogs.com/blog/761101/201807/761101-20180731094405012-1522115623.png) + +#### 3.Check the ssh service is started. + +![avatar](https://cdn.nlark.com/lark/0/2018/png/123186/1532952211218-fa86e7f1-3701-4cb9-bebd-c19bffed324c.png) + +#### 4.Get virtual machine IP + +![avatar](https://cdn.nlark.com/lark/0/2018/png/123186/1532952133466-32a66208-df68-47c7-9124-6d9dc8597a7f.png) + +#### 5.ssh remote login + +``` +ssh usrname@IP +``` +![avatar](https://cdn.nlark.com/lark/0/2018/png/123186/1532952276881-e3af58c4-7f23-4f40-81e1-b9a187ec9f5d.png) + + + + +### File Sharing + +#### 1.Configure shared folder virtualbox -> settings -> shared folder + +![avatar](https://cdn.nlark.com/lark/0/2018/png/123186/1532952361541-99ba9094-7f9c-4a3b-ba7e-9822a2196ad7.png) + +#### 2.Install Guest Additions +If you want to use the shared folder feature that comes with VirtualBox, you must first install Guest Additions. + +Installation method: Top menu bar -> devices-> Install Guest Additions + +![avatar](https://cdn.nlark.com/lark/0/2018/png/123186/1532952462469-cbd198d6-ac62-4565-ae4c-e76e0ebd8355.png) + +set the first IDE slave channel + +![avatar](https://cdn.nlark.com/lark/0/2018/png/123186/1532952473708-06dcb074-1358-48d5-bcb1-ddd590461790.png) + +#### 3.File mount +If you encounter a problem with 'unknown filesystem type "vboxsf"', execute the following command: + +``` +apt-get update +apt-get install virtualbox-guest-utils +``` + +![avatar](https://cdn.nlark.com/lark/0/2018/png/123186/1532952524332-240c1604-9e57-464e-8132-fcecdc872906.png) + +Then restart the virtual machine and remount file. + +![avatar](https://cdn.nlark.com/lark/0/2018/png/123186/1532951986751-fb248032-4172-4a34-87ac-270fef0f75da.png) +