Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

266-1: add translate and fix bug #103

Closed
wants to merge 1 commit into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
86 changes: 86 additions & 0 deletions blog-en/pouch_with_diskquota.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
# PouchContainer with Diskquota

## What is diskquota

Diskquota is one kind of technology which is used to restrict filesystem's disk
usage. PouchContainer uses diskquota to limit filesystem disk space. We all know that
the way based on block devices could directly help limit disk space usage
easily via setting size of block device. While the way based on filesystem can
hardly do this. Diskquota is designed for limitting filesystem disk usage.
Currently PouchContainer supports diskquota which is based on graphdriver overlayfs.

Currently in underlying filesystems only ext4 and xfs support diskquota. In addition, there are three ways to make it: **user quota**, **group quota** and **project quota**.

There are two dimensions to limit disk usage:

- usage quota(block quota): setting disk usage limit for a filesystem directory(not for inode number);
- file quota(inode quota): restrict file or inode allocation.

PouchContainer only supports block quota now with no inode support temporarily.

## Diskquota in PouchContainer

Diskquota in PouchContainer relies on kernel version PouchContainer runs on. Here is a table
describing when each filesystem supports diskquota.

| | user/group quota | project quota
:-: | :-: | :-:
ext4 | \>= 2.6| \>= 4.5
xfs | \>= 2.6 | \>= 3.10

Although each filesystem in related kernel version supports diskquota, user still needs to install [quota-tools-4.04](https://nchc.dl.sourceforge.net/project/linuxquota/quota-tools/4.04/quota-4.04.tar.gz).
This quota tool has not packaged into PouchContainer rpm yet. We will do this in the future.

## Get Started

There are two ways in PouchContainer for a container to get involved in underlying filesystems. One is container rootfs, the other is container volume bind from host(outside of container) to inside. Both two dimensions are covered in PouchContainer.

### Container Rootfs diskquota

Users can set flag `--disk-quota` for a created container's rootfs to limit disk space usage, for example `--disk-quota 10g`. After setting this successfully, we can see rootfs size is 10GB via command `df -h`. And it shows that diskquota has taken effects.

```
$ pouch run -ti --disk-quota 10g registry.hub.docker.com/library/busybox:latest df -h
Filesystem Size Used Available Use% Mounted on
overlay 10.0G 24.0K 10.0G 0% /
tmpfs 64.0M 0 64.0M 0% /dev
shm 64.0M 0 64.0M 0% /dev/shm
tmpfs 64.0M 0 64.0M 0% /run
tmpfs 64.0M 0 64.0M 0% /proc/kcore
tmpfs 64.0M 0 64.0M 0% /proc/timer_list
tmpfs 64.0M 0 64.0M 0% /proc/sched_debug
tmpfs 1.9G 0 1.9G 0% /sys/firmware
tmpfs 1.9G 0 1.9G 0% /proc/scsi
```

### Volume Diskquota

Users can also setting volume's disk quota when creating one. It is quite easy to add a `--option` or `-o` flag to specify disk space limit to be desired number, for example `-o size=10g`.

After creating diskquota limited volume, users can bind this volume to a running container. In the following example, it executes command
`pouch run -ti -v volume-quota-test:/mnt registry.hub.docker.com/library/busybox:latest df -h`.
And in running container, directory `/mnt` is restricted to be size 10GB.

```
$ pouch volume create -n volume-quota-test -d local -o mount=/data/volume -o size=10g
Name: volume-quota-test
Scope:
Status: map[mount:/data/volume sifter:Default size:10g]
CreatedAt: 2018-3-24 13:35:08
Driver: local
Labels: map[]
Mountpoint: /data/volume/volume-quota-test

$ pouch run -ti -v volume-quota-test:/mnt registry.hub.docker.com/library/busybox:latest df -h
Filesystem Size Used Available Use% Mounted on
overlay 20.9G 212.9M 19.6G 1% /
tmpfs 64.0M 0 64.0M 0% /dev
shm 64.0M 0 64.0M 0% /dev/shm
tmpfs 64.0M 0 64.0M 0% /run
/dev/sdb2 10.0G 4.0K 10.0G 0% /mnt
tmpfs 64.0M 0 64.0M 0% /proc/kcore
tmpfs 64.0M 0 64.0M 0% /proc/timer_list
tmpfs 64.0M 0 64.0M 0% /proc/sched_debug
tmpfs 1.9G 0 1.9G 0% /sys/firmware
tmpfs 1.9G 0 1.9G 0% /proc/scsi
```
60 changes: 60 additions & 0 deletions blog-en/pouch_with_dragonfly.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
# PouchContainer with Dragonfly

Container technology is helpful to facilitate IT operation and maintenance, but the distribution of images has brought a huge challenge at the same time. The images can be as large as several Gibs, pulling images will be very slow, not to mention the situation lots of pulling requests simultaneously or low network bandwidth. Dragonfly can play an important role here, as a P2P technology, it provides very efficient distribution, avoiding images distribution as a bottleneck in container technology.

## What is Dragonfly

[Dragonfly](https://github.com/alibaba/Dragonfly#installation) is a P2P file distribution system. It solves the problems of time-consuming distribution, low success rate, and wasteful bandwidth in large-scale file distribution scenarios. Dragonfly significantly improves service capabilities such as data preheating, and large-scale container image distribution.

### composition

#### server

Server consists of cluster managers, it download files from source and constructs a P2P network.

#### client

Client concludes two part: dfget and proxy. dfget is a host-side tool for downloading files, proxy is used for intercepted the http requests and route them to dfget.

## Install dragonfly

1.install server

server can be deployed in two way, on the physical machine or on a container, the steps is simple in the [install dragonfly server](https://github.com/alibaba/Dragonfly/blob/master/docs/install_server.md)

2.install client

- Downloading [client package](https://github.com/alibaba/Dragonfly/blob/master/package/df-client.linux-amd64.tar.gz).
- untar the package `tar xzvf df-client.linux-amd64.tar.gz -C /usr/local/bin`, or you can untar it to any directory you like, but remember to add client path into PATH environment, `export PATH=$PATH:/usr/local/bin/df-client`.
- add server ip into client config file, `/etc/dragonfly.conf`, nodeIp is the host ip which you deploy server

```
[node]
address=nodeIp1,nodeIp2,...
```

about detail install information, you can find in [install dragonfly client](https://github.com/alibaba/Dragonfly/blob/master/docs/install_client.md)

## Run PouchContainer with dragonfly

1.start dragonfly proxy

Start dragonfly proxy, `/usr/local/bin/df-client/df-daemon --registry https://reg.docker.alibaba-inc.com`, you can add flag `--verbose` to get debug log. Dragonfly logs can be found in directory `~/.small-dragonfly/logs`.

More dragonfly usage information you can find in [dragonfly usage](https://github.com/alibaba/Dragonfly/blob/master/docs/usage.md)

2.add the following configuration in PouchContainer config file `/etc/pouch/config.json`

```
{
"image-proxy": "http://127.0.0.1:65001"
}
```

3.pull a image `reg.docker.alibaba-inc.com/base/busybox:latest`, you can find the following output in `~/.small-dragonfly/logs/dfdaemon.log`, which means dragonfly works.

```
time="2018-03-06 20:08:00" level=debug msg="pre access:http://storage.docker.aliyun-inc.com/docker/registry/v2/blobs/sha256/1b/1b5110ff48b0aa7112544e1666cc7199f812243ded4128f0a1b2be027c7 38bec/data?Expires=1520339335&OSSAccessKeyId=LTAIfYaNrksx0ktL&Signature=fVEYIQzIaXyqIcAhypbmzaUx5x8%3D"
time="2018-03-06 20:08:00" level=debug msg="post access:http://storage.docker.aliyun-inc.com"
time="2018-03-06 20:08:00" level=debug msg="pre access:http://storage.docker.aliyun-inc.com/docker/registry/v2/blobs/sha256/98/9869810a78644038c8d37a5a3344de0217cb37bcc2caa2313036c6948b0 ee8da/data?Expires=1520339335&OSSAccessKeyId=LTAIfYaNrksx0ktL&Signature=Tx7JYU07Gap8RfasvCe0JGAUCo4%3D"
```