docs: update wording of sync.md #5326
Draft
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
CaitinChen
commented
Nov 28, 2024
CaitinChen
commented
Nov 28, 2024
docs/en/guide/sync.md
Outdated
| The core of the layer-by-layer filtering mode is to first split the full path according to the directory level and combine it into multiple string sequences layer by layer. For example, the full path is `a1/b1/c1.txt`, and the resulting sequence is `a1`, `a1/b1`, `a1/b1/c1.txt`. Then each element in this sequence is regarded as the path in the full path filtering mode, and ["full path filtering"](#full-path-filtering-mode) is executed in sequence. | ||
|
|
||
| If an element matches a certain pattern, there will be two processing logics: | ||
| By default, `juicefs sync` performs incremental synchronization. It only overwrites files if their sizes are different. You can also use [`--update`](../reference/command_reference.md#sync) to overwrite files when the `mtime` of the source file has been updated. For scenarios with higher demand for data integrity, use [`--check-new`](../reference/command_reference.md#sync) or [`--check-all`](../reference/command_reference.md#sync) to perform byte-by-byte comparison between the source and the destination. |
docs/en/guide/sync.md
Outdated
| * If either end is a JuiceFS mount point, you can use [`juicefs stats`](../administration/fault_diagnosis_and_analysis.md#stats) to quickly check current IO status. | ||
| * If destination is a local disk, look for temporary files that end with `.tmp.xxx`, these are the temp files created by `sync`, they will be renamed upon transfer complete. Look for size changes in temp files to verify the current IO status. | ||
| * If both end are object storage services, use tools like `nethogs` to check network IO. | ||
| For data synchronization that involves JuiceFS, it is recommended to use the `jfs://` protocol instead of mounting JuiceFS and accessing its local directory. This approach bypasses the FUSE mount point and accesses JuiceFS directly. This process still requires the client configuration file. You should prepare it in advance using [`juicefs auth`](../reference/command_reference.md#auth). In large-scale scenarios, bypassing the FUSE mount point can save precious resources and improve data synchronization performance. When using the `jfs://` protocol, you can pass mounting parameters from `juicefs mount` to improve transfer performance, such as `--max-downloads`, `--max-uploads`, and `--buffer-size`. |
docs/en/guide/sync.md
Outdated
|
|
||
| - If the pattern is an exclude pattern, the "exclude" behavior will be returned directly as the final matching result; | ||
| - If the pattern is an include pattern, the subsequent patterns to be matched at this layer are skipped and the next layer is entered directly. | ||
| For full synchronization (where all files are synchronized regardless of their presence on the destination path), use [`--force-update`](../reference/command_reference.md#sync). |
docs/en/guide/sync.md
Outdated
|
|
||
| ### Directory structure and file permissions {#directory-structure-and-file-permissions} | ||
| * If you must use a FUSE mount point for data synchronization, for large file transfers, it is recommended to add the [`--flush-wait=60s`](../reference/command_reference.md#object-storage-options) option. This effectively helps reduce the number of slices and mitigates write amplification. For more information, see [troubleshooting write amplification](https://github.com/juicedata/docs/pull/administration/troubleshooting.md#write-amplification). |
docs/en/guide/sync.md
Outdated
|
|
||
| After sync, you can see all the files in [Object Storage A](#required-storages). | ||
| * Add the [`--verbose` or `--debug`](../reference/command_reference.md#global-options) option to the `juicefs sync` command to print debug logs. |
|
|
||
| Unlike the file-oriented disaster recovery backup, the purpose of creating a copy of JuiceFS data is to establish a mirror with exactly the same content and structure as the JuiceFS data storage. When the object storage in use fails, you can switch to the data copy by modifying the configurations. Note that only the file data of the JuiceFS file system is replicated, and the metadata stored in the metadata engine still needs to be backed up. | ||
| * Use [`juicefs stats`](../administration/fault_diagnosis_and_analysis.md#stats) to quickly check current I/O status. | ||
| * Review the [client log](../administration/fault_diagnosis_and_analysis.md#client-log) (default path: `/var/log/juicefs.log`) for [slow requests or timeout errors](../administration/troubleshooting.md#io-error-object-storage). |
CaitinChen
commented
Nov 28, 2024
|
I'll take over this pr and make changes to TOC to align most content with EE docs https://juicefs.com/docs/zh/cloud/guide/sync/ |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.