diff --git a/README.md b/README.md index 7dbcf1e..58d166e 100755 --- a/README.md +++ b/README.md @@ -26,6 +26,7 @@ - Latest gdrive api used i.e v3 - Pretty logging - Easy to install and update +- An additional sync script for background synchronisation jobs. Read [Synchronisation](#synchronisation) section for more info. ## Table of Contents @@ -44,9 +45,13 @@ - [Generating Oauth Credentials](#generating-oauth-credentials) - [First Run](#first-run) - [Upload](#upload) - - [Custom Flags](#custom-flags) + - [Upload Script Custom Flags](#upload-script-custom-flags) - [Multiple Inputs](#multiple-inputs) - [Resuming Interrupted Uploads](#resuming-interrupted-uploads) +- [Additional Usage](#additional-usage) + - [Synchronisation](#synchronisation) + - [Basic Usage](#basic-usage) + - [Sync Script Custom Flags](#sync-script-custom-flags) - [Uninstall](#Uninstall) - [Reporting Issues](#reporting-issues) - [Contributing](#contributing) @@ -93,19 +98,29 @@ The script explicitly requires the following programs: | file/mimetype | Mimetype generation for extension less files | | find | To find files and folders for recursive folder uploads | | xargs | For parallel uploading | +| mkdir | To create folders | +| rm | To remove files and folders | | grep | Miscellaneous | | sed | Miscellaneous | +| cat | Miscellaneous ( only sync.sh ) | +| diff | To detect new files in a folder ( only sync.sh ) | +| ps | To manage background jobs ( only sync.sh ) | +| tail | To show indefinite logs ( only sync.sh ) | ### Installation You can install the script by automatic installation script provided in the repository. +This will also install the synchronisation script provided in the repo. + Default values set by automatic installation script, which are changeable: **Repo:** `labbots/google-drive-upload` **Command name:** `gupload` +**Sync command name:** `gsync` + **Installation path:** `$HOME/.google-drive-upload` **Source:** `release` { can be `branch` } @@ -136,6 +151,10 @@ This section provides information on how to utilise the install.sh script for cu These are the flags that are available in the install.sh script: +
+ +Click to expand + - -i | --interactive Install script interactively, will ask for all the variables one by one. @@ -154,6 +173,8 @@ These are the flags that are available in the install.sh script: Custom command name, after installation, script will be available as the input argument. + To change sync command name, use install sh -c gupload sync='gsync' + --- - -r | --repo @@ -205,6 +226,7 @@ E.g: ```shell bash <(curl --compressed -s https://raw.githubusercontent.com/labbots/google-drive-upload/master/install.sh) -r username/reponame -p somepath -s shell_file -c command_name -B branch_name ``` +
### Updation @@ -277,7 +299,7 @@ If `gdrive_folder_name` is present on gdrive, then script will upload there, els Apart from basic usage, this script provides many flags for custom usecases, like parallel uploading, skipping upload of existing files, overwriting, etc. -### Custom Flags +### Upload Script Custom Flags These are the custom flags that are currently implemented: @@ -285,13 +307,11 @@ These are the custom flags that are currently implemented: Override default config file with custom config file. - Default Config: `"${HOME}/.googledrive.conf` + Default Config: `${HOME}/.googledrive.conf` If you want to change the default value of the config path, then use this format, - ```shell - gupload --config default=your_config_file_path - ``` + `gupload --config default=your_config_file_path` --- @@ -307,9 +327,7 @@ These are the custom flags that are currently implemented: If you want to change the default value of the rootdir stored in config, then use this format, - ```shell - gupload --root-dir default=root_folder_[id/url] - ``` + `gupload --root-dir default=root_folder_[id/url]` --- @@ -457,6 +475,148 @@ Uploads interrupted either due to bad internet connection or manual interruption - No progress bars for resumable uploads as it messes up with output. - You can interrupt many times you want, it will resume ( hopefully ). +## Additional Usage + +### Synchronisation + +This repo also provides an additional script ( [sync.sh](https://github.com/labbots/google-drive-upload/blob/master/sync.sh) ) to utilise upload.sh for synchronisation jobs, i.e background jobs. + +#### Basic Usage + +To create a sync job, just run + +`gsync folder_name -d gdrive_folder` + +Here, folder_name is the local folder you want to sync and gdrive_folder is google drive folder name. + +In the local folder, all the contents present or added in the future will be automatically uploaded. + +Note: Giving gdrive_folder is optional, if you don't specify a name with -d/--directory flags, then it will take the name of the local folder. + +Also, gdrive folder creation works in the same way as gupload command. + +Default wait time: 3 secs ( amount of time to wait before checking new files ). + +Default gupload arguments: None ( see -a/--arguments section below ). + +#### Sync Script Custom Flags + +Read this section thoroughly to fully utilise the sync script, feel free to open an issue if any doubts regarding the usage. + +
+ +Click to expand + +- -d | --directory + + Specify gdrive folder name, if not specified then local folder name is used. + + --- + +- -j | --jobs + + See all background jobs that were started and still running. + + Use -j/--jobs v/verbose to show additional information for jobs. + + Additional information includes: CPU usage & Memory usage and No. of failed & successful uploads. + + --- + +- -p | --pid + + Specify a pid number, used for --jobs or --kill or --info flags, multiple usage allowed. + + --- + +- -i | --info + + Print information for a specific job. These are the methods to do it: + + - By specifying local folder and gdrive folder of an existing job, + + e.g: `gsync local_folder -d gdrive folder -i` + + - By specifying pid number, + + e.g: `gsync -i -p pid_number` + + - To show info of multiple jobs, use this flag multiple times, + + e.g: `gsync -i pid1 -p pid2 -p pid3`. You can also use it with multiple inputs by adding this flag. + + --- + +- -k | --kill + + Kill background jobs, following are methods to do it: + + - By specifying local_folder and gdrive_folder, + + e.g. `gsync local_folder -d gdrive_folder -k`, will kill that specific job. + + - pid ( process id ) number can be used as an additional argument to kill a that specific job, + + e.g: `gsync -k -p pid_number`. + + - To kill multiple jobs, use this flag multiple times, + + e.g: `gsync -k pid1 -p pid2 -p pid3`. You can also using it with multiple inputs with this flag. + + - This flag can also be used to kill all the jobs, + + e.g: `gsync -k all`. This will stop all the background jobs running. + + --- + +- -t | --time time_in_seconds + + The amount of time that sync will wait before checking new files in the local folder given to sync job. + + e.g: `gsync -t 4 local_folder`, here 4 is the wait time. + + To set default time, use `gsync local_folder -t default=4`, it will stored in your default config. + + --- + +- -l | --logs + + To show the logs after starting a job or show log of existing job. + + - By specifying local_folder and gdrive_folder, + + e.g. `gsync local_folder -d gdrive_folder -l`, will show logs of that specific job. + + - pid ( process id ) number can be used as an additional argument to show logs of a specific job, + + e.g: `gsync -l -p pid_number`. + + Note: If used with multiple inputs or pid numbers, then only first pid/input log is shown, as it goes on indefinitely. + + --- + +- -a | --arguments + + As the script uses gupload, you can specify custom flags for background job, + + e.g: `gsync local_folder -a '-q -p 4 -d'` + + To set some arguments by default, use `gsync -a default='-q -p 4 -d'`. + + In this example, will skip existing files, 4 parallel upload in case of folder. + + --- + +- -d | --debug + + Display script command trace, use before all the flags to see maximum script trace. + + --- + +Note: Flags that use pid number as input should be used at last, if you are not intending to provide pid number, say in case of a folder name with positive integers. + +
+ ## Uninstall If you have followed the automatic method to install the script, then you can automatically uninstall the script. diff --git a/install.sh b/install.sh index 6dc8870..37ddd37 100755 --- a/install.sh +++ b/install.sh @@ -9,7 +9,10 @@ All flags are optional.\n Options:\n -i | --interactive - Install script interactively, will ask for all the varibles one by one.\nNote: This will disregard all arguments given with below flags.\n -p | --path - Custom path where you want to install script.\nDefault Path: %s/.google-drive-upload \n - -c | --cmd - Custom command name, after installation script will be available as the input argument.\nDefault Name: upload \n + -c | --cmd - Custom command name, after installation script will be available as the input argument. + To change sync command name, use %s -c gupload sync='gsync' + Default upload command: gupload + Default sync command: gsync\n -r | --repo - Upload script from your custom repo,e.g --repo labbots/google-drive-upload, make sure your repo file structure is same as official repo.\n -R | --release - Specify tag name for the github repo, applies to custom and default repo both.\n -B | --branch - Specify branch name for the github repo, applies to custom and default repo both.\n @@ -199,7 +202,7 @@ _get_latest_sha() { LATEST_SHA="$(curl --compressed -s https://api.github.com/repos/"${3:-${REPO}}"/releases/"${2:-${TYPE_VALUE}}" | _json_value tag_name)" ;; esac - echo "${LATEST_SHA}" + printf "%s\n" "${LATEST_SHA}" } ################################################### @@ -346,6 +349,7 @@ _update_config() { _variables() { REPO="labbots/google-drive-upload" COMMAND_NAME="gupload" + SYNC_COMMAND_NAME="gsync" INFO_PATH="${HOME}/.google-drive-upload" INSTALL_PATH="${HOME}/.google-drive-upload/bin" UTILS_FILE="utils.sh" @@ -357,7 +361,7 @@ _variables() { if [[ -r ${INFO_PATH}/google-drive-upload.info ]]; then source "${INFO_PATH}"/google-drive-upload.info fi - __VALUES_ARRAY=(REPO COMMAND_NAME INSTALL_PATH CONFIG TYPE TYPE_VALUE SHELL_RC) + __VALUES_ARRAY=(REPO COMMAND_NAME SYNC_COMMAND_NAME INSTALL_PATH CONFIG TYPE TYPE_VALUE SHELL_RC) } ################################################### @@ -389,9 +393,9 @@ _start_interactive() { } ################################################### -# Install the script +# Install the upload and sync script # Globals: 10 variables, 6 functions -# Variables - INSTALL_PATH, INFO_PATH, UTILS_FILE, COMMAND_NAME, SHELL_RC, CONFIG, +# Variables - INSTALL_PATH, INFO_PATH, UTILS_FILE, COMMAND_NAME, SYNC_COMMAND_NAME, SHELL_RC, CONFIG, # TYPE, TYPE_VALUE, REPO, __VALUES_ARRAY ( array ) # Functions - _print_center, _newline, _clear_line # _get_latest_sha, _update_config @@ -407,9 +411,11 @@ _install() { _clear_line 1 _print_center "justify" "Latest sha fetched." "=" && _print_center "justify" "Downloading script.." "-" if curl --compressed -Ls "https://raw.githubusercontent.com/${REPO}/${LATEST_CURRENT_SHA}/${UTILS_FILE}" -o "${INSTALL_PATH}/${UTILS_FILE}" && - curl --compressed -Ls "https://raw.githubusercontent.com/${REPO}/${LATEST_CURRENT_SHA}/upload.sh" -o "${INSTALL_PATH}/${COMMAND_NAME}"; then + curl --compressed -Ls "https://raw.githubusercontent.com/${REPO}/${LATEST_CURRENT_SHA}/upload.sh" -o "${INSTALL_PATH}/${COMMAND_NAME}" && + curl --compressed -Ls "https://raw.githubusercontent.com/${REPO}/${LATEST_CURRENT_SHA}/sync.sh" -o "${INSTALL_PATH}/${SYNC_COMMAND_NAME}"; then sed -i "2a UTILS_FILE=\"${INSTALL_PATH}/${UTILS_FILE}\"" "${INSTALL_PATH}/${COMMAND_NAME}" - chmod +x "${INSTALL_PATH}/${COMMAND_NAME}" + sed -i "2a UTILS_FILE=\"${INSTALL_PATH}/${UTILS_FILE}\"" "${INSTALL_PATH}/${SYNC_COMMAND_NAME}" + chmod +x "${INSTALL_PATH}"/* for i in "${__VALUES_ARRAY[@]}"; do _update_config "${i}" "${!i}" "${INFO_PATH}"/google-drive-upload.info done @@ -422,6 +428,7 @@ _install() { for _ in {1..3}; do _clear_line 1; done _print_center "justify" "Installed Successfully" "=" _print_center "normal" "[ Command name: ${COMMAND_NAME} ]" "=" + _print_center "normal" "[ Sync command name: ${SYNC_COMMAND_NAME} ]" "=" _print_center "justify" "To use the command, do" "-" _newline "\n" && _print_center "normal" "source ${SHELL_RC}" " " _print_center "normal" "or" " " @@ -458,9 +465,11 @@ _update() { else _print_center "justify" "Updating.." "-" if curl --compressed -Ls "https://raw.githubusercontent.com/${REPO}/${LATEST_CURRENT_SHA}/${UTILS_FILE}" -o "${INSTALL_PATH}/${UTILS_FILE}" && - curl --compressed -Ls "https://raw.githubusercontent.com/${REPO}/${LATEST_CURRENT_SHA}/upload.sh" -o "${INSTALL_PATH}/${COMMAND_NAME}"; then + curl --compressed -Ls "https://raw.githubusercontent.com/${REPO}/${LATEST_CURRENT_SHA}/upload.sh" -o "${INSTALL_PATH}/${COMMAND_NAME}" && + curl --compressed -Ls "https://raw.githubusercontent.com/${REPO}/${LATEST_CURRENT_SHA}/sync.sh" -o "${INSTALL_PATH}/${SYNC_COMMAND_NAME}"; then sed -i "2a UTILS_FILE=\"${INSTALL_PATH}/${UTILS_FILE}\"" "${INSTALL_PATH}/${COMMAND_NAME}" - chmod +x "${INSTALL_PATH}/${COMMAND_NAME}" + sed -i "2a UTILS_FILE=\"${INSTALL_PATH}/${UTILS_FILE}\"" "${INSTALL_PATH}/${SYNC_COMMAND_NAME}" + chmod +x "${INSTALL_PATH}"/* for i in "${__VALUES_ARRAY[@]}"; do _update_config "${i}" "${!i}" "${INFO_PATH}"/google-drive-upload.info done @@ -491,12 +500,15 @@ _update() { # Arguments: None # Result: read description # If cannot edit the SHELL_RC, then print message and exit +# Kill all sync jobs that are running ################################################### _uninstall() { _print_center "justify" "Uninstalling.." "-" __bak="source ${INFO_PATH}/google-drive-upload.binpath" if sed -i "s|${__bak}||g" "${SHELL_RC}"; then - rm -f "${INSTALL_PATH}"/{"${COMMAND_NAME}","${UTILS_FILE}"} + # Kill all sync jobs and remove sync folder + "${SYNC_COMMAND_NAME}" -k all &> /dev/null && rm -rf "${INFO_PATH}"/sync + rm -f "${INSTALL_PATH}"/{"${COMMAND_NAME}","${UTILS_FILE}","${SYNC_COMMAND_NAME}"} rm -f "${INFO_PATH}"/{google-drive-upload.info,google-drive-upload.binpath,google-drive-upload.configpath} _clear_line 1 _print_center "justify" "Uninstall complete." "=" @@ -548,6 +560,7 @@ _setup_arguments() { -c | --cmd) _check_longoptions "${1}" "${2}" COMMAND_NAME="${2}" && shift + { [[ ${2} = sync* ]] && SYNC_COMMAND_NAME="${2/sync=/}" && shift; } || : ;; -B | --branch) _check_longoptions "${1}" "${2}" diff --git a/sync.sh b/sync.sh new file mode 100755 index 0000000..f0c4cf4 --- /dev/null +++ b/sync.sh @@ -0,0 +1,546 @@ +#!/usr/bin/env bash +# Sync a FOLDER to google drive forever using labbots/google-drive-upload + +_usage() { + printf " +The script can be used to sync your local folder to google drive. + +Utilizes google-drive-upload bash scripts.\n +Usage:\n %s [options.. ]\n +Options:\n + -d | --directory - Gdrive foldername.\n + -k | --kill - to kill the background job using pid number ( -p flags ) or used with input, can be used multiple times.\n + -j | --jobs - See all background jobs that were started and still running.\n + Use --jobs v/verbose to more information for jobs.\n + -p | --pid - Specify a pid number, used for --jobs or --kill or --info flags, can be used multiple times.\n + -i | --info - See information about a specific sync using pid_number ( use -p flag ) or use with input, can be used multiple times.\n + -t | --time - Amount of time to wait before try to sync again in background.\n + To set wait time by default, use %s -t default='3'. Replace 3 with any positive integer.\n + -l | --logs - To show the logs after starting a job or show log of existing job. Can be used with pid number ( -p flag ). + Note: If multiple pid numbers or inputs are used, then will only show log of first input as it goes on forever. + -a | --arguments - Additional arguments for gupload commands. e.g: %s -a '-q -o -p 4 -d'.\n + To set some arguments by default, use %s -a default='-q -o -p 4 -d'.\n + -D | --debug - Display script command trace, use before all the flags to see maximum script trace.\n + -h | --help - Display usage instructions.\n" "${0##*/}" "${0##*/}" "${0##*/}" "${0##*/}" + exit +} + +_short_help() { + printf "No valid arguments provided, use -h/--help flag to see usage.\n" + exit +} + +################################################### +# Check if a pid exists by using ps +# Globals: None +# Arguments: 1 +# ${1} = pid number of a sync job +# Result: read description +################################################### +_check_pid() { + { ps -p "${1}" -o pid &> /dev/null && return 0; } || return 1 +} + +################################################### +# Show information about a specific sync job +# Globals: 1 variable, 2 functions +# Variable - SYNC_LIST +# Functions - _check_pid, _setup_loop_variables +# Arguments: 1 +# ${1} = pid number of a sync job +# ${2} = anything: Prints extra information ( optional ) +# ${3} = all information about a job ( optional ) +# Result: read description +################################################### +_get_job_info() { + declare input local_folder pid times extra + pid="${1}" + input="${3:-$(grep "${pid}" "${SYNC_LIST}" || :)}" + if [[ -n ${input} ]]; then + if _check_pid "${pid}"; then + printf "\n%s\n" "PID: ${pid}" + : "${input#*"|:_//_:|"}" && local_folder="${_/"|:_//_:|"*/}" + printf "Local Folder: %s\n" "${local_folder}" + printf "Drive Folder: %s\n" "${input/*"|:_//_:|"/}" + times="$(ps -p "${pid}" -o etimes --no-headers)" + printf "Running Since: %s\n" "$(_display_time "${times}")" + if [[ -n ${2} ]]; then + extra="$(ps -p "${pid}" -o %cpu,%mem --no-headers)" + printf "CPU usage:%s\n" "${extra% *}" + printf "Memory usage: %s\n" "${extra##* }" + _setup_loop_variables "${local_folder}" "${input/*"|:_//_:|"/}" + printf "Success: %s\n" "$(_count < "${SUCCESS}")" + printf "Failed: %s\n" "$(_count < "${ERROR}")" + fi + return 0 + else + return 1 + fi + else + return 11 + fi +} + +################################################### +# Remove a sync job information from database +# Globals: 2 variables, 2 functions +# Variables - SYNC_LIST, SYNC_DETAIL_DIR +# Functions - _get_job_info, _remove_job +# Arguments: 1 +# ${1} = pid number of a sync job +# Result: read description +################################################### +_remove_job() { + declare pid="${1}" input local_folder drive_folder + input="$(grep "${pid}" "${SYNC_LIST}")" + : "${input#*"|:_//_:|"}" && local_folder="${_/"|:_//_:|"*/}" + drive_folder="${input/*"|:_//_:|"/}" + new_list="$(grep -v "${pid}" "${SYNC_LIST}")" + printf "%s\n" "${new_list}" >| "${SYNC_LIST}" + rm -rf "${SYNC_DETAIL_DIR:?}/${drive_folder}${local_folder}" + # Cleanup dir if empty + if [[ -z $(find "${SYNC_DETAIL_DIR:?}/${drive_folder}" -type f) ]]; then + rm -rf "${SYNC_DETAIL_DIR:?}/${drive_folder}" + fi +} + +################################################### +# Kill a sync job and do _remove_job +# Globals: 1 function +# _remove_job +# Arguments: 1 +# ${1} = pid number of a sync job +# Result: read description +################################################### +_kill_job() { + declare pid="${1}" + kill -9 "${pid}" &> /dev/null + _remove_job "${pid}" + printf "Killed.\n" +} + +################################################### +# Show total no of sync jobs running +# Globals: 1 variable, 2 functions +# Variable - SYNC_LIST +# Functions - _get_job_info, _remove_job +# Arguments: 1 +# ${1} = v: Prints extra information ( optional ) +# Result: read description +################################################### +_show_jobs() { + declare list pid total=0 + list="$(grep -v '^$' "${SYNC_LIST}" || :)" + printf "%s\n" "${list}" >| "${SYNC_LIST}" + while read -r -u 4 line; do + if [[ -n ${line} ]]; then + : "${line/"|:_//_:|"*/}" && pid="${_/*: /}" + _get_job_info "${pid}" "${1}" "${line}" + { [[ ${?} = 1 ]] && _remove_job "${pid}"; } || : + no_task="printf" + ((total += 1)) + fi + done 4< "${SYNC_LIST}" + printf "\nTotal Jobs Running: %s\n" "${total}" + "${no_task:-:}" "For more info: %s -j/--jobs v/verbose\n" "${0##*/}" +} + +################################################### +# Setup required variables for a sync job +# Globals: 1 Variable +# SYNC_DETAIL_DIR +# Arguments: 1 +# ${1} = Local folder name which will be synced +# Result: read description +################################################### +_setup_loop_variables() { + declare folder="${1}" drive_folder="${2}" + DIRECTORY="${SYNC_DETAIL_DIR}/${drive_folder}${folder}" + PID_FILE="${DIRECTORY}/pid" + SUCCESS="${DIRECTORY}/success_list" + ERROR="${DIRECTORY}/failed_list" + LOGS="${DIRECTORY}/logs" +} + +################################################### +# Create folder and files for a sync job +# Globals: 4 variables +# DIRECTORY, PID_FILE, SUCCESS, ERROR +# Arguments: None +# Result: read description +################################################### +_setup_loop_files() { + mkdir -p "${DIRECTORY}" + for file in PID_FILE SUCCESS ERROR; do + printf "" >> "${!file}" + done +} + +################################################### +# Check for new files in the sync folder and upload it +# A list is generated everytime, success and error. +# Globals: 4 variables, 1 function +# Variables - SUCCESS, ERROR, COMMAND_NAME, ARGS, GDRIVE_FOLDER +# Function - _remove_array_duplicates +# Arguments: None +# Result: read description +################################################### +_check_and_upload() { + declare all initial final new + + mapfile -t initial < "${SUCCESS}" + + mapfile -t all <<< "$(cat "${SUCCESS}" "${ERROR}")" + { all+=(*) && [[ ${all[1]} = "*" ]] && return 0; } || : + + mapfile -t final <<< "$(_remove_array_duplicates "${all[@]}")" + + printf "" >| "${ERROR}" + + mapfile -t new <<< "$(diff \ + --new-line-format="%L" \ + --old-line-format="" \ + --unchanged-line-format="" \ + <(printf "%s\n" "${initial[@]}") <(printf "%s\n" "${final[@]}"))" + + if [[ -n ${new[0]} ]]; then + for i in "${new[@]}"; do + # shellcheck disable=SC2086 + if "${COMMAND_NAME}" "${i}" ${ARGS} -C "$GDRIVE_FOLDER"; then + printf "%s\n" "${i}" >> "${SUCCESS}" + else + printf "%s\n" "${i}" >> "${ERROR}" + fi + printf "\n" + done + else + printf "%s\n" "${initial[@]}" >| "${SUCCESS}" + fi +} + +################################################### +# Loop _check_and_upload function, sleep for sometime in between +# Globals: 1 variable, 1 function +# Variable - TIME_TO_SLEEP +# Function - _check_and_upload +# Arguments: None +# Result: read description +################################################### +_loop() { + while true; do + _check_and_upload + _bash_sleep "${SYNC_TIME_TO_SLEEP}" + done +} + +################################################### +# Start a new sync job by _loop function +# Print sync job information +# Globals: 6 variables, 1 function +# Variable - LOGS, PID_FILE, INPUT, GDRIVE_FOLDER, FOLDER, SYNC_LIST +# Function - _loop +# Arguments: None +# Result: read description +# Show logs at last and don't hangup if SHOW_LOGS is set +################################################### +_start_new_loop() { + _loop &> "${LOGS}" & + printf "%s\n" "$!" >| "${PID_FILE}" + PID="$(< "${PID_FILE}")" + printf "%b\n" "Job started.\nLocal Folder: ${INPUT}\nDrive Folder: ${GDRIVE_FOLDER}" + printf "%s\n" "PID: ${PID}" + printf "%b\n" "PID: ${PID}|:_//_:|${FOLDER}|:_//_:|${GDRIVE_FOLDER}" >> "${SYNC_LIST}" + { [[ -n ${SHOW_LOGS} ]] && tail -f "${LOGS}"; } || : +} + +################################################### +# Triggers in case either -j & -k or -l flag ( both -k|-j if with positive integer as argument ) +# Priority: -j > -i > -l > -k +# Globals: 5 variables, 6 functions +# Variables - JOB, SHOW_JOBS_VERBOSE, INFO_PID, LOG_PID, KILL_PID ( all array ) +# Functions - _check_pid, _setup_loop_variables +# _kill_job, _show_jobs, _get_job_info, _remove_job +# Arguments: None +# Result: show either job info, individual info or kill job(s) according to set global variables. +# Script exits after -j and -k if kill all is triggered ) +################################################### +_do_job() { + case "${JOB[*]}" in + *SHOW_JOBS*) + _show_jobs "${SHOW_JOBS_VERBOSE:-}" + exit + ;; + *KILL_ALL*) + PIDS="$(_show_jobs | grep -o 'PID:.*[0-9]' | sed "s/PID: //g" || :)" && total=0 + if [[ -n ${PIDS} ]]; then + for _pid in ${PIDS}; do + printf "PID: %s - " "${_pid##* }" + _kill_job "${_pid##* }" + ((total += 1)) + done + fi + printf "\nTotal Jobs Killed: %s\n" "${total}" + exit + ;; + *PIDS*) + for pid in "${ALL_PIDS[@]}"; do + if [[ ${JOB_TYPE} =~ INFO ]]; then + _get_job_info "${pid}" more + status="${?}" + if [[ ${status} != 0 ]]; then + { [[ ${status} = 1 ]] && _remove_job "${pid}"; } || : + printf "No job running with given PID ( %s ).\n" "${pid}" 1>&2 + fi + fi + if [[ ${JOB_TYPE} =~ SHOW_LOGS ]]; then + input="$(grep "${pid}" "${SYNC_LIST}" || :)" + if [[ -n ${input} ]]; then + if _check_pid "${pid}"; then + : "${input#*"|:_//_:|"}" && local_folder="${_/"|:_//_:|"*/}" + _setup_loop_variables "${local_folder}" "${input/*"|:_//_:|"/}" + tail -f "${LOGS}" + fi + else + printf "No job running with given PID ( %s ).\n" "${pid}" 1>&2 + fi + fi + if [[ ${JOB_TYPE} =~ KILL ]]; then + _get_job_info "${pid}" + status="${?}" + if [[ ${status} = 0 ]]; then + _kill_job "${pid}" + else + { [[ ${status} = 1 ]] && _remove_job "${pid}"; } || : + printf "No job running with given PID ( %s ).\n" "${pid}" 1>&2 + fi + fi + done + if [[ ${JOB_TYPE} =~ (INFO|SHOW_LOGS|KILL) ]]; then + exit + fi + ;; + esac +} + +################################################### +# Process all arguments given to the script +# Globals: 1 variable, 4 functions +# Variable - HOME +# Functions - _kill_jobs, _show_jobs, _get_job_info, _remove_array_duplicates +# Arguments: Many +# ${@} = Flags with arguments +# Result: On +# Success - Set all the variables +# Error - Print error message and exit +################################################### +_setup_arguments() { + [[ $# = 0 ]] && printf "%s: Missing arguments\n" "${FUNCNAME[0]}" && return 1 + declare -g SYNC_TIME_TO_SLEEP ARGS COMMAND_NAME DEBUG GDRIVE_FOLDER KILL SHOW_LOGS + + INFO_PATH="${HOME}/.google-drive-upload" + SYNC_DETAIL_DIR="${INFO_PATH}/sync" + SYNC_LIST="${SYNC_DETAIL_DIR}/sync_list" + mkdir -p "${SYNC_DETAIL_DIR}" && printf "" >> "${SYNC_LIST}" + + # Grab the first arg and shift, only if ${1} doesn't contain -. + { ! [[ ${1} = -* ]] && INPUT_ARRAY+=("${1}") && shift; } || : + + _check_longoptions() { + { [[ -z ${2} ]] && + printf '%s: %s: option requires an argument\nTry '"%s -h/--help"' for more information.\n' \ + "${0##*/}" "${1}" "${0##*/}" && exit 1; } || : + } + + while [[ $# -gt 0 ]]; do + case "${1}" in + -h | --help) + _usage + ;; + -D | --debug) + DEBUG="true" && export DEBUG + _check_debug + ;; + -d | --directory) + _check_longoptions "${1}" "${2}" + GDRIVE_FOLDER="${2}" && shift + ;; + -j | --jobs) + { [[ ${2} = v* ]] && SHOW_JOBS_VERBOSE="true" && shift; } || : + JOB=(SHOW_JOBS) + ;; + -p | --pid) + _check_longoptions "${1}" "${2}" + if [[ ${2} =~ ^([0-9]+)+$ ]]; then + ALL_PIDS+=("${2}") && shift + JOB+=(PIDS) + else + printf "-p/--pid only takes postive integer as arguments.\n" + exit 1 + fi + ;; + -i | --info) + JOB_TYPE+="INFO" + INFO="true" + ;; + -k | --kill) + JOB_TYPE+="KILL" + KILL="true" + if [[ ${2} = all ]]; then + JOB=(KILL_ALL) && shift + fi + ;; + -l | --logs) + JOB_TYPE+="SHOW_LOGS" + SHOW_LOGS="true" + ;; + -t | --time) + _check_longoptions "${1}" "${2}" + if [[ ${2} =~ ^([0-9]+)+$ ]]; then + { [[ ${2} = default* ]] && UPDATE_DEFAULT_TIME_TO_SLEEP="_update_config"; } || : + TO_SLEEP="${2/default=/}" && shift + else + printf "-t/--time only takes positive integers as arguments, min = 1, max = infinity.\n" + exit 1 + fi + ;; + -a | --arguments) + _check_longoptions "${1}" "${2}" + { [[ ${2} = default* ]] && UPDATE_DEFAULT_ARGS="_update_config"; } || : + ARGS+="${2/default=/} " && shift + ;; + '') + shorthelp + ;; + *) + # Check if user meant it to be a flag + if [[ ${1} = -* ]]; then + printf '%s: %s: Unknown option\nTry '"%s -h/--help"' for more information.\n' "${0##*/}" "${1}" "${0##*/}" && exit 1 + else + # If no "-" is detected in 1st arg, it adds to input + INPUT_ARRAY+=("${1}") + fi + ;; + esac + shift + done + + _do_job + + if [[ -z ${INPUT_ARRAY[0]} ]]; then + _short_help + else + # check if given input exists ( file/folder ) + for array in "${INPUT_ARRAY[@]}"; do + { [[ -d ${array} ]] && FINAL_INPUT_ARRAY+=("${array[@]}"); } || { + printf "\nError: Invalid Input ( %s ), no such directory.\n" "${array}" + exit 1 + } + done + fi + + mapfile -t FINAL_INPUT_ARRAY <<< "$(_remove_array_duplicates "${FINAL_INPUT_ARRAY[@]}")" +} + +################################################### +# Grab config variables and modify defaults if necessary +# Globals: 5 variables, 2 functions +# Variables - INFO_PATH, UPDATE_DEFAULT_CONFIG, DEFAULT_ARGS +# UPDATE_DEFAULT_ARGS, UPDATE_DEFAULT_TIME_TO_SLEEP, TIME_TO_SLEEP +# Functions - _print_center, _update_config +# Arguments: None +# Result: source .info file, grab COMMAND_NAME and CONFIG +# source CONFIG, update default values if required +################################################### +_config_variables() { + if [[ -f "${INFO_PATH}/google-drive-upload.info" ]]; then + # shellcheck source=/dev/null + source "${INFO_PATH}/google-drive-upload.info" + else + _print_center "justify" "google-drive-upload is not installed system wide." "=" 1>&2 + exit 1 + fi + + # Check if command exist, not necessary but just in case. + if ! type "${COMMAND_NAME}" &> /dev/null; then + printf "Error: %s is not installed, use -c/--command to specify.\n" "${COMMAND_NAME}" 1>&2 + exit 1 + fi + + ARGS+=" -q " + SYNC_TIME_TO_SLEEP="3" + # Config file is created automatically after first run + if [[ -r ${CONFIG} ]]; then + # shellcheck source=/dev/null + source "${CONFIG}" + if [[ -n ${UPDATE_DEFAULT_CONFIG} ]]; then + printf "%s\n" "${CONFIG}" >| "${INFO_PATH}/google-drive-upload.configpath" + fi + fi + SYNC_TIME_TO_SLEEP="${TO_SLEEP:-${SYNC_TIME_TO_SLEEP}}" + ARGS+=" ${SYNC_DEFAULT_ARGS:-} " + "${UPDATE_DEFAULT_ARGS:-:}" SYNC_DEFAULT_ARGS " ${ARGS} " "${CONFIG}" + "${UPDATE_DEFAULT_TIME_TO_SLEEP:-:}" SYNC_TIME_TO_SLEEP "${SYNC_TIME_TO_SLEEP}" "${CONFIG}" +} + +################################################### +# Process all the values in "${FINAL_INPUT_ARRAY[@]}" +# Globals: 20 variables, 15 functions +# Variables - FINAL_INPUT_ARRAY ( array ), GDRIVE_FOLDER, PID_FILE, SHOW_LOGS, LOGS +# Functions - _setup_loop_variables, _setup_loop_files, _start_new_loop, _check_pid, _kill_job +# _remove_job, _start_new_loop +# Arguments: None +# Result: Start the sync jobs for given folders, if running already, don't start new. +# If a pid is detected but not running, remove that job. +################################################### +_process_arguments() { + for INPUT in "${FINAL_INPUT_ARRAY[@]}"; do + CURRENT_FOLDER="$(pwd)" + FOLDER="$(cd "${INPUT}" && pwd)" || exit 1 + GDRIVE_FOLDER="${GDRIVE_FOLDER:-${FOLDER##*/}}" + _setup_loop_variables "${FOLDER}" "${GDRIVE_FOLDER}" && _setup_loop_files + cd "${FOLDER}" || exit 1 + PID="$(< "${PID_FILE}")" + if [[ -z ${PID} ]]; then + _start_new_loop + { [[ -n ${SHOW_LOGS} ]] && tail -f "${LOGS}"; } || : + else + if _check_pid "${PID}"; then + printf "%b\n" "Job is already running.." + if [[ -n ${INFO} ]]; then + _get_job_info "${PID}" more + else + printf "%b\n" "Local Folder: ${INPUT}\nDrive Folder: ${GDRIVE_FOLDER}" + printf "%s\n" "PID: ${PID}" + fi + if [[ -n ${KILL} ]]; then + _kill_job "${PID}" + exit + fi + { [[ -n ${SHOW_LOGS} ]] && tail -f "${LOGS}"; } || : + else + _remove_job "${PID}" + _start_new_loop + fi + fi + cd "${CURRENT_FOLDER}" || exit 1 + done +} + +main() { + [[ $# = 0 ]] && _short_help + + UTILS_FILE="${UTILS_FILE:-./utils.sh}" + if [[ -r ${UTILS_FILE} ]]; then + # shellcheck source=/dev/null + source "${UTILS_FILE}" || { printf "Error: Unable to source utils file ( %s ) .\n" "${UTILS_FILE}" 1>&2 && exit 1; } + else + printf "Error: Utils file ( %s ) not found\n" "${UTILS_FILE}" 1>&2 + exit 1 + fi + + _setup_arguments "${@}" + _config_variables + _process_arguments +} + +main "${@}" diff --git a/utils.sh b/utils.sh index 22ead0d..80509aa 100755 --- a/utils.sh +++ b/utils.sh @@ -183,6 +183,26 @@ _dirname() { printf '%s\n' "${tmp:-/}" } +################################################### +# Convert given time in seconds to readable form +# 110 to 1 minute(s) and 50 seconds +# Globals: None +# Arguments: 1 +# ${1} = Positive Integer ( time in seconds ) +# Result: read description +# Reference: +# https://stackoverflow.com/a/32164707 +################################################### +_display_time() { + declare T="${1}" + declare DAY="$((T / 60 / 60 / 24))" HR="$((T / 60 / 60 % 24))" MIN="$((T / 60 % 60))" SEC="$((T % 60))" + [[ ${DAY} -gt 0 ]] && printf '%d days ' "${DAY}" + [[ ${HR} -gt 0 ]] && printf '%d hrs ' "${HR}" + [[ ${MIN} -gt 0 ]] && printf '%d minute(s) ' "${MIN}" + [[ ${DAY} -gt 0 || ${HR} -gt 0 || ${MIN} -gt 0 ]] && printf 'and ' + printf '%d seconds\n' "${SEC}" +} + ################################################### # Extract ID from a googledrive folder/file url. # Globals: None