Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Vendor load-singlesshagent.sh script #409

Merged
merged 8 commits into from
Oct 18, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions stack/base/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -54,8 +54,7 @@ ENV AIIDA_USER_INSTITUTION Khedivial
# Install the load-singlesshagent.sh script as described here:
# https://aiida.readthedocs.io/projects/aiida-core/en/v2.0.0/howto/ssh.html#starting-the-ssh-agent
# The startup of this script is configured in the before-notebook.d/20_setup-ssh.sh file.
RUN wget --quiet --directory-prefix=/opt/bin/ \
"https://aiida.readthedocs.io/projects/aiida-core/en/latest/_downloads/4265ec5a42c3a3dba586dd460c0db95e/load-singlesshagent.sh"
COPY load-singlesshagent.sh /opt/bin/

# Add ~/.local/bin to PATH where the dependencies get installed via pip
ENV PATH=${PATH}:/home/${NB_USER}/.local/bin
Expand Down
35 changes: 18 additions & 17 deletions stack/base/before-notebook.d/10_prepare-home-config.sh
Original file line number Diff line number Diff line change
Expand Up @@ -6,20 +6,6 @@ if [[ ! -f /home/${NB_USER}/.bashrc ]]; then
cp /etc/skel/.bashrc /home/${NB_USER}/.bashrc
fi

# Set sshagent by source load-singlesshagent.sh script
# append the command text of source to .bashrc if the script /opt/bin/load-singlesshagen.sh is present
# and the command text is not already present in .bashrc
header="# Load singlesshagent on shell startup."
if [[ -f /opt/bin/load-singlesshagent.sh ]] && ! grep -q "${header}" /home/${NB_USER}/.bashrc; then
cat >> "/home/${NB_USER}/.bashrc" <<- EOF

${header}
if [ -f /opt/bin/load-singlesshagent.sh ]; then
source /opt/bin/load-singlesshagent.sh
fi
EOF
fi

if [[ ! -f /home/${NB_USER}/.profile ]]; then
cp /etc/skel/.profile /home/${NB_USER}/.profile
fi
Expand All @@ -40,12 +26,27 @@ fi
mkdir -p --mode=0700 /home/${NB_USER}/.ssh && \
touch /home/${NB_USER}/.ssh/known_hosts


if [[ ! -f /home/${NB_USER}/.ssh/id_rsa ]]; then
# Generate ssh key that works with `paramiko`
# See: https://aiida.readthedocs.io/projects/aiida-core/en/latest/get_started/computers.html#remote-computer-requirements
ssh-keygen -f /home/${NB_USER}/.ssh/id_rsa -t rsa -b 4096 -m PEM -N ''
fi

# Start the ssh-agent.
eval `ssh-agent`
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Based on the discussion, I think we should source the load-singlesshagent.sh script here. Otherwise I don't think the ssh-agent will be running when the container starts, since .bashrc is not evaluated AFAIK.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ha! I think the one of the ssh-agent mentioned in aiidalab/aiidalab-mfa-cscs#6 is from here!
I agree to source the load-singlesshagent.sh here. It avoids the duplicate ssh-agent I think.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, interesting. That would mean that it is started somewhere else before we get to this script. Maybe the Jupyter image that we use already starts it somewhere. In any case, source the script here is the right thing to do I think.

# Set sshagent by source load-singlesshagent.sh script
# append the command text of source to .bashrc if the script /opt/bin/load-singlesshagen.sh is present
# and the command text is not already present in .bashrc
header="# Load singlesshagent on shell startup."
if [[ -f /opt/bin/load-singlesshagent.sh ]] && ! grep -q "${header}" /home/${NB_USER}/.bashrc; then
cat >> "/home/${NB_USER}/.bashrc" <<- EOF

${header}
if [ -f /opt/bin/load-singlesshagent.sh ]; then
source /opt/bin/load-singlesshagent.sh
fi
EOF
fi

# load the ssh-agent and add the default key generated
# the return code can be non-zero if the ssh-agent is not running
# which will cause the notebook to fail to start so we need to ignore the return code
source /opt/bin/load-singlesshagent.sh || true
68 changes: 68 additions & 0 deletions stack/base/load-singlesshagent.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
# Make sure you source this script rather than executing it
if [[ "${BASH_SOURCE[0]}" == "${0}" ]]
then
echo "You need to source this script, stopping." >&2
exit 1
fi

# run as
# source load-singlesshagent.sh -v
# for verbose output

load_singlesshagent() {
local VERBOSE
local SSH_ENV
local SSH_ADD_OUTPUT
local SSHADD_RETVAL
local NUMKEYS

VERBOSE=false
if [ "$1" == "-v" ]
then
VERBOSE=true
fi

[ "$VERBOSE" == "true" ] && echo "Single SSH agent script [verbose mode]" >&2
SSH_ENV="$HOME/.ssh/agent-environment"
# Source SSH settings, if applicable
if [ -r "${SSH_ENV}" ]; then
# don't show the output of this source command
source "${SSH_ENV}" 1> /dev/null
[ "$VERBOSE" == "true" ] && echo "- sourcing existing environment" >&2
else
[ "$VERBOSE" == "true" ] && echo "- no existing environment to source" >&2
fi

SSH_ADD_OUTPUT=`ssh-add -l 2> /dev/null`
Copy link
Contributor Author

@danielhollas danielhollas Sep 27, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@unkcpz here's the small change to get rid of spurious errors.. Note that it is okay to redirect error to dev/null here since we check the return value of this ssh-add command below.

# Needed, the later 'test' calls will replace this
SSHADD_RETVAL="$?"
# Error code: 0: there are keys; 1: there are no keys; 2: cannot contact agent
if [ "$SSHADD_RETVAL" == "2" ]
then
[ "$VERBOSE" == "true" ] && echo " - unable to contact agent, creating a new one" >&2
(umask 066; ssh-agent > ${SSH_ENV})
source "${SSH_ENV}" 2> /dev/null
elif [ "$SSHADD_RETVAL" == "1" ]
then
[ "$VERBOSE" == "true" ] && echo " - ssh-agent found (${SSH_AGENT_PID}), no keys (I might want to add keys here)" >&2
# run ssh-add to add the default generate key `id_rsa` to the agent
ssh-add ~/.ssh/id_rsa 2> /dev/null
elif [ "$SSHADD_RETVAL" == "0" ]
then
NUMKEYS=`echo "$SSH_ADD_OUTPUT" | wc -l`
[ "$VERBOSE" == "true" ] && echo " - ssh-agent found (${SSH_AGENT_PID}) with $NUMKEYS keys" >&2
else
[ "$VERBOSE" == "true" ] && echo " - ssh-add replied with return code $SSHADD_RETVAL - I don't know what to do..." >&2
fi

[ "$VERBOSE" == "true" ] && echo "- Debugging, listing all ssh-agents for user $NB_USER:"
[ "$VERBOSE" == "true" ] && ps -U "$NB_USER" | grep --color=no '[s]sh-agent'
}

# Run with the requested verbosity
if [ "$1" == "-v" ]
then
load_singlesshagent -v
else
load_singlesshagent
fi
3 changes: 3 additions & 0 deletions stack/lab/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,9 @@ RUN mamba install --yes \
RUN echo "aiidalab==${AIIDALAB_VERSION}" >> /opt/requirements.txt
RUN conda config --system --add pinned_packages "aiidalab=${AIIDALAB_VERSION}"

# Enable aiidalab autocompletion
RUN echo 'eval "$(_AIIDALAB_COMPLETE=bash_source aiidalab)"' >> "${CONDA_DIR}/etc/conda/activate.d/activate_aiida_autocompletion.sh"

# Install the aiidalab-home app.
ARG AIIDALAB_HOME_VERSION
RUN git clone https://github.com/aiidalab/aiidalab-home && \
Expand Down
8 changes: 8 additions & 0 deletions tests/test-common.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,3 +11,11 @@ def test_verdi_status(aiidalab_exec, nb_user):
output = aiidalab_exec("verdi status", user=nb_user).decode().strip()
assert "Connected to RabbitMQ" in output
assert "Daemon is running" in output


def test_ssh_agent_is_running(aiidalab_exec, nb_user):
output = aiidalab_exec("ps aux | grep ssh-agent", user=nb_user).decode().strip()
assert "ssh-agent" in output

# also check only one ssh-agent process is running
assert len(output.splitlines()) == 1