Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update GSI analysis jobs to use COMIN/COMOUT #3092

Open
wants to merge 27 commits into
base: develop
Choose a base branch
from
Open

Update GSI analysis jobs to use COMIN/COMOUT #3092

Show file tree
Hide file tree
Changes from 19 commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
fcd075e
changing COM_ into COMIN_ or COMOUT_
Nov 8, 2024
c1d9f8c
Merge remote-tracking branch 'emc/develop' into feature/jgdas_enkf_se…
Nov 12, 2024
1ff5591
don't need -m 755 in mkdir
Nov 13, 2024
677b8ca
remove trailing space on line 37
Nov 13, 2024
01216a5
Merge branch 'develop' into feature/jgdas_enkf_select_obs_com_in_out
aerorahul Nov 13, 2024
ad1aa96
changing COM_ into COMIN_ or COMOUT_
mingshichen-noaa Nov 21, 2024
ebdb0ba
changing COM_ATMOS_ANALYSIS int COMOUT_ATMOS_ANALYSIS
mingshichen-noaa Nov 21, 2024
3337cc4
changing COM_ into COMIN_ or COMOUT_
mingshichen-noaa Nov 21, 2024
5abb226
modified JGLOBAL_ATMOS_ANALYSIS_CALC
mingshichen-noaa Nov 21, 2024
f0be27e
changing COMOUT_ATMOS_RESTART into COMIN_ATMOS_RESTART
mingshichen-noaa Nov 21, 2024
4fa5030
changing COM_ into COMOUT
mingshichen-noaa Nov 21, 2024
7a956aa
changing COM_ into COMIN_ or COMOUT_
mingshichen-noaa Nov 21, 2024
ca1829b
COMOUT_OBS into COMIN_OBS
mingshichen-noaa Nov 21, 2024
4988539
Merge branch 'develop' into feature/jgdas_enkf_select_obs_com_in_out
mingshichen-noaa Nov 21, 2024
1213950
Merge branch 'develop' into feature/jgdas_enkf_select_obs_com_in_out
mingshichen-noaa Nov 25, 2024
20917eb
Merge branch 'develop' into feature/jgdas_enkf_select_obs_com_in_out
mingshichen-noaa Nov 26, 2024
594f983
Merge branch 'develop' into feature/jgdas_enkf_select_obs_com_in_out
mingshichen-noaa Nov 26, 2024
0303a9c
Merge branch 'develop' into feature/jgdas_enkf_select_obs_com_in_out
mingshichen-noaa Nov 27, 2024
4d94381
Merge branch 'develop' into feature/jgdas_enkf_select_obs_com_in_out
mingshichen-noaa Dec 2, 2024
0826c3e
changing COMIN_ATMOS_RESTART into COMOUT_ATMOS_RESTART
mingshichen-noaa Dec 3, 2024
67acf61
Merge branch 'develop' into feature/jgdas_enkf_select_obs_com_in_out
mingshichen-noaa Dec 3, 2024
fa937ef
remove line 137 because COMIN_ATMOS_RESTART is an input directory
mingshichen-noaa Dec 3, 2024
18eb1f4
Merge branch 'develop' into feature/jgdas_enkf_select_obs_com_in_out
mingshichen-noaa Dec 4, 2024
4c05e20
Merge branch 'develop' into feature/jgdas_enkf_select_obs_com_in_out
mingshichen-noaa Dec 5, 2024
4329a19
Merge branch 'develop' into feature/jgdas_enkf_select_obs_com_in_out
mingshichen-noaa Dec 9, 2024
bc771e9
Merge branch 'develop' into feature/jgdas_enkf_select_obs_com_in_out
mingshichen-noaa Dec 9, 2024
df83c7b
missing \
mingshichen-noaa Dec 10, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
64 changes: 34 additions & 30 deletions jobs/JGDAS_ENKF_SELECT_OBS
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,21 +32,25 @@ GPREFIX_DET="${GDUMP}.t${gcyc}z."
export GSUFFIX=".ensmean.nc"

# Generate COM variables from templates
RUN=${RUN/enkf} YMD=${PDY} HH=${cyc} declare_from_tmpl -rx COM_OBS
MEMDIR='ensstat' YMD=${PDY} HH=${cyc} declare_from_tmpl -rx COM_ATMOS_ANALYSIS
declare -rx COM_ATMOS_ANALYSIS_ENS="${COM_ATMOS_ANALYSIS}"
RUN=${RUN/enkf} YMD=${PDY} HH=${cyc} declare_from_tmpl -rx \
COMIN_OBS:COM_OBS_TMPL
MEMDIR='ensstat' YMD=${PDY} HH=${cyc} declare_from_tmpl -rx \
COMOUT_ATMOS_ANALYSIS:COM_ATMOS_ANALYSIS_TMPL
declare -rx COMOUT_ATMOS_ANALYSIS_ENS="${COMOUT_ATMOS_ANALYSIS}"

RUN=${RUN/enkf} YMD=${PDY} HH=${cyc} declare_from_tmpl -r COM_ATMOS_ANALYSIS_DET:COM_ATMOS_ANALYSIS_TMPL
RUN=${RUN/enkf} YMD=${PDY} HH=${cyc} declare_from_tmpl -r \
COMOUT_ATMOS_ANALYSIS_DET:COM_ATMOS_ANALYSIS_TMPL

MEMDIR='ensstat' RUN=${GDUMP_ENS} YMD=${gPDY} HH=${gcyc} declare_from_tmpl -rx \
COM_ATMOS_ANALYSIS_PREV:COM_ATMOS_ANALYSIS_TMPL \
COM_ATMOS_HISTORY_PREV:COM_ATMOS_HISTORY_TMPL \
COMIN_ATMOS_ANALYSIS_PREV:COM_ATMOS_ANALYSIS_TMPL \
COMIN_ATMOS_HISTORY_PREV:COM_ATMOS_HISTORY_TMPL

RUN="${GDUMP}" YMD=${gPDY} HH=${gcyc} declare_from_tmpl -r COM_ATMOS_ANALYSIS_DET_PREV:COM_ATMOS_ANALYSIS_TMPL
RUN="${GDUMP}" YMD=${gPDY} HH=${gcyc} declare_from_tmpl -r \
COMIN_ATMOS_ANALYSIS_DET_PREV:COM_ATMOS_ANALYSIS_TMPL

mkdir -m 775 -p "${COM_ATMOS_ANALYSIS}"
mkdir -p "${COMOUT_ATMOS_ANALYSIS}"

export ATMGES_ENSMEAN="${COM_ATMOS_HISTORY_PREV}/${GPREFIX}atmf006${GSUFFIX}"
export ATMGES_ENSMEAN="${COMIN_ATMOS_HISTORY_PREV}/${GPREFIX}atmf006${GSUFFIX}"
if [[ ! -f ${ATMGES_ENSMEAN} ]]; then
echo "FATAL ERROR: FILE MISSING: ATMGES_ENSMEAN = ${ATMGES_ENSMEAN}"
exit 1
Expand All @@ -61,44 +65,44 @@ status=$?
export LEVS

# Link observational data
export PREPQC="${COM_OBS}/${OPREFIX}prepbufr"
export PREPQC="${COMIN_OBS}/${OPREFIX}prepbufr"
if [[ ! -f ${PREPQC} ]]; then
echo "WARNING: Global PREPBUFR FILE ${PREPQC} MISSING"
fi
export TCVITL="${COM_OBS}/${APREFIX_DET}syndata.tcvitals.tm00"
export TCVITL="${COMIN_OBS}/${APREFIX_DET}syndata.tcvitals.tm00"
if [[ ${DONST} = "YES" ]]; then
export NSSTBF="${COM_OBS}/${OPREFIX}nsstbufr"
export NSSTBF="${COMIN_OBS}/${OPREFIX}nsstbufr"
fi
export PREPQCPF="${COM_OBS}/${OPREFIX}prepbufr.acft_profiles"
export PREPQCPF="${COMIN_OBS}/${OPREFIX}prepbufr.acft_profiles"

# Deterministic analysis and increment files
export SFCANL="${COM_ATMOS_ANALYSIS_DET}/${APREFIX_DET}sfcanl.nc"
export DTFANL="${COM_ATMOS_ANALYSIS_DET}/${APREFIX_DET}dtfanl.nc"
export ATMANL="${COM_ATMOS_ANALYSIS_DET}/${APREFIX_DET}atmanl.nc"
export ATMINC="${COM_ATMOS_ANALYSIS_DET}/${APREFIX_DET}atminc.nc"
export SFCANL="${COMOUT_ATMOS_ANALYSIS_DET}/${APREFIX_DET}sfcanl.nc"
export DTFANL="${COMOUT_ATMOS_ANALYSIS_DET}/${APREFIX_DET}dtfanl.nc"
export ATMANL="${COMOUT_ATMOS_ANALYSIS_DET}/${APREFIX_DET}atmanl.nc"
export ATMINC="${COMOUT_ATMOS_ANALYSIS_DET}/${APREFIX_DET}atminc.nc"

# Guess Bias correction coefficients related to control
export GBIAS=${COM_ATMOS_ANALYSIS_DET_PREV}/${GPREFIX_DET}abias
export GBIASPC=${COM_ATMOS_ANALYSIS_DET_PREV}/${GPREFIX_DET}abias_pc
export GBIASAIR=${COM_ATMOS_ANALYSIS_DET_PREV}/${GPREFIX_DET}abias_air
export GRADSTAT=${COM_ATMOS_ANALYSIS_DET_PREV}/${GPREFIX_DET}radstat
export GBIAS=${COMIN_ATMOS_ANALYSIS_DET_PREV}/${GPREFIX_DET}abias
export GBIASPC=${COMIN_ATMOS_ANALYSIS_DET_PREV}/${GPREFIX_DET}abias_pc
export GBIASAIR=${COMIN_ATMOS_ANALYSIS_DET_PREV}/${GPREFIX_DET}abias_air
export GRADSTAT=${COMIN_ATMOS_ANALYSIS_DET_PREV}/${GPREFIX_DET}radstat

# Bias correction coefficients related to ensemble mean
export ABIAS="${COM_ATMOS_ANALYSIS}/${APREFIX}abias.ensmean"
export ABIASPC="${COM_ATMOS_ANALYSIS}/${APREFIX}abias_pc.ensmean"
export ABIASAIR="${COM_ATMOS_ANALYSIS}/${APREFIX}abias_air.ensmean"
export ABIASe="${COM_ATMOS_ANALYSIS}/${APREFIX}abias_int.ensmean"
export ABIAS="${COMOUT_ATMOS_ANALYSIS}/${APREFIX}abias.ensmean"
export ABIASPC="${COMOUT_ATMOS_ANALYSIS}/${APREFIX}abias_pc.ensmean"
export ABIASAIR="${COMOUT_ATMOS_ANALYSIS}/${APREFIX}abias_air.ensmean"
export ABIASe="${COMOUT_ATMOS_ANALYSIS}/${APREFIX}abias_int.ensmean"

# Diagnostics related to ensemble mean
export GSISTAT="${COM_ATMOS_ANALYSIS}/${APREFIX}gsistat.ensmean"
export CNVSTAT="${COM_ATMOS_ANALYSIS}/${APREFIX}cnvstat.ensmean"
export OZNSTAT="${COM_ATMOS_ANALYSIS}/${APREFIX}oznstat.ensmean"
export RADSTAT="${COM_ATMOS_ANALYSIS}/${APREFIX}radstat.ensmean"
export GSISTAT="${COMOUT_ATMOS_ANALYSIS}/${APREFIX}gsistat.ensmean"
export CNVSTAT="${COMOUT_ATMOS_ANALYSIS}/${APREFIX}cnvstat.ensmean"
export OZNSTAT="${COMOUT_ATMOS_ANALYSIS}/${APREFIX}oznstat.ensmean"
export RADSTAT="${COMOUT_ATMOS_ANALYSIS}/${APREFIX}radstat.ensmean"

# Select observations based on ensemble mean
export RUN_SELECT="YES"
export USE_SELECT="NO"
export SELECT_OBS="${COM_ATMOS_ANALYSIS}/${APREFIX}obsinput.ensmean"
export SELECT_OBS="${COMOUT_ATMOS_ANALYSIS}/${APREFIX}obsinput.ensmean"

export DIAG_SUFFIX="_ensmean"

Expand Down
30 changes: 16 additions & 14 deletions jobs/JGLOBAL_ATMOS_ANALYSIS
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,18 +31,20 @@ export APREFIX="${rCDUMP}.t${cyc}z."
export GPREFIX_ENS="${GDUMP_ENS}.t${gcyc}z."

# Generate COM variables from templates
YMD=${PDY} HH=${cyc} declare_from_tmpl -rx COM_OBS COM_ATMOS_ANALYSIS
YMD=${PDY} HH=${cyc} declare_from_tmpl -rx \
COMIN_OBS:COM_OBS_TMPL \
COMOUT_ATMOS_ANALYSIS:COM_ATMOS_ANALYSIS_TMPL

RUN=${GDUMP} YMD=${gPDY} HH=${gcyc} declare_from_tmpl -rx \
COM_ATMOS_ANALYSIS_PREV:COM_ATMOS_ANALYSIS_TMPL \
COM_ATMOS_HISTORY_PREV:COM_ATMOS_HISTORY_TMPL
COMIN_ATMOS_ANALYSIS_PREV:COM_ATMOS_ANALYSIS_TMPL \
COMIN_ATMOS_HISTORY_PREV:COM_ATMOS_HISTORY_TMPL

MEMDIR='ensstat' RUN=${GDUMP_ENS} YMD=${gPDY} HH=${gcyc} declare_from_tmpl -rx \
COM_ATMOS_HISTORY_ENS_PREV:COM_ATMOS_HISTORY_TMPL
COMIN_ATMOS_HISTORY_ENS_PREV:COM_ATMOS_HISTORY_TMPL

mkdir -m 775 -p "${COM_ATMOS_ANALYSIS}"
mkdir -p "${COMOUT_ATMOS_ANALYSIS}"

export ATMGES="${COM_ATMOS_HISTORY_PREV}/${GPREFIX}atmf006.nc"
export ATMGES="${COMIN_ATMOS_HISTORY_PREV}/${GPREFIX}atmf006.nc"
if [ ! -f ${ATMGES} ]; then
echo "FATAL ERROR: FILE MISSING: ATMGES = ${ATMGES}"
exit 1
Expand All @@ -55,7 +57,7 @@ status=$?


if [ ${DOHYBVAR} = "YES" ]; then
export ATMGES_ENSMEAN="${COM_ATMOS_HISTORY_ENS_PREV}/${GPREFIX_ENS}atmf006.ensmean.nc"
export ATMGES_ENSMEAN="${COMIN_ATMOS_HISTORY_ENS_PREV}/${GPREFIX_ENS}atmf006.ensmean.nc"
if [ ! -f ${ATMGES_ENSMEAN} ]; then
echo "FATAL ERROR: FILE MISSING: ATMGES_ENSMEAN = ${ATMGES_ENSMEAN}"
exit 2
Expand All @@ -64,17 +66,17 @@ fi


# Link observational data
export PREPQC="${COM_OBS}/${OPREFIX}prepbufr"
export PREPQC="${COMIN_OBS}/${OPREFIX}prepbufr"
if [[ ! -f ${PREPQC} ]]; then
echo "WARNING: Global PREPBUFR FILE ${PREPQC} MISSING"
fi
export TCVITL="${COM_OBS}/${OPREFIX}syndata.tcvitals.tm00"
export TCVITL="${COMIN_OBS}/${OPREFIX}syndata.tcvitals.tm00"
if [[ ${DONST} = "YES" ]]; then
if [[ ${MAKE_NSSTBUFR} == "YES" ]]; then
export NSSTBF="${COM_OBS}/${OPREFIX}nsstbufr"
export NSSTBF="${COMIN_OBS}/${OPREFIX}nsstbufr"
fi
fi
export PREPQCPF="${COM_OBS}/${OPREFIX}prepbufr.acft_profiles"
export PREPQCPF="${COMIN_OBS}/${OPREFIX}prepbufr.acft_profiles"

# Copy fix file for obsproc # TODO: Why is this necessary?
if [[ ${RUN} = "gfs" ]]; then
Expand All @@ -95,9 +97,9 @@ status=$?
# Send Alerts
##############################################
if [ ${SENDDBN} = YES -a ${RUN} = gdas ] ; then
${DBNROOT}/bin/dbn_alert MODEL GDAS_MSC_abias ${job} ${COM_ATMOS_ANALYSIS}/${APREFIX}abias
${DBNROOT}/bin/dbn_alert MODEL GDAS_MSC_abias_pc ${job} ${COM_ATMOS_ANALYSIS}/${APREFIX}abias_pc
${DBNROOT}/bin/dbn_alert MODEL GDAS_MSC_abias_air ${job} ${COM_ATMOS_ANALYSIS}/${APREFIX}abias_air
${DBNROOT}/bin/dbn_alert MODEL GDAS_MSC_abias ${job} ${COMOUT_ATMOS_ANALYSIS}/${APREFIX}abias
${DBNROOT}/bin/dbn_alert MODEL GDAS_MSC_abias_pc ${job} ${COMOUT_ATMOS_ANALYSIS}/${APREFIX}abias_pc
${DBNROOT}/bin/dbn_alert MODEL GDAS_MSC_abias_air ${job} ${COMOUT_ATMOS_ANALYSIS}/${APREFIX}abias_air
fi


Expand Down
13 changes: 8 additions & 5 deletions jobs/JGLOBAL_ATMOS_ANALYSIS_CALC
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,16 +28,19 @@ export GPREFIX="${GDUMP}.t${gcyc}z."
export APREFIX="${RUN}.t${cyc}z."
export GPREFIX_ENS="${GDUMP_ENS}.t${gcyc}z."

RUN=${rCDUMP} YMD=${PDY} HH=${cyc} declare_from_tmpl -rx COM_OBS
RUN=${rCDUMP} YMD=${PDY} HH=${cyc} declare_from_tmpl -rx \
COMIN_OBS:COM_OBS_TMPL

YMD=${PDY} HH=${cyc} declare_from_tmpl -rx COM_ATMOS_ANALYSIS COM_ATMOS_RESTART
YMD=${PDY} HH=${cyc} declare_from_tmpl -rx
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mingshichen-noaa
This line is causing the failure. Please fix

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@aerorahul

After I checked codes in jobs/JGLOBAL_ATMOS_ANALYSIS_CALC from mingshichen-noaa:feature/jgdas_enkf_select_obs_com_in_out, the codes had been modified as follows:
30
31 RUN=${rCDUMP} YMD=${PDY} HH=${cyc} declare_from_tmpl -rx
32 COMIN_OBS:COM_OBS_TMPL
33
34 YMD=${PDY} HH=${cyc} declare_from_tmpl -rx
35 COMOUT_ATMOS_ANALYSIS:COM_ATMOS_ANALYSIS_TMPL
36 COMIN_ATMOS_RESTART:COM_ATMOS_RESTART_TMPL
37
38 RUN=${GDUMP} YMD=${gPDY} HH=${gcyc} declare_from_tmpl -rx
39 COMIN_OBS_PREV:COM_OBS_TMPL
40 COMIN_ATMOS_HISTORY_PREV:COM_ATMOS_HISTORY_TMPL
41

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
YMD=${PDY} HH=${cyc} declare_from_tmpl -rx
YMD=${PDY} HH=${cyc} declare_from_tmpl -rx \

This is the fix needed.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@aerorahul
I have changed line 34 as follows:
34 YMD=${PDY} HH=${cyc} declare_from_tmpl -rx
35 COMOUT_ATMOS_ANALYSIS:COM_ATMOS_ANALYSIS_TMPL
36 COMIN_ATMOS_RESTART:COM_ATMOS_RESTART_TMPL

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@aerorahul
My changes are located the branch: mingshichen-noaa:feature/jgdas_enkf_select_obs_com_in_out

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please push them

COMOUT_ATMOS_ANALYSIS:COM_ATMOS_ANALYSIS_TMPL \
COMIN_ATMOS_RESTART:COM_ATMOS_RESTART_TMPL

RUN=${GDUMP} YMD=${gPDY} HH=${gcyc} declare_from_tmpl -rx \
COM_OBS_PREV:COM_OBS_TMPL \
COM_ATMOS_HISTORY_PREV:COM_ATMOS_HISTORY_TMPL
COMIN_OBS_PREV:COM_OBS_TMPL \
COMIN_ATMOS_HISTORY_PREV:COM_ATMOS_HISTORY_TMPL


export ATMGES="${COM_ATMOS_HISTORY_PREV}/${GPREFIX}atmf006.nc"
export ATMGES="${COMIN_ATMOS_HISTORY_PREV}/${GPREFIX}atmf006.nc"
if [ ! -f ${ATMGES} ]; then
echo "FATAL ERROR: FILE MISSING: ATMGES = ${ATMGES}"
exit 1
Expand Down
Loading