Merge dev/sfs into develop by NeilBarton-NOAA · Pull Request #4763 · NOAA-EMC/global-workflow

NeilBarton-NOAA · 2026-04-07T21:37:10Z

Description

Draft PR to show changes between dev/sfs and develop. dev/sfs is tested bi-weekly(ish) with the CI test and code standards.

Any comments in changes is appreciated with the goal of easing the merge to develop

Major GW-updates include:

SFS products
SFS post jobs to create products
SFS initialization with GFSv17
Model Namelist Changes for SFS

Postprocessing for SFS products is dependent on #4733

Type of change

Bug fix (fixes something broken)
New feature (adds functionality)
Maintenance (code refactor, clean-up, new CI test, etc.)

Change characteristics

Is this change expected to change outputs (e.g. value changes to existing outputs, new files stored in COM, files removed from COM, filename changes, additions/subtractions to archives)? YES/NO (If YES, please indicate to which system(s))
- GFS
- GEFS
- SFS
- GCAFS
Is this a breaking change (a change in existing functionality)? NO
Does this change require a documentation update?
Does this change require an update to any of the following submodules? YES/NO (If YES, please add a link to any PRs that are pending.)
- EMC verif-global
- GDAS
- GFS-utils
- GSI
- GSI-monitor
- GSI-utils
- UFS-utils
- UFS-weather-model
- wxflow

How has this been tested?

CI tests run on ursa and c6

Checklist

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

github-actions · 2026-04-08T14:08:54Z

ush/process_atmos_daily.sh

+#---------------------------------------------------------
+# LOOP THROUGH EACH DAY OF THE MONTH (1 to month_days)
+#---------------------------------------------------------
+for ((d = 1; d <= ${month_days}; d++)); do


[shellcheck] _{reported by reviewdog 🐶}
$/${} is unnecessary on arithmetic variables. SC2004

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

github-actions · 2026-04-08T14:10:24Z

ush/process_atmos_daily.sh

+    all_inst_exist=true
+    list_inst=""
+    for hr in $((end_of_day_fhr - 24)) $((end_of_day_fhr - 18)) $((end_of_day_fhr - 12)) $((end_of_day_fhr - 6)) $((end_of_day_fhr)); do
+        fpath="${COMIN_ATMOS_MASTER}/sfs.t${cyc}z.master.f$(printf "%03d" $hr).grib2"


[shellcheck] _{reported by reviewdog 🐶}
Prefer double quoting even when variables don't contain special characters. SC2248

github-actions · 2026-04-08T14:10:24Z

ush/process_atmos_daily.sh

+    all_inst_exist=true
+    list_inst=""
+    for hr in $((end_of_day_fhr - 24)) $((end_of_day_fhr - 18)) $((end_of_day_fhr - 12)) $((end_of_day_fhr - 6)) $((end_of_day_fhr)); do
+        fpath="${COMIN_ATMOS_MASTER}/sfs.t${cyc}z.master.f$(printf "%03d" $hr).grib2"


[shellcheck] _{reported by reviewdog 🐶}
Prefer putting braces around variable references even when not strictly required. SC2250

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

github-actions · 2026-04-08T14:11:36Z

ush/process_atmos_daily.sh

+
+    if [ "$all_inst_exist" = true ]; then
+        raw_tmp_inst="${tmp_inst_work_dir}/day_${d}_raw.grb"
+        ${GMERGE} "${raw_tmp_inst}" ${list_inst}


📝 [shellcheck] _{reported by reviewdog 🐶}
Double quote to prevent globbing and word splitting. SC2086

github-actions · 2026-04-08T14:11:36Z

ush/process_atmos_daily.sh

+    mapfile -t acc_files < <(ls -v "${tmp_acc_work_dir}"/daily_acc*.grb 2>/dev/null)
+
+    if [ ${#acc_files[@]} -gt 0 ]; then
+        echo "INFO: Task $i merging ${#acc_files[@]} days for ACC using array expansion."


[shellcheck] _{reported by reviewdog 🐶}
Prefer putting braces around variable references even when not strictly required. SC2250

github-actions · 2026-04-08T14:11:36Z

ush/process_atmos_daily.sh

+    mapfile -t inst_files < <(ls -v "${tmp_inst_work_dir}"/daily_inst*.grb 2>/dev/null)
+
+    if [ ${#inst_files[@]} -gt 0 ]; then
+        echo "INFO: Task $i merging ${#inst_files[@]} days for INST using array expansion."


[shellcheck] _{reported by reviewdog 🐶}
Prefer putting braces around variable references even when not strictly required. SC2250

github-actions · 2026-04-08T14:12:34Z

ush/process_atmos_daily.sh

+
+    if [ "$all_inst_exist" = true ]; then
+        raw_tmp_inst="${tmp_inst_work_dir}/day_${d}_raw.grb"
+        ${GMERGE} "${raw_tmp_inst}" ${list_inst}


[shellcheck (suggestion)] _{reported by reviewdog 🐶}

Suggested change

${GMERGE} "${raw_tmp_inst}" ${list_inst}

${GMERGE} "${raw_tmp_inst}" "${list_inst}"

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

github-actions · 2026-04-08T14:22:49Z

ush/process_atmos_daily.sh

+    end_of_day_fhr=$(((day_offset + d) * 24))
+
+    # Skip if we exceed the total available forecast length
+    if [[ "$end_of_day_fhr" -gt "$lastfhr" ]]; then 


[shfmt] _{reported by reviewdog 🐶}

Suggested change

if [[ "$end_of_day_fhr" -gt "$lastfhr" ]]; then

if [[ "$end_of_day_fhr" -gt "$lastfhr" ]]; then

github-actions · 2026-04-08T14:23:47Z

ush/process_atmos_daily.sh

+    end_of_day_fhr=$(((day_offset + d) * 24))
+
+    # Skip if we exceed the total available forecast length
+    if [[ "$end_of_day_fhr" -gt "$lastfhr" ]]; then 


[shellcheck] _{reported by reviewdog 🐶}
Prefer putting braces around variable references even when not strictly required. SC2250

github-actions · 2026-04-08T14:23:47Z

ush/process_atmos_daily.sh

+    end_of_day_fhr=$(((day_offset + d) * 24))
+
+    # Skip if we exceed the total available forecast length
+    if [[ "$end_of_day_fhr" -gt "$lastfhr" ]]; then 


[shellcheck] _{reported by reviewdog 🐶}
Prefer putting braces around variable references even when not strictly required. SC2250

github-actions · 2026-04-08T14:23:47Z

ush/process_atmos_daily.sh

+        list_acc+="${fpath} "
+    done
+
+    if [[ "$all_acc_exist" = true ]]; then


[shellcheck] _{reported by reviewdog 🐶}
Prefer putting braces around variable references even when not strictly required. SC2250

github-actions · 2026-04-08T14:24:51Z

ush/process_atmos_daily.sh

+    if [[ "$end_of_day_fhr" -gt "$lastfhr" ]]; then 
+        echo "DEBUG: Skipping Day $d because $end_of_day_fhr > $lastfhr"


[shellcheck (suggestion)] _{reported by reviewdog 🐶}

Suggested change

if [[ "$end_of_day_fhr" -gt "$lastfhr" ]]; then

echo "DEBUG: Skipping Day $d because $end_of_day_fhr > $lastfhr"

if [[ "${end_of_day_fhr}" -gt "${lastfhr}" ]]; then

echo "DEBUG: Skipping Day ${d} because ${end_of_day_fhr} > ${lastfhr}"

github-actions · 2026-04-08T14:24:51Z

ush/process_atmos_daily.sh

+        list_acc+="${fpath} "
+    done
+
+    if [[ "$all_acc_exist" = true ]]; then


[shellcheck (suggestion)] _{reported by reviewdog 🐶}

Suggested change

if [[ "$all_acc_exist" = true ]]; then

if [[ "${all_acc_exist}" = true ]]; then

github-actions · 2026-04-08T14:24:51Z

ush/process_atmos_daily.sh

+    all_inst_exist=true
+    list_inst=""
+    for hr in $((end_of_day_fhr - 24)) $((end_of_day_fhr - 18)) $((end_of_day_fhr - 12)) $((end_of_day_fhr - 6)) $((end_of_day_fhr)); do
+        fpath="${COMIN_ATMOS_MASTER}/sfs.t${cyc}z.master.f$(printf "%03d" $hr).grib2"


[shellcheck (suggestion)] _{reported by reviewdog 🐶}

Suggested change

fpath="${COMIN_ATMOS_MASTER}/sfs.t${cyc}z.master.f$(printf "%03d" $hr).grib2"

fpath="${COMIN_ATMOS_MASTER}/sfs.t${cyc}z.master.f$(printf "%03d" "${hr}").grib2"

github-actions · 2026-04-08T14:24:51Z

ush/process_atmos_daily.sh

+    mapfile -t acc_files < <(ls -v "${tmp_acc_work_dir}"/daily_acc*.grb 2> /dev/null)
+
+    if [[ ${#acc_files[@]} -gt 0 ]]; then
+        echo "INFO: Task $i merging ${#acc_files[@]} days for ACC using array expansion."


[shellcheck (suggestion)] _{reported by reviewdog 🐶}

Suggested change

echo "INFO: Task $i merging ${#acc_files[@]} days for ACC using array expansion."

echo "INFO: Task ${i} merging ${#acc_files[@]} days for ACC using array expansion."

github-actions · 2026-04-08T14:24:51Z

ush/process_atmos_daily.sh

+    mapfile -t inst_files < <(ls -v "${tmp_inst_work_dir}"/daily_inst*.grb 2> /dev/null)
+
+    if [[ ${#inst_files[@]} -gt 0 ]]; then
+        echo "INFO: Task $i merging ${#inst_files[@]} days for INST using array expansion."


[shellcheck (suggestion)] _{reported by reviewdog 🐶}

Suggested change

echo "INFO: Task $i merging ${#inst_files[@]} days for INST using array expansion."

echo "INFO: Task ${i} merging ${#inst_files[@]} days for INST using array expansion."

github-actions · 2026-04-08T14:32:56Z

dev/scripts/exglobal_atmos_post.sh

+for ((i = 0; i < exp_months; i++ )); do
+


[shfmt] _{reported by reviewdog 🐶}

Suggested change

for ((i = 0; i < exp_months; i++ )); do

for ((i = 0; i < exp_months; i++)); do

github-actions · 2026-04-08T14:32:56Z

dev/scripts/exglobal_atmos_post.sh

+    days_left=$(( total_fcst_days - m_start_day ))
+
+    if [[ "${days_left}" -le 0 ]]; then break; fi # No more data 


[shfmt] _{reported by reviewdog 🐶}

Suggested change

days_left=$(( total_fcst_days - m_start_day ))

if [[ "${days_left}" -le 0 ]]; then break; fi # No more data

days_left=$((total_fcst_days - m_start_day))

if [[ "${days_left}" -le 0 ]]; then break; fi # No more data

github-actions · 2026-04-08T14:32:56Z

dev/scripts/exglobal_atmos_post.sh

+# 3. Dynamically count tasks and execute
+if [[ -s "${cmdfile_s2}" ]]; then
+    n_tasks=$(wc -l < "${cmdfile_s2}")
+    echo "INFO: Launching Stage 2 MPMD with ${n_tasks} months." 


[shfmt] _{reported by reviewdog 🐶}

Suggested change

echo "INFO: Launching Stage 2 MPMD with ${n_tasks} months."

echo "INFO: Launching Stage 2 MPMD with ${n_tasks} months."

github-actions · 2026-04-08T14:33:59Z

dev/scripts/exglobal_atmos_post.sh

+month_days_in_year=("31" "28" "31" "30" "31" "30" "31" "31" "30" "31" "30" "31")
+# Leap Year Check
+
+if [[ $((yy_init % 4)) -eq 0 ]] && ([[ $((yy_init % 100)) -ne 0 ]] || [[ $((yy_init % 400)) -eq 0 ]]); then


[shellcheck] _{reported by reviewdog 🐶}
Use { ..; } instead of (..) to avoid subshell overhead. SC2235

github-actions · 2026-04-08T14:33:59Z

dev/scripts/exglobal_atmos_post.sh

+fi
+
+cmdfile_s3="${DATA}/mpmd_s3_monthly.txt"
+> "${cmdfile_s3}"


⚠️ [shellcheck] _{reported by reviewdog 🐶}
This redirection doesn't have a command. Move to its command (or use 'true' as no-op). SC2188

github-actions · 2026-04-08T14:33:59Z

dev/scripts/exglobal_atmos_post.sh

+done
+
+if [[ -s "${cmdfile_s3}" ]]; then
+    echo "INFO: Launching $(wc -l < ${cmdfile_s3}) monthly mean tasks in parallel."


📝 [shellcheck] _{reported by reviewdog 🐶}
Double quote to prevent globbing and word splitting. SC2086

Yangxing Zheng and others added 30 commits November 18, 2025 12:53

changes for generating sfs products

c1a11ec

pycodestyles

fedddb6

comments removal

21960de

adding sfs atmos post

20592a0

update atmos/ocn/ice post tasks

10499e7

pycode styles

fc4d495

pycode styles

6a5c525

pycodes styles

44d6f0b

updates atmos post

5dc2404

remove copy ocean zgrid manually

d4efa78

update mom6

a3d80d5

Merge remote-tracking branch 'origin/develop' into dev/sfs

c4bd8f8

add arch_tars task

ae39180

update monthly mean file name

837f4ce

change where the model executable is saved

f1b7571

Merge remote-tracking branch 'origin/develop' into dev/sfs

7e76ae0

update wgrib2 version

05ffa2c

add computing wvel

66a5fff

add ocean monthly product yaml file

d58af18

add ocean daily product yaml file

5ac05d4

create ice daily product yaml

cfcccbb

create ice monthly product yaml

47f3de2

update atmos post

6ee609b

update atmos post

937f66d

update post jobs

b8c3eaf

update atmos master grib yaml

afc9f56

update sfs archival yaml

5cd911d

update 6hr_avg yaml

226c1ee

update ocean monthly history yaml

c49b087

add more ocean fields

2a58577

NeilBarton-NOAA and others added 14 commits April 8, 2026 09:46

Update dev/scripts/exglobal_atmos_post.sh

b6dc754

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

Update dev/scripts/exglobal_atmos_post.sh

98a29b0

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

Update dev/scripts/exglobal_atmos_post.sh

1e01ce0

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

Update dev/scripts/exglobal_atmos_post.sh

bab100e

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

Update dev/scripts/exglobal_atmos_post.sh

6bd8ee3

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

Update dev/scripts/exglobal_atmos_post.sh

9e024b1

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

Update dev/scripts/exglobal_atmos_post.sh

35a2350

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

Update dev/scripts/exglobal_ocn_post.sh

e28089d

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

Update ush/forecast_postdet.sh

6930701

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

Update ush/forecast_postdet.sh

b62ca92

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

Update ush/process_atmos_daily.sh

196bdb4

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

Update ush/process_atmos_daily.sh

edea486

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

Update dev/jobs/JGLOBAL_OCN_POST

a10e1f1

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

Update ush/process_atmos_daily.sh

aa781b9

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

github-actions bot reviewed Apr 8, 2026

View reviewed changes

Update ush/process_atmos_daily.sh

6836086

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

github-actions bot reviewed Apr 8, 2026

View reviewed changes

NeilBarton-NOAA and others added 3 commits April 8, 2026 10:10

Update ush/process_atmos_daily.sh

9b1b1db

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

Update ush/process_atmos_daily.sh

32b26be

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

Update ush/process_atmos_daily.sh

1c4ae0d

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

github-actions bot reviewed Apr 8, 2026

View reviewed changes

NeilBarton-NOAA and others added 2 commits April 8, 2026 10:13

Update ush/process_atmos_daily.sh

7a89b0c

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

updates for style guidance

79601bd

github-actions bot reviewed Apr 8, 2026

View reviewed changes

style guide changes

657aa7e

github-actions bot reviewed Apr 8, 2026

View reviewed changes

	${GMERGE} "${raw_tmp_inst}" ${list_inst}
	${GMERGE} "${raw_tmp_inst}" "${list_inst}"

	if [[ "$end_of_day_fhr" -gt "$lastfhr" ]]; then
	if [[ "$end_of_day_fhr" -gt "$lastfhr" ]]; then

		if [[ "$end_of_day_fhr" -gt "$lastfhr" ]]; then
		echo "DEBUG: Skipping Day $d because $end_of_day_fhr > $lastfhr"

	if [[ "$all_acc_exist" = true ]]; then
	if [[ "${all_acc_exist}" = true ]]; then

	fpath="${COMIN_ATMOS_MASTER}/sfs.t${cyc}z.master.f$(printf "%03d" $hr).grib2"
	fpath="${COMIN_ATMOS_MASTER}/sfs.t${cyc}z.master.f$(printf "%03d" "${hr}").grib2"

	echo "INFO: Task $i merging ${#acc_files[@]} days for ACC using array expansion."
	echo "INFO: Task ${i} merging ${#acc_files[@]} days for ACC using array expansion."

	echo "INFO: Task $i merging ${#inst_files[@]} days for INST using array expansion."
	echo "INFO: Task ${i} merging ${#inst_files[@]} days for INST using array expansion."

	for ((i = 0; i < exp_months; i++ )); do

	for ((i = 0; i < exp_months; i++)); do

		days_left=$(( total_fcst_days - m_start_day ))

		if [[ "${days_left}" -le 0 ]]; then break; fi # No more data

	echo "INFO: Launching Stage 2 MPMD with ${n_tasks} months."
	echo "INFO: Launching Stage 2 MPMD with ${n_tasks} months."

Conversation

NeilBarton-NOAA commented Apr 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Type of change

Change characteristics

How has this been tested?

Checklist

Uh oh!

github-actions bot Apr 8, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions bot Apr 8, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions bot Apr 8, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions bot Apr 8, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions bot Apr 8, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions bot Apr 8, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions bot Apr 8, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions bot Apr 8, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions bot Apr 8, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions bot Apr 8, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions bot Apr 8, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions bot Apr 8, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions bot Apr 8, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions bot Apr 8, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions bot Apr 8, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions bot Apr 8, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions bot Apr 8, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions bot Apr 8, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions bot Apr 8, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions bot Apr 8, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions bot Apr 8, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions bot Apr 8, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

NeilBarton-NOAA commented Apr 7, 2026 •

edited

Loading