[develop] Eliminate Wave Postprocessing Warnings and Suppress Creation of Empty Output Directories#4752
Conversation
… track_dir_out is created before archiving outputs
…reamline file handling in analysis stats
This reverts commit fc2c357.
…directory creation only when files are added to the archive
…section and ensure tarball is always copied to COM in jedi.py
… analysis_stats.py
AntonMFernando-NOAA
changed the title
|
@JessicaMeixner-NOAA @DavidHuber-NOAA
|
I'm pretty surprised these files are identical. Can you share the two paths? |
develop: Ran this command: |
|
Thanks for sharing paths @AntonMFernando-NOAA ! |
|
Okay - well I see in the log file where the second wgrib2 command was called, but the packing is still the same which seems odd to me: |
My assumption is that the original input GRIB2 file uses c2 packing, since the command preserves the same packing as the input file. I tested this using simple packing, and it produced a larger file, so this seems like a reasonable alternative. |
|
AntonMFernando-NOAA
requested review from
AndrewEichmann-NOAA,
BrianCurtis-NOAA,
CoryMartin-NOAA,
DavidHuber-NOAA,
DavidNew-NOAA,
TravisElless-NOAA,
guillaumevernieres and
sbanihash
as code owners
AndrewEichmann-NOAA
left a comment
AndrewEichmann-NOAA
left a comment
There was a problem hiding this comment.
These seem to be getting rid of directories that are still used for the coupled marine
|
|
||
| mkdir -m 775 -p "${COMOUT_OCEAN_ANALYSIS}" | ||
| mkdir -m 775 -p "${COMOUT_ICE_ANALYSIS}" | ||
| mkdir -m 775 -p "${COMOUT_ICE_RESTART}" |
There was a problem hiding this comment.
This is still used
There was a problem hiding this comment.
@AndrewEichmann-NOAA JGLOBAL_MARINE_ANALYSIS_FINALIZE declares the variable but never actually uses it. The real usage happens in the FORECAST job via CICE_out() in ush/forecast_postdet.sh.
This PR, included the directory creation inside if [[ -s "${cmdfile}" ]] (L780 and L910). This means the directory is only created when there are actual CICE or MOM6 restart files to copy.
|
|
||
| declare -rx COMOUT_CONF="${ROTDIR}/${RUN}.${PDY}/${cyc}/ensstat/conf" | ||
|
|
||
| if [[ ! -d "${COMOUT_OCEAN_LETKF}" ]]; then mkdir -p "${COMOUT_OCEAN_LETKF}"; fi |
There was a problem hiding this comment.
This is not always used right now, but is when marine LETKF runs. This is expected to be the default
There was a problem hiding this comment.
@AndrewEichmann-NOAA These directories are created by the Python workflow engine (via the YAML mkdir: list). So removing the shell mkdir has no effect.
JessicaMeixner-NOAA
left a comment
JessicaMeixner-NOAA
left a comment
There was a problem hiding this comment.
Thanks for sorting out the wave postprocessing warnings.
We might need to make a couple adjustments for that in the dev/gfsv17 branch.
Thanks, @JessicaMeixner-NOAA! Let me know if you want me to add these adjustments to dev/gfsv17 branch. |
|
Launching CI on Ursa. |
emcbot
added
CI-Ursa-Ready
6 participants
Description
The wave postprocessing step should complete without warnings. This PR addresses issues that caused warnings during wave postprocessing, specifically impacting WCOSS2.
Resolves warnings in wavepostsbs #4136
This PR also updates the workflow to avoid creating empty output directories such as
model,atmos,ocean,ice,wave,med,inputandproducts. This reduces clutter and ensures only directories with content are created.Resolves Only create "restart" folder in gfs.YYYYMMDD/model/component/ if you are saving the last restart #4642
Resolves Remove empty model/[ocean/ice]/input folders #4641
Note: Empty
bumpandcyclone trackdirectories are still created. These will be addressed in a separate PR, which will require updates to the GDAS App and TC_Tracker files.Type of change
Change characteristics
How has this been tested?
hera(optional WCOSS2) and compare the results with the develop branch to confirm that the only difference is the absence of empty directories.Checklist