Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

XIOS: Determine Axes and Domains from NetCDF file #732

Open
jwallwork23 opened this issue Nov 11, 2024 · 2 comments
Open

XIOS: Determine Axes and Domains from NetCDF file #732

jwallwork23 opened this issue Nov 11, 2024 · 2 comments
Assignees
Labels
ICCS Tasks or reviews for the ICCS team

Comments

@jwallwork23
Copy link
Contributor

Thinking more about what's required for #552, I realised there's a key missing piece for reading files.

Usually, XIOS reads in an XML configuration file along with the NetCDF file that contains the data. The XML file says how to set up the various attributes, e.g., Axes, Domains, Fields. If we want to go ahead with using the XIOS API instead of XML files then we need to be able to read such information from NetCDF files directly. This could be done by borrowing from the existing I/O implementation, for example.

On the other hand, am I right in thinking that the only use case for having XIOS read from file is for restarts? In the case of restarts, you presumably already know how the dimensions and fields are laid out so could reproduce the XIOS attributes without needing to read them.

Any thoughts on this @timspainNERSC @einola?

@jwallwork23 jwallwork23 added the ICCS Tasks or reviews for the ICCS team label Nov 11, 2024
@jwallwork23 jwallwork23 self-assigned this Nov 11, 2024
@jwallwork23 jwallwork23 moved this from Todo to Discussion in neXtSIM_DG overview Nov 11, 2024
@timspainNERSC
Copy link
Collaborator

The ultimate plan is of course to replace my ad-hoc IO with XIOS. So, below is how file reading currently works.

  1. The restart file is briefly opened to read the contents of structure/type. This is used to select with implementation of StructureModule will be used to read the file. The file is then closed.
  2. (Assuming ParaGrid from here on). The restart file is opened and the dimension data is read. Up until this point, the model has no defined size or land map. This is all determined from the restart file at this point in the start-up phase.
  3. The contents of the file are then read, using the sizes just obtained, and every variable contained in the restart file is added to a name->data map contained in a ModelState object.
  4. The restart file is closed and the ModelState object passed to the cascade of classes that make up the model.

Additionally there is reading forcing data from forcing files, which contain many time samples. Currently the forcing files are fixed to the same grid as the model fields, but the hope is that XIOS or OASIS will allow us to use the files provided in the reanalysis datasets.

The summary is that the current method is to read the dimensions and fields as part of the input process, and not to have them predefined in the configuration file, if possible. And definitely not to set them at compile time.

@jwallwork23
Copy link
Contributor Author

Thanks for the description here @timspainNERSC, it's really useful! I'll implement file writing first and see what comes up there before coming back to this to implement file reading.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ICCS Tasks or reviews for the ICCS team
Projects
Status: Discussion
Development

No branches or pull requests

2 participants