gridded: dimensions and variable exclusion

[smwoodman]

Addresses #226

[smwoodman]

@jklymak a couple fyis to explain my thinking. Happy to adjust these as you like. Sorry, not sure how to directly embed lines from the pr in the comment

ds['time_1970'] = ds.longitude.copy()

Changing this from ds.temperature removes temperature as a 'required' variable for this function to run

profile_direction

Profile direction felt useful to add, as an indicator of if a profile was while the glider was ascending or descending. Since it was only binned by profile, it seemed easiest to put direction with the time/lat/lon binning

exclude_vars , exclude_vars += list(dsout.keys()) + ["depth"]

It felt simplest to let the user pass a list of variables (exclude_var) to ignore when gridding, rather than eg looking for a new 'grid_ignore: true' key in the deploymentyaml file. This line, along with the check if (k in exclude_vars) or ('time' in k) or ('profile' in k): ... continue should be consistent with the current make_gridfiles behavior. It also ensures that both a) all vars already in dsout, like lat/lon or whichever else are added in the future, and b) all time/profile variables, aren't gridded

try: dsout['waypoint_latitude'].attrs.pop('standard_name') ...

profile_time_start and profile_time_end are guaranteed to be in dsout, since they're made in this function. Thus, I moved them to the top of the try statement, in case waypoint_latitude and waypoint_longitude aren't in dsout

[smwoodman]

One other question. Is there a reason that dsout['profile'].attrs['cf_role'] = 'profile_id' is the only profile attribute? Could there be a line such as dsout['profile'].attrs = ds.profile_index.attrs to bring along that info?

[smwoodman]

One other question. Is there a reason that dsout['profile'].attrs['cf_role'] = 'profile_id' is the only profile attribute? Could there be a line such as dsout['profile'].attrs = ds.profile_index.attrs to bring along that info?

@jklymak sorry for the ping, but I wanted to check in about both this question, and the pr as a whole?

[jklymak]

This looks to be gridding longitude and latitude by deafult now? Or are they in the default exclude_vars?

I wonder if for non-coordinate data (eg temperature etc) we should really be setting whether to grid it or not in the yaml instead of specifying a list here. eg in the yaml say grid_exclude: True if we don't want it to be gridded, but we do want it in the time series?

[smwoodman]

This looks to be gridding longitude and latitude by deafult now? Or are they in the default exclude_vars?

longitude and latitude (along with time, profile_direction, profile_time_start, and profile_time_end) are all guaranteed to be part of dsout. Thus, they're always added to exclude_vars in exclude_vars += list(dsout.keys()) + ["depth"]

[smwoodman]

I wonder if for non-coordinate data (eg temperature etc) we should really be setting whether to grid it or not in the yaml instead of specifying a list here. eg in the yaml say grid_exclude: True if we don't want it to be gridded, but we do want it in the time series?

A grid_exclude boolean key feels totally reasonable to me. It's also more consistent with the function checking for the 'average_method' key. I'll update the pr now

[jklymak]

OK, this seems good. I was just wondering though if we wanted to allow the user to exclude a timeseries variable from being gridded at this stage by adding a property to the yaml. But maybe that can be a follow up?

[jklymak]

@smwoodman I was waiting to understand this...

[smwoodman]

@jklymak wanted to check in about this, as I'm currently working on some code that uses these features.

But, please please let me know if you're busy at the moment, and prefer that either I ping someone else or we pick this back up at a later date!