Time mapper doesn't handle time zones from dsgrid

[daniel-thom]

dsgrid creates geography records with time zones like EasternPrevailing, CentralPrevailing, etc: https://github.com/dsgrid/dsgrid-project-StandardScenarios/blob/main/dsgrid_project/dimensions/counties.csv

chronify uses pandas for time zone conversions and doesn't know about these. What should we do?

E               pytz.exceptions.UnknownTimeZoneError: 'PacificPrevailing'

env/lib/python3.11/site-packages/pytz/__init__.py:188: UnknownTimeZoneError
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> entering PDB >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PDB post_mortem (IO-capturing turned off) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> /Users/dthom/repos/secondary-dsgrid/env/lib/python3.11/site-packages/pytz/__init__.py(188)timezone()
-> raise UnknownTimeZoneError(zone)
(Pdb) up
> /Users/dthom/repos/secondary-dsgrid/timezones.pyx(170)pandas._libs.tslibs.timezones.maybe_get_tz()
(Pdb) 
> /Users/dthom/repos/secondary-dsgrid/timezones.pyx(145)pandas._libs.tslibs.timezones.maybe_get_tz()
(Pdb) 
> /Users/dthom/repos/secondary-dsgrid/env/lib/python3.11/site-packages/pandas/core/arrays/datetimes.py(911)tz_convert()
-> tz = timezones.maybe_get_tz(tz)
(Pdb) 
> /Users/dthom/repos/secondary-dsgrid/env/lib/python3.11/site-packages/pandas/core/indexes/datetimes.py(283)tz_convert()
-> arr = self._data.tz_convert(tz)
(Pdb) 
> /Users/dthom/repos/secondary-dsgrid/env/lib/python3.11/site-packages/pandas/core/indexes/accessors.py(132)_delegate_method()
-> result = method(*args, **kwargs)
(Pdb) 
> /Users/dthom/repos/secondary-dsgrid/env/lib/python3.11/site-packages/pandas/core/accessor.py(112)f()
-> return self._delegate_method(name, *args, **kwargs)
(Pdb) 
> /Users/dthom/repos/chronify/src/chronify/representative_time_range_generator.py(64)create_tz_aware_mapping_dataframe()
-> dft["timestamp_tmp"] = dft[timestamp_col].dt.tz_convert(tz)
(Pdb) ll
 54  	   def create_tz_aware_mapping_dataframe(
 55  	       self,
 56  	       df: pd.DataFrame,
 57  	       timestamp_col: str,
 58  	       time_zones: list[str],
 59  	   ) -> pd.DataFrame:
 60  	       """Create time zone-aware time mapping dataframe."""
 61  	       dfm = []
 62  	       for tz in time_zones:
 63  	           dft = df.copy()
 64  ->	           dft["timestamp_tmp"] = dft[timestamp_col].dt.tz_convert(tz)
 65  	           dft = self._handler.add_time_attribute_columns(dft, "timestamp_tmp")

[lixiliu]

Either:
[1] Remove the use of these enums from dsgrid and rely instead on a tz library that pandas uses, which is dateutil: https://dateutil.readthedocs.io/en/stable/tz.html
[2] Convert the enums to pandas-friendly field values
[3] Build in Chronify capability to accept these enums.

These enums contain values for the pyspark API and can be used to go between pairs of standard time and prevailing time. They were useful for mapping industrial time. Let's see what @JensZack ends up implementing. I am inclined to go with the cleanest solution in that if these enums are no longer needed, we go with [1].

[lixiliu]

Decision:

• Remove enums
• Allow users to pass in any field values from zoneinfo.ZoneInfo, add this check in dsgrid
• For fixing dsgrid-project legacy datasets, use GMT offsets (e.g., "Etc/+xx") to replace standard time time zones (tz_name of enums).
• Apply bullet 3 to convert old repo content. No input data or generated data is affected.

@elainethale - does this work for you?

[elainethale]

Yes, that works. Thanks!