feat: grid serialization #100
Conversation
Signed-off-by: jaapschoutenalliander <[email protected]>
Signed-off-by: jaapschoutenalliander <[email protected]>
Signed-off-by: jaapschoutenalliander <[email protected]>
Signed-off-by: jaapschoutenalliander <[email protected]>
Signed-off-by: jaapschoutenalliander <[email protected]>
Signed-off-by: jaapschoutenalliander <[email protected]>
Signed-off-by: jaapschoutenalliander <[email protected]>
Signed-off-by: jaapschoutenalliander <[email protected]>
Signed-off-by: jaapschoutenalliander <[email protected]>
Signed-off-by: jaapschoutenalliander <[email protected]>
Signed-off-by: jaapschoutenalliander <[email protected]>
Signed-off-by: jaapschoutenalliander <[email protected]>
I'd vote for |
|
|
||
| try: | ||
| array_field = grid.find_array_field(getattr(grid, attr_name).__class__) | ||
| matched_columns = { |
There was a problem hiding this comment.
perhaps we can re-use PowerGridModelInterface._match_dtypes here (will have to move the code)
There was a problem hiding this comment.
Interesting, that could work. However, that function compares two dtype objects and here we have one dtype object and a dict with columns as keys. So they do not match completely. Given the low complexity building a shared function might not be worthwile
| return path | ||
|
|
||
|
|
||
| def load_grid_from_json(path: Path, target_grid_class=None): |
There was a problem hiding this comment.
perhaps just:
| def load_grid_from_json(path: Path, target_grid_class=None): | |
| def deserialize(path: Path, target_grid_class=None): |
There was a problem hiding this comment.
For now that would work, but if we later add another serialisation option having load_grid_from... works better
| logger.warning(f"Failed to restore '{attr_name}': {e}") | ||
|
|
||
|
|
||
| def save_grid_to_json( |
There was a problem hiding this comment.
perhaps just
| def save_grid_to_json( | |
| def serialize( |
| if isinstance(field_value, (int, float, str, bool)): | ||
| serialized_data[field.name] = field_value | ||
|
|
||
| if not isinstance(field_value, FancyArray) or field_value.size == 0: |
There was a problem hiding this comment.
I would crash here if not FancyArray for now instead of skipping. Or maybe this should be an option to skip?
| continue | ||
|
|
||
| if not issubclass(getattr(grid, attr_name).__class__, FancyArray): | ||
| setattr(grid, attr_name, attr_values) |
There was a problem hiding this comment.
Should we check that it is the correct type as well?
Now you could set a bool for something that is typed as a string.
There was a problem hiding this comment.
I now get the type from the grid and make sure to cast it to the expected type (if possible). I will also add test for sad flow
| assert loaded_grid.node.size == basic_grid.node.size | ||
| assert loaded_grid.line.size == basic_grid.line.size | ||
| assert list(loaded_grid.node.id) == list(basic_grid.node.id) |
There was a problem hiding this comment.
Cant we just use, with assert_array_equal from numpy.testing:
(also applicable to other tests in this file)
| assert loaded_grid.node.size == basic_grid.node.size | |
| assert loaded_grid.line.size == basic_grid.line.size | |
| assert list(loaded_grid.node.id) == list(basic_grid.node.id) | |
| assert_array_equal(loaded_grid.node.data, basic_grid.node.data) | |
| assert_array_equal(loaded_grid.line.data, basic_grid.line.data) |
There was a problem hiding this comment.
I updated to use the array_equal from pgm_ds where applicable
There was a problem hiding this comment.
Ik mis tests voor wanneer het crasht.
Co-authored-by: Vincent Koppen <[email protected]> Signed-off-by: Jaap Schouten <[email protected]>
Signed-off-by: jaapschoutenalliander <[email protected]>
Signed-off-by: jaapschoutenalliander <[email protected]>
Signed-off-by: jaapschoutenalliander <[email protected]>
Signed-off-by: jaapschoutenalliander <[email protected]>
|
| txt_lines = f.readlines() | ||
| return TextSource(grid_class=cls).load_from_txt(*txt_lines) | ||
|
|
||
| def to_json(self, path: Path, **kwargs) -> Path: |
There was a problem hiding this comment.
this way, we'll need two extra methods for each serialization method we add in the future. I'd rather name this method serialize and add a method parameter once we include other serialization methods.
future example:
def serialize(self, path: Path, method: Literal["json", "msgpack", "parquet"], **kwargs):There was a problem hiding this comment.
That sounds good
3 participants
This pull request introduces robust JSON serialization and deserialization support for
Gridobjects, enabling interoperability and extension handling in the power grid modeling library. It adds new serialization utilities, integrates them into theGridAPI, and provides comprehensive unit tests to ensure correct roundtrips and compatibility across extended types