ENH: Enable pytables to round-trip with StringDtype #60663
Conversation
| if dtype == "str[python]": | ||
| dtype = StringDtype("python", np.nan) | ||
| elif dtype == "string[python]": | ||
| dtype = StringDtype("python", NA) | ||
| elif dtype == "str[pyarrow]": | ||
| dtype = StringDtype("pyarrow", np.nan) | ||
| else: | ||
| assert dtype == "string[pyarrow]" | ||
| dtype = StringDtype("pyarrow", NA) |
There was a problem hiding this comment.
Not sure if there is a better approach here.
There was a problem hiding this comment.
StringDtype.construct_from_string?
There was a problem hiding this comment.
This wouldn't allow round-tripping if you e.g. write out a Python-backed string with NaN-semantics, and read it in an environment with PyArrow installed.
| vlarr = self._handle.create_vlarray(self.group, key, _tables().ObjectAtom()) | ||
| vlarr.append(value.to_numpy()) | ||
| node = getattr(self.group, key) | ||
| if value.dtype == StringDtype("python", np.nan): |
There was a problem hiding this comment.
Do we need all the branches here or can you just do str(values.dtype)?
There was a problem hiding this comment.
Same as #60663 (comment)
| if using_string_dtype() and is_string_array(values, skipna=True): | ||
| if ( | ||
| using_string_dtype() | ||
| and isinstance(values, np.ndarray) |
There was a problem hiding this comment.
I'm assuming read_array can return a non-ndarray type here? Does this prevent against pyarrow arrays?
There was a problem hiding this comment.
I'm assuming
read_arraycan return a non-ndarray type here?
Correct - datetime and string currently.
doc/source/whatsnew/vX.X.X.rstfile if fixing a bug or adding a new feature.Replaces #60625
Writing out with object dtype still infers
strwheninfer_string=True.