Correct Unexpected floats when reading LI L2 LFL #2998
Conversation
…he _FillValue is not into the attributes
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #2998 +/- ##
==========================================
+ Coverage 96.08% 96.09% +0.01%
==========================================
Files 377 377
Lines 55150 55167 +17
==========================================
+ Hits 52992 53014 +22
+ Misses 2158 2153 -5
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
Pull Request Test Coverage Report for Build 12184523570Warning: This coverage report may be inaccurate.This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.
Details
💛 - Coveralls |
…tests to check the dtype improve the mock datas
ameraner
added
enhancement
| @@ -158,11 +158,9 @@ def get_array_on_fci_grid(self, data_array: xr.DataArray): | |||
| data_2d = da.map_blocks(_np_add_at_wrapper, data_2d, (rows, cols), data_array, | |||
| dtype=data_array.dtype, | |||
| chunks=(LI_GRID_SHAPE[0], LI_GRID_SHAPE[1])) | |||
| data_2d = da.where(data_2d > 0, data_2d, np.nan) | |||
|
|
|||
| data_2d = da.where(data_2d > 0, data_2d, np.nan).astype(np.float32) | |||
There was a problem hiding this comment.
This would prevent one unnecessary upcasting:
| data_2d = da.where(data_2d > 0, data_2d, np.nan).astype(np.float32) | |
| data_2d = da.where(data_2d > 0, data_2d, np.float32(np.nan)) | |
There was a problem hiding this comment.
@pnuu casting a nan who is a float32 is not enough to convert all the arrays into a float32 . For example if data_2d is an int32 the method where will convert it to a float64. To prevent it I have used the method astype(np.float32).
There was a problem hiding this comment.
Ok, that makes sense I guess.
But for integer data the data would still be converted to floats, which seemed to be the original problem reported #2854 , right? So should there be separate handling for the integer data?
There was a problem hiding this comment.
For this specific case, handling the accumulated LI 2-d data arrays, we need floats since we need to support NaN values. So I'm ok with the solution here that avoids float64.
The problem in the original issue was that some integer variables, that do not have a FillValue attribute, were still being casted unnecessarily to float; that problem is fixed by the other modification of this PR in apply_fill_value here https://github.com/pytroll/satpy/pull/2998/files#diff-3b2bff08b4001ec6f72cca67791cc4322b38e0db97e68f2109791093e56e6052R445
Makes sure that the method apply_fill_value is not applied when
_FillaValueis not into the attributes. It is fixing the bug mentioned there #2854 .