Run Synsat on ICON Data for SEVIRI
Here, we will
load ICON test data (limited area simulation, stored in separate files, from IFCES2 project)
run Synsat on ICON for SEVIRI and
finally store the Synsats into a netcdf file
The workflow is similar to 02-Run-Synsat-for-ERA5-Data.ipynb
Setup Env and Load Libraries
[ ]:
import os, sys
os.environ['RTTOV_PYTHON_WRAPPER'] = '/work/bb1262/tools/rttov/rttov-v13.2/wrapper'
import warnings
warnings.filterwarnings("ignore")
[2]:
import synsatipy
from synsatipy.synsat import SynSat
import synsatipy.synsat_example_data as synsat_example_data
Steps towards Synsat
Step I: Initialize Synsat class
[3]:
s = SynSat( synsat_instrument = 'SEVIRI' )
... [synsat] set cloud / aerosol file to /work/bb1262/tools/rttov/rttov-v13.2/rtcoef_rttov13/cldaer_visir/sccldcoef_msg_3_seviri.dat
... [synsat] load coefficient file /work/bb1262/tools/rttov/rttov-v13.2/rtcoef_rttov13/rttov13pred54L/rtcoef_msg_3_seviri_o3.dat
Load successful >>>>> inst_id : 1, nchannels : 6.
2025/05/07 08:20:14 Load coefficients:
2025/05/07 08:20:14 /work/bb1262/tools/rttov/rttov-v13.2/rtcoef_rttov13/rttov13pred54L/rtcoef_msg_3_seviri_o3.dat
2025/05/07 08:20:14 /work/bb1262/tools/rttov/rttov-v13.2/rtcoef_rttov13/cldaer_visir/sccldcoef_msg_3_seviri.dat
Step II: Load Example Data
[4]:
iconname = synsat_example_data.get_example_data( 'icon02' )
s.load( iconname )
... [synsat] read data from file /work/bb1376/data/icon/atlantic-cases/paulette/ifces2-atlanXL-20200907-exp021/POSTPROC//3d_full_base_DOM02_ML_20200912T000000Z_regrid7km.nc
The ICON data have been loaded into a data handler. You can inspect the data as follows:
[5]:
icon = s.synsat.data_handler.input_data
[6]:
print(icon)
<xarray.Dataset> Size: 2GB
Dimensions: (lon: 1094, lat: 767, lev: 70, time: 1)
Coordinates:
* lon (lon) float32 4kB -85.0 -84.94 -84.87 ... -15.18 -15.11 -15.05
* lat (lat) float32 3kB 0.0 0.06 0.12 0.18 ... 45.78 45.84 45.9 45.96
* height (lev) float64 560B 1.0 2.0 3.0 4.0 5.0 ... 66.0 67.0 68.0 69.0 70.0
* time (time) datetime64[ns] 8B 2020-09-12
Dimensions without coordinates: lev
Data variables:
p (time, lev, lat, lon) float32 235MB dask.array<chunksize=(1, 39, 767, 1094), meta=np.ndarray>
t (time, lev, lat, lon) float32 235MB dask.array<chunksize=(1, 39, 767, 1094), meta=np.ndarray>
q (time, lev, lat, lon) float32 235MB dask.array<chunksize=(1, 39, 767, 1094), meta=np.ndarray>
clwc (time, lev, lat, lon) float32 235MB dask.array<chunksize=(1, 39, 767, 1094), meta=np.ndarray>
ciwc (time, lev, lat, lon) float32 235MB dask.array<chunksize=(1, 39, 767, 1094), meta=np.ndarray>
cswc (time, lev, lat, lon) float32 235MB dask.array<chunksize=(1, 39, 767, 1094), meta=np.ndarray>
SKT (time, lat, lon) float32 3MB dask.array<chunksize=(1, 767, 1094), meta=np.ndarray>
T2M (time, lat, lon) float32 3MB dask.array<chunksize=(1, 767, 1094), meta=np.ndarray>
SP (time, lat, lon) float32 3MB dask.array<chunksize=(1, 767, 1094), meta=np.ndarray>
cc (time, lev, lat, lon) float32 235MB dask.array<chunksize=(1, 39, 767, 1094), meta=np.ndarray>
The actual ICON data are not really small. At this time, however, the data are not loaded into memory.
Maybe this is really too large for our small example here. Let’s redo the input and subset the ICON data!
Step IIb: Subset and Thean Load Again Example Data
[7]:
iconname = synsat_example_data.get_example_data( 'icon01' )
s.load( iconname, isel = {'lon': slice(0,None,4), 'lat': slice(0,None, 4)} )
... [synsat] read data from file /work/bb1376/data/icon/atlantic-cases/paulette/ifces2-atlanXL-20200907-exp021/POSTPROC//3d_full_base_DOM01_ML_20200912T000000Z_regrid7km.nc
The ICON data have been loaded into a data handler. You can inspect the data as follows:
[8]:
icon = s.synsat.data_handler.input_data
[9]:
print(icon)
<xarray.Dataset> Size: 104MB
Dimensions: (lon: 274, lat: 192, lev: 70, time: 1)
Coordinates:
* lon (lon) float32 1kB -85.0 -84.74 -84.49 ... -15.62 -15.37 -15.11
* lat (lat) float32 768B 0.0 0.24 0.48 0.72 ... 45.12 45.36 45.6 45.84
* height (lev) float64 560B 1.0 2.0 3.0 4.0 5.0 ... 66.0 67.0 68.0 69.0 70.0
* time (time) datetime64[ns] 8B 2020-09-12
Dimensions without coordinates: lev
Data variables:
p (time, lev, lat, lon) float32 15MB dask.array<chunksize=(1, 39, 192, 274), meta=np.ndarray>
t (time, lev, lat, lon) float32 15MB dask.array<chunksize=(1, 39, 192, 274), meta=np.ndarray>
q (time, lev, lat, lon) float32 15MB dask.array<chunksize=(1, 39, 192, 274), meta=np.ndarray>
clwc (time, lev, lat, lon) float32 15MB dask.array<chunksize=(1, 39, 192, 274), meta=np.ndarray>
ciwc (time, lev, lat, lon) float32 15MB dask.array<chunksize=(1, 39, 192, 274), meta=np.ndarray>
cswc (time, lev, lat, lon) float32 15MB dask.array<chunksize=(1, 39, 192, 274), meta=np.ndarray>
SKT (time, lat, lon) float32 210kB dask.array<chunksize=(1, 192, 274), meta=np.ndarray>
T2M (time, lat, lon) float32 210kB dask.array<chunksize=(1, 192, 274), meta=np.ndarray>
SP (time, lat, lon) float32 210kB dask.array<chunksize=(1, 192, 274), meta=np.ndarray>
cc (time, lev, lat, lon) float32 15MB dask.array<chunksize=(1, 39, 192, 274), meta=np.ndarray>
Yes, much better and smaller now!
Step III: Setup Run Options and Start Execution
[10]:
s._options.Nthreads=4
s._options.NprofsPerCall = 4000
[11]:
%%time
s.run( chunked = True )
... [synsat] running 0/14 chunk with {'profile': slice(0, 4000, None)}
IR emissivity atlas loaded successfully
Atlas deallocated.
2025/05/07 08:20:24 Running RTTOV using nthreads = 4 and nprofs_per_call = 4000
... [synsat] running 1/14 chunk with {'profile': slice(4000, 8000, None)}
IR emissivity atlas loaded successfully
Atlas deallocated.
2025/05/07 08:20:28 Running RTTOV using nthreads = 4 and nprofs_per_call = 4000
... [synsat] running 2/14 chunk with {'profile': slice(8000, 12000, None)}
IR emissivity atlas loaded successfully
Atlas deallocated.
2025/05/07 08:20:34 Running RTTOV using nthreads = 4 and nprofs_per_call = 4000
... [synsat] running 3/14 chunk with {'profile': slice(12000, 16000, None)}
IR emissivity atlas loaded successfully
Atlas deallocated.
2025/05/07 08:20:38 Running RTTOV using nthreads = 4 and nprofs_per_call = 4000
... [synsat] running 4/14 chunk with {'profile': slice(16000, 20000, None)}
IR emissivity atlas loaded successfully
Atlas deallocated.
2025/05/07 08:20:42 Running RTTOV using nthreads = 4 and nprofs_per_call = 4000
... [synsat] running 5/14 chunk with {'profile': slice(20000, 24000, None)}
IR emissivity atlas loaded successfully
Atlas deallocated.
2025/05/07 08:20:46 Running RTTOV using nthreads = 4 and nprofs_per_call = 4000
2025/05/07 08:20:47 rttov_check_reg_limits.F90
Input water vapour profile exceeds upper coef limit (profile number = 989)
2025/05/07 08:20:47 Limit = 20.3040
2025/05/07 08:20:47 p (hPa) = 97.1505
2025/05/07 08:20:47 Value = 20.7966
... [synsat] running 6/14 chunk with {'profile': slice(24000, 28000, None)}
IR emissivity atlas loaded successfully
Atlas deallocated.
2025/05/07 08:20:51 Running RTTOV using nthreads = 4 and nprofs_per_call = 4000
... [synsat] running 7/14 chunk with {'profile': slice(28000, 32000, None)}
IR emissivity atlas loaded successfully
Atlas deallocated.
2025/05/07 08:20:56 Running RTTOV using nthreads = 4 and nprofs_per_call = 4000
... [synsat] running 8/14 chunk with {'profile': slice(32000, 36000, None)}
IR emissivity atlas loaded successfully
Atlas deallocated.
2025/05/07 08:21:02 Running RTTOV using nthreads = 4 and nprofs_per_call = 4000
... [synsat] running 9/14 chunk with {'profile': slice(36000, 40000, None)}
IR emissivity atlas loaded successfully
Atlas deallocated.
2025/05/07 08:21:06 Running RTTOV using nthreads = 4 and nprofs_per_call = 4000
... [synsat] running 10/14 chunk with {'profile': slice(40000, 44000, None)}
IR emissivity atlas loaded successfully
Atlas deallocated.
2025/05/07 08:21:11 Running RTTOV using nthreads = 4 and nprofs_per_call = 4000
... [synsat] running 11/14 chunk with {'profile': slice(44000, 48000, None)}
IR emissivity atlas loaded successfully
Atlas deallocated.
2025/05/07 08:21:16 Running RTTOV using nthreads = 4 and nprofs_per_call = 4000
... [synsat] running 12/14 chunk with {'profile': slice(48000, 52000, None)}
IR emissivity atlas loaded successfully
Atlas deallocated.
2025/05/07 08:21:20 Running RTTOV using nthreads = 4 and nprofs_per_call = 4000
... [synsat] running 13/14 chunk with {'profile': slice(52000, None, None)}
IR emissivity atlas loaded successfully
Atlas deallocated.
2025/05/07 08:21:24 Running RTTOV using nthreads = 4 and nprofs_per_call = 4000
CPU times: user 1min 13s, sys: 52.6 s, total: 2min 5s
Wall time: 1min 6s
Step IV: Extract the Output
[12]:
icon_synsat = s.extract_output()
[13]:
print( icon_synsat )
<xarray.Dataset> Size: 3MB
Dimensions: (time: 1, lon: 274, lat: 192)
Coordinates:
* time (time) datetime64[ns] 8B 2020-09-12
* lon (lon) float32 1kB -85.0 -84.74 -84.49 ... -15.62 -15.37 -15.11
* lat (lat) float32 768B 0.0 0.24 0.48 0.72 ... 45.12 45.36 45.6 45.84
Data variables:
bt062 (time, lon, lat) float64 421kB 226.2 226.2 226.1 ... 233.6 232.2
bt073 (time, lon, lat) float64 421kB 244.4 244.3 244.2 ... 257.2 256.5
bt087 (time, lon, lat) float64 421kB 278.9 278.9 278.8 ... 285.9 286.0
bt108 (time, lon, lat) float64 421kB 282.8 282.7 282.6 ... 287.9 288.1
bt120 (time, lon, lat) float64 421kB 280.9 280.8 280.8 ... 287.1 287.4
bt134 (time, lon, lat) float64 421kB 252.2 252.1 252.1 ... 264.1 264.3
Attributes:
author: Fabian Senf
contact: senf@tropos.de
institution: Leibniz Institute for Tropospheric Research
creation_time: 2025-05-07 08:21:24.578753
synsat_version: 0.1
synsat_githash: 4f8b856c0aceda7924506f3c05b728f26ad0190a
license: CC-BY SA 3.0
_local_software_path: /home/b/b380352/proj/2022-01_RTTOV-Synsat/synsatip...
input_filename: /work/bb1376/data/icon/atlantic-cases/paulette/ifc...
Step V: Store Data
[14]:
outdir = './Data'
if not os.path.isdir( outdir ):
os.makedirs( outdir )
outfile = f'{outdir}/seviri_synsat_icon_example_data.nc'
print(f'... store synsat at {outfile}')
icon_synsat.to_netcdf(outfile)
... store synsat at ./Data/seviri_synsat_icon_example_data.nc