The short-term prediction of precipitation is critical in many areas of life. Recently, a large body of work was devoted to forecasting radar reflectivity images. The radar images are available only in areas with ground weather radars. Thus, we aim to predict high-resolution precipitation from lower-resolution satellite radiance images. A neural network called WeatherFusionNet is employed to predict severe rain up to eight hours in advance. WeatherFusionNet is a U-Net architecture that fuses three different ways to process the satellite data; predicting future satellite frames, extracting rain information from the current frames, and using the input sequence directly. Using the presented method, we achieved 1st place in the NeurIPS 2022 Weather4Cast Core challenge. The code and trained parameters are available at \url{https://github.com/Datalab-FIT-CTU/weather4cast-2022}.

Accurate and timely rain prediction is crucial for decision making and is also a challenging task. This paper presents a solution which won the 2 nd prize in the Weather4cast 2022 NeurIPS competition using 3D U-Nets and EarthFormers for 8-hour probabilistic rain prediction based on multi-band satellite images. The spatial context effect of the input satellite image has been deeply explored and optimal context range has been found. Based on the imbalanced rain distribution, we trained multiple models with different loss functions. To further improve the model performance, multi-model ensemble and threshold optimization were used to produce the final probabilistic rain prediction. Experiment results and leaderboard scores demonstrate that optimal spatial context, combined loss function, multi-model ensemble, and threshold optimization all provide modest model gain. A permutation test was used to analyze the effect of each satellite band on rain prediction, and results show that satellite bands signifying cloudtop phase (8.7 um) and cloud-top height (10.8 and 13.4 um) are the best predictors for rain prediction. The source code is available at https://github.com/bugsuse/weather4cast-2022-stage2.

Accurately forecasting the weather is an important task, as many real-world processes and decisions depend on future meteorological conditions. The NeurIPS 2022 challenge entitled Weather4cast poses the problem of predicting rainfall events for the next eight hours given the preceding hour of satellite observations as a context. Motivated by the recent success of transformer-based architectures in computer vision, we implement and propose two methodologies based on this architecture to tackle this challenge. We find that ensembling different transformers with some baseline models achieves the best performance we could measure on the unseen test data. Our approach has been ranked 3rd in the competition.

This paper presents a solution to the Weather4cast 2022 Challenge Stage 2. The goal of the challenge is to forecast future high-resolution rainfall events obtained from ground radar using low-resolution multiband satellite images. We suggest a solution that performs data preprocessing appropriate to the challenge and then predicts rainfall movies using a novel RainUNet. RainUNet is a hierarchical U-shaped network with temporal-wise separable block (TS block) using a decoupled large kernel 3D convolution to improve the prediction performance. Various evaluation metrics show that our solution is effective compared to the baseline method. The source codes are available at https://github.com/jinyxp/Weather4cast-2022

Traditional weather forecasting relies on domain expertise and computationally intensive numerical simulation systems. Recently, with the development of a data-driven approach, weather forecasting based on deep learning has been receiving attention. Deep learning-based weather forecasting has made stunning progress, from various backbone studies using CNN, RNN, and Transformer to training strategies using weather observations datasets with auxiliary inputs. All of this progress has contributed to the field of weather forecasting; however, many elements and complex structures of deep learning models prevent us from reaching physical interpretations. This paper proposes a SImple baseline with a spatiotemporal context Aggregation Network (SIANet) that achieved state-of-the-art in 4 parts of 5 benchmarks of W4C22. This simple but efficient structure uses only satellite images and CNNs in an end-to-end fashion without using a multi-model ensemble or fine-tuning. This simplicity of SIANet can be used as a solid baseline that can be easily applied in weather forecasting using deep learning.

本文介绍了作者在天气4播出阶段1中使用的神经网络模型，其中目标是预测基于卫星天气数据图像的时间演变。该网络基于编码器 - 预测架构利用所通用的经常性单元（GU），残差块和具有类似U-Net类似的快捷方式的契约/扩展架构。还介绍了利用剩余块代替卷积的GRU变体。提出了模型的示例预测和评估度量。这些表明，该模型可以保留第一个预测的输入的尖锐特征，而后来的预测变得更模糊以反映不变的不确定性。

能量供应和需求受到气象条件的影响。随着对可再生能源的需求增加，精确天气预报的相关性增加。能源提供者和决策者要求天气信息进行明智的选择，并根据业务目标建立最佳计划。由于最近应用于卫星图像的深度学习技术，使用遥感数据的天气预报也是主要进步的主题。本文通过基于U-Net的架构调查了荷兰沿海海洋元素的多个步骤框架预测。来自哥白尼观察计划的每小时数据在2年内跨过跨越2年的时间，用于培训模型并进行预测，包括季节性预测。我们提出了U-Net架构的变化，并使用剩余连接，并行卷积和不对称卷积进一步扩展了这一新颖模型，以便引入三种额外的架构。特别是，我们表明，配备有平行和不对称卷积的架构以及跳过连接优于其他三个讨论的模型。

太阳能现在是历史上最便宜的电力形式。不幸的是，由于其变异性，显着提高栅格的太阳能的一部分仍然具有挑战性，这使得电力的供需平衡更加困难。虽然热发电机坡度 - 它们可以改变输出的最高速率 - 是有限的，太阳能的坡度基本上是无限的。因此，准确的近期太阳能预测或垂圈，对于提供预警来调整热发电机输出，以响应于太阳能变化来调整热发电机，以确保平衡供需。为了解决问题，本文开发了使用自我监督学习的丰富和易于使用的多光谱卫星数据的太阳能垂圈的一般模型。具体而言，我们使用卷积神经网络（CNN）和长短期内存网络（LSTM）开发深度自动回归模型，这些模型在多个位置训练全球培训，以预测最近推出的最近收集的时空数据的未来观察-R系列卫星。我们的模型估计了基于卫星观测的未来的太阳辐照度，我们向较小的场地特定的太阳能数据培训的回归模型提供，以提供近期太阳能光伏（PV）预测，其考虑了现场特征的特征。我们评估了我们在25个太阳能场所的不同覆盖区域和预测视野的方法，并表明我们的方法利用地面真理观察结果产生靠近模型的错误。

Deep learning-based weather prediction models have advanced significantly in recent years. However, data-driven models based on deep learning are difficult to apply to real-world applications because they are vulnerable to spatial-temporal shifts. A weather prediction task is especially susceptible to spatial-temporal shifts when the model is overfitted to locality and seasonality. In this paper, we propose a training strategy to make the weather prediction model robust to spatial-temporal shifts. We first analyze the effect of hyperparameters and augmentations of the existing training strategy on the spatial-temporal shift robustness of the model. Next, we propose an optimal combination of hyperparameters and augmentation based on the analysis results and a test-time augmentation. We performed all experiments on the W4C22 Transfer dataset and achieved the 1st performance.

The Weather4Cast competition (hosted by NeurIPS 2022) required competitors to predict super-resolution rain movies in various regions of Europe when low-resolution satellite contexts covering wider regions are given. In this paper, we show that a general baseline 3D U-Net can be significantly improved with region-conditioned layers as well as orthogonality regularizations on 1x1x1 convolutional layers. Additionally, we facilitate the generalization with a bag of training strategies: mixup data augmentation, self-distillation, and feature-wise linear modulation (FiLM). Presented modifications outperform the baseline algorithms (3D U-Net) by up to 19.54% with less than 1% additional parameters, which won the 4th place in the core test leaderboard.

前所未有的访问多时间卫星图像，为各种地球观察任务开辟了新的视角。其中，农业包裹的像素精确的Panoptic分割具有重大的经济和环境影响。虽然研究人员对单张图像进行了探索了这个问题，但我们争辩说，随着图像的时间序列更好地寻址作物候选的复杂时间模式。在本文中，我们介绍了卫星图像时间序列（坐着）的Panoptic分割的第一端到端，单级方法（坐姿）。该模块可以与我们的新型图像序列编码网络相结合，依赖于时间自我关注，以提取丰富和自适应的多尺度时空特征。我们还介绍了Pastis，第一个开放式访问坐在Panoptic注释的数据集。我们展示了对多个竞争架构的语义细分的编码器的优越性，并建立了坐在的第一封Panoptic细分状态。我们的实施和痛苦是公开的。

提出了一个深度学习模型，以便在未来60分钟的五分钟时间分辨率下以闪电的形式出现。该模型基于反复横向的结构，该结构使其能够识别并预测对流的时空发展，包括雷暴细胞的运动，生长和衰变。预测是在固定网格上执行的，而无需使用风暴对象检测和跟踪。从瑞士和周围的区域收集的输入数据包括地面雷达数据，可见/红外卫星数据以及衍生的云产品，闪电检测，数值天气预测和数字高程模型数据。我们分析了不同的替代损失功能，班级加权策略和模型特征，为将来的研究提供了指南，以最佳地选择损失功能，并正确校准其模型的概率预测。基于这些分析，我们在这项研究中使用焦点损失，但得出结论，它仅在交叉熵方面提供了较小的好处，如果模型的重新校准不实用，这是一个可行的选择。该模型在60分钟的现有周期内实现了0.45的像素临界成功指数（CSI）为0.45，以预测8 km的闪电发生，范围从5分钟的CSI到5分钟的提前时间到CSI到CSI的0.32在A处。收货时间60分钟。

Fusing satellite imagery acquired with different sensors has been a long-standing challenge of Earth observation, particularly across different modalities such as optical and Synthetic Aperture Radar (SAR) images. Here, we explore the joint analysis of imagery from different sensors in the light of representation learning: we propose to learn a joint embedding of multiple satellite sensors within a deep neural network. Our application problem is the monitoring of lake ice on Alpine lakes. To reach the temporal resolution requirement of the Swiss Global Climate Observing System (GCOS) office, we combine three image sources: Sentinel-1 SAR (S1-SAR), Terra MODIS, and Suomi-NPP VIIRS. The large gaps between the optical and SAR domains and between the sensor resolutions make this a challenging instance of the sensor fusion problem. Our approach can be classified as a late fusion that is learned in a data-driven manner. The proposed network architecture has separate encoding branches for each image sensor, which feed into a single latent embedding. I.e., a common feature representation shared by all inputs, such that subsequent processing steps deliver comparable output irrespective of which sort of input image was used. By fusing satellite data, we map lake ice at a temporal resolution of < 1.5 days. The network produces spatially explicit lake ice maps with pixel-wise accuracies > 91% (respectively, mIoU scores > 60%) and generalises well across different lakes and winters. Moreover, it sets a new state-of-the-art for determining the important ice-on and ice-off dates for the target lakes, in many cases meeting the GCOS requirement.

The goal of precipitation nowcasting is to predict the future rainfall intensity in a local region over a relatively short period of time. Very few previous studies have examined this crucial and challenging weather forecasting problem from the machine learning perspective. In this paper, we formulate precipitation nowcasting as a spatiotemporal sequence forecasting problem in which both the input and the prediction target are spatiotemporal sequences. By extending the fully connected LSTM (FC-LSTM) to have convolutional structures in both the input-to-state and state-to-state transitions, we propose the convolutional LSTM (ConvLSTM) and use it to build an end-to-end trainable model for the precipitation nowcasting problem. Experiments show that our ConvLSTM network captures spatiotemporal correlations better and consistently outperforms FC-LSTM and the state-of-theart operational ROVER algorithm for precipitation nowcasting.

降雨数据的时间和空间分辨率对于环境建模研究至关重要，在环境建模研究中，其时空的变异性被视为主要因素。来自不同遥感仪器（例如雷达，卫星）的降雨产品具有不同的时空分辨率，因为它们的感应能力和后处理方法的差异。在这项研究中，我们开发了一种深度学习方法，以增加降雨数据，并增加时间分辨率，以补充相对较低的分辨率产品。我们提出了基于卷积神经网络（CNN）的神经网络体系结构，以改善基于雷达的降雨产品的时间分辨率，并将提出的模型与基于光流的插值方法和CNN基线模型进行比较。这项研究中提出的方法可用于增强降雨图，并以更好的时间分辨率和2D降雨图序列中缺失的框架进行插补，以支持水文和洪水预测研究。

准确地估算主要山区盆地中的积雪对于水资源经理来说至关重要，以便做出影响当地和全球经济，野生动植物和公共政策的决策。目前，此估计需要多个配备LIDAR的飞机飞行或原位测量值，两者均昂贵，稀疏和对可访问区域有偏见。在本文中，我们证明了来自多个，公开可用的卫星和天气数据源的空间和时间信息的融合，可以估算关键山区的积雪。我们的多源模型的表现优于单源估计值5.0英寸RMSE，并且优于稀疏的原位测量值的估计值1.2英寸RMSE。

Forecasting the state of vegetation in response to climate and weather events is a major challenge. Its implementation will prove crucial in predicting crop yield, forest damage, or more generally the impact on ecosystems services relevant for socio-economic functioning, which if absent can lead to humanitarian disasters. Vegetation status depends on weather and environmental conditions that modulate complex ecological processes taking place at several timescales. Interactions between vegetation and different environmental drivers express responses at instantaneous but also time-lagged effects, often showing an emerging spatial context at landscape and regional scales. We formulate the land surface forecasting task as a strongly guided video prediction task where the objective is to forecast the vegetation developing at very fine resolution using topography and weather variables to guide the prediction. We use a Convolutional LSTM (ConvLSTM) architecture to address this task and predict changes in the vegetation state in Africa using Sentinel-2 satellite NDVI, having ERA5 weather reanalysis, SMAP satellite measurements, and topography (DEM of SRTMv4.1) as variables to guide the prediction. Ours results highlight how ConvLSTM models can not only forecast the seasonal evolution of NDVI at high resolution, but also the differential impacts of weather anomalies over the baselines. The model is able to predict different vegetation types, even those with very high NDVI variability during target length, which is promising to support anticipatory actions in the context of drought-related disasters.

降水预测是一项重要的科学挑战，对社会产生广泛影响。从历史上看，这项挑战是使用数值天气预测（NWP）模型解决的，该模型基于基于物理的模拟。最近，许多作品提出了一种替代方法，使用端到端深度学习（DL）模型来替代基于物理的NWP。尽管这些DL方法显示出提高的性能和计算效率，但它们在长期预测中表现出局限性，并且缺乏NWP模型的解释性。在这项工作中，我们提出了一个混合NWP-DL工作流程，以填补独立NWP和DL方法之间的空白。在此工作流程下，NWP输出被馈入深层模型，该模型后处理数据以产生精致的降水预测。使用自动气象站（AWS）观测值作为地面真相标签，对深层模型进行了监督训练。这可以实现两全其美，甚至可以从NWP技术的未来改进中受益。为了促进朝这个方向进行研究，我们提出了一个专注于朝鲜半岛的新型数据集，该数据集称为KOMET（KOMEN（KOREA气象数据集），由NWP预测和AWS观察组成。对于NWP，我们使用全局数据同化和预测系统-KOREA集成模型（GDAPS-KIM）。

亚马逊森林中对森林砍伐的估计是挑战任务，因为该地区的规模巨大和直接人类通道的难度。但是，这是一个至关重要的问题，因为森林砍伐会导致严重的环境问题，例如全球气候变化，生物多样性降低等。为了有效解决这些问题，卫星图像将是估计亚马逊森林砍伐的一个很好的选择。通过光学图像和合成孔径雷达（SAR）图像的组合，无论天气条件如何，都可以观察到如此庞大的区域。在本文中，我们提出了一种准确的森林砍伐估计方法，并使用常规的UNET和全面的数据处理。Sentinel-1，Sentinel-2和Landsat 8的各种渠道被精心选择并用于训练深层神经网络。通过提出的方法，以很高的精度成功估计了新的查询的森林砍伐状态。

将间歇性可再生能源集成到大量的电网中是具有挑战性的。旨在解决这一困难的建立良好的方法涉及即将到来的能源供应可变性以适应电网的响应。在太阳能中，可以在全天空摄像机（前方30分钟）和卫星观测（提前6小时）的不同时间尺度上预测由遮挡云引起的短期变化。在这项研究中，我们将这两种互补的观点集成到单个机器学习框架中的云覆盖物上，以改善时间内（最高60分钟）的辐照度预测。确定性和概率预测均在不同的天气条件（晴朗，多云，阴天）以及不同的输入配置（天空图像，卫星观测和/或过去的辐照度值）中进行评估。我们的结果表明，混合模型在晴朗的条件下有益于预测，并改善了长期预测。这项研究为将来的新颖方法奠定了基础，即在单个学习框架中将天空图像和卫星观测结合起来，以推动太阳现象。