2.1. Snow and Ice Disaster Database

The Snow and Ice Disaster Database is a collection of newspaper articles compiled by the Snow and Ice Research Center, National Research Institute for Earth Science and Disaster Resilience, since fiscal year (FY) 2000 ⁶. The database summarizes situations reported in newspapers, as well as accidents and damage, organized by prefecture and type of snow and ice disaster. On the Snow and Ice Disaster Database website, users can choose either “Map View” or “List View”; when “List View” is selected, the target year, prefecture, and type of snow and ice disaster can be specified. “Across Japan” and “All Disaster Types” can also be selected. For the selected range, a table displays the date, location, circumstances and causes of each accident, number of victims, and type of economic loss. In this study, information on the dates and locations of roof-snow-falling incidents was extracted across Japan for each year from FY2019 onward and used as training data. In the disaster database, roof snow falling is in the “falling snow from artifact” category. From this category, we excluded incidents unrelated to buildings, such as snow falling from power lines, and used the remaining cases as training data. The training data included not only cases of snow sliding off a roof, but also instances where snow cornices collapsed or snow removal operations caused the roof snow to fall. The number of roof-snow-falling incidents per year from FY2019 to FY2024 was 1, 58, 51, 22, 8, and 21, respectively. As of March 2026, the website provides data through FY2022; data for FY2023 and FY2024 are not yet publicly available, but are held by the Snow and Ice Research Center and were used in this study.

The number of cases that could be used as training data was limited because the area covered by the YukioroSignal was restricted to a few prefectures in the early years ³. Consequently, 131 incident cases up to FY2024 fell within the YukioroSignal coverage area and were available for analysis.

2.2. Use of SNOWPACK Output Data Calculated by YukioroSignal

YukioroSignal expanded its coverage year by year. Following a previous report ³, the system was extended to western Japan and is currently applied to all heavy snowfall areas, except around Mt. Fuji. In the Chugoku region, for prefectures such as Shimane and Tottori, where most of the prefecture is designated as a heavy snowfall area, the YukioroSignal covers the entire prefecture. In contrast, for prefectures such as Hyogo, Okayama, and Hiroshima, where only mountainous areas are designated as heavy snowfall areas, the municipalities in heavy snowfall areas are included. Similarly, municipalities in Kyoto, Shiga, and Tochigi were selected to cover heavy snowfall areas. YukioroSignal coverage is shown in Fig. 1, together with the year of application for each prefecture. Within this coverage area, snow stratigraphy was calculated using SNOWPACK ^2,3 at locations where snow depth observations were available. The model uses snow depth data from a quasi-real-time snow distribution monitoring system ²⁶ and meteorological inputs from nearby Automated Meteorological Data Acquisition System (AMeDAS) stations. SNOWPACK simulations were performed at 1,762 observation points in this area.

For the roof-snow-falling cases described in Section 2.1, we identified the names of municipalities and regions from individual newspaper articles, searched for those names on a “Calculation site for estimating the post-snow removal snow load” ², and selected the nearest observation point. However, we excluded locations where snow depth observations were recorded daily and selected locations where observations were taken hourly.

Figure 2(a) shows a case study of an accident site in Yamagata Prefecture. Snow depth increased by approximately 70 cm due to heavy snowfall on December 23, and the upper part of the snow cover then metamorphosed into Melt Forms as temperatures rose and rainfall occurred the following day. In this study, “wet-type roof snow falling” is defined as roof snow falling associated with an increase in wet snow due to rising temperatures and rainfall, as in this case.

Figure 2(b) shows a case at an accident site in Hokkaido, where the roof snow fell under dry conditions, with no liquid water in the snow cover. This type of roof snow falling is defined as “dry-type roof snow falling.”

For classification, a case was classified as wet type if the upper 15 cm of the snow cover on the day of occurrence contained any layer with a volumetric water content greater than 2% and as dry type otherwise. Of the 131 cases included in the study, 49 were wet and 82 were dry.

Note that since this definition is not based on the LWC at the interface between the roof and the snowpack, a case will be classified as “dry type” even if snow melts due to heat from the building, provided there is no snowmelt or rainfall caused by weather conditions.

2.3. Construction of Machine Learning Models

When constructing the machine learning models, the meteorological data and SNOWPACK outputs corresponding to each roof-snow-falling accident were used as explanatory variables. The explanatory variables included (i) meteorological inputs to SNOWPACK (air temperature, relative humidity, wind speed, wind direction, solar radiation, long-wave radiation, precipitation, and snow depth) and (ii) all SNOWPACK output variables in the .met file generated using SNOWPACK version 3.7.0 including snow water equivalent, snowpack runoff, snowpack stability index, sensible heat flux, and latent heat flux. Because there are many output variables, please refer to the SNOWPACK website ²⁷ for all parameter names.

Furthermore, to reflect information on snowpack stratigraphy, the snow profile data were processed into the following scalar values: average temperature across all layers, average density, average LWC, maximum LWC, depth of wet snow, proportion of wet snow, average LWC and proportion of wet snow in the top 15 cm, and average LWC and proportion of wet snow in the bottom 30 cm.

The selection of these processed scalar values followed the methodology of Hendrick et al. ²⁵, who combined SNOWPACK outputs with machine learning to develop a wet snow avalanche prediction method. In their study, occurrence and non-occurrence days were used as the target variables. Because roof snow falling is influenced by the characteristics of individual houses, it is strictly speaking different from the occurrence of wet snow avalanches. However, because there is no information on the specific roof characteristics of individual houses and it is impossible to estimate the temperature conditions at the interface between the snowpack and roof, we used Hendrick et al.’s input variables, which are based on natural snow but have a proven track record. Because this model employs many explanatory variables, we believe that the differences in importance derived from the training results will yield a machine learning model that is distinct from that used for wet snow avalanches.

In this study, the meteorological and snow data were converted into daily averages and used as explanatory variables. The target variables were defined as dangerous days and safe days. Dangerous days were defined as days on which accidents occurred, whereas safe days were defined as the period from the beginning of the winter season to the day before an accident. Unlike pre-accident days, post-accident days cannot be classified as safe because houses with roof snow after an accident day may still be at risk of roof snow falling. However, they cannot be classified as dangerous days because there are no records of roof snow falling on these days. Therefore, post-accident days were excluded from the training data. A Random Forest model was constructed. Meteorological data used as input to SNOWPACK and SNOWPACK outputs were available at hourly resolution. Because the machine learning model used daily data, both were averaged to obtain daily values.

Because dry- and wet-type roof-snow-falling events occur under different snow conditions, separate Random Forest models were constructed, although the same explanatory variables were used.

The machine learning model was produced using Python’s scikit-learn library. Missing values were imputed using SimpleImputer and scalar values were standardized using StandardScaler. The RandomForestClassifier was used with the number of estimators (n_estimators) set to 100 to ensure that 100 “safe” or “dangerous” outputs were generated and random_state was set to a specific value to prevent different results from being generated each time.

3.1. Importance of the Features

The Random Forest model provides information on the importance of each explanatory variable. Table 1 lists the 10 most important variables for the wet- and dry-type cases. For the wet-type roof snow falling (left in Table 1), variables related to liquid water in the upper snowpack showed the highest importance, including the proportion of wet snow (wet ratio) in the top 15 cm (No.1) and the total water content (total LWC) in the top 15 cm (No.3). Variables associated with melting and temperature rise, such as the melt-freeze part of the internal energy change (No.2) and the internal energy change (No.4), also had high importance, which is consistent with the idea that rising temperatures and rainfall increase the risk of roof snow falling. Variables commonly used in wet-snow avalanche risk assessment—the stability index and snowpack runoff—were ranked Nos.9 and 10, respectively. Although these variables were within the top 10, their importance was not particularly high compared with the top-ranked variables. The top three variables had importance values above 0.03, reflecting the characteristic conditions under which wet-type roof snow falling occurs.

For dry-type roof snow falling (right in Table 1), only snow water equivalent (SWE) exceeded an importance value of 0.03, and many of the top-ranked variables had importance values between 0.02 and 0.03. Features related to the positions at the minimum for various types of stability indices (Nos.2, 6, and 7) and features related to snow amount (Nos.1, 4, and 8) showed high importance. These results may provide useful reference information for inferring the conditions under which dry-type roof snow falling is likely to occur, and for identifying factors that may increase its risk.

3.2. Accuracy Verification in the Fitting Phase

When constructing the Random Forest model, 100 decision trees were used to assess danger levels. The model output was obtained using the predict_proba function of the RandomForestClassifier, and the output value for the dangerous class was used as a quantified danger score ranging from 0 to 1. This score was used as a relative indicator of danger and was not interpreted as the calibrated probability of roof snow falling. Fig. 3 shows the SNOWPACK results for the selected cases together with the temporal changes in the danger score estimated by the Random Forest model.

In the case of roof snow falling at one accident site in Niigata Prefecture (wet-type; Fig. 3(a)), the danger level increased during snowmelt but did not exceed 0.2 before the accident. In contrast, it reached 0.5 on January 13, the day of the accident, correctly indicating that roof snow falling occurred on a day with elevated danger. For the dry-type case at one accident site in Hokkaido (Fig. 3(b)), the danger level increased substantially even though no major change in snow cover was evident on the day of the accident. Examination of the explanatory variables suggested that the daily average wind speed on the day of the accident was higher than that on the preceding days, reaching 7.8 m/s.

Daily changes in danger levels were obtained for all cases. To evaluate accuracy, an optimal threshold is required to binarize the danger level into “safe” and “dangerous.” Therefore, a threshold sweep curve was created by varying the threshold from 0 to 1 (Fig. 4). The gray curve represents the rate at which the actual safe days were correctly identified, which increased as the threshold increased (i.e., fewer false alarms). The black curve represents the rate at which the actual dangerous days were correctly identified, which increased as the threshold decreased (i.e., fewer misses).

For wet-type cases, the intersection of the two curves was adopted as the threshold balancing the two hit rates, resulting in an optimal threshold of 0.05. The threshold is considerably smaller than 0.5 because the number of safe days is much larger than the number of dangerous days. This class imbalance poses a challenge for model training and evaluation; however, the dataset is not sufficiently large to downsample the majority class by discarding many safe-day samples. Therefore, we proceeded with the analysis as is, and left improvements for future work (see Discussion). Using a threshold of 0.05, the results were 47 hits, 139 false alarms, 2 misses, and 3279 correct no-event classifications. This indicates that if approximately two to three false alarms are allowed per roof-snow-falling event, risk can be detected with a success rate of over 90%.

For dry-type cases, the optimal threshold was 0.03, yielding 78 hits, 312 false alarms, four misses, and 5059 correct no-event judgments. This indicates that, if approximately three to four false alarms are allowed per roof-snow-falling event, the danger of roof snow falling can be detected, with a success rate of more than 90%. Overall, the fitting phase achieved relatively good accuracy in hazard estimation. The evaluation-data performance of the proposed models is evaluated in the following section.

3.3. Exploratory Evaluation of Model Performance

Because the results in Section 3.2 were evaluated using the same data used for training (i.e., without a separate test set), they do not directly indicate practical usefulness in real-time applications. Therefore, we conducted an exploratory evaluation by using separate temporal data.

In this study, the intended use case is that a model trained on past roof-snow-falling data is used to estimate real-time roof-snow-falling hazard. To reflect this assumption, data up to FY2023 were used for training, and data from FY2024 were used for evaluation. Fig. 5 shows the threshold sweep curves for the test data for wet-type roof snow falling. At the selected operating threshold of 0.08, defined by the intersection of the two curves, the test results were 6 hits, 1 miss, 59 false alarms, and 371 correct no-event judgments. It should be noted that because this threshold was selected using the FY2024 evaluation data, the results should be regarded as a post-hoc exploratory assessment rather than a fully independent test of generalization performance.

As an example, Fig. 6 shows the temporal changes in the estimated roof-snow-falling hazard during a period of frequent snowfall in Aomori Prefecture on January 6, 2025. A roof-snow-falling accident occurred when the danger level reached 0.4. This correctly reproduced the increase in danger, because the danger level was lower before that date. As shown in Figs. 2(a) and 3(a), this was a typical wet-type roof-snow-falling case, in which the upper part of the snow cover metamorphosed into Melt Forms due to melting or rainfall when fresh snow was predominant. In such a typical wet-type case, the hazard increase can be estimated correctly even when using data not included in the training.

In contrast, for dry-type roof snow falling, the test results were three hits, three misses, 200 false alarms, and 146 correct no-event judgments, indicating poor model performance and suggesting that a different approach is required (no figure).

Based on these results, we will continue to improve the practical usefulness of wet-type roof-snow-falling estimation through additional case-by-case verifications. For dry-type roof snow falling, it is necessary to begin by examining the conditions associated with roof-snow-falling accidents, for example, by analyzing the importance of the explanatory variables. In practice, wet-type roof snow falling is often considered dangerous when the temperature rises or rainfall occurs; however, no clear indicator has been established for dry-type roof snow falling, and estimating the danger level remains challenging.

3.4. Construction of Roof-Snow-Falling Hazard Prediction System

Based on the threshold-sweep analysis of the machine learning model described in Section 3.3 (exploratory evaluation), we assigned colors to the danger level in accordance with ISO 22324, consistent with the color scheme used in YukioroSignal. The color categories are operational categories for displaying danger scores and should not be interpreted as probability-calibrated categories of roof-snow-falling occurrence. The color categories were defined as follows:

1. ・

   Green: 0 to 0.02, very low danger score for roof-snow-falling events.

2. ・

   Yellow-green: over 0.02 to 0.05, low danger score.

3. ・

   Yellow: over 0.05 to 0.10, moderate-low danger score.

4. ・

   Orange: over 0.10 to 0.40, moderate danger score.

5. ・

   Red: over 0.40, high danger score.

Using a machine learning model trained on all cases available through the FY2023 winter and the color settings above, we applied the model to the FY2024 winter YukioroSignal data, which cover most of the heavy snowfall areas (Fig. 1), and calculated the spatial distribution of the roof-snow-falling hazard at each snow-depth observation point. As an example, the hazard distribution on January 7, 2025, is shown in Fig. 7. Roof-snow-falling accidents occurred in Hirosaki City and Fujisaki Town in Aomori Prefecture on that day, and in Aomori City on the following day (January 8); most areas in Aomori Prefecture were classified as yellow or orange. Although there is still room for improvement, the system reproduced the increase in roof-snow-falling hazard in certain cases, suggesting the potential for practical application with further refinement.

4.1. Practicality and Limitations at the Present Stage

To evaluate the reproducibility of increases in risk of roof snow falling for practical use, we conducted an analysis using a test data split. The results showed some accuracy for the wet type, but poor accuracy for the dry type. In the full-winter analysis, many snow-covered areas tended to be assigned high danger levels for the wet type during late winter, that is, the snowmelt period. However, in practice, the risk is not necessarily high because snow removal and prior natural roof snow falling (not associated with accidents) can reduce the remaining roof snow; these effects are not captured in the current approach. In addition, because the study dataset did not include post-event periods after the accidental roof snow falling, the risk during the snowmelt season was not fully examined, and high-risk levels persisted because of the sustained LWC in the snowpack. Therefore, the proposed system may be useful for estimating wet-type roof-snow-falling hazards during severe winters.

While roof shape, slope, roofing materials, and snow guards are the major factors influencing roof snow falling, the inability to incorporate detailed conditions specific to each roof leads to limitations in accuracy. This model cannot be used to directly determine whether roof snow will fall from a specific house; rather, it is designed to estimate the meteorological and snow conditions that cause roof snow to fall on a regional or wide-area basis. Even if the model indicates a high risk, accidents are less likely to occur if a specific house is equipped with snow guards, has already had snow removed from its roof, or has already experienced roof snow falling. These circumstances can lead to false positives. The validation results indicate that determining an acceptable number of false positives per case for a given accuracy rate is necessary before this information is used operationally.

If it becomes possible to incorporate information on roof-snow-falling observations from specific houses, obtained using webcams and similar devices, along with roof slopes and shapes derived from 3D urban models, aerial LiDAR surveys, and municipal data, these data could enable the assessment of risk levels for individual houses in the future.

With regard to the “dry type,” verification of its generalizability has not yet reached a practical stage. According to the classification defined in Section 2.2, it is believed that even among cases classified as “dry type,” there are cases where roof snow fell due to the underside of the snowpack becoming wet as a result of heat transfer from the roof. Additionally, some cases classified as “dry type” may be associated with snow-cornice collapse, temperature inhomogeneity on roof surfaces, or roof snow falling as a result of snow removal. The poor model performance of the “dry type” is thought to stem from the difficulty of making estimates using meteorological data and the structure of natural snow accumulation as features, due to the involvement of individual house-specific conditions. More sophisticated approaches are required to achieve practical applications, such as analyzing observational data on the presence or absence of roof snow falling on houses with known structures in cold regions. Alternatively, it would be useful to focus the analysis on cases of snow falling from roofs classified as “dry type” that occurred under low-temperature conditions where it is unlikely that the interface had become wet.

Despite these limitations, the feature importance results of the dry-type model may provide useful clues regarding the conditions associated with dry-type accidents. The highly ranked variables were mainly related to snow amount, such as SWE and modeled snow depth, and snow instability, such as the values and positions of the stability indices. Snow-amount-related variables may be important because a larger amount of roof snow can increase the impact force and burial potential when a fall occurs, making accidents more likely. Instability-related variables may reflect unstable snow conditions caused by recent snowfall or weak layers, which may also favor the local failure of roof snow or the collapse of overhanging snow cornices. However, these interpretations should be regarded as hypotheses because the present model does not include roof-specific factors, such as roof shape, slope, roofing material, or snow guards.

Future improvements include refining the machine learning approach and increasing the amount of training data, as discussed in the next section; the incorporation of roof-specific information, as noted above, remains an important long-term issue.

4.2. Challenges in Machine Learning

In this study, only the Random Forest approach was evaluated. Random Forests were adopted because they have been used in related studies on wet-snow avalanche forecasting ²⁵ and often perform well with limited training data and mixed, potentially nonlinear explanatory variables, while also providing interpretable feature importance. Model accuracy may be improved by testing and comparing multiple machine learning methods (e.g., \(k\)-nearest neighbors, Support Vector Machines, XGBoost), as well as by further refining the Random Forest model. In addition, the model accuracy could be improved through hyperparameter optimization, such as by tuning the number and depth of individual decision trees, the number of features considered at each split, and related settings. However, because the method is still under development and the amount of training data remains limited, we consider it premature to pursue detailed hyperparameter optimization or comparisons with other machine learning models at this stage. These analyses will be addressed in future studies as the dataset and methods are further developed.

In this study, the analysis was conducted primarily from the perspective of snow and ice research. Further development of this method, including collaborations with researchers specializing in machine learning and artificial intelligence, is important for improving its applicability.

4.3. Addressing Insufficient Training Data

Although a certain level of accuracy was achieved in the fitting phase of this study, there is room for improvement in the performance on the evaluation data, indicating that further accuracy improvements are necessary.

A major factor limiting the accuracy is the lack of training data. Therefore, a primary step toward improving the accuracy in future work is to increase the amount of training data. Possible approaches include collecting additional roof-snow-falling cases, incorporating more historical roof-snow-falling accident data, and increasing the effective size of the training dataset through data augmentation.

In this study, incidents were collected from newspaper articles and used as the training data. The dataset can be expanded by incorporating additional cases beyond newspaper-based records. If possible, it would be desirable to increase the dataset using non-public data related to roof snow falling; however, obtaining such data has practical hurdles, including the need for cooperation from local governments.

Under the current method, days with no accident records are classified as “safe,” even if unrecorded roof-snow-falling events occurred. This limits the accuracy of the risk estimation. An approach to avoid this is to identify roof-snow-falling event dates from webcam images and use the resulting information for analysis. Although events detected in this manner are not necessarily associated with accidents, this approach provides a direct means of determining whether an event has occurred. In addition, developing methods to detect the presence or absence of events from roof conditions using satellite or other remote-sensing data would also help increase data availability.

The Snow and Ice Disaster Database has compiled records of roof-snow-falling incidents since FY2000, which could be used to increase the amount of training data. In this case, retrospective (hindcast) calculations are required because these records predate the start of the YukioroSignal. Although historical AMeDAS data are readily available, snow-depth observations from other agencies are not available online. In addition, snow depth data from Snow Distribution Monitoring System ²⁶ have only been available since 2014, which limits the application of the same calculation method. Alternative approaches include estimating snow depth without direct observations by applying elevation corrections to AMeDAS sites or using reanalysis data, such as JRA-55. However, because these approaches involve input data generation that differs from YukioroSignal, it is necessary to consider appropriate adjustments that account for these input differences.

When labeling days as safe or dangerous in the present approach, the number of safe days was much larger than the number of dangerous days, resulting in a class imbalance. As the amount of training data is limited, increasing the number of dangerous samples through data expansion may be preferable to downsampling the majority safe-day class to address this imbalance.

To increase the dataset, we recalculated SNOWPACK for the training cases using precipitation as input instead of snow depth (with snowfall estimated via rain/snow discrimination) and applied temperature perturbations of 1 °C and −1 °C. Including the original data increased the number of samples by a factor of four. As a result, the operating threshold (the intersection point of the threshold-sweep curves) increased, and the fitting-phase accuracy improved; however, the performance on the evaluation data decreased, indicating a stronger tendency toward overfitting. This may be because the augmented samples were highly similar to each other, and improved strategies for data augmentation are required. In the future, we plan to test other methods to address this imbalance, such as the synthetic minority oversampling technique (SMOTE). Furthermore, if the method discussed in this section is successfully implemented and a large amount of training data can be obtained, we will consider methods for addressing the imbalance using techniques such as random undersampling.