single-jc.php

JACIII Vol.30 No.6 pp. 1876-1883
(2026)

Research Paper:

Ship Propulsion Performance Estimation from Limited Hull Form Images Using Self-Supervised Pretraining with Decoder-Side Masking

Taian Kigawa*, Tomoki Hamagami*,†, Ichiro Amaya** ORCID Icon, and Naoya Okamoto**

*Yokohama National University
79-1 Tokiwadai, Hodogaya-ku, Yokohama, Kanagawa 240-8501, Japan

†Corresponding author

**Japan Marine United Corporation
1-3 Kumozu-kokan-cho, Tsu, Mie 514-0398, Japan

Received:
November 28, 2025
Accepted:
June 26, 2026
Published:
November 20, 2026
Keywords:
ship performance estimation, self-supervised learning, masked image modeling, limited datasets
Abstract

Efficient estimation of ship propulsion performance from hull form geometry is essential for data-driven ship design. However, limited hull data due to confidentiality and manufacturing constraints hinder stable training of deep learning models. To address this challenge, we propose a self-supervised pretraining framework, salience-based adaptive decoder-side masking (SBADM), for estimating performance from a limited number of hull-form images. SBADM integrates two techniques: decoder-side masking, which preserves complete geometric information during encoding, and salience-based adaptive masking (SBAM), which dynamically adjusts masking ratios and regions based on token salience. These mechanisms enable effective pretraining even with small sample sizes. Experiments using 299 hull geometries provided by Japan Marine United Corporation show that SBADM improves estimation accuracy, particularly for the form factor (1+K) and the highly sensitive thrust deduction factor (1-t), confirming its effectiveness in enhancing feature representation and prediction robustness.

Attention focus on ship stern by SBADM

Attention focus on ship stern by SBADM

Cite this article as:
T. Kigawa, T. Hamagami, I. Amaya, and N. Okamoto, “Ship Propulsion Performance Estimation from Limited Hull Form Images Using Self-Supervised Pretraining with Decoder-Side Masking,” J. Adv. Comput. Intell. Intell. Inform., Vol.30 No.6, pp. 1876-1883, 2026.
Data files:

1. Introduction

In the maritime industry, improving ship propulsion performance is of paramount importance, given that most international trade relies on marine transportation. Improving the efficiency of ship propulsion performance estimation directly contributes to reducing the time and computational cost required for ship design, thereby facilitating faster and more sustainable development. Traditionally, the evaluation of ship hull performance has relied on computational fluid dynamics (CFD), which provides accurate flow predictions but requires substantial computational resources—often several hours per hull design. Consequently, there is an increasing demand for data-driven methods capable of estimating propulsion performance rapidly and accurately without reliance on CFD simulations.

Recent studies have increasingly explored machine learning approaches to predict ship propulsion performance from numerical parameters or geometric data, highlighting the promise of deep learning as a rapid alternative to traditional CFD-based analyses 1,2,3,4. However, a critical research gap remains in the high data-dependency of these conventional deep learning models. In practical ship design, the availability of high-quality ship hull geometry data is extremely limited due to strict industrial confidentiality and manufacturing constraints. Consequently, deep learning models trained from scratch under such small-sample conditions suffer severely from training instability, overfitting, and poor generalization on unseen hull configurations.

To address this challenge, this study focuses on self-supervised pretraining methods that enable robust feature extraction from limited data. Specifically, we propose a decoder-side masking (DSM) framework, a novel variant of masked image modeling (MIM) designed for limited-data learning. Unlike conventional MIM, which masks input tokens before the encoder, DSM applies masking only at the decoder stage. This design preserves complete input information during encoding, stabilizing feature representation and mitigating the loss of critical geometric details. Furthermore, we extend DSM with a salience-based adaptive masking (SBAM) mechanism, resulting in the proposed salience-based adaptive decoder-side masking (SBADM) method. SBADM dynamically adjusts both the masking ratio and the masked regions according to image salience, allowing the model to emphasize visually significant hull regions while suppressing redundant information. This adaptive mechanism enhances the learning of spatially meaningful representations even under limited-data conditions.

The dataset used in this study consists of 299 ship hulls provided by Japan Marine United Corporation (JMU), each represented as a depth map of size \((1,64,512)\) derived from polygonal mesh models. Three performance indicators are considered: the form factor \((1+K)\), thrust deduction \((1-t)\), and wake fraction \((1-w)\). Although these indicators are partially interdependent, each is predicted individually in this study to maximize estimation accuracy.

The main contributions of this paper are as follows:

  1. 1.

    We demonstrate the effectiveness of self-supervised pretraining using vision transformer (ViT) architectures in extracting generalizable features from limited ship hull image data.

  2. 2.

    We propose a DSM strategy for MIM that improves stability and feature representation under limited-data conditions.

  3. 3.

    We introduce SBADM, which dynamically adjusts mask ratios and regions based on image salience, further enhancing performance in downstream regression tasks.

The remainder of this paper is organized as follows: Section 2 reviews related work on machine-learning-based ship propulsion performance estimation and self-supervised image modeling; Section 3 describes the proposed method, including DSM and SBADM; Section 4 presents the experimental setup, results, and discussion; and Section 5 concludes the paper with future perspectives.

3. Proposed Method

3.1. Problem Definition

The objective of this study is to estimate propulsion performance indices from ship hull geometry images under small-sample conditions. The input to the model is a depth-map image representing the hull surface, and the output consists of three propulsion-related indices: form factor \((1+K)\), thrust deduction \((1-t)\), and wake fraction \((1-w)\). Since the number of available hull samples is limited, an effective feature extraction method that can generalize from a small dataset is required. To achieve this, we adopt a self-supervised pretraining framework based on MIM and introduce an enhanced variant called DSM.

Conventional MIM methods, such as MAE 7 and SimMIM 8, apply masking to the input tokens before feeding them into the encoder. However, this approach discards a large portion of spatial information at the encoder stage, which is particularly problematic when the dataset is small. As a result, the encoder cannot stably learn global shape representations. The proposed DSM addresses this issue by removing masking from the encoder input and instead applying it only before the decoder, thereby preserving full input information during encoding and stabilizing feature extraction.

Furthermore, to improve the adaptability of the masking strategy, we integrate DSM with the SBAM framework 10, resulting in the proposed SBADM. SBADM dynamically adjusts both the masking ratio and masked token locations based on visual salience, enabling the model to emphasize geometrically significant regions of the hull while reducing redundancy.

3.2. Overview of the Proposed SBADM Framework

figure

Fig. 1. Overview of the proposed method (SBADM).

Figure 1 illustrates the overall structure of the proposed method. The input image is divided into non-overlapping patches and embedded into tokens \(\mathbf{X} \in \mathbb{R}^{B \times L \times D}\), where \(B\) is the batch size, \(L\) is the number of patches, and \(D\) is the embedding dimension. These tokens are processed by the encoder \(f_{\mathrm{enc}}\), and the resulting feature tokens \(\mathbf{Z}\) are used to compute token salience and determine adaptive masking parameters.

The overall process is formulated as follows:

\begin{align} \mathbf{Z} &= f_{\mathrm{enc}}\left(\mathbf{X}\right), \label{eq:z} \tag{1} \end{align}
\begin{align} \hat{\mathbf{A}} &= \rho\left(\mathbf{Z}\mathbf{Z}^{\top}\right), \label{eq:affinity} \tag{2} \end{align}
\begin{align} S_i &= \mathrm{MinMax}\!\left(\sum_{j=1}^{L} \hat{A}_{j,i}\right), \label{eq:salience} \tag{3} \end{align}
where \(\rho(\cdot)\) denotes SoftMax normalization applied row-wise, and \(\mathrm{MinMax}(\cdot)\) denotes element-wise min–max normalization applied across all tokens. \(\hat{\mathbf{A}} \in \mathbb{R}^{B\times L\times L}\) represents the affinity matrix expressing pairwise similarity among tokens, and \(S_i\) measures how much token \(i\) is attended to by others.

To prevent overly deterministic masking, a random perturbation matrix \(\boldsymbol{\eta}\) sampled from a uniform distribution \(U([0, 0.5))^{B\times L}\) is added to the salience values to obtain the adjusted salience:

\begin{equation} \tilde{\mathbf{S}} = \mathbf{S} + \boldsymbol{\eta}. \label{eq:s_tilde} \tag{4} \end{equation}

Next, the adaptive masking ratio \(R_{\mathrm{dyna}}\) is determined according to the distribution of salience values:

\begin{equation} R_{\mathrm{dyna}} = r - \Delta r + 2\Delta r \times \mathrm{mean}\left(1_{\mathbf{S} > \delta}\right), \label{eq:amr} \tag{5} \end{equation}
where \(r\) is the baseline masking ratio, \(\Delta r\) is the fluctuation range, and \(\delta\) is the salience threshold. The indicator function \(1_{\mathbf{S}>\delta}\) equals one for tokens whose salience exceeds \(\delta\). This formulation allows the model to dynamically adjust the masking ratio for each image based on its internal salience distribution.

3.3. Decoder-Side Masking and Reconstruction

Based on the computed dynamic masking ratio, the masking function \(\Phi_{\mathrm{mask}}\) is applied to generate the masked token sequence \(\mathbf{Z}_{\mathrm{masked}}\):

\begin{align} \mathbf{Z}_{\mathrm{masked}} &= \Phi_{\mathrm{mask}}\left(\mathbf{Z}, R_{\mathrm{dyna}}\right), \label{eq:masking} \tag{6} \end{align}
\begin{align} \hat{\mathbf{X}} &= f_{\mathrm{dec}}\left(\mathbf{Z}_{\mathrm{masked}}\right), \label{eq:decoder} \tag{7} \end{align}
where \(f_{\mathrm{dec}}\) denotes the decoder network. The decoder reconstructs the input image using the partially masked tokens. Unlike conventional MIM, in DSM and SBADM the encoder receives all tokens, while only the decoder processes masked inputs, ensuring that the learned representations are stable and information-rich.

The reconstruction loss is computed in the same manner as standard MIM:

\begin{equation} L_{\mathrm{SBADM}} = \frac{1}{\displaystyle\sum_{i,j} M_{i,j}} \sum_{i,j} M_{i,j} \left\lVert \hat{\mathbf{X}}_{i,j} - \bar{\mathbf{X}}_{i,j} \right\rVert_2^2, \label{eq:loss} \tag{8} \end{equation}
where \(\bar{\mathbf{X}}\) is the normalized version of the input tokens, standardized by mean \(\mu(\mathbf{X})\) and standard deviation \(\sigma(\mathbf{X})\).

3.4. Expected Advantages

The proposed DSM and SBADM frameworks offer several advantages for small-sample learning.

First, applying masking at the decoder side reduces information loss during encoding, enabling the model to extract more stable and generalizable representations.

Second, the adaptive salience-driven masking emphasizes perceptually and geometrically important hull regions, improving the efficiency of feature learning.

Finally, the dynamic adjustment of mask ratio according to image content enhances learning robustness by automatically regulating reconstruction difficulty throughout pretraining.

Through these mechanisms, SBADM achieves both stable pretraining and superior downstream regression accuracy, even when trained on a limited dataset of ship hull images.

4. Experiments

4.1. Experimental Setup

4.1.1. Model Configuration

The proposed method was implemented based on the ViT architecture and trained under the MIM framework. The encoder consisted of four transformer blocks with an embedding dimension of 512, and the decoder comprised two transformer blocks with a dimension of 256. To verify the effectiveness of the proposed method, we compared DSM and SBADM with baseline approaches, including standard MIM, SBAM pretraining, and a baseline ViT trained from scratch.

4.1.2. Dataset

The dataset used in this study was provided by JMU and consists of 299 ship hulls. Each hull was originally modeled as a polygonal mesh and then converted into a depth map image with a resolution of \((1, 64, 512)\), where the single channel represents the depth value of the hull surface. While deep learning architectures like ViT typically require tens of thousands of samples for stable supervised training, acquiring such large-scale hull data is practically impossible due to the extremely high computational cost of CFD simulations and strict industrial confidentiality. Therefore, this dataset of 299 samples represents a highly “limited” data scenario typical of real-world maritime engineering.

Three propulsion performance indices were used as regression targets: the form factor \((1+K)\), thrust deduction \((1-t)\), and wake fraction \((1-w)\). These indices are commonly employed in ship hydrodynamics and were predicted independently in this work to maximize precision. The dataset was divided using 5-fold cross-validation to ensure statistical robustness and to rigorously evaluate the generalization ability of the models on unseen configurations, mitigating the bias of using data from a single industrial source. Each fold contained distinct training and validation sets, and all models were evaluated independently on each fold.

4.1.3. Evaluation Metrics

The model performance was quantitatively assessed using the root mean square percentage error (RMSPE), defined as:

\begin{equation} \mathrm{RMSPE}~\mbox{[}\%\mbox{]} = \sqrt{\frac{1}{n}\sum_{i=1}^{n}\left(\frac{y_i - \hat{y}_i}{y_i}\right)^2}\times 100, \label{eq:rmspe} \tag{9} \end{equation}
where \(n\) denotes the number of samples, \(y_i\) the ground truth, and \(\hat{y}_i\) the predicted value. An RMSPE below 1% is considered satisfactory for practical industrial prediction of propulsion indices.

4.1.4. Implementation Details

All experiments were conducted using PyTorch on an NVIDIA GPU environment. Both pretraining and fine-tuning were performed for 2000 epochs with a batch size of 16. The Adam optimizer was used with an initial learning rate of \(1.0\times10^{-4}\). The input images were divided into non-overlapping patches of size \((16, 32)\). For SBADM, the baseline masking ratio \(r\) and variation range \(\Delta r\) were initialized at 0.5 and 0.25, respectively, and the salience threshold \(\delta\) was set to 0.7.

4.2. Results and Discussion

4.2.1. Performance Comparison

Table 1 summarizes the RMSPE results (mean \(\pm\) standard deviation) for each performance index across different pretraining strategies.

When masking was applied on the encoder side as in conventional MIM, the estimation accuracy degraded as the masking ratio increased, indicating that excessive information loss hindered feature learning. In contrast, applying masking on the decoder side consistently improved performance, with the best results obtained with a masking ratio of 75%. This confirms that preserving complete encoder input helps extract stable and generalizable representations even under small-sample conditions.

While DSM with a 75% mask rate achieves the lowest RMSPE for the wake fraction \((1-w)\), the proposed SBADM achieved the best accuracy for the form factor \((1+K)\) and the thrust deduction factor \((1-t)\). For \((1-w)\), SBADM still yields a significant error reduction compared to the scratch-trained model and all encoder-side masking baselines. Notably, unlike plain DSM which requires manual, task-specific grid searches to find the optimal masking rate (e.g., 75%), SBADM automatically and adaptively optimizes the masking configuration per image. This comprehensive improvement can be attributed to two key factors: (1) the stabilization of encoder feature extraction by DSM, and (2) the adaptive emphasis on geometrically and physically salient regions through the salience weighting of SBAM.

Notably, for the thrust deduction factor \((1-t)\), SBADM (0.603\(\pm\)0.104) successfully outperformed both the standard ViT trained from scratch (0.648\(\pm\)0.154) and plain DSM (0.660\(\pm\)0.105). This demonstrates that SBADM effectively captures the highly sensitive localized geometric features required for precise thrust deduction estimation, overcoming the limitations of conventional approaches. Overall, SBADM demonstrates highly competitive performance and superior robustness across the key propulsion indices compared with existing pretraining strategies, offering a more practical framework that eliminates manual hyperparameter tuning.

Table 1. RMSPE (mean \(\pm\) standard deviation) of each propulsion performance index for different training methods.
Method Masking side Mask rate [%] \(1+K\) \(1-t\) \(1-w\)
Scratch N/A N/A 1.063\(\pm\)0.255 0.648\(\pm\)0.154 1.808\(\pm\)0.155
MIM Encoder 25 1.146\(\pm\)0.201 0.738\(\pm\)0.110 1.919\(\pm\)0.196
MIM Encoder 50 1.545\(\pm\)0.176 0.823\(\pm\)0.082 2.479\(\pm\)0.233
MIM Encoder 75 2.356\(\pm\)0.316 1.004\(\pm\)0.079 3.592\(\pm\)0.578
DSM Decoder 25 1.025\(\pm\)0.182 0.801\(\pm\)0.120 2.117\(\pm\)0.147
DSM Decoder 50 0.852\(\pm\)0.225 0.657\(\pm\)0.130 1.645\(\pm\)0.222
DSM Decoder 75 0.851\(\pm\)0.165 0.660\(\pm\)0.105 1.485\(\pm\)0.142
SBAM Encoder N/A 1.331\(\pm\)0.246 0.676\(\pm\)0.104 2.229\(\pm\)0.277
SBADM Decoder N/A 0.820\(\pm\)0.222 0.603\(\pm\)0.104 1.620\(\pm\)0.206
figure

Fig. 2. Visualization of attention maps for different pretraining methods and propulsion indices.

4.2.2. Visualization of Attention Maps

Figure 2 visualizes the attention maps of models trained under different strategies, comparing both the pretraining and fine-tuning phases.

As observed in the pretraining phase, the conventional encoder-side masking approach (SBAM) exhibits dispersed and irregular attention. This phenomenon, often referred to as “attention collapse,” occurs because excessive information loss at the encoder stage prevents the model from learning coherent global structures. Consequently, this scattered attention propagates into the fine-tuning phase, which is particularly evident in the highly irregular and fragmented map for the thrust deduction factor \((1-t)\).

In contrast, the decoder-side masking approaches (DSM and SBADM) preserve complete geometric information during encoding, resulting in stable and highly concentrated attention patterns from the pretraining phase onward. Specifically, SBADM demonstrates the most intense and localized focus on the stern region (corresponding to the leftmost 20% area of the hull images) across all three propulsion indices during fine-tuning.

From a quantitative perspective, while the attention weights of scratch and SBAM are widely scattered across the entire hull length (yielding high spatial variance), SBADM exhibits a stark numerical contrast, focusing nearly all high-magnitude attention coefficients strictly onto the localized stern geometry. Because this specific region critically dictates the complex wake fields and propeller-hull interactions, this precise spatial focus physically explains why SBADM achieves superior quantitative performance—especially the marked improvement in \((1-t)\)—over the scratch-trained baseline and standard MIM methods. This visual evidence confirms that the proposed method effectively guides the network to consistently focus on physically meaningful hydrodynamic regions.

figure

Fig. 3. Comparison of mask probability distributions (left) and mask ratio histograms (right) between the baseline SBADM (patch size \(16\times32\)) and SBADM with smaller patches (patch size \(8\times16\)).

4.3. Sensitivity Analysis and Model Construction

To evaluate the robustness of SBADM, we conducted a sensitivity analysis on the hyper-parameters \(r\), \(\Delta r\), and \(\delta\). As shown in the mask ratio histograms in Fig. 3 (right), the dynamically adjusted masking ratio does not fluctuate randomly but consistently converges to a highly sharp, single peak tightly concentrated around 70% during pretraining. The estimation accuracy remained robust within a reasonable range of these parameters (e.g., \(r \in [0.4, 0.6]\)).

We also investigated the impact of patch size (model construction) by comparing the baseline \((16, 32)\) with a finer resolution of \((8, 16)\), denoted as \(\mathrm{SBADM}_{\mathrm{smallpatch}}\). The finer patches resulted in an increased RMSPE (e.g., from 0.820% to 0.973% for \(1+K\)). This degradation can be quantitatively and visually interpreted by Fig. 3. While the baseline SBADM maintains a well-structured masking pattern concentrated in the central hull region with an extremely narrow mask ratio distribution (peaking sharply at 0.7), the distribution for the smaller patch size becomes overly dispersed. Quantitatively, the histogram for \(\mathrm{SBADM}_{\mathrm{smallpatch}}\) exhibits a significantly wider numerical variance (spreading from 0.55 to 0.85), which indicates substantial learning instability during the self-supervised phase.

This evidence strongly supports the occurrence of “feature fragmentation,” where the increased number of tokens makes it difficult for the self-supervised task to learn global geometric relationships within a limited dataset. Thus, the patch size of \((16, 32)\) was found to be optimal for balancing local detail and global structural consistency.

4.4. Summary of Experimental Findings

The experimental results demonstrate that:

  1. 1.

    DSM effectively stabilizes feature extraction by preventing information loss during the encoder phase, which is critical for learning from small-sample datasets.

  2. 2.

    SBADM significantly enhances estimation accuracy—achieving the lowest RMSPE for the form factor \((1+K)\) and the highly sensitive thrust deduction factor \((1-t)\)—by dynamically focusing on geometrically and physically significant hull regions, successfully outperforming conventional approaches.

  3. 3.

    The proposed framework successfully avoids the “attention collapse” observed in conventional MIM during pretraining, maintaining a stable and numerically concentrated spatial focus on the stern region.

  4. 4.

    The model is robust against hyper-parameter variations, and its dynamic mask ratio consistently converges to a narrow distribution, optimally balancing local detail and global structural consistency while avoiding the “feature fragmentation” that occurs with overly fine patch sizes.

Overall, these findings confirm that SBADM provides a highly suitable, robust, and physically interpretable self-supervised pretraining strategy for industrial ship design datasets with limited sample sizes.

5. Conclusion

This study proposed a self-supervised pretraining framework, termed SBADM, for estimating ship propulsion performance from a limited number of hull form images. The proposed framework integrates two key concepts: (1) DSM, which applies masking only at the decoder stage to preserve complete geometric information during encoding, and (2) SBAM, which dynamically adjusts mask ratios and positions according to token salience. By combining these mechanisms, SBADM achieves stable pretraining and improved feature extraction even under small-sample conditions.

Experimental results demonstrated that the proposed method significantly improved estimation accuracy and training stability compared to conventional encoder-side MIM approaches and scratch-trained baselines. Notably, SBADM achieved the lowest RMSPE for the form factor \((1+K)\) and the highly sensitive thrust deduction factor \((1-t)\), while maintaining highly competitive and robust generalization performance for the wake fraction \((1-w)\) without requiring task-specific manual tuning of masking rates. Furthermore, visualization of attention maps confirmed that SBADM successfully prevents “attention collapse” during pretraining, maintaining a precise spatial focus on the stern region critical for hydrodynamic performance. Extensive sensitivity analyses also verified the model’s robustness to hyper-parameter variations and demonstrated that the proposed optimal patch size effectively prevents “feature fragmentation,” ensuring stable learning from limited data. Future work will extend this study in several directions. First, a multi-task learning framework will be developed to simultaneously estimate multiple propulsion indices within a unified model. Second, data augmentation and generative modeling techniques—such as diffusion models or VQ-VAE-based synthesis—will be investigated to further enhance pretraining robustness. Finally, the proposed method will be validated on higher-resolution datasets and diverse hull configurations to evaluate its scalability and general applicability in real-world ship design.

Overall, the findings confirm that SBADM provides an effective and physically interpretable SSL strategy for small-scale hull-form datasets and offers a promising direction toward accelerating data-driven ship design and performance optimization.

References
  1. [1] J. M. Walker, A. Coraddu, and L. Oneto, “Data-driven models for yacht hull resistance optimization: Exploring geometric parameters beyond the boundaries of the Delft Systematic Yacht Hull Series,” IEEE Access, Vol.12, pp. 76102-76120, 2024. https://doi.org/10.1109/ACCESS.2024.3404495
  2. [2] Y. Ao, Y. Li, J. Gong, and S. Li, “An artificial intelligence-aided design (AIAD) of ship hull structures,” J. of Ocean Engineering and Science, Vol.8, No.1, pp. 15-32, 2023. https://doi.org/10.1016/j.joes.2021.11.003
  3. [3] Y.-C. Kim, K.-S. Kim, S. Yeon, Y.-Y. Lee, G.-D. Kim, and M. Kim, “Power prediction method for ships using data regression models,” J. of Marine Science and Engineering, Vol.11, No.10, Article No.1961, 2023. https://doi.org/10.3390/jmse11101961
  4. [4] L. T. Trinh, T. Hamagami, and N. Okamoto, “3D ship hull design direct optimization using generative adversarial network,” J. Adv. Comput. Intell. Intell. Inform., Vol.28, No.3, pp. 693-703, 2024. https://doi.org/10.20965/jaciii.2024.p0693
  5. [5] T. Chen, S. Kornblith, M. Norouzi, and G. Hinton, “A simple framework for contrastive learning of visual representations,” arXiv:2002.05709, 2020. https://arxiv.org/abs/2002.05709
  6. [6] K. He, H. Fan, Y. Wu, S. Xie, and R. Girshick, “Momentum contrast for unsupervised visual representation learning,” arXiv:1911.05722, 2020. https://arxiv.org/abs/1911.05722
  7. [7] K. He, X. Chen, S. Xie, Y. Li, P. Dollár, and R. Girshick, “Masked autoencoders are scalable vision learners,” arXiv:2111.06377, 2021. https://arxiv.org/abs/2111.06377
  8. [8] Z. Xie, Z. Zhang, Y. Cao, Y. Lin, J. Bao, Z. Yao, Q. Dai, and H. Hu, “SimMIM: A simple framework for masked image modeling,” arXiv:2111.09886, 2022. https://arxiv.org/abs/2111.09886
  9. [9] A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, and N. Houlsby, “An image is worth 16×16 words: Transformers for image recognition at scale,” arXiv:2010.11929, 2021. https://arxiv.org/abs/2010.11929
  10. [10] H. Choi, H. Park, K. M. Yi, S. Cha, and D. Min, “Salience-based adaptive masking: Revisiting token dynamics for enhanced pre-training,” arXiv:2404.08327, 2024. https://arxiv.org/abs/2404.08327

*This site is desgined based on HTML5 and CSS3 for modern browsers, e.g. Chrome, Firefox, Safari, Edge, Opera.

Last updated on Nov. 20, 2026