A. Studies on the Semantic Gap in U-Net

Although early research did not explicitly identify the semantic gap in U-Nets, these studies acknowledged the significant role of efficient skip connection schemes in improving U-Net’s segmentation precision. For instance, Attention U-Net [20] employs attention gates instead of the direct skip connection (DSC) (as shown in Fig. 1(b)) to guide feature fusion between the decoder and encoder, effectively highlighting relevant regions while suppressing unimportant background areas. UNet++ [21] introduces a dense connection scheme to replace the DSC, resulting in superior segmentation performance compared to U-Net. UNet3+ [22] designs a full-scale connection scheme to supplant the DSC, achieving advanced performance across multiple databases. These methods enhance segmentation precision by improving the skip connection scheme, yet they do not explicitly address the underlying issue of the semantic gap.

Fig. 1.

Comparison of skip connection schemes between the proposed USCT-UNet and other models.

Show All

It was not until 2020 that MultiResUNet [25] suggested the potential existence of a semantic gap between corresponding layers of the encoder and decoder, based on observations of U-Net’s segmentation results across five public databases. To address this semantic gap, they designed the residual path (Res Path) with convolutional blocks instead of DSC, as illustrated in Fig. 1(c). These additional nonlinear operations aimed to mitigate the semantic gap. Subsequently, in 2022, UCTransNet [26] confirmed the existence of this semantic gap through systematic analysis and noted that the DSC scheme could impair model performance. They proposed using the channel Transformer (CTrans) as a substitute for the DSC to further reduce the semantic gap, as shown in Fig. 1(d).

In 2023, the literature [35] employed a Transformer-based cross-layer feature enhancement module that fuses feature sets from neighboring encoder layers and integrates them with decoder feature sets. This approach enhances low-layer features through cross-layer feature learning, effectively mitigating semantic gaps and improving medical image segmentation performance. In 2024, the literature [36] introduced a composite attention module combining channel and spatial attention, featuring a three-branch structure with double squeeze-and-excitation blocks, convolutional blocks, and batch normalization. This module replaces the DSC and reduces the semantic gap across different layers between the encoder and decoder.

Although these studies have acknowledged the existence of the semantic gap between the decoder and encoder, they have not provided a comprehensive quantitative measurement or detailed analysis of it. Furthermore, they continue to use a uniform skip connection scheme, suggesting that there is still room for improvement in segmentation performance. To address this, we developed the U-shaped skip connection (USC) solution, which allocates a variable number of MCFT blocks based on the semantic gap laws at different layers between the encoder and decoder, as illustrated in Fig. 1(a).

B. Transformer-Based Segmentation Methods

The Transformer is a neural network architecture that leverages a self-attention mechanism and exhibits exceptional global modeling capabilities [14]. Recently, the Vision Transformer (ViT) [37], which has demonstrated impressive results in image recognition tasks, has prompted research into replacing convolutional neural networks (CNNs) with ViT for certain medical image segmentation tasks [15], [16]. For instance, TransUnet [17] integrates ViT to construct the encoder for U-Net, while Swin-Unet [18] employs the Swin Transformer (SwinT) [38] to develop a Transformer-based U-Net. Similarly, DS-TransUNet [19] introduces a dual-scale encoder to capture both coarse- and fine-grained image features. It is worth noting that all the aforementioned methods retain the DSC component.

In a recent study [39], integrating SwinT into U-Net, along with spatial interaction, feature compression, and relationship aggregation blocks, enhanced the representation of irregularly shaped tumors. Other research [40] utilized SwinT as an encoder to extract image features and designed a cascading upsampling block as a decoder to optimize segmentation results. Additionally, some studies [41] proposed a hybrid network combining CNNs as the encoder and SwinT as the decoder for medical image segmentation, achieving remarkable performance. In another study [35], researchers used SwinT for the encoder and CNNs for the decoder, introducing a cross-layer feature enhancement module and a spatial channel squeeze-excitation module to enhance feature learning across different layers. Despite these advancements, these approaches primarily focus on enhancing the encoder and decoder of U-Net without effectively addressing the semantic gap issue by improving the skip connetion.

A. Databases

In this section, we quantitatively analyze the semantic gap between the layers of the encoder and decoder, assessing their impact on U-Net segmentation performance across four challenging datasets: DSB-2018, ISIC-2018, GlaS, and MoNuSeg. These datasets were chosen because they offer diverse and comprehensive cases for various biomedical imaging challenges, which can effectively indicate the robustness and generalization capabilities of segmentation models. Details regarding each dataset and the distribution of images among the training, validation, and testing sets are outlined in TABLE I.

1. The 2018 Data Science Bowl Challenge (DSB-2018) dataset [29] presents a wide variety of nuclei images, offering challenges due to the diversity in the structure and morphology of the nuclei. This diversity necessitates that segmentation models capture precise semantic information, thereby supporting the resolution of semantic gap.

2. The GlaS dataset [32] focuses on the segmentation of glandular structures in colon histology images, encompassing glands with diverse shapes, sizes, and appearances. These irregular shapes pose a challenge to segmentation models, requiring them to capture precise semantic information of glandular contours, making the dataset an ideal platform for testing model performance in segmenting complex and irregularly images.

3. The MoNuSeg dataset [33], [34] includes nuclear segmentation images from multiple organs, such as the liver, kidney, prostate, and breast. This diversity in data necessitates that segmentation models possess strong generalization capabilities, enabling them to perform well across various organ and tissue types. By employing a multi-organ dataset, we aim to evaluate the model’s ability to bridge semantic gap.

4. The Lesion Boundary Segmentation Challenge dataset (ISIC-2018) [30], [31] focuses on skin lesion segmentation, encompassing a variety of shapes, sizes, and locations. This diversity aids in assessing the capability of segmentation models to handle complex lesion features, capture variable semantic information, and address the semantic gap issue.

TABLE I Details of the Medical Segmentation Databases

B. Implementation Details and Assessment Metrics

The experiments were conducted using a standalone NVIDIA GeForce RTX 3090 Ti tensor core GPU, an 8-core CPU, and 24 GB RAM, employing the PyTorch framework. To prevent overfitting, data augmentation strategies such as image flipping and random rotation were used. Before being input into the network, the resolution of all images was standardized to $448\times 448$ pixels. Model training was performed using the Adam optimizer with an initial learning rate of 0.001, employing cross-entropy and Dice loss functions. A 5-fold cross-validation was applied to enhance the robustness of the results.

The model’s performance was evaluated using the Dice coefficient (Dice), the mean intersection over union (MIoU), and the Hausdorff distance (HD) [42]. The Dice coefficient, a metric used to assess sample similarity, measures the ratio of twice the intersection of two sets to their total size. MIoU represents the average ratio of the intersection to the union of the ground truth and predicted sets. The formulas for calculating Dice and MIoU are as follows:

View Source\begin{align*} Dice & = \frac {2*TP}{FP + 2*TP + FN} \times 100\%, \ \tag {1}\\ MIoU & = \frac {1}{k + 1}\sum \limits _{i = 0}^{k} {\frac {TP}{FP + TP + FN}} \times 100\%,\ \tag {2}\end{align*}

The Hausdorff distance (HD) quantifies the spatial separation between the ground truth and predicted sets within a metric space. A smaller HD value indicates a closer correspondence between the model prediction and the actual target region. The formula for calculating HD is as follows:

View Source\begin{align*} & {d_{H}}(X,Y) \\ & = \max \left \{{{\begin{array}{cccccccccccccccccccc} {\max \limits _{x \in X} \left ({{\mathop {\min d(x,y)}\limits _{y \in Y} }}\right ),}& {\max \limits _{y \in Y} \left ({{\mathop {\min d(y,x}\limits _{x \in X} }}\right )} \end{array}}}\right \}, \tag {3}\end{align*}

C. Quantitative Results and Analysis

This analysis is conducted for various skip connection configurations, as depicted in Fig. 2. The Dice, MIoU, and HD results across the four datasets indicate that the DSC scheme reduces model’s segmentation performance, whether applied individually or in combination. The performance drop is particularly pronounced when SC1 and SC2 are used separately. This is due to the complex and variable morphology of target regions in these datasets, which exacerbates semantic gap at corresponding layers between the encoder and decoder. The DSC scheme failed to effectively address this issue.

Fig. 2.

Quantifying the influence of semantic gap on U-Net’s segmentation performance across multiple assessment metrics.

Show All

The semantic gap between different layers of the encoder and decoder are quantified, as shown in TABLEs II, III, and IV. Here, “None” indicates the absence of skip connections, “SC1” implies that only the first level of skip connections is preserved, and “SC4+SC1” means that the first and fourth levels of skip connections are maintained. In this study, the scenario denoted as “None” is used as a reference point to gauge the impact of the semantic gap. In this configuration, there are no skip connections. Notably, skip connections in U-Net typically bypass certain layers, forwarding the output from one layer directly to a subsequent layer, thus preserving spatial information that might otherwise be lost during the pooling operation. However, in the “None” scenario, this bypassing mechanism is absent. Consequently, there is no direct transfer of information between the corresponding layers of the encoder and decoder, effectively eradicating the opportunity for a semantic gap to emerge. Thus, the “None” scenario acts as a control condition, forming a baseline against which the impacts of the semantic gap in other scenarios incorporating skip connections can be measured and compared. This setup allows for a comprehensive understanding of the semantic gap’s impacts under different circumstances. In these TABLEs, “+” indicates improvement, while “-” signifies decline. A systematic analysis of the quantified semantic gap results reveals three critical findings.

TABLE II Quantitative Analysis of Semantic Gap Influence on Dice Metric at Different Layers

TABLE III Quantitative Analysis of Semantic Gap Influence on MIoU Metric at Different Layers

TABLE IV Quantitative Analysis of Semantic Gap Impact on HD Metric at Different Layers

A. Overall Architecture

To address the challenges posed by the semantic gap, this study introduces the USCT-UNet architecture, an extension of the U-Net. The structure of the proposed USCT-UNet is illustrated in Fig. 3. In this architecture, the U-shaped skip connection (USC) replaces the direct skip connection (DSC), and the fusion of features from the decoder and USC is achieved through the SCCA module in the decoder. The process begins with an image $X \in {\mathbb {R}^{H \times W \times 3}}$ (where H and W represent the height and width of the image, respectively) being fed into the encoder for representation learning, resulting in feature maps $\textbf {E}_{k}$ for $k = 1, \cdot \cdot \cdot,4$ . These feature maps then undergo feature embedding to generate $\textbf {T}_{k}$ , which are then fed into the USC for semantic disambiguation, yielding the features $\textbf {O}_{k}$ . Simultaneously, $\textbf {E}_{4}$ undergoes pooling and convolution operations to obtain $\textbf {E}_{5}$ , which is fed into the decoder, producing the feature maps $\textbf {D}_{k}$ . Finally, $\textbf {O}_{k}$ and $\textbf {D}_{k}$ are fused through the SCCA module in the decoder, and the output features from the final layer of the decoder undergo a convolution operation to produce the segmentation result $Y \in {\mathbb {R}^{H \times W \times 1}}$ . Further details are provided in below and in Algorithms 1 and 2.

Fig. 3.

Schematic of the overall structure of USCT-UNet. The DSC is replaced with the USC, built using MCFT blocks. A SCCA module is introduced to guide the fusion of feature information between the decoder and the output of the USC.

Show All

B. U-Shaped Skip Connection

In response to the observed characteristics of the semantic gap, this study introduces the USC scheme using the MCFT to replace the DSC scheme, thereby eliminating the semantic gaps between the encoder and decoder, as shown in Fig. 3. Specifically, the severity of the semantic gap determines the number of MCFT blocks used to address it. For example, “SC1,” with the most pronounced semantic gap, consists of four MCFT blocks. Similarly, “SC2” contains three MCFT blocks, “SC3” has two, and “SC4,” with the least severe semantic gap, includes a single MCFT block. The output from each encoder layer, $\textbf {E}_{k}$ , is converted into a two-dimensional matrix and then input into the USC for multichannel feature fusion and multiscale modeling. The USC output feature, $\textbf {O}_{k}$ , is fused with the decoder output feature, $\textbf {D}_{k}$ , within the SCCA module in the decoder, thereby restoring the image’s fine-grained details.

C. Multichannel Fusion Transformer

To address the semantic gap between the encoder and decoder, we employ a multichannel fusion Transformer (MCFT) for multiscale global modeling on the output features of each encoder layer, $\textbf {E}_{k}$ . As shown in Fig. 3, the MCFT block consists of a feature embedding layer, a linear layer, two normalization layers (LN), a multihead channel self-attention (MCSA) mechanism, a dropout layer, and an improved multilayer perceptron (IMLP).

2) Multihead Channel Self-Attention:

We then linearly map $\textbf {T}_{k}$ to the query vector ${\textbf {Q}_{k}}\in {\mathbb {R}^{d \times {C_{k}}}}$ , and the channel-wise concatenation of $\textbf {T}_{\Sigma }$ to the key vector $\textbf {K} \in {\mathbb {R}^{d \times {C_{\Sigma } }}}$ and value vector $\textbf {V} \in {\mathbb {R}^{d \times {C_{\Sigma } }}}$ . This is expressed as

$$\begin{equation*} \ {\textbf {Q}_{k}} = {\textbf {T}_{k}}{\textbf {W}_{Q_{k}}}, \textbf {K} = {\textbf {T}_{\Sigma }}{\textbf {W}_{K}}, \textbf {V} = {\textbf {T}_{\Sigma }}{\textbf {W}_{V}},\ \tag {4}\end{equation*}$$ View Source\begin{equation*} \ {\textbf {Q}_{k}} = {\textbf {T}_{k}}{\textbf {W}_{Q_{k}}}, \textbf {K} = {\textbf {T}_{\Sigma }}{\textbf {W}_{K}}, \textbf {V} = {\textbf {T}_{\Sigma }}{\textbf {W}_{V}},\ \tag {4}\end{equation*} where ${\textbf {W}_{Q_{k}}} \in {\mathbb {R}^{d \times {C_{k}}}}$ , ${\textbf {W}_{K}} \in {\mathbb {R}^{d \times {C_{\Sigma } }}}$ , and ${\textbf {W}_{V}} \in {\mathbb {R}^{d \times {C_{\Sigma } }}}$ are the weights for $\textbf {Q}_{k}$ , K, and V, respectively. Additionally, ${C_{\Sigma } } = \mathrm {Concat}({C_{1}},{C_{2}},{C_{3}},{C_{4}})$ and ${\textbf {T}_{\Sigma } } = \mathrm {Concat} ({\textbf {T}_{1}},{\textbf {T}_{2}},{\textbf {T}_{3}},{\textbf {T}_{4}})$ .

After LN, $\textbf {Q}_{k}$ , K, and V are processed through the CSA for attention calculation. The output $\textbf {CSA}_{k}$ is defined as

$$\begin{equation*} \ {\textbf {CSA}_{k}} = {\text {SoftMax}}\left ({{{\mathrm { SN}}\left ({{\frac {\textbf {Q}_{_{k}}^{\mathrm { T}}\textbf {K}}{\sqrt {C_{\Sigma }} }}}\right )}}\right ){\textbf {V}^{\mathrm { T}}},\ \tag {5}\end{equation*}$$ View Source\begin{equation*} \ {\textbf {CSA}_{k}} = {\text {SoftMax}}\left ({{{\mathrm { SN}}\left ({{\frac {\textbf {Q}_{_{k}}^{\mathrm { T}}\textbf {K}}{\sqrt {C_{\Sigma }} }}}\right )}}\right ){\textbf {V}^{\mathrm { T}}},\ \tag {5}\end{equation*} where $\mathrm {SN}(\cdot)$ denotes switch normalization (SN) [43]. $\textbf {Q}_{_{k}}^{\mathrm { T}}$ and $\textbf {V}^{\mathrm { T}}$ are the transposes of $\textbf {Q}_{k}$ and V, respectively.

MCSA is formed by combining multiple CSA, which can be computed in parallel to obtain the output result. With m heads, the output result $\textbf {MCSA}_{k}$ of the MCSA is computed as

$$\begin{equation*} {\textbf {MCSA}_{k}} = \frac {\textbf {CSA}_{_{k}}^{1} + \textbf {CSA}_{_{k}}^{2} +, \cdot \cdot \cdot, + \textbf {CSA}_{_{k}}^{m}}{m} + {\textbf {T}_{k}},\ \tag {6}\end{equation*}$$ View Source\begin{equation*} {\textbf {MCSA}_{k}} = \frac {\textbf {CSA}_{_{k}}^{1} + \textbf {CSA}_{_{k}}^{2} +, \cdot \cdot \cdot, + \textbf {CSA}_{_{k}}^{m}}{m} + {\textbf {T}_{k}},\ \tag {6}\end{equation*}

Then, applying the IMLP and residual operator, we derive the final output feature ${\textbf {O}_{k}} \in {\mathbb {R}^{d \times {C_{k}}}}$ as

$$\begin{equation*} \ {\textbf {O}_{k}} = {\mathrm { IMLP}}\left ({{{\mathrm { LN}}\left ({{\textbf {MCSA}_{k}}}\right )}}\right ) + {\mathrm { LN}}\left ({{\textbf {MCSA}_{k}}}\right ),\ \tag {7}\end{equation*}$$ View Source\begin{equation*} \ {\textbf {O}_{k}} = {\mathrm { IMLP}}\left ({{{\mathrm { LN}}\left ({{\textbf {MCSA}_{k}}}\right )}}\right ) + {\mathrm { LN}}\left ({{\textbf {MCSA}_{k}}}\right ),\ \tag {7}\end{equation*} By repeating Eqs. (5–​7) L times, L MCFT blocks can be assembled. This work assigns a different number of MCFT blocks to each skip connection, corresponding to the severity of the semantic gap.

D. Improved Multilayer Perceptron

The original multilayer perceptron (MLP) used in the Transformer structure employs a two-layer linear mapping mechanism, which may not fully capture the complex shape features present in medical images. To address this limitation, we have developed an improved multilayer perceptron (MLP) with three linear layers, incorporating batch normalization BN in the middle layer and a dropout layer, as shown in Fig. 4.

Fig. 4.

Comparison between the improved MLP and the original MLP.

Show All

The feature sequence from the MCSA is first processed by a linear layer, transforming it into a 784-dimensional feature sequence. After undergoing BN and Gaussian Error Linear Unit (GELU) activation, the sequence enters next linear layer, retaining the same dimensionality. The sequence then passes through another round of BN and GELU activation before entering third linear layer, which reduces its dimensionality back to the original size. Following a dropout layer, the final output provides a comprehensive representation of the complex semantic information in the medical image.

E. Spatial Channel Cross-Attention

In the architecture shown in Fig. 3, we design a spatial channel cross-attention (SCCA) module to facilitate the fusion of features from the USC module and the corresponding decoder layer. Specifically, the decoder output features$\textbf {D}_{k}$ are first upscaled using a convolution operation and then passed through a feature embedding layer to obtain the two-dimensional patch token ${\textbf {T}_{k}} \in {\mathbb {R}^{d * {C_{k}}}}$ , which is then linearly mapped to the query vector ${\textbf {Q}_{k}} \in {\mathbb {R}^{d * {C_{k}}}}$ using Eq. (4). Simultaneously, the features $\textbf {O}_{k}$ from the USC module are linearly mapped to the key vector $\textbf {K} \in {\mathbb {R}^{d * {C_{k}}}}$ and the value vector $\textbf {V} \in {\mathbb {R}^{d * {C_{k}}}}$ through Eq. (4). The query vector $\textbf {Q}_{k}$ and the value vector V are then transposed to obtain $\textbf {Q}^{\mathrm { T}}_{_{k}}$ and $\textbf {V}^{\mathrm { T}}$ , respectively, flipping the spatial and channel information of the feature maps. Next, $\textbf {Q}^{\mathrm { T}}_{_{k}}$ , K, $\textbf {V}^{\mathrm { T}}$ are applied in Eq. (8) for the cross-attention computation. The resulting output feature map $\textbf {M}_{k}$ is then reconstructed and multiplied with $\textbf {D}_{k}$ to update $\textbf {D}_{k}$ , signifying the fusion of information from the USC module and the corresponding decoder layer.

View Source\begin{equation*} \textbf {M}_{k} = \text {SoftMax}\left ({{{\mathrm { SN}}\left ({{\frac {\textbf {Q}^{\mathrm { T}}_{_{k}}\textbf {K}}{\sqrt {C_{k}}}}}\right ) }}\right )\textbf {V}^{\mathrm { T}} \tag {8}\end{equation*}

F. Computational Complexity

This study designs the USC and SCCA modules based on the self-attention mechanism. Both modules employ softmax attention as described in Eq. (5) and Eq. (8), which involves matrix operations to compute the similarity between all Q-K pairs, resulting in a complexity of $\mathrm {O}({N_{k}^{2}}\cdot d)$ . In the U-Net architecture, convolution operations are the primary source of computational complexity, which can be expressed as:

View Source\begin{equation*} \mathrm {O}(H \times W \times {\kappa ^{2}} \times 2C_{in} \times C_{out}), \tag {9}\end{equation*}

A. K-Fold Cross-Validation

To rigorously evaluate the segmentation precision of the U-Net and our proposed USCT-UNet models, we conducted 5-fold cross-validation experiments across four distinct datasets. The results are summarized in TABLE V, which indicates that the USCT-UNet consistently outperforms the U-Net in terms of segmentation performance across all datasets. For instance, for ISIC-2018 dataset, the USCT-UNet achieved a 4.79% improvement in Dice, a 5.70% increase in MIoU, and a reduction of 2.15 in HD. Similarly, for GlaS dataset, the USCT-UNet demonstrated a 2.24% enhancement in Dice, a 3.46% enhancement in MIoU, and a 3.26 decrease in HD. Additionally, the USCT-UNet showed substantial improvements in these evaluation metrics for both DSB-2018 and MoNuSeg datasets.

TABLE V Results of 5-Fold Cross-Validation

B. Ablation Studies

We conducted ablation study using four challenging datasets, with the results presented in TABLE VI. In these experiments, we systematically evaluated the effects of the USC module, the SCCA module, and their combination on model performance. The results demonstrate that the USC module significantly enhances model performance across all four datasets. For instance, on ISIC-2018 dataset, the Dice for the UNet+USC model improves to 87.79%, which is 3.92% higher than that of the UNet of 83.87%. On DSB-2018 dataset, the MIoU for the UNet+USC model improves to 82.08%, which is 2.85% higher than that of the UNet of 79.23%. Similarly, incorporating the SCCA module substantially improves model performance. For example, on MoNuSeg dataset, the MIoU for the UNet+SCCA model reaches 65.81, which is 3.48% higher than that of the UNet of 62.33%. On GlaS dataset, the HD for the UNet+USC model decreases to 26.49, which is 1.60 higher than that of the UNet of 28.09.

TABLE VI Results of The Ablation Experiments to Assess The Contribution of USCT-UNet’s Components on Four Datasets

The model’s performance reaches its optimal level when both the USC and SCCA modules are combined. For instance, on GlaS dataset, the Dice and MIoU for the UNet+USC+SCCA model improve to 91.16% and 84.44%, On DSB-2018 dataset, the Dice and MIoU rise to 89.12% and 83.15%, respectively. These enhancements demonstrate that the USC module effectively mitigates the negative impact of the semantic gap, while the SCCA module proficiently integrates features from both the USC and the decoder. However, these performance gains come at the cost of increased computational complexity. Its parameters and FLOPs increase to 31.15M and 62.05G, respectively, which may affect the model’s real-time performance and deployment efficiency.

We further investigate the role of the USC module in USCT-UNet, specifically how it addresses the semantic gap between the decoder and encoder. This section includes a detailed ablation study of the USC scheme using four datasets, with results provided in Fig. 5. The Dice, MIoU, and HD results across the four datasets demonstrate that the USC scheme significantly enhances model’s segmentation performance. The improvements are particularly pronounced on more challenging datasets, such as MoNuSeg and ISIC-2018, due to the morphological diversity and complexity of the target regions in these datasets. These factors make it more difficult for segmentation models to capture accurate semantic information, whereas our USC scheme effectively overcomes this challenge.

Fig. 5.

Results of the ablation experiments to assess the contribution of the USC options across multiple assessment metrics.

Show All

The effectiveness of the proposed USC module in addressing the semantic gap between different layers of the encoder and decoder has been quantified through the data presented in TABLEs VII, VIII, and IX. In these TABLEs, “+” indicates improvement. These results demonstrate significant improvements in the model’s ability to handle semantic mismatches across layers. The findings reveal three significant improvements facilitated by the USC scheme compared to the DSC scheme (as detailed in TABLEs II, III, and IV).

1. Across the four datasets, all variations of the USC scheme positively effect the segmentation process. This underscores the efficacy of USCT-UNet in bridging the semantic gap between corresponding layers of the decoder and encoder. For instance, on DSB-2018 dataset, compared to the configuration without skip connections (“None”), “SC1,” “SC2,” “SC3,” and “SC4” increased Dice by 1.79%, 1.62%, 1.02%, and 1.87%, respectively. On ISIC-2018 dataset, they enhanced MIoU by 2.48%, 2.19%, 2.15%, and 2.34%, respectively.

2. The USC module eliminates the adverse impacts associated with the direct skip connection (DSC) while maximizing its benefits. For example, on MoNuSeg dataset, transitioning from “SC4” to “SC4+SC1,” “SC4+SC1+SC2,” and “SC4+SC1+SC2+SC3” increased Dice from 1.42% to 3.28%, 4.07%, and 5.60%, respectively. On GlaS dataset, they reduced HD from 1.16 to 1.01, 1.07, and 1.47, respectively.

3. The USC module significantly improved the model’s segmentation performance. Specifically, on GlaS dataset, USCT-UNet achieved a Dice of 91.16%, a MIoU of 84.44%, and a HD of 24.83. On DSB-2018 dataset, the model achieved a Dice of 89.12%, a MIoU of 83.15%, and a HD of 15.11. These results underscore the effectiveness of the USC scheme in bridging the semantic gap between the decoder and encoder.

TABLE VII Results of the Ablation Experiments to Assess the Contribution of the USC Options on Dice Metric

TABLE VIII Results of the Ablation Experiments to Assess the Contribution of the USC Options on MIoU Metric

TABLE IX Results of the Ablation Experiments to Assess the Contribution of the USC Options on HD Metric

C. Segmentation Results Visualisation

In this section, we present visual comparisons of the segmentation results produced by the proposed USCT-UNet and benchmark models to demonstrate USCT-UNet’s superiority in medical image segmentation tasks. These results are depicted in Fig. 6, where the red boxes indicate areas where USCT-UNet outperforms the other models in segmentation effectiveness. USCT-UNet clearly delivers superior segmentation outcomes that more closely resemble the ground truth compared to U-Net and UCTransNet. As illustrated in Fig. 5, USCT-UNet accurately highlights the correct pathological regions while suppressing false positives and delineates continuous boundary edges, demonstrating its effectiveness in precise and robust segmentation.

Fig. 6.

Qualitative comparative results on four challenging medical image databases.

Show All

D. Comparison With State-of-the-Art Related Methods

In this subsection, we comprehensively analyze the segmentation performance, parameters, and computational complexity of USCT-UNet across four datasets. We compare its performance with the latest CNN and Transformer-based methods, as detailed in TABLEs X and XI. The results indicate that USCT-UNet achieves the highest scores in both Dice and MIoU metrics, with values of 89.12% and 83.15% on DSB-2018, 88.66% and 81.62% on ISIC-2018, 91.16% and 84.44% on GlaS, and 80.07% and 66.99% on MoNuSeg, representing the highest values across all datasets. Additionally, compared to methods such as HRNetV2 [44], TransFuse [49], UDTransNet [53], UCTransNet [26] and LCAMix [52], our USCT-UNet demonstrates design efficiency by significantly improving precision while maintaining a low parameters (31.15M Param) and computational complexity (62.05G FLOPs), highlighting its potential for broad clinical deployment and application.

TABLE X Comparison With the Latest Relevant Methods on DSB-2018 and ISIC-2018

TABLE XI Comparison With the Latest Relevant Methods on GlaS And MoNuSeg