Introduction

In the contemporary era, computer technology, the Internet of Things and multimedia technology have achieved remarkable development results. This development has had a significant impact on all areas of society, especially in the field of information security1. Given the background of globalization and digitalization, the internet has become an important platform for information dissemination. However, problems such as data leakage, information tampering and privacy infringement are becoming increasingly severe. Information security is related not only to the protection of personal privacy but also to the national economy, livelihood protection, political stability and national security. Therefore, it has become one of the important directions of national strategic research.

With the continuous development of cryptography research, modern encryption technology systems have become increasingly mature. Many efficient encryption algorithms have been widely applied, such as symmetric encryption algorithms such as DES2 and AES3,4, as well as asymmetric encryption algorithms such as RSA5,6. However, when processing digital images, these traditional algorithms typically handle image data as ordinary binary streams. This model fails to fully exploit the unique characteristics of images, such as large data volume, high redundancy, and strong interpixel correlation, which limits processing performance. This limitation leads to the inefficiency and lack of security of algorithms such as AES and DES in image encryption. To address potential security threats during the transmission and storage of images, many experts and scholars have proposed different image encryption algorithms. Among them, encryption methods based on chaotic systems have received widespread attention because of their excellent security performance.

Chaos is a type of nonlinear and complex phenomenon commonly found in nature that can effectively explain the characteristics of irregularity and dependence. Since Lorenz7 proposed chaos theory in the 1960s, significant progress has been made in related research, and it has had a profound impact on multiple disciplinary fields8,9. In 1998, Fridrich10 first introduced the chaotic system into the field of image encryption, promoting the extensive application of chaos-based methods. Chaotic systems have become important tools in image encryption in recent years because of their complexity and randomness. These properties have attracted a great deal of research exploring the potential of chaotic systems for enhancing the complexity of cryptographic keys. For example, Lu11 improved the sine map and proposed a 1D chaotic system; Jackson12 proposed a 2D chaotic system by combining the sine map and logistic map; Wang et al.13 designed a novel 2D crossed hyperchaotic sine-modulation-logistic map. Even if a low-dimensional chaotic system has a straightforward structure, its key space is limited, and its fault tolerance is insufficient. In contrast, owing to their unpredictable trajectories and more complex dynamic behavior, high-dimensional chaotic systems exhibit greater promise in image encryption. High-dimensional chaotic systems have progressively gained attention in recent years14,15,16. For example, in 1999, Chen proposed the Chen chaotic system17. Compared with the Lorenz system, it has greater attractor complexity in the phase space and is more suitable for use in encryption algorithms. Li et al.18 constructed a dual chaotic system by improving the logistic chaotic system and combining it with the hyperchaotic Chen chaotic system, further increasing the complexity and randomness of the encryption scheme. Zhou et al.19 designed a multiparameter-coupled 4D chaotic system, which significantly increased the key space and enhanced the encryption security by extending the parameters. Owing to its rich dynamical behavior and large key space, the 4D chaotic system exhibited a good balance in image encryption: It offers high security while maintaining appropriate computational complexity. Compared with low-dimensional systems, it is more secure; compared with five-dimensional and higher-dimensional systems, its computational complexity is lower. Therefore, to increase computational efficiency, attain a higher Lyapunov exponent and expand the key space, devising a 4D chaotic system with complex dynamic behavior is crucial.

In image encryption, chaotic systems are typically used in two key stages, scrambling and diffusion, to strengthen the security of the algorithm. Diffusion increases the strength of encryption by altering the pixel values, and scrambling upsets the spatial structure by moving the pixel locations. Classic scrambling methods such as Arnold scrambling, Zigzag scrambling and Hilbert scrambling20,21,22 have been widely used but are vulnerable to attacks because of their simple algorithms. To address this, researchers have proposed various novel encryption methods, such as DNA operation23,24, compressed sensing25,26,27, S-box replacement28,29,30, the shuffling algorithm31,32, and deep learning33,34. DNA technology stands out among these techniques because of its enormous parallelism, low power consumption, and vast storage capacity. For example, Rahul35 proposed an improved DNA encoding scheme that can simultaneously encode 4-bit binary numbers, significantly reducing data processing costs and improving encryption efficiency. However, the complicated key generation process of this method increases the encryption time. Wu36 designed a novel DNA algorithm that combines cyclic crossover and mutation and increases the complexity of encryption by integrating the scrambling and diffusion functions; however, its resistance to clipping attacks still needs further verification. Although existing image encryption schemes have made some progress by optimizing the scrambling and diffusion stages, most of them rely on single-layer encryption techniques, such as pixel-level encryption or bit-level diffusion. This single level of encryption cannot comprehensively guarantee the security of encrypted images. Therefore, combining pixel-level encryption and bit-level diffusion to construct a multilayer scrambling diffusion algorithm can significantly improve image encryption security.

A 4D hyperchaotic system is proposed in this study on the basis of these concerns, and an image encryption strategy combining Thorp shuffle and dequeuing is constructed on the basis of this system. The primary contributions of this study are outlined below.

  1. (1).

    An improved four-dimensional hyperchaotic system is proposed. Through dynamic analysis, it is verified that this system has higher Lyapunov values and more complex dynamic behaviors. The system can concurrently produce four chaotic sequences, significantly reducing the system’s iteration time and improving computational efficiency.

  2. (2).

    A double-ended Thorp scrambling method is proposed, which combines Thorp Shuffle with the dequeue operation. This method involves multiple rounds of scrambling processes. Each round is carried out on different pseudorandom sequences, and each round contains multiple pseudorandom sequences, effectively enhancing the security of the encryption algorithm.

  3. (3).

    A fourth-order bit‒plane diffusion method is proposed by combining DNA encoding and multiplane diffusion. Pixels are stratified according to bit planes, which reduces the correlation between adjacent pixels. Moreover, this diffusion method establishes connections between different pixel bit planes, enabling even minor changes to generate different diffusion effects and enhancing the security of the algorithm.

The following sections of this paper are organized as follows. Section II describes the classical Chen hyperchaotic system, the Thorp shuffle process, and the DNA operation method. Section III provides a detailed analysis of the structure and performance of the improved chaotic system. Section IV presents the image encryption algorithm utilizing the improved chaotic system. Section V evaluates the performance of the algorithm through simulation experiments and compares it with existing algorithms. Section VI concludes the paper.

Relevant knowledge

Chen hyperchaotic system

In 2005, Li37 constructed a 4D Chen hyperchaotic system on the basis of the Chen chaotic system via state feedback control. The mathematical model is as follows:

$$\begin{array}{c}\left\{\begin{array}{c}\dot{x}=a\left(y-x\right)+w\\ \dot{y}=bx-xz+cy\\ \dot{z}=xy-dz \\ \dot{w}=yz+ew \end{array}\right.,\end{array}$$
(1)

where x, y, z, and w are state variables and where a, b, c, d, and r are control parameters. When a = 35, b = 7, c = 12, d = 3, and e ε(0.085, 0.798], with initial values of (0.1, 0.1, 0.1, 0.1), the system is in the hyperchaotic state.

Thorp shuffle

The Thorp shuffle algorithm38 is a pseudorandom permutation method used for data encryption in finite space. Its basic principle is as follows: Suppose there are I cards (I is an even number), which are evenly divided into two piles, left and right. The drawing order is determined by tossing a coin: when the result shows heads, one card is taken from the bottom of each of the left and right piles in turn; if the result is tails, one card is taken from the right pile, then one from the left pile. After the process is repeated I/2 times, a round of shuffling is complete. To enhance the encryption effect, the card shuffling process can be repeated as many times as needed.

Expressed slightly more algebraically, for each round of shuffling (r = 1, 2, …, R), the cards at positions x and x + I/2 are moved either to positions 2 × and 2x + 1 or to positions 2x + 1 and 2x, and the specific order is determined by the coin-tossing result \(c \in (0,1)\). Figure 1 shows the Thorp shuffle process.

Fig. 1
figure 1

Thorp shuffle.

DNA encoding and operation

DNA is composed of four bases: adenine (A), cytosine (C), guanine (G) and thymine (T)39,40. Among these nucleobases, A and T specifically form stable complementary base pairs, as do C and G. In the field of image encryption, the pixel values range from 0–255 and can be converted to 8-bit binary numbers. In the binary system, 0 and 1 form pairs that are complementary to one another. Consequently, pairs such as 00 and 11, 01 and 10 also become complementary. On the basis of this complementary principle, 00, 10, 01, and 11 are mapped to four DNA bases. With this method, each pixel can be represented by a DNA sequence of length 4.

There are eight DNA encoding rules (Table 1). The same pixel value can generate different DNA encoding results on the basis of different rules. DNA operations include mainly addition, subtraction, and XOR operations (Table 2, Table 3 and Table 4), which are widely used in the encryption process.

Table 1 Rules of DNA encoding.
Table 2 DNA addition operation.
Table 3 DNA subtraction operation.
Table 4 DNA XOR operation.

Propose and analyze an improved 4D hyperchaotic system

This section proposes an improved 4D hyperchaotic system, and the system is comprehensively analyzed from the aspects of the phase diagram, Lyapunov exponent, fractal dimension, and NIST test.

Improved 4D Chen hyperchaotic system

Chaotic systems have a wide range of applications in image encryption because of their initial condition sensitivity and ergodicity. In contrast to low-dimensional chaotic systems, hyperchaotic systems can provide larger key spaces and more complex dynamic behaviors. To address the performance limitations imposed by the restricted control parameter ranges of the Chen hyperchaotic system and overcome the issue of traditional 4D hyperchaotic systems containing only two positive Lyapunov exponents, this section proposes an enhanced 4D hyperchaotic system with three positive Lyapunov exponents. Building upon the original 4D Chen hyperchaotic system, we reconstruct the nonlinear terms in its second, third, and fourth dimensions to develop an improved hyperchaotic system incorporating multidimensional nonlinear couplings. The mathematical model is formulated as follows:

$$\begin{array}{c}\left\{\begin{array}{c}\dot{x}=a\left(y-x\right)+w \\ \dot{y}=bxz-x{z}^{2}+cy\\ \dot{z}={x}^{2}+{y}^{2}-dz \\ \dot{w}=xz+yz-ew \end{array}\right.,\end{array}$$
(2)

where x, y, z, and w are state variables and where a, b, c, d, and e are control parameters.

The improved system’s phase diagram is shown in Fig. 2 when a = 28, b = 17, c = 13, d = 21.8, and e = 5, with initial values of (0.1, 0.1, 0.1, 0.1). Compared with the phase diagram of the traditional 4D Chen hyperchaotic system (Fig. 2(f)), the improved system’s phase diagram has a more complex structure and a wider dynamic range. This finding shows that the improved system is more suitable for encryption-related applications and has excellent chaotic characteristics.

Fig. 2
figure 2

Phase diagrams.

Lyapunov exponential and fractal dimension analysis

One of the most important metrics for determining how sensitive a chaotic system is to its initial conditions is the Lyapunov exponent (LE). When a = 28, b = 17, c = 13, d = 21.8, and e = -0.98, Fig. 3 shows the LE of the improved system proposed in this paper. In a 4D chaotic system, when the system includes two or more positive LE values, the system enters a hyperchaotic state. At this time, it is significantly more difficult to predict the system trajectory41. Figure 3(b) shows the LE values for the 4D Chen hyperchaotic system.

Fig. 3
figure 3

Lyapunov exponents.

The calculations reveal that the LE values of the improved system are LE1 = 11.28, LE2 = 0.765, LE3 = 0.048, and LE4 = -47.91. The system exhibits complex chaotic dynamics characterized by three positive Lyapunov exponents, where LE3 approaches zero while the total sum of LEs remains negative. In comparison, the improved system has a maximum LE value of 11.3, whereas the 4D Chen hyperchaotic system has a maximum LE of 0.458. This clearly shows that the improved system possesses more powerful chaotic features.

In addition, Fig. 4 shows the effects of different parameters (a [25, 35], b [15, 20] and c [8, 18]) on the chaotic characteristics of the system. Figure 4(b) shows that when b is approximately 13, the system transforms from the periodic state to the hyperchaotic state. Figures 4(a) and 4(c) show that the system consistently displays chaotic behavior across the specified parameter range. Compared with Fig. 4(d), the improved system clearly has a larger chaotic range and a higher LE, which further proves its excellent chaotic properties.

Fig. 4
figure 4

Single-parameter LE

Figure 5 shows the heatmap of the maximum LE for the system with a = 28, b (0, 20), c (0, 20), d = 21.8, and e = -0.98. Brighter colors represent larger LE values, and darker colors represent smaller LE values (shown by the color scale of Fig. 5(a)). These findings indicate that the improved 4D hyperchaotic system has a broader chaotic region.

Fig. 5
figure 5

Maximum LE of two parameters.

The Lyapunov fractal dimension is an important method for determining the existence of chaotic attractors, and its definition is as follows:

$$\begin{array}{c}{D}_{L}=j+\frac{{\sum }_{i=1}^{j}{LE}_{j}}{\left|{LE}_{j+1}\right|}\end{array}$$
(3)

where LEj is the jth LE value and j is the maximum value of i, \({\sum }_{i=1}^{j}{LE}_{j}>0\).

If DL is fractional, it indicates the presence of chaotic attractors in the system. By calculating the LE value of the improved system, the fractal dimension is DL = 3.253, indicating that the system has aperiodic orbits, chaotic attractors, and highly complex dynamic behavior.

Table 5 presents a comparison of the Lyapunov exponents and Lyapunov fractal dimensions between the improved 4D hyperchaotic system and the existing 4D chaotic systems. The improved 4D hyperchaotic system proposed in this paper has larger maximum LE values and DL values. This indicates that the new system exhibits superior hyperchaotic performance, which is more conducive to the generation of random sequences.

Table 5 Comparison of Lyapunov exponents and Lyapunov fractal dimensions.

Dissipative and equilibrium analysis

The dissipation property of the improved system can be determined via the following equation:

$$\begin{array}{c}\nabla V=\frac{\partial \dot{x}}{\partial x}+\frac{\partial \dot{y}}{\partial y}+\frac{\partial \dot{z}}{\partial z}+\frac{\partial \dot{w}}{\partial w}=-a+c-d-e=-33.82<0\end{array}$$
(4)

when parameter c < a + d + e, the system is dissipative and converges exponentially as e(-a+c-d-e)t. When t → ∞, all volume elements of the system’s trajectory will shrink to zero at the exponential rate -a + c-d-e, indicating the existence of an equilibrium point for the system.

Equilibrium point analysis is a key step in the study of the dynamic properties of chaotic systems. In general, systems with multiple unstable equilibrium points exhibit more complex dynamic behaviors and more obvious chaotic characteristics. By processing the chaotic system in Eq. (2), the following formula is obtained:

$$\begin{array}{c}\left\{\begin{array}{c}a\left(y-x\right)+w=0\\ bxz-x{z}^{2}+cy=0\\ {x}^{2}+{y}^{2}-dz=0\\ xz+yz-ew=0\end{array}\right.,\end{array}$$
(5)

The coefficient matrix A is shown in Eq. (6).

$$\begin{array}{c}A=\left(\begin{array}{cccc}-a& a& 0& 1\\ z\left(b-z\right)& c& x\left(b-z\right)-xz& 0\\ 2x& 2y& -d& 0\\ z& z& x+y& -e\end{array}\right)\end{array}$$
(6)

The system has seven equilibrium points when the control parameters a = 28, b = 17, c = 13, d = 21.8 and e = -0.98, denoted as S1, S2, S3, S4, S5, S6 and S7.

$${S}_{1}=\left(\text{0,0},\text{0,0}\right),$$
$${S}_{2}=\left(-2.8277i, -2.6874i, -0.69809, -3.9287i\right),$$
$${S}_{3}=\left(2.8277i, 2.6874i, -0.69809, 3.9287i\right),$$
$${S}_{4}=\left(2.1553-0.55226i, 22.094+0.68351i,22.569+1.2762i, -558.28-34.602i\right),$$
$${S}_{5}=\left(-2.1553+0.55226i, -22.094-0.68351i,22.569+1.2762i, 558.28+34.602i\right),$$
$${S}_{6}=\left(-2.1553-0.55226i, -22.094+0.68351i,22.569-1.2762i, 558.28-34.602i\right),$$
$${S}_{7}=\left(2.1553+0.55226i, 22.094-0.68351i,22.569-1.2762i, -558.28+34.602i\right),$$

Taking the equilibrium point S1 = (0,0,0,0) as an example, the system is linearized to obtain the Jacobian matrix as follows:

$$\begin{array}{c}{J}_{0}=\left[\begin{array}{cccc}-a& a& 0& 1\\ z\left(b-z\right)& c& x\left(b-z\right)-xz& 0\\ 2x& 2y& -d& 0\\ z& z& x+y& -e\end{array}\right]=\left[\begin{array}{cccc}-28& 28& 0& 1\\ 0& 13& 0& 0\\ 0& 0& -21.8& 0\\ 0& 0& 0& 0.98\end{array}\right]\end{array}$$
(7)

The characteristic equation of J0 is

$$\begin{array}{c}\left|\lambda I-{J}_{0}\right|=0\end{array}$$
(8)

The corresponding eigenvalues are obtained as follows:

$$\begin{array}{c}\left\{\begin{array}{c}{\lambda }_{1}=-28 \\ {\lambda }_{2}=-21.8\\ {\lambda }_{3}=0.98 \\ {\lambda }_{4}=13 \end{array}\right.\end{array}$$
(9)

When the real part of the eigenvalues of the equilibrium point is less than 0, the equilibrium point is stable. Table 6 shows the eigenvalues of each equilibrium point and their corresponding stability classifications, and it can be seen that the system contains seven unstable equilibrium points.

Table 6 Eigenvalues of the equilibrium point.

Initial value sensitivity analysis

Chaotic systems exhibit extreme sensitivity to minor variations in initial conditions. The experiment is performed by adding a minor perturbation of size 10–14 at the initial conditions of x0 = 0.1, y0 = 0.1, z0 = 0.1, and w0 = 0.1. The results are shown in Fig. 6. Significant trajectory variations result from small perturbations, further confirming the system’s sensitivity to initial values.

Fig. 6
figure 6

Initial value sensitivity.

Sample entropy analysis

Sample entropy (SE) is a statistical metric that quantifies the complexity and regularity of time series data, and its value is positively correlated with series complexity. Therefore, SE is commonly used to analyze the intricacies of pseudorandom sequences produced by chaotic systems and to evaluate the performance of chaotic systems. Superior chaotic features are reflected in a higher SE value, which denotes a chaotic sequence with less regularity and greater complexity. Within the ranges of \(a \in (28,38)\) and \(c \in (11,14)\) parameters, the SE value of the improved 4D hyperchaotic system is significantly greater than that of the 4D Chen hyperchaotic system (Fig. 7). This indicates that the improved system generates sequences of greater complexity, further demonstrating its superior chaotic performance and potential application in high-security scenarios.

Fig. 7
figure 7

Comparison of the Ses.

Bifurcation diagram analysis

The bifurcation diagram can reflect the changes in the dynamic behavior of a chaotic system through the transformation of control parameters. Figure 8 shows a comparison of the bifurcation diagrams of the improved 4D hyperchaotic system (Fig. 8(a)) and the 4D Chen hyperchaotic system (Fig. 8(b)) within the same parameter range (e (-1,1)). By comparison, the bifurcation diagram of the improved 4D hyperchaotic system is relatively uniformly distributed within the given interval, indicating that it remains in a chaotic state throughout this parameter range. In contrast, the blue points in the bifurcation diagram of the Chen chaotic system are unevenly distributed, with obvious blank areas, suggesting that it is not in a chaotic state throughout the given interval. Thus, the improved 4D hyperchaotic system has a broader hyperchaotic range and better chaotic performance.

Fig. 8
figure 8

Bifurcation diagram.

NIST test

The NIST test suite is a standardized tool for evaluating the randomness of pseudorandom sequences. This suite of 15 tests can be used to detect binary sequences of arbitrary length. If the P value of a test exceeds 0.01, the sequence is considered to have passed that randomness test.

In this paper, the pseudorandom sequences generated by the proposed hyperchaotic system are tested via NIST, as shown in Table 7. The experimental results show that the P-value of all the sequences exceeds 0.01, effectively passing each test item. This shows that the pseudorandom sequences are highly random and can satisfy the image encryption randomness criterion.

Table 7 NIST test results.

Circuit design

To further verify the effectiveness of the improved four-dimensional hyperchaotic system proposed in this paper, this section designs the circuit of the chaotic system via the circuit simulation software Multisim 14.3. In this circuit simulation experiment, operational amplifiers of the model LM324AD, linear resistors, and capacitors are used for circuit integration to achieve addition, subtraction, and inverse integration operations. Moreover, a multiplier simulator AD633 with a multiplication factor of 0.1 is introduced to construct the nonlinear terms. The analog circuit is shown in Fig. 9.

Fig. 9
figure 9

Simulation circuit diagram.

When the control variables are a = 28, b = 17, c = 13, d = 21.8, and e = 5, as shown in Fig. 2, the range of state variables is much larger than the linear range (\(\pm\) 15 V) of the operational amplifier. To ensure that the state variables meet this range, the state variables are compressed, that is, (x,y,z,w) = (10x,10y,10z,10w). Substituting the control variables and the transformed state variables into Eq. (2), we can obtain:

$$\begin{array}{c}\left\{\begin{array}{c}\dot{x}=28y-28x+w \\ \dot{y}=170xz-100x{z}^{2}+13y\\ \dot{z}=10{x}^{2}+10{y}^{2}-21.8z \\ \dot{w}=10xz+10yz-5y \end{array}\right.\end{array}$$
(10)

With the time constant τ0 = 100, perform a time-scale transformation to obtain Eq. (11):

$$\begin{array}{c}\left\{\begin{array}{c}\dot{x}=2800y-2800x+100w \\ \dot{y}=17000xz-10000x{z}^{2}+1300y\\ \dot{z}=10{00x}^{2}+10{00y}^{2}-2180z \\ \dot{w}=1000xz+1000yz-500y \end{array}\right.\end{array}$$
(11)

The state equations of the circuit can be obtained through the circuit schematic diagram and Kirchhoff’s laws, which are presented in Eq. (12):

$$\begin{array}{c}\left\{\begin{array}{c}\dot{x}=\frac{-x}{{C}_{1}{R}_{5}}+\frac{y}{{C}_{1}{R}_{4}}+\frac{w}{{C}_{1}{R}_{1}} \\ \dot{y}=\frac{y}{{C}_{2}{R}_{10}}+\frac{xz}{{C}_{2}{R}_{9}}+\frac{-x{z}^{2}}{{C}_{2}{R}_{6}} \\ \dot{z}=\frac{-z}{{C}_{3}{R}_{15}}+\frac{{y}^{2}}{{C}_{3}{R}_{14}}+\frac{{x}^{2}}{{C}_{3}{R}_{11}}\\ \dot{w}=\frac{xz}{{C}_{4}{R}_{19}}+\frac{-w}{{C}_{4}{R}_{20}}+\frac{yz}{{C}_{4}{R}_{16}}\end{array}\right.\end{array}$$
(12)

The capacitance values are set as C1 = C2 = C3 = C4 = 100nF, and the feedback resistance values are R2 = R3 = R7 = R8 = R12 = R13 = R17 = R18 = 1kΩ. By comparing Eq. (11) and Eq. (12), the integral resistance values can be obtained as follows: R1 = 100Kω, R4 = R5 = 3.75kΩ, R6 = 1kΩ, R9 = 0.588kΩ, R10 = 7.69kΩ, R11 = R12 = R16 = R19 = 10kΩ, R15 = 4.59kΩ, R20 = 20kΩ. Figure 10 shows the operation results of the simulation circuit on the oscilloscope, which are consistent with the corresponding phase plane diagrams in Fig. 2. This finding verifies the effectiveness and practical application ability of the improved 4D chaotic system proposed in this paper.

Fig. 10
figure 10

Circuit simulation results.

Encryption scheme

This section proposes an image scrambling method combined with a double-ended Thorp shuffle with an improved chaotic system and a bit-plane diffusion method based on DNA encoding. The proposed encryption technique consists of three primary phases: key design and chaotic sequence generation, double-ended Thorpe scrambling and fourth-order bit-plane diffusion. Figure 11 shows the encryption procedure.

Fig. 11
figure 11

Flow diagram of the encryption scheme.

Key design and pseudorandom sequence generation

Hash algorithms are irreversible algorithms, and minor alterations in the input can result in considerable differences in the output, which is similar to the characteristics of chaotic systems. Therefore, hash algorithms are crucial in the key generation process. In this encryption scheme, the image pixel data serve as the initial input, upon which the 256-bit binary hash value K is computed through the SHA-256 algorithm. Owing to the different pixel information of different images, their hash values also differ. Then, 32 groups of 8 bits each are created from the hash value K, denoted as K = {k1,k2,…,k32}, which are used to generate E = {e1,e2,…,e8}. In this way, we ensure that each image has a unique and image-related chaotic initial value, thus enhancing the uniqueness and security of the key.

$$\begin{array}{c}\left\{\begin{array}{c}{e}_{1}={k}_{1}+{k}_{2}+\dots +{k}_{8} \\ {e}_{2}={k}_{1}\oplus {k}_{2}\oplus \cdots \oplus {k}_{8} \\ {e}_{3}={k}_{9}+{k}_{10}+\dots +{k}_{16 }\\ {e}_{4}={k}_{9}\oplus {k}_{10}\oplus \cdots \oplus {k}_{16 }\\ {e}_{5}={k}_{17}+{k}_{18}+\dots +{k}_{24} \\ {e}_{6}={k}_{17}\oplus {k}_{18}\oplus \cdots \oplus {k}_{24}\\ {e}_{7}={k}_{25}+{k}_{26}+\dots +{k}_{32} \\ {e}_{8}={k}_{25}\oplus {k}_{26}\oplus \cdots {k}_{32} \end{array}\right.\end{array}$$
(13)

H = {h1,h2,…,h8} is obtained from E, and the hash value K is as follows:

$$\begin{array}{c}\left\{\begin{array}{c}{h}_{1}={({e}_{2}\times K}_{1}\times {K}_{9}\times {K}_{17}\times {K}_{25})/256+{e}_{1}/{e}_{2}\\ {h}_{2}={({e}_{4}\times K}_{2}\times {K}_{10}\times {K}_{18}\times {K}_{26})/256+{e}_{3}/{e}_{4}\\ {h}_{3}={({e}_{6}\times K}_{3}\times {K}_{11}\times {K}_{19}\times {K}_{27})/256+{e}_{5}/{e}_{6}\\ {h}_{4}={({e}_{8}\times K}_{4}\times {K}_{12}\times {K}_{20}\times {K}_{28})/256+{e}_{7}/{e}_{8}\\ {h}_{5}={({e}_{2}+e}_{4})/256+{e}_{1}/{e}_{2}\\ {h}_{6}={({e}_{4}+e}_{6})/256+{e}_{3}/{e}_{4}\\ {h}_{7}={({e}_{6}+e}_{8})/256+{e}_{5}/{e}_{6}\\ {h}_{8}={({e}_{8}+e}_{2})/256+{e}_{7}/{e}_{8}\end{array}\right.\end{array}$$
(14)

The keys x0, y0, z0 and w0 are then generated and computed as follows:

$$\begin{array}{c}\left\{\begin{array}{c}{x}_{0}=\frac{(mod\left({h}_{1}+{h}_{5}\right)\times {10}^{4},256)}{256}\\ {y}_{0}=\frac{(mod\left({h}_{2}+{h}_{6}\right)\times {10}^{4},256)}{256}\\ {z}_{0}=\frac{\left(mod\left({h}_{3}+{h}_{7}\right)\times {10}^{4},256\right)}{256}\\ {w}_{0}=\frac{(mod\left({h}_{4}+{h}_{8}\right)\times {10}^{4},256)}{256}\end{array}\right.\end{array}$$
(15)

Using x0, y0, z0 and w0 as the initial values of Eq. (2), four chaotic sequences with length M × N + 1000 are iteratively generated. To eliminate transient effects, the first 1000 iterations are discarded, yielding four pseudorandom sequences of length M × N denoted X, Y, Z, and W.

$$\begin{array}{c}\left\{\begin{array}{c}X=\left\{x\left(1\right),x\left(2\right),\dots ,x\left(MN\right)\right\} \\ Y=\left\{y\left(1\right),y\left(2\right),\dots ,y\left(MN\right)\right\} \\ Z=\left\{z\left(1\right),z\left(2\right),\dots ,z\left(MN\right)\right\} \\ W=\left\{w\left(1\right),w\left(2\right),\dots ,w\left(MN\right)\right\}\end{array}\right.\end{array}$$
(16)

According to Eq. (17), the pseudorandom sequences X, Y, Z and W are converted into four pseudorandom sequences X1, Y1, Z1, and W1.

$$\begin{array}{c}\left\{\begin{array}{c}{X}_{1}=mod\left(floor\left(x\left(i\right)\times {10}^{4}\right),2\right) \\ {Y}_{1}=mod\left(floor\left(y\left(i\right)\times {10}^{4}\right),8\right)+1 \\ {Z}_{1}=mod\left(floor\left(z\left(i\right)\times {10}^{4}\right),8\right)+1 \\ {W}_{1}=mod\left(floor\left(y\left(i\right)\times {10}^{4}\right),256\right)+1\end{array}\right.\end{array}$$
(17)

Double-ended Thorp scrambling

Digital images typically exhibit a high degree of correlation between adjacent pixels; therefore, an enhanced encryption technique is needed to break this connection and increase the security of the algorithm. This paper constructs a double-ended Thorp scrambling technique that integrates Thorp shuffle and dequeue. In the Thorp shuffle algorithm, mixing a card evenly into I cards usually takes O(rlogI) steps, where r represents the number of shuffling rounds. In this paper, r is defined as 1, and the (logMN-2) steps of the scrambling process are performed. During a single round of shuffling, each step corresponds to a different pseudorandom sequence, which further enhances the safety of the scrambling process.

The following is a list of the specific steps in the scrambling encryption process:

  • Step 1: An M × N image matrix A1 is created from the original image A. If the image size does not satisfy the form of 2n, the image is padded with zeros at the edges to meet the condition of 2n.

  • Step 2: According to the method in Section "Key design and pseudorandom sequence generation", determine the hash value K of image matrix A1, and acquire the initial values of parameters x0, y0, z0 and w0. The improved 4D hyperchaotic system undergoes M × N + 1000 iterations. The first 1000 iterations are omitted to eliminate transient impacts, resulting in the generation of four pseudorandom sequences X, Y, Z and W with a length of M × N.

  • Step 3: Sort the pseudorandom sequence X in increasing order to generate the index sequence X0. According to the index sequence X0, image matrix A1 undergoes index scrambling to produce the scrambled image matrix A11. The scrambled image matrix is converted into a one-dimensional sequence B = {b11,b21,…bMN}.

  • Step 4: Process the pseudorandom sequence X to obtain the pseudorandom sequence X1 via Eq. (17). The pseudorandom sequence X1 is divided into two subsequences: XU = {x1(1),x1(2),…,x1(MN/2)} and XV = {x1(MN/2 + 1), x1(MN/2 + 2), …, x1(MN)}.

  • Step 5: Drawing on the parity of the scrambling step number k where k [1,logMN-2], the splitting operations for XU and XV are carried out in the following manner. When the scrambling step number k is odd, the pseudorandom sequence XU is divided into 2(k-1) groups of subsequences on average. To achieve this division, we first calculate the total number of elements in XU, which is M × N/2. Then, we evenly distribute these elements among the 2(k-1) groups. Each group contains s = (M × N)/2k elements. For the ith group, XUi = {XU((i-1)s + 1), XU((i-1)s + 2),…, XU (is)}. Conversely, when the scrambling step number k is even, the pseudorandom sequence XV is divided into 2(k-1) groups of subsequences on average; for the ith group, XVi = {XV((i-1)s + 1), XV((i-1)s + 2), …, XV(is)}.

  • Step 6: Divide the one-dimensional sequence B into two equal halves: B1 = {b1, b2,…b(MN/2)} and B2 = {b(MN/2+1),b(MN/2+2),…bMN}. First, create an empty dequeue with a size of M × N. This dequeue allows data to enter and exit from both ends, and we will put data into it later. Next, on the basis of the pseudorandom sequence XU1, the first-layer scrambling operation is performed on B1 and B2. The specific rules are as follows:

  • When the jth value in XU1 is 0, put the element at the jth position in B1 into the dequeue from the right side. At the same time, the jth element from the right in B2 is placed into the dequeue from the left side.

  • When the jth value in XU1 is 1, put the element at the jth position in B1 into the dequeue from the left side. At the same time, put the jth element from the right in B2 into the dequeue from the right side.

  • Keep operating according to the above rules until all the elements in B1 and B2 are successfully placed into the dequeue. At this point, save this complete dequeue and name it queue T.

  • Step 7: In the second round, the result T from the previous round is evenly divided into four sequences: T1, T2, T3, and T4. T1 and T2 are grouped together, and T3 and T4 are grouped together for the scrambling operation. First, create a dequeue with a size of (M × N)/2. The first group is scrambled according to the pseudorandom sequence XV1. When XV1(j) = 0, the jth element of T1 is placed into the dequeue from the right side. Simultaneously, the jth element from the right of T2 is placed into the dequeue from the left side. When XV1(j) = 1, the jth element of T1 is placed into the dequeue from the left side. Simultaneously, the jth element from the right of T2 is placed into the dequeue from the right side. The same operation is performed on the second group according to the pseudorandom sequence XV2. After all the elements are filled into the double-ended queues, the two double-ended queues are combined to form a sequence with a size of M × N.

  • Step 8: Repeat the above steps. In each round of the scrambling process, first, the corresponding pseudorandom sequence is generated according to the number of scrambling rounds. Then, the results of the previous round of scrambling are evenly divided into 2 k sequences. These sequences are grouped in pairs in order, and each pair of sequences is filled into a dequeue according to the pseudorandom sequence. Finally, the scrambled results are merged into a sequence with a size of M × N. This process continues until the logMN-2 scrambling steps are completed. After all scrambling operations are completed, the final sequence is rearranged in column order and converted into an M × N image matrix C.

Figure 12 shows the matrix scrambling steps, taking a 4 × 4 matrix as an example. In the case of 16 pixels, to obtain the final scrambled matrix, 2 layers of scrambling (log2(16)-2 = 2) have to be performed.

Fig. 12
figure 12

Double-ended Thorp scrambling.

Fourth-order bit-plane diffusion

To guarantee that every pixel value significantly affects the security of the final encrypted image, image diffusion attempts to change the pixel values by diffusing the variations in the original image to each pixel. Bit-level permutation can alter the location of a pixel and its associated value in the bit plane. This study presents a fourth-order bit-plane diffusion approach in conjunction with DNA encoding to further increase the security of the encrypted image as follows.

  • Step 1: Each pixel value in image matrix C is transformed to 8-bit binary form to obtain image matrix C1.

  • Step 2: Eq. (17) is applied to process the chaotic sequences Y and Z to generate random integer sequences between 1 and 8. These sequences form the encoding rule sequence Y1 and decoding rule sequence Z1, where each element corresponds to one of the eight rules for encoding the DNA shown in Table 1. Similarly, Eq. (17) is used to process the chaotic sequence W, obtaining a random integer sequence W1 with values ranging from 1 to 256. W1 is converted into a binary matrix, and then every two binary digits are grouped together and encoded via DNA encoding rule 1. This generates a matrix consisting of four DNA base combinations. From the DNA base matrix, the first bit of each four-base group is extracted to construct the diffusion matrix W11. Similarly, the second, third and fourth bits are sequentially extracted to construct the diffusion matrices W12, W13 and W14.

  • Step 3: According to the coding rule sequence Y1, perform DNA encoding on C1 from left to right. Each pixel’s eight-bit binary number in the matrix is encoded via the same encoding method, resulting in a matrix C2 composed of DNA base sequences.

  • Step 4: Four DNA bases are extracted one by one from matrix C2, and four DNA matrices C21, C22, C23 and C24 are extracted. are constructed. The diffusion operations are performed on these matrices according to Eq. (18), resulting in the diffused DNA matrices C31, C32, C33 and C34.

    $$C_{3k} (i,j) = \left\{ \begin{gathered} C_{2k} (i,j) - W_{1k} (i,j) + C_{24} (i,j),i = 1,j = 1,k = 1 \hfill \\ C_{2k} (i,j) - W_{1k} (i,j) + C_{3(k - 1)} (M,N),i = 1,j = 1,k \ne 1 \hfill \\ C_{2k} (i,j) - W_{1k} (i,j) + C_{3k} (i - 1,N),i \ne 1,j = 1 \hfill \\ C_{2k} (i,j) - W_{1k} (i,j) + C_{3k} (i,j - 1),i \ne 1 \hfill \\ \end{gathered} \right.$$
    (18)
  • Step 5: Combine the four diffused DNA matrices C31, C32, C33, and C34 in sequence to form a matrix where each position contains four DNA bases. The DNA sequence is decoded back into binary form via the decoding rule sequence Z1 and then converted into a decimal matrix C4, which represents the diffused encrypted image matrix.

Figure 13 shows the specific diffusion encryption process.

Fig. 13
figure 13

Fourth-order bit-plane diffusion.

Security analysis

This section presents simulations and tests of the proposed encryption algorithm via MATLAB R2018b. From the USC-SIPI image library, seven grayscale test images were selected: “Baboon”, “Goldhill”, “Pepper”, “Boat”, “House”, “Lake” and “Finger”. The sizes of the images “Baboon”, “Goldhill” and “Peppers” are 512 × 512, and the sizes of the images “Boat”, “House”, “Lake” and “Finger” are 256 × 256.

Figure 14 shows the plain, cipher, and decrypted images obtained via the proposed method. As illustrated in the figure, the cipher image shows an irregular noise pattern, and the plain image information cannot be extracted. The efficacy and dependability of the algorithm in both the encryption and decryption procedures are fully demonstrated by the decrypted image’s capacity to effectively restore the plain image.

Fig. 14
figure 14

Simulation results.

Analysis of the key space and sensitivity

Key sensitivity analysis serves to assess how minor alterations in the encryption key affect the outcome of the encryption process. In this study, the “Boat” image is utilized as a case in point: the encrypted key is decrypted after adding a small perturbation of 10–15. As a result, the plain image cannot be recovered. Figure 15 shows the decryption outcomes across various keys, which demonstrates that the algorithm has a high degree of sensitivity to key changes.

Fig. 15
figure 15

Key sensitivity analysis.

To resist brute-force cracking attacks, image encryption algorithms need to have a sufficiently large key space. Research indicates45 that the key space length should be greater than 2100. When the computational accuracy of this algorithm is set to 10–15, for the initial values x0, y0, z0, w0 and parameters a, b, c, d, e, the key space can be estimated as (1015)9 = 10135 ≈ 2448. The results indicate that the proposed algorithm’s key space is sufficient for resisting brute-force attacks.

Statistical analysis

Analysis of histograms

Histograms are intuitive tools for assessing the pixel distribution in an image. Specifically, the histogram comparison of the plain and cipher images is presented in Fig. 16(a) and (c). The polar histograms of the plain and cipher images are further analyzed in Fig. 16(b) and (d), which provide additional insights into the pixel distribution characteristics in a different geometric representation. The experimental results show that the plain image has an uneven pixel distribution, with significant fluctuations in the histogram, whereas the cipher image has a highly uniform pixel distribution, and its histogram shows no distinct peaks. The information of the cipher image is difficult to obtain through a statistical attack, and the effectiveness of the encryption algorithm has been verified.

Fig. 16
figure 16

Histogram (a) histogram of plaintext images Baboon, Goldhill, Pepper, Boat, House, Lake, and Finger; (b) polar histogram of each corresponding plaintext image; (c) histogram of each corresponding ciphertext image; (d) polar histogram of each corresponding ciphertext image.

\({{\varvec{\chi}}}^{2}\) Test

The \({\chi }^{2}\) test performs a quantitative analysis of histogram uniformity. Its calculation formula is as follows:

$$\begin{array}{c}{\chi }^{2}={\sum }_{i=0}^{256}\frac{({O}_{i}-{E}_{i}{)}^{2}}{{E}_{i}},\end{array}$$
(19)

where Oi is the frequency of gray values in the histogram and \({E}_{i}=\frac{M\times N}{256}\) is the expected frequency of grayscale values. The criterion for evaluating histogram homogeneity is \({\chi }^{2}\le 293.2478\) when the significance level is α = 0.05. A lower \({\chi }^{2}\) suggests a more uniform distribution in the histogram46. The \({\chi }^{2}\) values of several cipher pictures are shown in Table 8. All cipher images passed the \({\chi }^{2}\) test, demonstrating that their histogram distributions are quite uniform and can effectively withstand statistical attacks.

Table 8 \({\chi }^{2}\) Test.

Correlation analysis of adjacent pixels

In plain images, adjacent pixels typically exhibit high spatial and grayscale correlations. An effective encryption algorithm should break this correlation. The following formula is used to calculate the adjacent pixel correlation coefficient γ:

$$\begin{array}{c}\left\{\begin{array}{c}{r}_{xy}=\frac{cov\left(x,y\right)}{\sqrt{D\left(x\right)D\left(y\right)}}\\ E\left(x\right)=\frac{1}{N}\sum_{i=1}^{N}{x}_{i}\\ D\left(x\right)=\frac{1}{N}\sum_{i=1}^{N}{[{x}_{i}-E(x)]}^{2}\\ cov\left(x,y\right)=E\left(x-E\left(x\right)\right)(y-E(y))\end{array}\right.\end{array}$$
(20)

where cov(x, y) is the covariance between gray values x and y, rxy is the correlation coefficient, E() is the chosen sample’s expectation, and D() is the chosen sample’s variance. Figure 17 shows the correlation analysis findings when the “Baboon” image is used as an example. For the plain image, there is a highly apparent linear relationship between adjacent pixels. In contrast, for the cipher image, there is almost no discernible correlation between adjacent pixels. This significant difference verifies the excellent anti-correlation analysis ability of the algorithm.

Fig. 17
figure 17

Baboon image correlation analysis.

Table 9 shows the correlation coefficients between adjacent pixels in the cipher and plain images. The correlation coefficient between neighboring pixels before encryption is close to 1, and after encryption, it is close to 0, which shows that this method can greatly reduce the correlation of the image.

Table 9 Adjacent pixel correlation coefficients.

The correlation coefficients between the proposed algorithm and other algorithms are compared in Table 10. The outcomes demonstrate that the suggested algorithm performs better at reducing the correlation between adjacent pixels.

Table 10 Comparison of Baboon cipher image correlation coefficients.

Information entropy and local Shannon entropy analysis

The information entropy measures the randomness of image pixel values and is defined as:

$$\begin{array}{c}H\left(S\right)=\sum_{i=0}^{N}p\left({s}_{i}\right){log}_{2}\frac{1}{p\left({s}_{i}\right)}\end{array}$$
(21)

where p(si) is the likelihood that the gray value si will appear in the image, and N is the image’s gray level. The information entropy is positively correlated with the unpredictability of the pixel values, with a theoretical value of 8 for grayscale images. In practice, the stronger the algorithm’s encryption effect is, the closer the information entropy is to 8.52 Table 11 shows the information entropy of different plain images and cipher images and provides a comprehensive comparative analysis of other methods. According to the study’s results, this algorithm’s cipher image’s information entropy is greater than that of previous encryption methods, approaches the theoretical value of 8, and can successfully fend off an information entropy attack.

Table 11 Information entropy comparison.

The local unpredictability of the image’s pixel values is measured by the local Shannon entropy, which is defined as follows:

$$\begin{array}{c}\overline{{H }_{k,{T}_{B}}\left(S\right)}=\sum_{i=1}^{k}\frac{H\left({S}_{i}\right)}{k}\end{array}$$
(22)

where Si is the cipher image information entropy, TB is the number of pixels in each block, and k is the number of image blocks. Taking k = 30 and TB = 1936, the cipher picture passes the local information entropy test when α = 0.001, and the local information entropy value falls between (7.901515698, 7.903422936)54. Table 12 shows various cipher image local information entropy test results. All cipher images passed the test, indicating that the cipher images exhibit a high level of randomness and can effectively defend against statistical analysis attacks.

Table 12 Test of the local Shannon entropy.

Floating frequency analysis

Floating frequency analysis is used to evaluate the global randomization capability of an encryption method. Specifically, it measures the security of the encryption method by examining whether the algorithm can generate uniformly distributed random data in all regions of a plain image. A higher floating frequency value with a more uniform distribution indicates stronger security and resistance to statistical analysis55. If localized regions manifest low-frequency peaks in the analysis results, this indicates insufficient randomization in those areas during encryption. Using a 256 × 256 grayscale image as an example, this section partitions the encrypted image into statistical windows of 256 elements each on the basis of row and column data. The Row Floating Frequency (RFF) and Column Floating Frequency (CFF) are analyzed by quantifying the dispersion of the pixel distributions within each window. The specific computational procedure is outlined as follows:

  1. 1.

    For all rows and columns of the image, windows containing 256 consecutive pixels each are selected.

  2. 2.

    The number of unique pixel values in each window is counted, and RFF and CFF are calculated on the basis of their corresponding frequencies.

  3. 3.

    Compute the mean values of RFF and CFF and generate corresponding visualizations of their frequency distributions.

Figure 18 shows the comparison of RFF between plain and cipher images via four grayscale examples: “Boat”, “House”, “Lake” and “Finger”. After the mean floating frequency is converted into a percentage form, the RFFs of each plain image are 34% (89.35/256), 28%, 40% and 50%, and the RFFs of the corresponding cipher images are all increased to 63%. The results show that the percentage increase in the RFF of each channel reaches 29%, 35%, 23% and 13%, respectively, during the conversion from the plain image to the cipher image. This finding demonstrates that the proposed encryption algorithm achieves better random distribution characteristics in the row dimension, which is in line with the requirement that the encrypted image should have high randomness.

Fig. 18
figure 18

Row floating frequency.

Similarly, Fig. 19 shows the CFF comparison between plain and cipher images using the same four grayscale examples: “Boat”, “House”, “Lake”, and “Finger”. The analysis reveals that the CFF of cipher images is significantly greater than that of plain images, with a notable increase in the number of unique pixel values per column. The mean floating frequency percentages for the images increase. They range from 40%, 30%, 41%, and 52% in plain images to 63% in cipher images. This finding verifies that the proposed encryption algorithm also fulfills the high randomness requirements for encrypted images in the column dimension.

Fig. 19
figure 19

Column floating frequency.

Differential attack analysis

As a selective plaintext attack, a differential attack centers on making small modifications to the plain image to generate two cipher images and then analyzes the differences between the two to break the encryption algorithm. The uniform average change intensity (UACI) and pixel value change rate (NPCR), which are described below, are crucial metrics for assessing an algorithm’s resistance to differential attacks:

$$\begin{array}{c}\left\{\begin{array}{c}NPCR\left({C}_{1},{C}_{2}\right)=\frac{\sum_{i,j}D\left(i,j\right)}{M\times N}\times 100\%\\ D\left(i,j\right)=\left\{\begin{array}{c}1,{C}_{1}\left(i,j\right)\ne {C}_{2}\left(i,j\right)\\ 0,{C}_{1}\left(i,j\right)={C}_{2}\left(i,j\right)\end{array}\right.\end{array}\right.\end{array}$$
(23)
$$\begin{array}{c}UACI\left({C}_{1},{C}_{2}\right)=\frac{1}{M\times N}\left[\sum_{i,j}\frac{\left|{C}_{1}\left(i,j\right)-{C}_{2}\left(i,j\right)\right|}{Q}\right]\times 100\%\end{array}$$
(24)

where C1 and C2 are cipher images encrypted by different plain images having a 1 bit difference and where Q represents the maximum value of the pixel. When the significance level α = 0.05, for 8-bit grayscale images, the theoretical value of the NPCR is 99.5693%, and the theoretical interval of the UACI is (33.2824%, 33.6447%)47. Table 13 shows the NPCR and UACI results. A comparison of the suggested novel algorithm with other current algorithms is shown in Table 14. The outcome reveals that when contrasted with the test results of other algorithms, the values of the proposed algorithm are closer to the theoretical values. Additionally, our proposed algorithm can effectively fend off differential attacks.

Table 13 NPCR and UACI tests.
Table 14 Comparison of the NPCR and UACI tests.

Analysis of clipping and noise attack

During data transfer, the cipher image might be discarded due to various factors or subjected to noise attack. To preserve the quality and integrity of the decrypted image, the image encryption algorithm should possess the capacity to withstand different types of attacks. Even when the cipher image is under attack, valuable information can still be recovered following decryption. This experiment uses the “Boat” image as an example for analysis: the cipher image was decrypted after adding salt and pepper noise with intensities of 0.01, 0.05 and 0.1. The result still maintained high image quality and retained the important information and characteristics of the image.

Figure 20 shows the decryption results after different salt and pepper noise attacks, confirming the ability of the suggested algorithm to cope with noise attacks.

Fig. 20
figure 20

Noise attacks of varying intensity.

Figure 21 shows the cipher image and the corresponding decrypted image with missing values of 1/16, 1/8, 1/4 and 1/2 in the middle part. Critical parts of the image can be restored via decryption, demonstrating the algorithm’s effectiveness against clipping attacks.

Fig. 21
figure 21

Clipping attack of different sizes.

In image encryption, the peak signal-to-noise ratio (PSNR) is an important indicator for measuring the quality loss of cipher images. The PSNR measures the amount of visual distortion that occurs during the encryption and decryption procedures by comparing the signal-to-noise ratios of the plain and decrypted images. A greater PSNR value means that there is less distortion and a decrease in the gap between the plain and decrypted images56. This indicates that the algorithm is more effective in maintaining image quality. The mean square error (MSE) quantitatively assesses the pixel-level discrepancy between the plain and decrypted images. A greater difference between the decrypted and plain images is indicated by a larger MSE value. The combined evaluation of PSNR and MSE allows a comprehensive assessment of the impact of encryption algorithms on image quality.

$$\begin{array}{c}MSE=\frac{1}{M\times N}{\sum }_{i=1}^{N}{\sum }_{j=1}^{M}{\left(P\left(i,j\right)-I\left(i,j\right)\right)}^{2},\end{array}$$
(25)
$$\begin{array}{c}PSNR=10\times {log}_{10}\left(\frac{{255}^{2}}{MSE}\right),\end{array}$$
(26)

Here, I is the decrypted image, and P is the plain image.

Table 15 shows the PSNR and MSE values of the noise attack and the clipping attack. The results show that the decrypted image maintains high PSNR values and low MSE values even when subjected to noise or clipping attacks with greater intensity. This indicates that the encryption algorithm is highly resistant to cropping and noise attacks. It can preserve the encrypted image’s visual quality while successfully preserving its essential details and attributes.

Table 15 PSNRs and MSEs under noise attack and clipping attack.

Complexity analysis

Complexity analysis is an important component in the evaluation of image encryption algorithms. Low-complexity algorithms can significantly reduce the encryption processing time, thereby reducing device computational resource consumption and improving real-time processing capability. To exclude the performance differences among different computers, we analyze only the complexity theoretically. For a grayscale image of size M × N, the double-ended Thorpe scrambling process involves a total of log(M × N-2) rounds of operations. Each round requires traversing M × N pixels. Thus, its time complexity is O((M × N) log(M × N)). In the diffusion process, four channels are computed in parallel with a time complexity of O(M × N). Therefore, the overall complexity of the proposed algorithm is O((M × N) log(M × N)).

Conclusions

This study establishes a four-dimensional hyperchaotic system and validates its remarkable chaotic attributes via a range of techniques, such as phase diagrams, Lyapunov exponents, dissipation properties, equilibrium point analysis, initial value sensitivity, and NIST tests. On this basis, a novel scrambling method integrates Thorp scrambling and dequeues. Additionally, a fourth-order bit-plane diffusion strategy is developed on the basis of DNA encoding. Together, these two components constitute a comprehensive image encryption scheme. Security analysis and simulation experiments are used to fully evaluate the proposed encryption algorithms. The findings indicate that the algorithm excels in image encryption, demonstrating robust resistance to prevalent security threats. This greatly increases the unpredictability and randomness of encrypted data. Furthermore, the algorithm suggested in this research clearly outperforms current techniques in terms of image encryption security, demonstrating robust resistance to decryption attempts. Moreover, compared with existing methods, the algorithm proposed in this paper has a clear advantage in terms of image encryption security, showing strong resistance to decryption attempts.

However, the research in this paper has focused mainly on grayscale images, and the application scope still has certain limitations. Future research will further expand the algorithm’s application to color image encryption and optimize the algorithm on the basis of practical needs to improve the encryption speed and computational efficiency, thereby meeting a broader range of image encryption application scenarios.