In order to better understand the technical solutions, the technical solutions will be described in detail with reference to the drawings in the specification and specific embodiments.

Embodiment 1

Integrated energy systems (IESs), as a new energy supply mode based on energy utilization, conversion and transmission technologies, realize flexible conversion, efficient distribution and organic coordination of various types of energy by coupling multiple energy forms such as electricity, gas, cold, heat, and water. In IESs, various types of energy are supplied complementarily through a complex coupling mechanism, so that factors affecting various types of loads requirements in IESs are more complex, and it is also more difficult to guarantee balanced supply requirements of various types of energy. In order to make an accurate short-term forecasting of the multi-energy loads in the IESs, the following method is proposed.

In a first aspect, the disclosure first provides a multi-energy integrated short-term load forecasting method. Referring to FIG. 1, the method includes:

- S1. performing pretreatment and correlation calculation on acquired relevant data of multi-energy integrated short-term load forecasting to determine input data of a multi-energy integrated short-term load forecasting model, and performing sample classification on the input data; wherein the relevant data of multi-energy integrated short-term load forecasting includes multi-energy load historical data and external environment historical data corresponding to the multi-energy load historical data;
- S2. acquiring a multi-energy integrated short-term load forecasting model constructed based on encoder-decoder; wherein the encoder includes multiple layers of temporal convolutional networks having multi-head self-attention mechanism and rotary position embedding;
- S3. training the multi-energy integrated short-term load forecasting model based on the input data after the sample classification, and performing the multi-energy integrated short-term load forecasting by the trained multi-energy integrated short-term load forecasting model.

It can be seen that in the present embodiment, after obtaining the relevant data of multi-energy integrated short-term load forecasting, the sample classification data is used to train the multi-energy integrated short-term load forecasting model with multiple layers of temporal convolutional networks having multi-head self-attention mechanism and rotary position embedding, and finally the trained model is used to carry out the multi-energy integrated short-term load forecasting. The present embodiment can fully mine the coupling feature between multi-energy loads, improve the accuracy of multi-energy integrated short-term load forecasting, and further improve the management level and service efficiency of integrated energy demand side.

The implementation process of an embodiment of the present disclosure will be described in detail with reference to FIG. 1 and the explanation of the specific steps of S1-S3. Referring to FIG. 1, the specific process of a multi-energy integrated short-term load forecasting method in this embodiment is as follows:

This embodiment takes the three energy loads of water, electricity, and gas (that is, multi-energy load) as an example, and uses the data of the past 7 days to forecast the hourly multi-energy load data of the next 7 days, i.e., the forecasting step length is 168 (24 h*7). The specific steps of the multi-energy integrated short-term load forecasting method in this embodiment are as follows:

- S1. Pretreatment and correlation calculation are performed on acquired relevant data of multi-energy integrated short-term load forecasting to determine input data of a multi-energy integrated short-term load forecasting model, and sample classification is performed on the input data. The relevant data of multi-energy integrated short-term load forecasting includes multi-energy load historical data and external environment historical data corresponding to the multi-energy load historical data.
- 1) The relevant data of multi-energy integrated short-term load forecasting is acquired. The relevant data includes multi-energy load historical data and external environment historical data corresponding to the multi-energy load historical data. In the present embodiment, among the selected relevant data, the external environment historical data provided by the weather station is sufficient, with a total of 12 variables, including temperature, humidity, wind speed, etc.
- 2) The pretreatment is performed on the multi-energy load historical data and external environment historical data. The operations such as missing value filling and normalization processing are performed on the multi-energy load historical data and the external environment historical data. After pre-cleaning the historical data of multi-energy loads and filling the missing values, all the multi-energy load historical data is sorted into a data set with a time interval of 1 h to form the multi-energy load historical data. For the missing external environmental historical data, such as temperature, humidity, and wind speed, measures need to be taken to make up for the missing values of these meteorological data, and the external environmental historical data sequence can be obtained after pretreating operations such as missing value filling and normalization processing.
- 3) A correlation coefficient of the multi-energy load historical data sequence is calculated to determine the input data of the multi-energy integrated short-term load forecasting model. Specifically, a correlation inside the multi-energy load historical data sequence and a correlation between the multi-energy load historical data and the external environment historical data are compared. If the correlation inside the multi-energy load historical data sequence is greater than the correlation between the multi-energy load historical data and the external environment historical data, the multi-energy load historical data sequence is used as an input only. On the contrary, if the correlation inside the multi-energy load historical data sequence is greater than 0.2 but less than the correlation between the multi-energy load historical data and the external environment historical data, then the multi-energy load historical data sequence and the external environment historical data sequence are simultaneously used as inputs of the multi-energy integrated short-term load forecasting. In this embodiment, Pearson correlation coefficient is selected for correlation evaluation, and then the environmental variables with coefficient less than 0.2 are selected and discarded. In this embodiment, the final selected input variables include electric power load, gas load, water load, the temperature, and humidity data of the corresponding period.
- 4) The above pretreated input data are classified into a training set, a validation set and a test set. In this embodiment, the data set is divided into training set, validation set and test set according to the proportion of 70%, 20%, and 10% respectively.
- S2. A multi-energy integrated short-term load forecasting model constructed based on encoder-decoder is acquired. The encoder includes multiple layers of temporal convolutional networks having multi-head self-attention mechanism and rotary position embedding.

The multi-energy integrated short-term load forecasting model provided by the present embodiment is constructed based on encoder-decoder, and the encoder is used to extract the input feature of the input data of multi-energy integrated short-term load forecasting. The encoder is composed of multiple layers of temporal convolutional networks having multi-head self-attention mechanism and rotary position embedding.

S21. The input data of multi-energy integrated short-term load forecasting is converted into a feature matrix.

In the network embedding layer of the multi-energy integrated short-term load forecasting model, the input data of multi-energy integrated short-term load forecasting is converted into a feature matrix, which represents the numerical information and time information of the input data, and the feature matrix of the input sequence X={x₀, x₁, . . . , x_t} is obtained based on the input data. In this embodiment, the input sequence is X={x₀, x₁, . . . , x₁₆₈}, and the dimension of the feature matrix is L_x×512.

The representation of numerical information in the feature matrix means that the input variable

x
t
i

is encoded as a vector

u
t
i

of d=512 dimensions by converting the input data vector into a matrix through convolution. The representation of time information refers to encoding the time corresponding to each group of multi-energy load historical data in the input sequence into time vector according to date, time, working day or not. Specifically, in the embedding layer, each global timestamp is embedded by SE_(L_x_×(t−1)+i). Finally, the numerical information and time information of the input data are integrated as follows:

X
t
i

=

α
⁢

u
t
i

+

S
⁢

E

(

L
x

×

(

t
-
1

)

+
i

)

,

wherein, i∈{1, . . . , L_x}, L_xis the dimension of the input variable, α is the coefficient to balance the size between scalar projection and local/global embedded data. After the embedding layer feature encoding, the input sequence X={x₀, x₁, . . . , x_t} is shaped into a set of feature matrices X^en={X₁^en, X₂^en, . . . , X_t^en}, where

X
j

e
⁢
n

=

(

X
j
1

,

X
j
2

,
…

,

X
j

L
x

)

∈

▯

L
x

×
d

,

j
∈

{

1
,
…

,
t

}

.

In this embodiment, i∈{1, . . . ,L_X} is i∈{1,2,3,4,5}. After passing through the embedding layer, the input sequence X={x₀, x₁, . . . , x₁₆₈} is shaped into a set

X

e
⁢
n

,

X

t

e
⁢
n

=

(

X
t
1

,
 

X
t
2

,
 

X
t
3

,
 

X
t
4

,
 

X
t
5

)

∈

▯

5
×
5
⁢
1
⁢
2

.

S22. The encoder is constructed by temporal convolutional networks having self-attention mechanism and rotary position embedding, and based on the encoder, a coupling feature of the input data of multi-energy integrated short-term load forecasting mining is mined.

An encoder is constructed based on the temporal convolutional networks having multi-head self-attention mechanism. The encoder is used to extract the hidden features of the feature matrix, that is, the coupling information features between data. When constructing the encoder, this embodiment extracts the features of the input data (feature matrix) by stacking the coding layers of the L temporal convolutional networks (TCN), where the expansion coefficient r=2^l. In addition, each encoding layer also contains the multi-head attention mechanisms for feature extraction, that is, multiple attention mechanisms for parallel operation. In particular, in order to extract the coupling features of the multi-energy load historical data sequence, this embodiment introduces multi-head self-attention mechanism with rotary position embedding into TCN under the encoder-decoder deep learning framework to improve the forecasting effect of multi-energy load forecasting model, where the encoder reads the feature matrix

X
j

e
⁢
n

∈

X

e
⁢
n

and encodes it into a vector with fixed dimensions H, and

H
=

f
⁡
(

X
j

e
⁢
n

)

is the encoding state of the vector

X
j

e
⁢
n

=

(

X
j
1

,

X
j
2

,
 
…

,

X
j
n

)

∈

▯

n
×
d

,

j
∈

{

1
,
…

,
t

}

.

In this embodiment, when constructing the encoder, the input features are extracted by stacking the L=3 of encoding layers of TCNs, and each encoding layer also contains a multi-head self-attention mechanism for feature extraction. The encoder reads the input feature matrix

X
i

∈

{

X
1

e
⁢
n

,

X
2

e
⁢
n

,
…

,

X
t

e
⁢
n

}

and encodes it into a vector H with 512 dimensions, and

H
=

f
⁡
(

X
j

e
⁢
n

)

is the encoding state of the vector

X
j
en

=

(

X
j
1

,

X
j
2

,

X
j
3

,

X
j
4

,

X
j
5

)

∈

▯

5
×
512

.

The specific encoding process is as follows:

S211. An attention matrix is constructed based on the feature matrix, and the attention matrix includes query matrix, key matrix and value matrix.

In the self-attention mechanism, three matrices, Q, K, and V are obtained through the following operations based on the feature matrix:

[

Q

K

V

]

=

[

X
t

X
t

X
t

]

T

[

W
Q

W
K

W
V

]

,

wherein,

Q
∈

▯

L
Q

×
d

,

K
∈

▯

L
K

×
d

,

and
⁢

V

∈

▯

L
×
d

respectively represents the query matrix, key matrix and value matrix,

W
Q

∈

▯

d
×
d

,

W
K

∈

▯

d
×
d

,

and
⁢

W
V

∈

▯

d
×
d

respectively represents the weight parameter matrix of the query matrix, key matrix and value matrix. In this embodiment, when L=3, the query matrix, key matrix, and value matrix are

Q
∈

▯

5
×
5
⁢
1
⁢
2

,

K
∈

▯

5
×
5
⁢
1
⁢
2

,

and
⁢

V

∈

▯

5
×
12

respectively; the weight matrices of query matrix, key matrix and value matrix are

W
Q

∈

▯

5
⁢
1
⁢
2
×
5
⁢
1
⁢
2

,

W
K

∈

▯

5
⁢
1
⁢
2
×
5
⁢
1
⁢
2

,

and
⁢

W
V

∈

▯

5
⁢
1
⁢
2
×
5
⁢
1
⁢
2

,

respectively. In addition, during parameter initialization in this embodiment, the weight parameters are randomly initialized.

S222. Rotary position embedding is added to the query matrix and the key matrix.

Rotary position embedding is added to the attention matrix to identify the specific position of the feature. Specifically, in this embodiment, when d=512, the attention mechanism with rotary position embedding is applied to both the query matrix and the key matrix in the following way:

f
q

(

X
m

⁢

W
Q

,
m

)

=

R

Θ
,
m

d

⁢

X
m

⁢

W
Q

=

R

Θ
,
m

5
⁢
1
⁢
2

⁢

X
m

⁢

W
Q

,

f
k

(

X
n

⁢

W
K

,
n

)

=

R

Θ
,
n

d

⁢

X
n

⁢

W
K

=

R

Θ
,
n

5
⁢
1
⁢
2

⁢

X
n

⁢

W
K

,

- wherein,

R

Θ
,
m

d

=

[

M
1

M
2

⋱

M

d
/
2

]

=

R

Θ
,
m

512

=

[

M
1

M
2

⋱

M

512
/
2

]

.

And the elements M_rin the R_⊖,m^dis:

M
r

=

[

cos
⁢

m
⁢

θ
r

-

sin

⁢

m
⁢

θ
r

sin
⁢

m
⁢

θ
r

cos
⁢

m
⁢

θ
r

]

- and

Θ
=

{

θ
r

=

1
⁢
0
⁢
0
⁢
0
⁢

0

-
2

⁢

(

r
-
1

)

/
d

,

r
∈

[

1
,
2
,
…

,

d
/
2

]

}

=

{

θ
r

=

1
⁢
0
⁢
0
⁢
0
⁢

0

-
2

⁢

(

r
-
1

)

/
5

⁢
1
⁢
2

,

r
∈

[

1
,
2
,
…

,

512
/
2

]

}

,

and
⁢

⁢

f

q
,
k

(
·
)

⁢

(

i
.
e
.

,

f
q

⁢

and
⁢

f
k

)

is the function that combines the respective position information m and n into the input feature.

S223. An attention score between the query matrix and the key matrix after adding rotary position embedding is calculated.

The attention score is calculated based on the attention matrix with position information. The attention score (A_i,j) between the i-th query element and the j-th key element is calculated as:

A

i
,
j

=

(

Q
p

⁢

K
p
T

)

i
,
j

=

(

[

f
q

(

X
m

⁢

W
Q

,
m

)

]

[

f
k

(

X
n

⁢

W
K

,
n

)

]

T

)

i
,
j

,

- wherein, Q_pand K_pare the query matrix and key matrix with position information respectively.

S224. The query matrix is updated according to a sparsity measure of the query matrix to obtain an attention query matrix.

The sparsity measure of the query attention matrix is the attention matrix with large query weight (that is, large attention score). The weight parameter with the best score ranking is retained, and the other parameters are replaced by the parameters corresponding to the global average score. In this embodiment, the weight parameters of the top 96 scores are retained, and the rest are replaced by the weight parameters corresponding to the global average score. Specifically, the attention query matrix Q is obtained by calculating the sparsity M(q_i, K) of the query matrix in the following way:

M
_

(

q
i

,
K

)

=

max
j

{

q
j

⁢

k
j
T

d

}

-

1

L
K

⁢

∑

j
=
1

L
κ

q
j

⁢

k
j
T

d

,

- wherein, the attention query matrix Q has the same size as the query matrix Q, but Q only contains the query results of the i-th query element q_iunder the maximum mean sparsity measure; q_irepresents an element in the query matrix Q.

S225. An attention mechanism is executed on the key matrix, the value matrix and the updated attention query matrix to mine the coupling feature of the input data of multi-energy integrated short-term load forecasting.

The attention mechanism is executed again on the attention matrix with large attention calculation score (i.e., large weight). The attention mechanism is executed in the following way:

A
⁡
(

Q
p

,
K
,
V

)

=

Softmax
⁢

(

Q
_

⁢

K
T

d

)

⁢

V
.

S23. A redundant feature in the coupling feature is eliminated based on a distillation operation.

In order to eliminate the redundancy in the feature mapping V caused by the encoder after the multi-head attention mechanism, and thus eliminate the redundant feature caused by data expansion, the following distillation operation are performed between the j-th encoding layer and the (j+1)-th encoding layer (i.e., between every two encoding layers):

X

t
,

j
+
1

=

MaxPool
⁡
(

ELU
⁡
(

Conv
⁢
1
⁢

d
⁡
(

[

X

t
,
j

]

AB

)

)

)

,

- wherein,

[

x

t
,
j

]

A
⁢
B

represents the attention matrix after the above attention mechanism operation, and Conv1d(·) represents the execution of 1×1 convolution filter in the time dimension by ELU(·). All stacked outputs are connected through the expansion convolution of TCN, and the final output H of the encoder is obtained.

S24. A decoder is constructed. The decoder is arranged for obtaining an output data of multi-energy integrated short-term load forecasting after the coupling feature from which the redundant feature have been eliminated is decoded by the decoder.

The decoder is constructed. The decoder has two input sources: the feature matrix output by the encoder and the time sequence information to be forecasted. Through performing self-learning on the time sequence needing to be forecasted and performing cross learning on the time sequence and the feature matrix, the load sequence needing to be forecasted is decoded. In the decoding process, the decoder decodes H into target output sequence Y={Y₀, Y₁, . . . , Y_t}, measure the weight parameters of the feature map by combining the self-attention mechanism and the cross-attention mechanism, and generate the forecasting output elements in real time. Similar to the encoder, residual connection is used between each decoder, and then layer normalization is performed. The following vector is input to the decoder:

X
t
de

=

Concat
⁡
(

X
t
token

,

X
t
0

)

∈

▯

(

L
token

+

L
y

)

×
d

,

- wherein,

X
t

t
⁢
o
⁢
k
⁢
e
⁢
n

∈

▯

L
token

×
d

is the starting mark of the decoder input,

X
t
0

∈

▯

L
token

×
d

is the placeholder of the target sequence, and is initialized to 0.

In this embodiment, the number of layers of the decoder is set to 2, and the decoder is used to decode H into the target output sequence Y={Y₀, Y₁, . . . , Y₁₆₈}. Then the decoder input:

X
t

d
⁢
e

=

Concat
⁡
(

X
t

t
⁢
o
⁢
k
⁢
e
⁢
n

,

X
t
0

)

∈

▯

(

L
token

+

L
y

)

×
d

X
t

d
⁢
e

=

Concat
⁡
(

X
t

t
⁢
o
⁢
k
⁢
e
⁢
n

,

X
t
0

)

∈

▯

(

84
+

1
⁢
6
⁢
8

)

×
5
⁢
1
⁢
2

,

- wherein, the starting mark

X
t

t
⁢
o
⁢
k
⁢
e
⁢
n

∈

▯

L
token

×
d

of the decoder input is

X
t

t
⁢
o
⁢
k
⁢
e
⁢
n

∈

▯

8
⁢
4
×
5
⁢
1
⁢
2

,

- and the placeholder

X
t
0

∈

▯

L
token

×
d

of the target sequence is

X
t
0

∈

▯

8
⁢
4
×
5
⁢
1
⁢
2

.

S3. The multi-energy integrated short-term load forecasting model is trained based on the input data after the sample classification, and the multi-energy integrated short-term load forecasting is performed by using the trained multi-energy integrated short-term load forecasting model.

The encoder and decoder in the multi-energy integrated short-term load forecasting model are jointly trained. During training, the output of the encoder is also input into the decoder. After the decoder completes the output, the encoder and decoder are updated with parameters according to the output results. After the model is trained once with the data in the training set, the data of the validation set is used to forecast and save the model parameters and forecasting errors. In this way, the iteration is carried out and the forecasting errors are compared in the verification set each time. Finally, only the model parameters with the lowest error are retained. The model with the lowest error is used to test the forecasting accuracy of the model in the test set, and finally the conditional probability of the output sequence is maximized. In the model training, the loss function is taken as the optimization objective. This embodiment adopted mean square error (MSE) as the loss function:

MSE

=

1
N

⁢

∑

i
=
1

N

(

P
i

-

P
^

i

)

2

.

In the present embodiment,

MSE

=

1
N

⁢

∑

i
=
1

N

(

P
i

-

P
ˆ

i

)

2

=

MSE

=

1

1
⁢
6
⁢
8

⁢

∑

i
=
1

1
⁢
6
⁢
8

(

P
i

-

P
ˆ

i

)

2

,

wherein, P_iis the true value of the actual short-term load; {circumflex over (P)}_iis the forecasting value.

Considering that the performance of the Adam optimizer is good, the present embodiment chooses to apply the optimizer to the processed model training and iteratively correct the weight parameters until the forecasting error does not decline.

After the multi-energy integrated short-term load forecasting model is trained, the input data of multi-energy integrated short-term load forecasting can be input into the trained model to achieve the multi-energy integrated short-term load forecasting. The final result is the target output sequence Y={Y₀, Y₁, . . . , Y_t} of short-term load forecasting value, in which Y₀, Y₁, . . . , Y_tcorresponds to the hourly forecasting value of multi-energy integrated short-term load forecasting respectively.

So far, the whole process of the multi-energy integrated short-term load forecasting method has been completed.

In order to illustrate the effectiveness of the method in the present embodiment for the multi-energy integrated short-term load forecasting, the following experiments are carried out to prove it.

The forecasting results of single-energy load and multi-energy load are compared. In single-energy load forecasting, the input of the model does not consider the other two load factors. The root mean square error (RMSE) and mean square error (MSE) are used as the evaluation criteria of the forecasting model:

MSE

=

1

1
⁢
6
⁢
8

⁢

∑

i
=
1

1
⁢
6
⁢
8

(

P
i

-

P
ˆ

i

)

2

,

RMSE
=

1

1
⁢
6
⁢
8

⁢

∑

i
=
1

1
⁢
6
⁢
8

(

P
i

-

P
ˆ

i

)

2

,

- wherein, P_iand {circumflex over (P)}_iare actual load data and forecasting load data respectively. The comparison results are shown in Table 1:

TABLE 1

Comparison of results of single-energy load forecasting and multi-

energy load forecasting by the method in the present embodiment

Electricity
Gas
Water

Method
MSE
RMSE
MSE
RMSE
MSE
RMSE

Single-energy
1.0370
1.0183
0.6526
0.8078
0.5349
0.7314

load

forecasting

Multi-energy
0.9819
0.9828
0.6013
0.7754
0.5227
0.7230

load

forecasting

It can be seen that the forecasting accuracy of load forecasting considering multi-energy coupling feature is better than that of single-energy load forecasting, and the method proposed in the present embodiment can obtain the forecasting results of all loads such as electricity, gas, and water. at one time, and the model training takes less time.

In order to verify the effectiveness of the multi-energy load short-term forecasting method in the present embodiment, the forecasting error of the current commonly used methods is selected as the main comparison index to compare the effects of the forecasting methods, as shown in Table 2 below:

TABLE 2

Comparison of the results of multi-energy load

forecasting between the method in the present

embodiment and the existing conventional methods

Electricity
Gas
Water

Method
MSE
RMSE
MSE
RMSE
MSE
RMSE

TCN
1.3560
1.1645
0.7225
0.8500
0.6981
0.8355

Informer
0.9704
0.9851
0.5531
0.7437
0.5702
0.7551

Proposed
0.9659
0.9828
0.6010
0.7753
0.5164
0.7186

It can be seen from the above table that the accuracy of the method proposed in the present embodiment is better than other methods for multi-energy load short-term forecasting.

Embodiment 2

In a second aspect, the disclosure also provides a multi-energy integrated short-term load forecasting system. Referring to FIG. 2, the system includes:

- a data acquisition module, configured to acquire relevant data of multi-energy integrated short-term load forecasting, wherein the relevant data of multi-energy integrated short-term load forecasting includes multi-energy load historical data and external environment historical data corresponding to the multi-energy load historical data;
- a data pretreating module, configured to perform pretreatment and correlation calculation on acquired relevant data of multi-energy integrated short-term load forecasting to determine input data of a multi-energy integrated short-term load forecasting model, and perform sample classification on the input data;
- a forecasting model construction module, configured to acquire a multi-energy integrated short-term load forecasting model constructed based on encoder-decoder; wherein the multi-energy integrated short-term load forecasting model includes a feature encoding module and a feature decoding module, the feature encoding module includes performing time feature encoding, sequence position feature encoding and data feature encoding on the input data, performing feature learning by encoded matrixes, and outputting a feature matrix; the feature decoding module is used to decode a load sequence needing to be forecasted by performing self-learning on a time sequence needing to be forecasted and performing cross learning on the time sequence and the feature matrix; the encoder includes multiple layers of temporal convolutional networks having multi-head self-attention mechanism and rotary position embedding;
- a model training module, configured to train the multi-energy integrated short-term load forecasting model based on the input data after sample classification; and
- a forecasting result output module, configured to perform multi-energy integrated short-term load forecasting by the trained multi-energy integrated short-term load forecasting model and output a forecasting result.

Optionally, a step of the data pretreating module performing pretreatment and correlation calculation on acquired relevant data of multi-energy integrated short-term load forecasting to determine input data of a multi-energy integrated short-term load forecasting model, and performing sample classification on the input data includes: