The segmented image may be divided into N grid areas using, for example, the same method used for dividing the preset guide image. The N grid areas of the segmented image may respectively correspond to the N grid areas of the preset guide image obtained by dividing the preset guide image.

The electronic device may obtain the optimization value for the input image based on a preset class label of each pixel point of M first grid areas and a predicted class label of each pixel point of M second grid areas among the N grid areas of the segmented image.

In this case, the M first grid areas and the M second grid areas may each be positioned corresponding to the preset guide image and the segmented image. In other words, the M first grid areas may have a one-to-one correspondence with the M second grid areas. The electronic device may determine the M second grid areas among the plurality of grid areas of the segmented image based on the respective positions of the M first grid areas.

For M grid area pairs of the M first grid areas and the M second grid areas (a grid area pair being a first grid area paired with its corresponding second grid area), the electronic device may calculate loss function values respectively corresponding to the grid area pairs. The loss function value of a given grid area pair may be calculated based on a preset class label of a pixel point of the first grid area in the given grid area pair and a predicted class label of a pixel point of the second grid area in the given grid area pair. The electronic device may obtain M loss function values by calculating loss function values respectively corresponding to the M grid area pairs.

A grid area pair may include one first grid area paired with one second grid area that corresponds to the first grid area. For example, when the first grid area is positioned at (a, b) (a positional index among the plurality of grid areas of the guide image), the second grid area may correspond to the first grid area in that the second grid area is also positioned at (a, b) (a positional index among the plurality of grid areas of the segmented image).

The electronic device may determine the optimization value for the input image based on the M loss function values.

For example, under the assumption that the preset guide image includes 8 anchor points, the electronic device may calculate a loss function value based on a preset class label corresponding to each pixel point of each of the first grid areas respectively including the 8 anchor points and a predicted class label corresponding to each pixel point of each of second grid areas respectively corresponding to the first grid areas. In this case, the electronic device may obtain 8 loss function values corresponding to a total of 8 grid area pairs. The electronic device may calculate a loss function value corresponding to a grid area pair including a first grid area and a second grid area.

The electronic device may calculate a regularization function value based on a predicted class label of each pixel point of the plurality of grid areas included in the segmented image and a preset class label corresponding to each pixel point of the M first grid areas.

For example, the electronic device may calculate a regularization function value based on the predicted class label of each pixel point of the plurality of grid areas included in the segmented image and a preset class label value corresponding to each pixel point of each of the 8 first grid areas under the assumption of the foregoing example. The electronic device may regularize generated image data by using a regularization function and increase the stability and speed of the generated image data through such regularization.

The electronic device may determine the optimization value for the input image based on the regularization function value and the M loss function values. Under the assumption of the foregoing example, the electronic device may determine the optimization value according to the regularization function value and 8 loss function values corresponding to 8 grid area pairs including 8 first grid areas and 8 second grid areas corresponding to the 8 first grid areas.

The electronic device may calculate a loss function value by using a cross-entropy loss function. The electronic device may calculate the cross-entropy loss function according to Equation 1.

L
=

-

1
Q

⁢

∑
i

∑

c
=
1

T

y
ic

⁢

log
⁡
(

p
ic

)

Equation
⁢

1

In Equation 1, L denotes the cross-entropy loss function, Q denotes the number of pixel points included in each first grid area, and T denotes the number of identifiable class labels in the target model. For example, as described above, T may be 10, and a class label c may be an integer of 0 to T−1. When a preset class label value of an ith pixel point of a first grid area is c, y_ic=1, or otherwise, y_ic=0. p_icis a probability, predicted by the target model, that a class label of an ith pixel point of a second grid area is c.

The electronic device may calculate the sum of loss function values respectively corresponding to the M first grid areas. For example, under the assumption that the preset guide image includes 8 anchor points, the electronic device may calculate the sum of 8 loss function values corresponding to the 8 first grid areas that include the 8 anchor points. The electronic device may calculate the sum of the loss function values corresponding to the M first grid areas according to Equation 2.

∑
M

L
⁡
(

x
^

M

,

y
M

)

Equation
⁢

2

In Equation 2, {circumflex over (x)} denotes random noise (e.g., the input image) according to a Gaussian distribution of an input target model, M denotes the number of anchor points of a preset guide image, y_Mdenotes a first grid area including an Mth anchor point of the preset guide image, {circumflex over (x)}_Mdenotes a second grid area corresponding to y_Mof the random noise, and L({circumflex over (x)}_M, y_M) denotes a cross-entropy loss function between the first grid area including the Mth anchor point and the second grid area corresponding to the first grid area in the random noise.

The electronic device may calculate a total distribution value corresponding to the noise image (or the input image) based on a predicted class label of each pixel point thereof. The electronic device may calculate an L2-norm value corresponding to the first grid area based on a preset class label corresponding to each pixel point of each of the M first grid areas and based on the predicted class label.

Under the assumption that the preset guide image includes 8 anchor points, the electronic device may calculate an L2-norm value corresponding to the preset class label corresponding to each pixel point of each of the first grid areas and the predicted class label for the first grid areas respectively including 8 anchor points. The electronic device may obtain 8 L2-norm values respectively corresponding to the 8 first grid areas.

The electronic device may determine a regularization function value according to the 8 L2-norm values and according to the total distribution corresponding to the noise image (or the input image). For example, the electronic device may determine the regularization function value according to the 8 L2-norm values (respectively corresponding to the 8 first grid areas) and according to the total distribution corresponding to the noise image.

The electronic device may calculate the L2-norm values and the total distribution corresponding to the noise image according to Equation 3.

R

l
⁢
2

=

1
Q

⁢

∑

i
=
1

Q

[

y
i

-

F
i

]

2

Equation
⁢

3

R
tv

=

E
⁢

{

∑

i
=
1

S

[

X
i

-

E
⁡
(
x
)

]

2

}

In Equation 3, R_l2denotes an L2-norm value of a first grid area, Q denotes the number of pixel points included in each of the M first grid areas, y_idenotes a preset class label value corresponding to an ith pixel point in the first grid area, and F_idenotes a predicted class label corresponding to the ith pixel point of the first grid area.

In Equation 3, R_tvdenotes the total distribution corresponding to the noise image, S denotes the number of pixel points included in the noise image, X_idenotes a predicted class label corresponding to each of the pixel points included in the noise image, and E(x) denotes an average value of predicted class labels corresponding to the S pixel points.

The electronic device may calculate the sum of the L2-norm values corresponding to the M first grid areas. For example, under the assumption that the preset guide image includes 8 anchor points, the electronic device may calculate the sum of 8 L2-norm values corresponding to 8 first grid areas including the 8 anchor points.

The electronic device may calculate the sum of the L2-norm values corresponding to the M first grid areas according to Equation 4.

∑
M

R

l
⁢
2

(

x
^

M

,

y
M

)

Equation
⁢

4

In Equation 4, {circumflex over (x)} denotes random noise (e.g., the input image) according to the Gaussian distribution that is input to the target model, M denotes the number of anchor points of the preset guide image, y_Mdenotes the first grid area including the Mth anchor point of the preset guide image, {circumflex over (x)}_Mdenotes the second grid area (in the random noise) corresponding to y_M, and R_l2({circumflex over (x)}_M, y_M) denotes an L2-norm value between the first grid area including the Mth anchor point and the second grid area corresponding to the first grid area.

The electronic device may calculate a regularization function according to Equation 5.

R
⁡
(

x
^

)

=

α
tv

⁢

R
tv

(

x
^

)

+

α

l
⁢
2

⁢

R

l
⁢
2

(

x
^

)

=

α
tv

⁢

R
tv

(

x
^

)

+

α

l
⁢
2

⁢

∑
M

R

l
⁢
2

(

x
^

M

,

y
M

)

Equation
⁢

5

In Equation 5, R({circumflex over (x)}) denotes the regularization function, {circumflex over (x)} denotes the random noise according to the Gaussian distribution of the input target model, R_tv({circumflex over (x)}) denotes the total distribution corresponding to the noise image described above, and R_l2({circumflex over (x)}) denotes the sum of the L2-norm values corresponding to the M first grid areas described above. α_tvand α_tvboth are proportional coefficients and unchanging constants that may be set depending on actual demands.

In operation 130, the electronic device may optimize the input image based on the optimization value when the optimization value exceeds a set threshold value. The electronic device may obtain the optimized input image based on the optimization value. The threshold value may be set depending on actual demands and may be 0.001, for example.

The electronic device may calculate the optimization value according to Equation 6.

∑
M

L
⁡
(

x
^

M

,

y
M

)

+

R
⁡
(

x
^

)

=

∑
M

L
⁡
(

x
^

M

,

y
M

)

+

α
tv

⁢

R
tv

(

x
^

)

+

α

l
⁢
2

⁢

∑
M

L
⁡
(

x
^

M

,

y
M

)

Equation
⁢

6

The electronic device may repeatedly perform operations 110, 120, and 130 included in the data generation method of FIG. 1 until the optimization value is less than or equal to the threshold value by using the optimized input image.

FIG. 2 illustrates an example of repeatedly performing a data generation method, according to one or more embodiments.

In operation 210, an electronic device may generate a segmented image including class labels respectively predicted for the pixels of an input image. Specifically, the input image may be inputted to a target model which segments the input image (predicts the labels of the respective pixels).

For example, an input size of the target model may be 512*512, the number of identifiable class labels may be 32. The target model may be configured to receive square input data or non-square input data. For example, the target model may be a multi-target segmentation model (i.e., may predict multiple classes). The input image may be a random noise image according to a Gaussian distribution.

In operation 220, the electronic device may determine a second grid area of the segmented image corresponding to a first grid area. The first grid area may be set (selected), from among grid areas of a set guide image, based on having an anchor point.

The segmented image may include class labels predicted for all respective pixel points in the noise image (e.g., the input image). The segmented image may be divided into a plurality of grid areas, e.g., using the same method used to divide the guide image. The electronic device may determine a second grid area of the input image corresponding to the first grid area of the guide image. The first grid area of the guide image may correspond one-to-one to the second grid area of the input image.

In operation 230, the electronic device may calculate a loss function value and a regularization function value based on a class label of the first grid area and a class label of the second grid area.

For example, the electronic device may calculate the loss function value by using a cross-entropy loss function. The electronic device may determine the regularization function value according to an L2-norm value of the second grid area (corresponding to the first grid area) and according to a total distribution corresponding to the input image.

In operation 240, the electronic device may calculate an optimization value for the input image based on the loss function value and the regularization function value, for example, according to Equation 6 above.

In operation 250, the electronic device may determine whether a number of performed repetitions (iterations) of operations 210, 220, 230, 240, and 260 has reached a set number or whether the optimization value is less than or equal to a threshold value. For example, the set number of iterations may be 100.

When determined in operation 250 that the number of repetitions has not reached the set number and that the optimization value exceeds the threshold value, the electronic device may perform operation 260 and optimize the input image based on the optimization value. An example of optimizing an input image based on an optimization value is described with reference to FIG. 10.

When operation 210 is performed after operation 260, the electronic device may use the optimized input image (as per the previous iteration/repetition) as an input image for the new/current iteration/repetition. For example, after operation 260, the electronic device may input the optimized input image to the target model and may perform operation 210.

The electronic device may update the noise image (e.g., the input image) through repeated updates. The noise image after the updates may be used as a noise image of an input target model for the next repetitive update.

For example, the electronic device may perform an operation of setting a guide image. The electronic device may set the guide image by setting a preset class label to each pixel point of the first grid area that includes an anchor point.

When the number of repetitions reached the set number, or when the optimization value is less than or equal to the threshold value in operation 250, the electronic device may output the updated input image as an optimized input image. When these repeated updates are completed, the electronic device may use the final thus-obtained optimized image (the image after the repeated updates) as an image for compressing the target model.

The electronic device may repeatedly perform the method of FIG. 2 to output the set number of optimized input images. For example, the electronic device may set/generate a new guide image (e.g., a random noise image) whenever outputting an optimized input image (e.g., with each iteration/repetition of the loop of FIG. 2). The electronic device may perform the method of FIG. 2 by using a set guide image and an input image and may output/generate the set number of optimized input images.

The electronic device may repeatedly perform the method of FIG. 2 until obtaining the pieces of quantized data to be used for the static quantization of an image segmentation model (e.g., the target model). For example, (dynamic or static) quantization of an image segmentation model and quantized data, a description of (dynamic or static) quantization of an image segmentation model of a known image segmentation model and known quantized data can be applied.

With respect to the image segmentation model, the position of an anchor point and the size of a grid area may be fixed (or determined in any way that is independent of the content of the input image), but a class label of the grid area including the anchor point may be configured depending on actual needs. For example, under the assumption that the number of identifiable class labels is 10 in the image segmentation model, when resetting the preset guide image, the electronic device may randomly select one class label from among ten class labels, that is, class labels 0 to 9, for grid areas respectively including anchor points, and may set the selected class label as the class label of the grid area including the anchor point.

FIG. 3 illustrates an example of setting a guide image, according to one or more embodiments.

Referring to FIG. 3, the electronic device may determine a first grid area based on a set anchor point among a plurality of grid areas of (divisions of) the guide image in operation 310. The number of anchor points may be determined based on the size of input data that an image segmentation model is configured to receive (e.g., at an input layer). For example, the electronic device may set candidate items of the number of the anchor points to 1, 2, 4, or 8. The positions of the anchor points may be evenly distributed (e.g., generally of a uniform distance from each other).

In operation 320, the electronic device may set a class label for each pixel of the first grid area. That is, each pixel of the first grid area may be assigned the same class label.

More specifically, the electronic device may add a preset class label to each pixel point of the first grid area that includes an anchor point. For example, the electronic device may randomly select class labels for respective first grid areas that include respective anchor points and may set the pixels of the first grid areas to be the corresponding random class labels. Put another way, each first grid area having an anchor point may have its own random class label which its pixel points are set to. Thus, filling first grid areas that include an anchor point may maximally diversify a target of each anchor point and may provide diversity of the data generated by differentiating the first grid area including each anchor point.

In addition, when the electronic device adds a class label to each pixel point of a first grid area that includes an anchor point, the added class label may be a class label that is also a label that the image segmentation model (e.g., a target model) is configured to identify. Moreover, each class label may correspond to a class label color and image content.

For example, a specific class label may be 1, the class label color corresponding to the class label 1 may be green, and the image content corresponding to the class label 1 may be a puppy. As another example, a specific class label may be 3, the class label color corresponding to the class label 3 may be red, and the image content corresponding to the class label 3 may be a painting.

FIGS. 4, 5, 6, and 7 illustrate examples of an anchor point, according to one or more embodiments.

FIGS. 4, 5, 6, and 7 illustrate four examples of types of anchor point settings. FIG. 4 shows an example of setting only one anchor point a, FIG. 5 shows an example of setting two anchor points b1 and b2, FIG. 6 shows an example of setting four anchor points c1, c2, c3, and c4, and FIG. 7 shows an example of setting eight anchor points d1, d2, d3, d4, d5, d6, d7, and d8.

To increase the pieces of information for generating data and to increase a difference between anchor points, an electronic device may obtain a guide image by randomly assigning a preset class label to each of the M first grid areas that includes an anchor point. The electronic device may set the same class label to all pixel points of one first grid area.

For example, the electronic device may fill each pixel point of each first grid area including an anchor point with a respective randomly selected class label. The electronic device may maximally diversify a target of each anchor point and may ensure the diversity of data generated by differentiating a grid area including each anchor point.

A target model may be trained with a dataset for image segmentation. Different datasets may respectively correspond to the different numbers and types of class labels. For example, the number of class labels corresponding to a specific dataset may be 10, and the class label values thereof may be 0 to 9. In another example, the number of class label values corresponding to another dataset may be 100, and the class labels thereof may be 0 to 99.

When the electronic device adds a preset class label to each pixel point of a first grid area that includes an anchor point, the added preset class label may be a class label that the target model is configured to be capable of identifying. For example, under the assumption that the target model is trained with a dataset in which the number of class labels is 10, the electronic device may set one of 0 to 9 (any of the ten class label values corresponding to the dataset) as the preset class label added to the first grid area including the anchor point.

Each class label may correspond to a class label color and image content. For example, a class label may be 1, the class label color corresponding to the class label 1 may be green, and the image content corresponding to the class label 1 may be a puppy. As another example, the class label may be 3, the class label color corresponding to the class label 3 may be red, and the image content corresponding to the class label 3 may be a piece of a painting.

FIGS. 8 and 9 illustrate examples adding a class label to a first grid area that includes an anchor point, according to one or more embodiments.

Referring to FIG. 8, a guide image may include 16*16 first grid areas, eight of which include eight anchor points e1, e2, e3, e4, e5, e6, e7, and e8.

FIG. 9 illustrates an example of adding different class labels respectively to the eight first grid areas that included the eight anchor points of FIG. 8. Specifically, different class label colors filling the eight first grid areas respectively including the eight anchor points indicates that different class labels are added to the eight first grid areas respectively including the eight anchor points. For example, the first grid areas that include the anchor points are modified so that the pixel points have the class label colors.

FIG. 10 illustrates an example of updating an input image, according to one or more embodiments.

FIG. 10 illustrates the updating (e.g., optimizing) of a noise image 1010 (e.g., the input image) in a repetitive update method. Referring to FIG. 10, an electronic device may calculate (e.g., iteratively refine) an optimization value in a repetitive update process in operation 1070 (e.g., as described with reference to FIG. 2), and then may propagate the calculated optimization value backward to a noise image. The electronic device may propagate the optimization value backward to the noise image through each layer of a target model 1020, starting from an output layer of the target model 1020, and may thus update the noise image by using the calculated optimization value. Details of this optimization process are described next.

The electronic device may output a segmented image 1030 by inputting an input image 1010 to the target model 1020. The target model 1020 may generate a feature map 1023 by using a convolutional layer 1021. The target model 1020 may output the segmented image 1030 by using a convolutional layer 1022. In other words, the target model 1020 may be, or may include, a convolutional neural network (CNN), which may include a fully connected layer that maps feature map(s) to predicted classes of pixels.

The electronic device may calculate a loss function value and regularization function value 1050 by using the segmented image 1030 and a guide image 1040. The electronic device may calculate an optimization value 1060 by using the loss function value and regularization function value 1050.

According to embodiments, the electronic device may update a color value, e.g., a red, green, and blue (RGB) value of each of a plurality of pixel points included in the input image 1010 based on the optimization value 1060. The electronic device may set an optimizer for the target model 1020 and may stack a specific layer (e.g., a weight of the convolutional layers 1021 and 1022 or random noise) to be updated based on the optimizer.

When propagating the optimization value backward in each layer starting from the output layer of the target model 1020, the electronic device may search for a layer of the target model 1020 that is to be updated by the optimizer. In some embodiments, layer optimization is not performed, that is to say, there is no search for a layer to be updated, rather, the electronic device only propagates the optimization value backward until the optimization value is propagated to the noise image. For example, in a process of each repetitive update, the target model 1020 may remain fixed, that is, weights of each of the layers of the target model 1020 may be fixed. The electronic device may adjust and update the noise image only without adjusting each layer of the target model 1020.

According to embodiments, a condition for ending repeated updates may be a current repetition number reaching a preset maximum number of repetitions. The preset maximum number of repetitions may be set depending on actual needs. For example, the preset maximum number of repetitions may be set to 100 repetitions. When the number of repetitions reaches 100, the electronic device may determine that the repeated updates end, and an updated noise image obtained after the repeated updates may be used as an image for compressing the target model 1020.

According to embodiments, the condition for ending the repeated updates may be an optimization value being less than or equal to a preset threshold value, which may be tested each time a repetition/iteration ends. The preset threshold value may be set depending on actual needs. For example, the threshold value may be 0.001. When the optimization value when a repeated update completes is determined to be less than or equal to 0.001, the electronic device may end the repeated updating, and the updated (e.g., optimized) noise image obtained after the repeated updating ends may be used as an image for compressing the target model 1020.

The compression of an image segmentation model (e.g., the target model 1020) may include quantization of the image segmentation model. Model quantization is a model compression technique of, for example, converting a floating-point value into an integer value. For example, when an original image segmentation model is a float32 model, one weight may be expressed as float32. An int8 model may be obtained from the float32 model by quantizing the float32 image segmentation model, in which one quantized weight may be expressed as int8. In this method, the size (amount of data) of the image segmentation model may decrease, and storage overhead may also decrease. As another example, a float64 weight may be converted to a float32 weight.

FIG. 11 illustrates an example data generation device 1100, according to one or more embodiments.

Referring to FIG. 11, the data generating device 1100 may include an input module 1101, an optimization value obtaining module 1102, an optimization module 1103, and a final optimization image obtaining module 1104.

The input module 1101 may obtain a segmented image by inputting an input image to a target model. The segmented image may include class label values of the respective pixel points of the input image, as predicted by the target model.

The input image may be a noise image. The noise image may be a random noise image according to a Gaussian distribution, but examples are not limited thereto. For example, the noise image may be an impulse noise image or a uniform noise image. For example, the noise image may be a visible black-and-white snowflake image. The target model may be an image segmentation model.

According to embodiments, the data generation device 1100 may further include a guide image obtaining module, a grid processing module, an anchor point setting module, and a label value assigning module.

Before obtaining the segmented image by inputting the input image to the target model, the data generation device 1100 may obtain a guide image by using the guide image obtaining module. The size of the guide image may be the same as an input size of the target model, and the guide image may be a blank image. The data generation device 1100 may perform grid processing on the guide image by using the grid processing module and may obtain a plurality of grid areas (e.g., the guide image may be divided into a grid having grid areas). The data generation device 1100 may perform grid processing on the guide image to obtain N grid areas.

For example, the input size of the target model may be 512*512, the number of class labels that the target model is configured to identify may be 32, and the data generation device 1100 may be configured to receive square input data or non-square input data. The data generation device 1100 may perform grid processing on the guide image and may convert the guide image into the designated size of a plurality of grid areas of whole data. The data generation device 1100 may perform grid processing on the guide image and may readily select data to be noticed. For example, N, that is, the number of grid areas, may be set as 16*16, and the number of pixels included in each of the grid areas may be 32*32.

The data generation device 1100 may set anchor points to M respective first grid areas among the N grid areas by using the anchor point setting module. The data generation device 1100 may set anchor points in some of the grid areas.

The data generation device 1100 may set the number of anchor points according to the input data size of the target model. For example, the data generation device 1100 may set candidate items of the number of the anchor points to 1, 2, 4, or 8. The position of the anchor points may be evenly distributed (e.g., in a rectilinear pattern).

The data generation device 1100 may set grid data that is to be noticed by setting an anchor point, and data in another position may be generated according to the contour of actual data, and thus, the completeness of the data may be ensured. For example, a preset class label corresponding to a specific anchor point may be 1, the color of a class label corresponding to the preset class label 1 may be green, and the image content corresponding to the preset class label 1 may be a puppy.

A complete contour of the puppy around a first grid area including an anchor point may be generated according to data on the actual shape of the puppy, besides a partial image of the puppy being displayed in the first grid area including the anchor point, in finally generated image data.

To increase the pieces of information for generating data and a difference between anchor points, the data generation device 1100 may obtain a preset guide image by randomly assigning a preset class label to each pixel point of each first grid area of the M first grid areas each including an anchor point by using the label value assigning module. For example, the data generation device 1100 may randomly select a class label and may fill a grid area including each anchor point to maximally diversify a target of each anchor point and to ensure the diversity of data generated by differentiating the grid area including each anchor point.

The data generation device 1100 may obtain an optimization value of the input image based on a preset class label of each pixel point of the M first grid areas among the N grid areas included in the preset guide image and the segmented image by using the optimization value obtaining module 1102. A random preset class label value may be pre-assigned to each of the M first grid areas. N>1 and 1≤M≤N, and N and M may each be an integer.

The data generation device 1100 may obtain the optimization value by using a backward propagation algorithm to which a gradient descent technique is applied. The backward propagation algorithm to which the gradient descent technique is applied may include an SGD technique, but examples are not limited thereto.

According to embodiments, the data generation device 1100 may divide the segmented image into N grid areas by using the optimization value obtaining module 1102. The dividing method of the segmented image may be the same as the dividing method (or a grid processing method) of the preset guide image. The N grid areas of the segmented image may correspond one-to-one to the N grid areas of the preset guide image.

The data generation device 1100 may obtain the optimization value for the input image based on a preset class label of each pixel point of M first grid areas and a predicted class label of each pixel point of M second grid areas among the N grid areas of the segmented image by using the optimization value obtaining module 1102.

The M first grid areas and the M second grid areas may each be positioned corresponding to the preset guide image and the segmented image, and the M first grid areas may correspond one-to-one to the M second grid areas.

According to embodiments, the data generation device 1100 may obtain M loss function values by calculating a loss function value corresponding to each grid area pair based on a preset class label of a pixel point of a first grid area and a predicted class label of a pixel point of a second grid area for M pairs of the M first grid areas and the M second grid areas by using the optimization value obtaining module 1102. Each grid area pair may include one first grid area and one second grid area corresponding to the first grid area. The data generation device 1100 may determine the optimization value for the input image based on the M loss function values.

For example, under the assumption that the guide image includes 8 anchor points, the data generation device 1100 may calculate a loss function value corresponding to a grid area pair including a first grid area and a second grid area based on a preset class label corresponding to each pixel point of each of the first grid areas respectively including the 8 anchor points and a predicted class label corresponding to each pixel point of each of second grid areas respectively corresponding to the first grid areas. The data generation device 1100 may obtain 8 loss function values corresponding to a total of 8 grid area pairs.

According to embodiments, the data generation device 1100 may calculate a regularization function value based on a predicted class label of each pixel point of the segmented image and a preset class label corresponding to each pixel point of a first grid area by using the optimization value obtaining module 1102.

For example, the data generation device 1100 may calculate the regularization function value based on the predicted class label of each pixel point of the segmented image and a preset class label corresponding to each pixel point of each of 8 first grid areas by using the optimization value obtaining module 1102. The data generation device 1100 may regularize generated image data based on a regularization function and increase the stability and speed of the generated image data through such regularization.

The data generation device 1100 may determine the optimization value for the input image based on the regularization function value and the M loss function values by using the optimization value obtaining module 1102. For example, the optimization value obtaining module 1102 may determine the optimization value according to the regularization function value and 8 loss function values corresponding to 8 grid area pairs including 8 first grid areas and 8 second grid areas corresponding to the 8 first grid areas.

For example, the data generation device 1100 may calculate the loss function values by using a cross-entropy loss function. For example, the data generation device 1100 may calculate the loss function values according to Equation 1 above.

The data generation device 1100 may calculate the sum of the loss function values according to Equation 2 above. For example, under the assumption that the preset guide image includes 8 anchor points, the data generation device 1100 may calculate the sum of 8 loss function values corresponding to 8 first grid areas including the 8 anchor points.

According to embodiments, the data generation device 1100 may calculate a total distribution value corresponding to the noise image based on a predicted class label of each pixel point. The data generation device 1100 may calculate an L2-norm value corresponding to a first grid area based on a preset class label corresponding to each pixel point of each of the M first grid areas and the predicted class label.

For example, under the assumption that the preset guide image includes 8 anchor points, the data generation device 1100 may calculate an L2-norm value corresponding to the preset class label corresponding to each pixel point of each of the first grid areas and the predicted class label for the first grid areas respectively including 8 anchor points. The data generation device 1100 may obtain 8 L2-norm values respectively corresponding to the 8 first grid areas.

The data generation device 1100 may determine a regularization function value according to the 8 L2-norm values respectively corresponding to the 8 first grid areas and the total distribution corresponding to the noise image. For example, the data generation device 1100 may determine the regularization function value according to the 8 L2-norm values respectively corresponding to the 8 first grid areas and the total distribution corresponding to the noise image.

For example, the data generation device 1100 may calculate the L2-norm values and the total distribution corresponding to the noise image according to Equation 3 above.

The data generation device 1100 may calculate the sum of the L2-norm values corresponding to the M first grid areas according to Equation 4 above. For example, under the assumption that the preset guide image includes 8 anchor points, the data generation device 1100 may calculate the sum of 8 L2-norm values corresponding to 8 first grid areas including the 8 anchor points.

The data generation device 1100 may calculate the regularization function according to Equation 5 above.

The regularization function described above may be expressed as the following equation.

R({circumflex over (x)})=α_tvR_tv({circumflex over (x)})+α_l2R_l2({circumflex over (x)})=α_tvR_tv({circumflex over (x)})+α_l2Σ_MR_l2({circumflex over (x)}_M, y_M)

When the optimization value does not satisfy a preset condition, the data generation device 1100 may optimize the input image based on the optimization value by using the optimization module 1103 and may obtain the optimized input image. The satisfying of the preset condition may mean that the optimization value is less than or equal to a preset threshold value. The preset threshold value may be set depending on actual demands, and may be set to 0.001, for example.

The data generation device 1100 may use the optimized input image as an input image for the next optimization until a final optimized image in which the optimization value satisfies the preset condition is obtained by using the final optimization image obtaining module 1104. The data generation device 1100 may update the noise image through repeated updates until the repeated updates end. Each noise image after the updates may be used as a noise image of an input target model for the next repetitive update. When the repeated updates are completed, the electronic device may use a final optimized image obtained after the repeated updates as an image for compressing the target model.

The data generation device 1100 may calculate the optimization value by using the loss function value and the regularization function value according to Equation 6 above.

According to embodiments, the data generation device 1100 may update an RGB value of each of a plurality of pixel points included in the input image based on the optimization value. The data generation device 1100 may set an optimizer for the target model and may stack a weight of a specific layer to be updated or random noise on the optimizer. When propagating the optimization value backward in each layer starting from an output layer of the target model, the data generation device 1100 may search for a layer that needs to be updated of the target model in the optimizer. When the layer that needs to be updated is not searched for, the data generation device 1100 may propagate the optimization value backward until the optimization value is propagated to the noise image.

For example, in a repetitive update process, the target model may always be fixed, and the data generation device 1100 may adjust and update the noise image without adjusting each layer of the target model.

FIG. 12 illustrates an example electronic device 1200, according to one or more embodiments.

Referring to FIG. 12, an electronic device 1200 may include at least one memory 1201 and at least one processor 1202, and a data generation method according to embodiments may be executed by the at least one memory 1201 storing instructions and the at least one processor 1202 executing the instructions.

For example, the electronic device 1200 may be a personal computer (PC), a tablet device, a personal digital assistant (PDA), a smartphone, or other devices for executing the above commands. In this case, the electronic device 1200 may not necessarily be a single electronic device 1200 and may be a device or a set of circuits for executing the commands (or a command set) individually or commonly described above. The electronic device 1200 may also be a part of an integrated control system or a system administrator or may be implemented as a portable electronic device that is locally or remotely (e.g., through wireless transmission) connected through an interface.