FIG. 3 and FIG. 4 show distributions of elements for each channel and each patch of an input matrix.

As described above, a wide variety of images can be input to the image processing device. In addition, each color distribution of each image may be very different from each other. As shown in FIG. 3 as an example, when the embedding module 23 as shown in (b) of FIG. 3 receives the first and second images (Image 1, Image 2, as shown in (a) of FIG. 3) respectively to obtain an embedding matrix, element value distributions for each channel in the embedding matrix may be very different from each other. Referring to FIG. 3, it can be seen that in the first and second images (Image 1, Image 2), the element value distribution of the blue channel is very wide, whereas the element value distribution of the red channel is very narrow and dense. In the graph (c) of FIG. 3, the direction of the horizontal axis represents the element value, and the direction of the vertical axis represents the appearance frequency. As such, even though the distribution of element values for each channel is very different, if a single quantizer is applied and quantized in the same level range, a quantization error appears very large. If the quantizer performs quantization by setting a quantization level range based on the blue channel, element values of the red channel are almost always quantized to the same value, and no difference occurs in quantization values. On the other hand, if the quantization range is set based on the red channel, most of the element values of the blue channel are clipped, resulting in a very large quantization error. That is, element values smaller than the quantization range and element values larger than the quantization range are not distinguished from each other. In addition, in the case of an artificial neural network, even though the frequency of occurrence is low, the maximum and minimum values may have a large effect on the performance of the neural network.

Therefore, as shown in (d) and (e) of FIG. 3, if quantization is performed by applying a plurality of quantizers having different quantization ranges for each channel, quantization error can be greatly reduced. Accordingly, in the present disclosure, the quantization module (QM), which operates as a quantization apparatus, selects the most suitable quantizer for a channel among a plurality of quantizers having different quantization ranges for each channel according to the element distribution for each channel, and performs quantization with the selected quantizer, thereby reducing quantization error.

Similarly, when an image, as shown in (a) of FIG. 4, is divided into a plurality of patches, pixels included in the same patch are adjacent to each other, so pixel data is often similar, but pixel data included in different patches may be very different, as shown in (b) of FIG. 4. Accordingly, as shown in (c) of FIG. 4, distribution of patch data for different patches may also appear very different. Accordingly, when each patch is input by constructing a row of an input matrix in the embedding module 23, data distribution for each row may be very different. In addition, if they are quantized with the same quantizer and the attention scores (a) are calculated in units of rows representing patches, this also causes performance degradation due to quantization errors.

Accordingly, in the present disclosure, the quantization module (QM), which operates as a quantization apparatus, may select the most suitable quantizer for a channel among a plurality of quantizers having different quantization ranges for each channel according to the element distribution for each patch, so that the quantization errors can be reduced.

Here, for convenience of understanding, the patch is expressed separately from the channel, but the division according to the patch in the input matrix can also be referred to as a channel that distinguishes the elements of the input matrix, and in some cases, data for each pixel composed of a depth direction vector in a 3D matrix may be divided into channels.

In the present disclosure, channels may be set differently according to characteristics of input matrices or configurations of weight matrices in consideration of a multiplication operation between a quantized input matrix and a quantized weight matrix performed after quantization.

FIG. 5 shows an example of a detailed configuration of the quantization module of FIG. 2, and FIG. 6 is a diagram for explaining an operation of selecting a quantizer by the quantizer selection module of FIG. 5.

Referring to FIG. 5, in the present disclosure, the quantization module (QM) operating as a quantization apparatus may include a channel selection module 31, a distribution analysis module 33, a quantizer selection module 35, and a quantizer determination module 37.

The channel selection module 31 selects a channel according to the channel configuration in the input matrix (X). As described above, the channel (X_c) may be a color channel for dividing data for each RGB color, or may be a class for each patch. In addition, it may be a channel in which colors for each patch are divided, or when the input matrix is 3-dimensional, a channel in which vectors in a depth direction are divided. The channel selection module 31 may divide the input matrix (x) into a plurality of channels (X_c) according to the set channel division method, select and transmit each of the divided plurality of channels (X_c) to the distribution analysis module 33.

The distribution analysis module 33 analyzes the distribution of element values (x) of the selected channel (X_c). As shown in FIG. 6, the distribution analysis module 33 may check the maximum value (max(X_c)) and minimum value (min(X_c)) of the element values (x) included in each channel (X_c). In addition, the reason why the distribution analysis module 33 checks the maximum value (max(X_c)) and minimum value (min(X_c)) among the element values (x) of the channel (X_c) is, as described above, to prevent clipping by including both the maximum value (max(X_c)) and the minimum value (min(X_c)) in the quantization range as much as possible since the maximum value (max(X_c)) and minimum value (min(X_c)) of the elements in the channel (X_c) may have a great effect on the performance of the artificial neural network.

When the maximum value (max(X_c)) and minimum value (min(X_c)) of the elements are checked for each channel (X_c) by the distribution analysis module 33, as shown at the bottom of FIG. 6, the quantizer selection module 35 selects a quantizer (Q_i) having a quantization range most similar to the checked maximum value (max(X_c)) and minimum value (min(X_c)) among a plurality of quantizers (Q₁~Q_G) having different quantization ranges provided in the quantizer determination module 37. At this time, the quantizer selection module 35 may calculate the error distance (d(X_c, Q_i)) between each of the upper limit (u_i) and lower limit (l_i) of the quantization range of each of the plurality of quantizers (Q₁~Q_G) and the maximum value (max(X_c)) and minimum value (min(X_c)) of the channel (X_c) using Equation 1, and select a quantizer (Q_i) having a minimum calculated error distance (d(X_c, Q_i)).

d
⁡
(

X
c

,

Q
i

)

=

(

max
⁡
(

X
c

)

 
-
 

u
í

)

2

+

(

min
⁡
(

X
c

)

 
-
 

l
i

)

2

[

Equation
⁢

1

]

Then, the quantizer determination module 37 quantizes an element value of a corresponding channel (or patch) (X_c) in an input matrix (x) by using a quantizer (Q_i) selected by the quantizer selection module 35 among a plurality of quantizers (Q₁~Q_G). Accordingly, the quantizer determination module 37 may quantize the element value of each channel (or patch) (X_c) with quantizers having different quantization ranges.

In the above, it has been described that the quantizer determination module 37 includes a plurality of quantizers (Q₁~Q_G) and the quantizer selection module 35 selects a quantizer (Q_i) corresponding to an element value distribution (here, the maximum value (max(X_c)) and minimum value (min(X_c))) of a channel (X_c) from among the plurality of quantizers (Q₁~Q_G) provided in the quantizer determination module 37. However, in many cases, the quantizer is configured such that the quantization range is adjustable. In this case, the quantizer determination module 37 may include one quantizer, and the quantizer selection module 35 may instead be configured to adjust the quantization range of the quantizer for each channel (X_c).

As described above, when the quantization module (QM) performs quantization with quantizers having different quantization ranges for each channel (X_c), quantization errors can be reduced. However, an operation using a matrix quantized in a different quantization range for each channel (X_c) must be normally performed.

Accordingly, a method of operating a matrix quantized with different quantization ranges for each channel (X_c) will be described below.

FIG. 7 is a diagram for explaining a method of operating a matrix quantized differently for each channel.

In FIG. 7, (a) shows a conventional matrix multiplication operation method, and (b) and (c) are diagrams for explaining a method of operating a matrix differently quantized for each channel according to the present disclosure.

Referring to (a) of FIG. 7, when matrix multiplication is performed by quantizing the input matrix (M) and the weight matrix (N), the quantizer quantizes the input matrix (M) and the weight matrix (N) to obtain a quantization input matrix (M) and a quantization weight matrix (N). In this case, the input matrix (M) and the weight matrix (N) may be quantized by a quantizer having scaling values (s_M, s_N), respectively. Here, the scaling values (s_M, s_N) are parameters for adjusting the quantization range of the quantizer. The scaling value (s_M) of the quantizer quantizing the input matrix (M) and the scaling value (s_N) of the quantizer quantizing the weight matrix (N) may be different from or identical to each other. However, the scaling values (s_M) of the quantizers that quantize all input values (m) of the input matrix (M) are the same, and the scaling values (s_N) of the quantizers that quantize all weights (n) of the weight matrix (N) are the same.

Accordingly, the quantization input matrix (M) and the quantization weight matrix (N) may be expressed as M=s_M·Q(M; s_M) and N=s_N·Q(N; s_N), respectively, as shown in (a) of FIG. 7. Here, Q(M; s_M) and Q(N; s_N) each denote quantization level input matrices in which each element of the input matrix (M) and the weight matrix (N) is converted into bit values according to the quantization level. Therefore, multiplication between the quantization input matrix (M) and the quantization weight matrix (N) can be calculated as in Equation 2.

M
¯

⁢

N
¯

=

s
M

⁢

s
N

·

Q
⁡
(

M
;

s
M

)

⁢

Q
⁡
(

N
;

s
N

)

[

Equation
⁢

2

]

That is, multiplication of the quantization input matrix (M) and the quantization weight matrix (N) may be performed by individually calculating the product of the scaling values (s_M, s_N) and the product of the quantization level matrices (Q(M; s_M), Q(N; s_N)), and then multiplying them again.

Meanwhile, in (b) of FIG. 7, a case of matrix multiplication of the weight matrix (N) and the input matrix (M) divided into multiple (here, two) channels (M₁, M₂) in the row direction is assumed. Here, the input matrix (M) divided in the row direction may be a division according to color channels in the embedding matrix, or may be a feature map in which the embedding matrix is already matrix-multiplied by another weight matrix. In addition, the weight matrix (N) may be a weight matrix (W) obtained through training.

All elements (n) of the weight matrix (N) are all quantized with a quantizer having the same scaling value (s_N), but here, for convenience of understanding, the weight matrix (N) is divided into two regions (N₁, N₂) that are multiplied by each of the two divided channels (M₁, M₂) of the input matrix (M). The weight matrix (N) may also be quantized by dividing the channels, but when the weight matrix (N) is a weight matrix (W), it is assumed here that the weight matrix (W) is not only obtained by training, but also that the elements (n) of the weight matrix (N) are quantized with the same quantizer in order to prevent errors during multiplication with the input matrix (M).

The first and second channels (M₁, M₂) of the quantization input matrix (M), which are quantized by quantizers having different scaling values

(

s
M
1

,

s
M
2

)

in the input matrix (M), may be represented by

M
_

1

=

s
M
1

·

Q
⁡
(

M
1

;

s
M
1

)

and

M
_

2

=

s
M
2

·

Q
⁡
(

M
2

;

s
M
2

)

,

respectively. In addition, corresponding to the quantized input matrix (MN) divided into the first and second channels (M₁, M₂), the first and second regions (N₁, N₂) of the quantization weight matrix (N) may be expressed as N₁=s_N·Q(N₁; s_N) and N₂=s_N·Q(N₂; s_N), respectively.

Therefore, multiplication between the quantization input matrix (M), quantized by different quantizers for each channel, and the quantization weight matrix (N), can be calculated as in Equation 3.

M
¯

⁢

N
¯

=

M
¯

1

⁢

N
¯

1

+

M
¯

2

⁢

N
¯

2

=

s
M
1

⁢

s
N

·

Q
⁡
(

M
1

;

s
M
1

)

⁢

Q
⁡
(

N
1

;

s
N

)

+

s
M
2

⁢

s
N

·

Q
⁡
(

M
2

;

s
M
2

)

⁢

Q
⁡
(

N
2

;

s
N

)

[

Equation
⁢

3

]

That is, matrix multiplication between the quantization input matrix (M), which is quantized differently for each channel (M₁, M₂), and the quantization weight matrix (N), may be performed by performing a simple matrix sum operation after each matrix multiplication according to the divided channels. Although an inter-matrix sum operation must be additionally performed after inter-matrix multiplication, the matrix multiplication operation basically consists of the sum operation after the inter-element multiplication, and the inter-matrix sum operation is not identical to this, but is performed similarly, so the overall operation complexity and amount of operation do not increase significantly compared to the matrix multiplication of the existing quantization matrix.

In addition, in (c) of FIG. 7, it is assumed that the input matrix (M) divided into a plurality of (here two) channels (M₁, M₂) in the column direction and the weight matrix (N) are matrix-multiplied. Here, the input matrix (M) divided in the column direction may be an attention matrix (A) obtained by matrix multiplication and softmax operation of the query matrix (Q) and the key matrix (A) in FIG. 2, and the weight matrix (N) may be a value matrix (V).

As described above, in the self-emphasis technique, the similarity between patches is calculated using the attention matrix (A), and each patch constitutes one row by the embedding module 23, so that the attention matrix (A) may be divided into a plurality in the column direction. In addition, all elements (n) of the weight matrix (N) are quantized with quantizers having the same scaling value (s_N), but for convenience of understanding, it is divided into two regions (N₁, N₂) where each of the divided channels (M₁, M₂) of the input matrix (M) is multiplied.

Also in (c) of FIG. 7, the first and second channels (M₁, M₂) of the quantization input matrix (M) quantized by quantizers having different scaling values

(

s
M
1

,

s
M
2

)

in the input matrix (M) can be expressed as

M
_

1

=

s
M
1

·

Q
⁡
(

M
1

;

s
M
1

)

and

M
_

2

=

s
M
2

·

Q
⁡
(

M
2

;

s
M
2

)

,

respectively. In addition, corresponding to the quantized input matrix (MN) divided into the first and second channels (M₁, M₂), the first and second regions (N₁, N₂) of the quantization weight matrix (N) may be expressed as N₁=s_N·Q(N₁; s_N) and N₂=s_N·Q(N₂; s_N), respectively.

However, in (c), since the channels extend in the row direction and are distinguished from each other in the column direction, multiplication between the quantization input matrix (M), quantized by different quantizers for each channel, and the quantization weight matrix (N), can be calculated as shown in Equation 4.

M
¯

⁢

N
_

=

Concat
(

M
_

1

⁢

N
_

1

,

M
¯

2

⁢

N
¯

2

)

[

Equation
⁢

4

]

That is, by performing each matrix multiplication according to the divided channels and then concatenating them, matrix multiplication between the quantization input matrix (M), which is quantized differently for each channel (M₁, M₂), and the quantization weight matrix (N), may be performed.

Here, since each matrix multiplication is performed according to the divided channels and concatenated, even if quantization is performed with different quantizers for each channel (M₁, M₂) in the input matrix (M), a matrix multiplication operation can be performed without an increase in computational complexity or amount of computation.

In a general quantization operation, if a matrix multiplication operation is performed by performing quantization using different quantizers having different quantization ranges, errors may occur due to differences in quantization ranges. However, the present disclosure is an invention for quantization for artificial neural networks, and as described above, in an artificial neural network, even if a matrix multiplication operation is performed by quantizing with different quantizers having different quantization ranges, if the same quantization operation is applied during training, the weight (w) of the weight matrix (W) is obtained by reflecting these characteristics, so that it is possible to greatly improve the operation efficiency due to quantization while minimizing the performance degradation of the artificial neural network.