Specifically, referring to FIG. 4, the highest accuracy is seen when the ratio of the Adam algorithm to the SGD algorithm is 3:7 among the accuracy according to a change in channel size when a kernel size is 3×3 and the number of convolutional layers is two (see FIG. 4A), the accuracy according to a change in channel size when the number of convolutional layers is four (see FIG. 4B), and the accuracy according to a change in channel size when the number of convolutional layers is six (see FIG. 4C).

Also, referring to FIG. 5, the highest accuracy is seen when the ratio of the Adam algorithm to the SGD algorithm is 3:7 among the accuracy according to a change in channel size when a kernel size is 5×5 and the number of convolutional layers is two (see FIG. 5A), the accuracy according to a change in channel size when the number of convolutional layers is four (see FIG. 5B), and the accuracy according to a change in channel size when the number of convolutional layers is six (see FIG. 5C).

Subsequently, the neural network device 10 may calculate an optimal learning rate for adjusting weights in BNN and an intensity of a kernel update using the learning rate included in the hyperparameters (S110). In other words, the neural network device 10 may calculate the same optimal learning rate of 0.03 when the kernel size is 3×3 and the kernel size is 5×5.

For example, the neural network device 10 may calculate the optimal learning rate of 0.03 having an accuracy of 99% by comparatively analyzing learning rates from 0.01 to 0.1 indicating how much learning is necessary at one time.

Specifically, referring to FIG. 6, the highest accuracy is seen when the learning rate is 0.03 among the accuracy according to a change in channel size when a kernel size is 3×3 and the number of convolutional layers is two (see FIG. 6A), the accuracy according to a change in channel size when the number of convolutional layers is four (see FIG. 6B), and the accuracy according to a change in channel size when the number of convolutional layers is six (see FIG. 6C).

Also, referring to FIG. 7, the highest accuracy is seen when the learning rate is 0.03 among the accuracy according to a change in channel size when a kernel size is 5×5 and the number of convolutional layers is two (see FIG. 7A), the accuracy according to a change in channel size when the number of convolutional layers is four (see FIG. 7B), and the accuracy according to a change in channel size when the number of convolutional layers is six (see FIG. 7C).

Subsequently, the neural network device 10 may calculate an optimal momentum by considering a momentum value in the BNN using the momentum included in the hyperparameters (S120). In other words, the neural network device may calculate an optimal momentum of 0.5 when a kernel size is 3×3 and calculate an optimal momentum of 0.6 when a kernel size is 5×5.

For example, to calculate an optimal momentum value for reducing a training time, the neural network device 10 may calculate an optimal momentum by considering different momentum values from 0.1 to 1.0, that is, a momentum of 0.5 when a kernel size is 3×3, and a momentum of 0.6 when a kernel size is 5×5. Specifically, referring to FIG. 8, the highest accuracy is seen when the momentum is 0.5 among the accuracy according to a change in channel size when a kernel size is 3×3 and the number of convolutional layers is two (see FIG. 8A), the accuracy according to a change in channel size when the number of convolutional layers is four (see FIG. 8B), and the accuracy according to a change in channel size when the number of convolutional layers is six (see FIG. 8C).

Also, referring to FIG. 9, the highest accuracy is seen when the momentum is 0.6 among the accuracy according to a change in channel size when a kernel size is and the number of convolutional layers is two (see FIG. 9A), the accuracy according to a change in channel size when the number of convolutional layers is four (see FIG. 9B), and the accuracy according to a change in channel size when the number of convolutional layers is six (see FIG. 9C).

Subsequently, the neural network device 10 may determine whether there is batch normalization (S130). In other words, the neural network device 10 may determine whether batch normalization is necessary in a convolutional layer.

For example, referring to FIG. 10, when the number of convolutional layers is four, a channel size is 9, and a kernel size 3×3, batch normalization may be required in each of the convolutional layers.

Here, batch normalization may be calculated according to Equation 1 below. Elements having scalar values may be independently calculated for normalization using Equation 1 for an n-dimensional input x={x(1), x(2), . . . , x(n)}.

x
^

(
k
)

=

x

(
k
)

-

𝔼
[

x

(
k
)

]

Var
[

x

(
k
)

]

[

Equation
⁢

1

]

When batch normalization is not included in each of the convolutional layers, referring to FIG. 11, there may be an accuracy difference of 8%, but the present disclosure is not limited thereto.

Subsequently, the neural network device 10 may determine whether there is a pooling layer (S140). In other words, the neural network device 10 may determine whether there is a pooling layer in the convolutional layers.

For example, referring to