This watermark image may need to (or should) ensure (or be configured to provide for) invisibility and robustness of a watermark in that the watermark may be subject to various attacks during a distribution process.

The existing studies often train the watermarking model for a single specific attack type. In this case, it is difficult to expect robustness against an attack type not used during learning.

Therefore, some example embodiments may provide a complex attack learning method for training a watermarking model for complex attacks of various combinations or order as learning methodology of minimizing (or reducing) degradation in visibility of a watermark image and maximizing (or increasing) robustness against a target attack.

The computer device 200 according to some example embodiments may provide a client with a content copy prevention service through connection to an exclusive application installed on the client or a website/mobile site related to the computer device 200. A computer-implemented watermarking model learning system may be configured in the computer device 200. For example, the watermarking model learning system may be implemented in the form of a program that independently operates or may be configured in an in-app form of a specific application to be operable on the specific application.

The processor 220 of the computer device 200 may be implemented as a component to perform the following watermarking model learning method. According to some example embodiments, components of the processor 220 may be selectively included in or excluded from the processor 220. Also, according to some example embodiments, the components of the processor 220 may be separated or merged for functional representation of the processor 220.

The processor 220 and the components of the processor 220 may control the computer device 200 to perform operations included in the following watermarking model learning method. For example, the processor 220 and the components of the processor 220 may be configured to execute an instruction according to a code of at least one program and a code of an OS included in the memory 210.

Here, the components of the processor 220 may be representations of different functions performed by the processor 220 in response to an instruction provided from a program code stored in the computer device 200.

The processor 220 may read a necessary (or given) instruction from the memory 210 to which instructions related to control of the computer device 200 are loaded. In this case, the read instruction may include an instruction for controlling the processor 220 to perform operations included in the following watermarking model learning method.

Operations included in the watermarking model learning method may be performed in order different from illustrated order. Some of the operations may be omitted or an additional process may be further included.

Operations included in the watermarking model learning method may be performed by the server 150. According to some example embodiments, at least some of the operations may be performed by the client.

FIG. 3 is a flowchart illustrating an example of a method performed by a computer device according to some example embodiments.

Referring to FIG. 3, in operation S310, the processor 220 may divide the entire epochs for learning (e.g., an entire number of iterations) of a watermarking model into at least two stages and may configure the different number of attack types as target attacks to be applied for each stage. According to some example embodiments, a respective threshold number of target attack types may be set for each of the at least two stages such that each stage may learn the watermarking model using a number of different target attack types up to (e.g., less than or equal to) the threshold number set for that stage. According to some example embodiments, the number of different target attack types for each stage may be selected (e.g., randomly, pseudorandomly, based on an input from an external source/operator) from a list of attack types or dataset (including a plurality of different target attack types) up to the threshold number set for that stage. In an epoch of early learning, robustness against an easy attack is learned and as the model is trained, model robustness may be gradually learned by making an attack scheme more complex and difficult. For example, the processor 220 may set a single attack without applying the attack as a complex type (e.g., a threshold number of one) in an early learning stage and may set a complex attack by gradually increasing the maximum (or highest) number of attacks for an attack type to be learned in a subsequent learning stage. For example, assuming that the entire epochs are divided into three stages, the processor 220 may set the maximum (or highest) number of attacks (e.g., threshold number) to 1 in an initial stage in which learning starts, may set the maximum (or highest) number of attacks (e.g., threshold number) to 2 in a subsequent stage, stage 2, and may set the maximum (or highest) number of attacks (e.g., threshold number) to 3 in a last learning stage, stage 3. In the case of applying a strong attack at the beginning of learning, a model may be collapsed. Therefore, robust learning of applying a single attack in the initial stage of learning and then gradually applying a stronger attack may be provided.

In operation S320, the processor 220 may configure a learning batch corresponding to an attack type set to a corresponding stage, when configuring a learning batch for each stage of the entire epochs. According to some example embodiments, training data (e.g., training images, video, etc.) may be selected and configured to obtain the learning batch for each stage. In particular, the processor 220 may configure the learning batch through random sampling. According to some example embodiments, references to “random” made herein may also refer to “pseudorandom.” Here, a portion of a predetermined (or alternatively, given) batch size may be configured with a learning sample to which a target attack is applied (hereinafter, referred to as ‘attack sample’) and the reset thereof may be configured with a learning sample to which the target attack is not applied (hereinafter, referred to as ‘original sample’). Some example embodiments may maintain some of learning batches as original samples to which an attack is not applied for model learning having high invisibility to a watermark. An original retention probability representing a portion of original samples, and an attack application probability representing a portion of attack samples each to which a target attack is applied, among all samples for learning batch configuration may each be preset (or alternatively, given). The processor 220 may configure the learning batch that includes the original samples and the attack samples based on the original retention probability and the attack application probability. Some example embodiments may guarantee superiority of invisibility (e.g., may improve the invisibility of the watermark) over a learning model using only an attack sample since an original sample to which an attack is not applied is mixed (e.g., mixed into, or included in, the learning batch) even when configuring a learning batch of a strong attack.

In operation S330, the processor 220 may perform learning of the watermarking model using a learning batch corresponding to an attack type of a corresponding stage for each stage of the entire epochs. According to some example embodiments, references to learning of the watermarking model may refer to training the watermarking model. The processor 220 may gradually perform learning of the watermarking model. Here, the processor 220 may train the watermarking model without a model collapse phenomenon by initially performing learning using a single attack and then sequentially performing learning using various complex attacks. Robustness against an attack to a watermark may be acquired by sequentially configuring the number of attack types and attack combination order in a different manner depending on a learning stage. Also, degradation in invisibility of watermark may be prevented (or reduced) by performing learning using a batch in which samples, each with an applied target attack and original samples each without an applied attack, are mixed. According to some example embodiments, an original image is input into the learned watermarking model to apply a watermark (e.g., including user ID information) to the original image to obtain a watermark image. According to some example embodiments, a watermark image is input into the learned watermarking model to extract the information contained in the watermark (e.g., the user ID information).

FIG. 4 illustrates an example of the entire structure of a watermarking model according to some example embodiments.

Referring to FIG. 4, a watermarking model 400 may include a watermark inserter 410 and/or a watermark extractor 420.

The watermark inserter 410 serves to insert user identification information 402 into an original image 401 as a watermark and to generate a watermark image 403, and the watermark extractor 420 serves to extract user identification information 405 inserted as watermark from a distribution image 404 for the watermark image 403.

The watermark image 403 with the watermark inserted for the original image 401 is distributed. Here, the distribution image 404 may represent an image that is distributed with various attacks, such as arbitrarily editing the watermark image 403 with another image (e.g., an image found on website, such as an illegal site or site at which unlawful distribution and/or editing of copyrighted media is performed).

A loss function for learning of the watermarking model 400 may be defined as Equation 1 below.

ℒ
=

λ

img
⁢
_
⁢
mse

⁢

ℒ

img
⁢
_
⁢
mse

+

λ
vgg

⁢

ℒ
vgg

+

λ

wm
⁢
_
⁢
mse

⁢

ℒ

wm
⁢
_
⁢
mse

[

Equation
⁢

1

]

Here, λ_{img_mse} [Image Omitted] _{img_mse}+λ_vgg_vggdenotes a first loss value for acquiring invisibility of watermark and λ_{wm_mse}_{wm_mse}denotes a second loss value for acquiring robustness of watermark.

That is, a mean square error (MSE) between the original image 401 and the watermark image 403 may be used as the first loss value and an MSE between the user identification information 402 inserted into the original image 401 and the user identification information 405 extracted from the distribution image 404 may be used as the second loss value.

The invisibility and the robustness for the watermark may be learned uniformly. The processor 220 may perform learning of the watermarking model 400 in a way that minimizes (or reduces) degradation in the invisibility of the watermark through the first loss value and maximizes (or increases) the robustness of the watermark against a target attack through the second loss value.

In some example embodiments, the processor 220 may perform some operations (e.g., the operations described herein as being performed by the watermarking model 400) by artificial intelligence and/or machine learning. As an example, the processing circuitry may implement an artificial neural network (e.g., the watermarking model 400) that is trained on a set of training data by, for example, a supervised, unsupervised, and/or reinforcement learning model, and wherein the processing circuitry may process a feature vector to provide output based upon the training. Such artificial neural networks may utilize a variety of artificial neural network organizational and processing models, such as convolutional neural networks (CNN), recurrent neural networks (RNN) optionally including long short-term memory (LSTM) units and/or gated recurrent units (GRU), stacking-based deep neural networks (S-DNN), state-space dynamic neural networks (S-SDNN), deconvolution networks, deep belief networks (DBN), and/or restricted Boltzmann machines (RBM). Alternatively or additionally, the processing circuitry may include other forms of artificial intelligence and/or machine learning, such as, for example, linear and/or logistic regression, statistical clustering, Bayesian classification, decision trees, dimensionality reduction such as principal component analysis, and expert systems; and/or combinations thereof, including ensembles such as random forests.

Herein, the machine learning model (e.g., the watermarking model 400) may have any structure that is trainable, e.g., with training data. For example, the machine learning model may include an artificial neural network, a decision tree, a support vector machine, a Bayesian network, a genetic algorithm, and/or the like. The machine learning model will now be described by mainly referring to an artificial neural network, but some example embodiments are not limited thereto. Non-limiting examples of the artificial neural network may include a convolution neural network (CNN), a region based convolution neural network (R-CNN), a region proposal network (RPN), a recurrent neural network (RNN), a stacking-based deep neural network (S-DNN), a state-space dynamic neural network (S-SDNN), a deconvolution network, a deep belief network (DBN), a restricted Boltzmann machine (RBM), a fully convolutional network, a long short-term memory (LSTM) network, a classification network, and/or the like.

FIGS. 5 and 6 illustrate examples of a target attack for learning of a watermarking model according to some example embodiments.

Examples of a target attack against the watermark image 403 may include an image quality degradation attack, an image conversion attack, a neural network-based conversion attack, and the like.

For example, the image quality degradation attack may include JPEG (JPEG image compression technique), resizing (loss due to interpolation when converting an image size), Gaussian filtering (filtering with Gaussian window), color quantization (expressing an image by reducing the number of pixels), leveling (color correction with gamma correction after defining a maximum (or highest)/minimum (or lowest) pixel value), noise (noise addition), and the like. The image conversion attack may include Gaussian blur (filtering with Gaussian window), gamma correction (color change), median blur (filtering with median blurring), translate (x, y-axis transformation), and the like. The neural network-based conversion attack may include Waifu2x (waifu-based denoiser), RealESRGAN (ESRGAN-based denoiser and super resolution), ASNet (Topaz AI) (Topaz-based denoiser), and the like.

As an example of image application for each attack type, an original sample, an attack sample to which a target attack is applied, and a residual sample (image in which color values of the attack sample are subtracted from the original sample) are shown in FIGS. 5 and 6.

For simultaneous (or contemporaneous) learning of the watermark inserter 410 and the watermark extractor 420 included in the watermarking model 400, a differentiable target attack may be used. In the case of JPEG, color quantization, and leveling including a non-differentiable area (quantization, rounding, flooring, etc.) among attack types, a corresponding area may be replaced with a differentiable area using an approximation function of Equation 2 and applied. According to some example embodiments, in order to apply a target attack, in areas where differentiation is not possible, such as quantization, rounding, and flooring, an approximation function (e.g., Equation 2) may be used to change it into a differentiable area. Accordingly, a non-differentiable area may be changed into a differentiable area through an approximation.

[
x
]

approx

=

[
x
]

+

(

x
+

[
x
]

)

3

[

Equation
⁢

2

]

Example of a pseudo code of a model learning method to maximally (or otherwise) secure (or improve) robustness against a target attack while minimizing (or reducing) degradation in visibility of watermark is as follows.

TABLE 1

n_epochs=100

batch_size=32

p1=0.5 # original retention probability

p2=0.5 # attack application probability

A={

‘JPEG’: (20, 99), # JPEG QF range

‘resize’: (0.3, 1.5), # resize scale range

‘leveling’: (0, 50), # range of number of pixels to be leveled

‘translate’: (−12, 12), # x, y pixel translation range

‘color_quantization’: (8, 40), # range of number of pixel quantization

‘gamma’: (0.3, 1.5), # gamma correction range

‘waifu2x’: (0, 3), # waifu2x denoising level range

‘gaussian’: (3, 5), # gaussian filtering window size range

‘median’: (3, 5), # median filtering window size range

‘noise’: (0.01, 0.04), # random noise std range

# ... etc

}

num_attacked=int(batch_size*(1-p1)*p2)

for epoch in range(n_epochs):

# maximum number of attacks applied

if epoch <= n_epochs/3:

a_n=1

elif epoch <=2* n_epochs/3:

a_n=2

else:

a_n=3

data=sample_data(batch_size)

attacked_data=data[:batch_size*(1-p1)]

attacks=random.sample(list(A.item( )), a_n)

for attack, (r1, r2) in attacks:

param=random.uniform(r1,r2)

data_index=random.sample(range(len(attacked_data)), num_attacked)

attacked_data[data_index]=attack(attacked_data[data_index], param)

# changed to attacked data

data[:batch_size*(1-p1)]=attacked_data

model.train(data)

Referring to the example of pseudo code of Table 1, the maximum (or highest) number of attacks for model learning in the entire epochs may be limited to 1 for epochs from a first epoch to ⅓ epochs (first learning stage), 2 for epochs from ⅓ epochs to ⅔ epochs (second learning stage), and 3 for remaining epochs (third learning stage).

The processor 220 may randomly (or pseudorandomly) select a sample index equal (or corresponding to) a batch size for each epoch through random (or pseudorandom) sampling from a dataset available for learning of the watermarking model. According to some example embodiments, processor 220 may randomly (or pseudorandomly) select sample indices until a number of samples is selected equal (or similar) to the batch size for each epoch (or learning stage).

The processor 220 may randomly (or pseudorandomly) select the number of attack types corresponding to the maximum (or highest) number of attacks in a corresponding learning stage from an attack type list for each learning stage. For example, the processor 220 may set one attack type, JPEG, as a target attack in a first learning stage and may set three attack types, JPEG, resize, and leveling, as target attacks in a third learning stage.