130, for example, when the AI learning model is a YOLO-based learning model.

According to an exemplary embodiment of the present disclosure, the method may include determining, by the determiner (630 of FIG. 6), one of the plurality of bounding boxes for each of the de-identification areas using non-maximum suppression (NMS).

The non-maximum suppression (NMS) may be configured to determine a bounding method of one of the plurality of bounding boxes for the same de-identification area by removing remaining bounding boxes, except only one bounding box with the highest score or confidence score among multiple bounding boxes for the same de-identification area and also removing bounding boxes that do not satisfy an intersection over union (IOU) threshold, for coordinate values of a plurality of bounding boxes output by the AI learning model.

According to an exemplary embodiment of the present disclosure, in S130, the de-identifier (640 in FIG. 6) may de-identify the de-identification area (by blurring the de-identification area or replacing the de-identification area with a composite image through DeepFake technique.

In one example, in S130, as shown in FIG. 3A and FIG. 3B, the de-identification areas 211, 212, 221, and 222 (FIG. 2) detected as pedestrian faces or vehicle license plates may be subjected to blurring processing through a blurring technique such as Average blur or Gaussian blur which are OpenCV Blur techniques to generate and obtain blurred pedestrian faces 310 and 320 and blurred vehicle license plates 330 and 340.

In another example, in S130, the de-identifier (640 in FIG. 6) may replace the de-identification areas 211, 212, 221, and 222 (FIG. 2) detected as pedestrian faces or vehicle license plates with composite images or generate the composite images through a DeepFake technique using a Generative Adversarial Network (GAN)-based model to de-identify the de-identification areas.

It may be required to train a recognition network for recognizing objects and spaces using image data de-identified through the process of FIG. 1 and to recognize the objects and the spaces from the image data in which the de-identification areas are de-identified using the trained recognition network. However, de-identification may affect the performance of recognition of recognizing objects and spaces from de-identified image data, and therefore, a method that does not affect recognition performance is needed, which will be described with reference to FIG. 4 and FIG. 5.

FIG. 4 is a flowchart of operations for training a recognition network and recognizing objects/spaces using de-identified image data, which shows a flowchart of operations for training a recognition network using de-identified image data by the method of FIG. 1, FIG. 2A, FIG. 2B and FIG. 2C, and FIG. 3A and FIG. 3B, and recognizing objects and spaces from the de-identified image data using the trained recognition network.

Referring to FIG. 4, a method according to an exemplary embodiment of the present disclosure may include receiving, by a receiver, de-identified image data obtained or generated through the process of FIG. 1, that is, de-identified image data, in which de-identification areas such as pedestrian faces and vehicle license plates have been subjected to de-identification processing, to train the recognition network and receiving, by the receiver, features of a teacher recognition network through knowledge distillation.

Accordingly, the recognition network may be trained by a training device using the training data received in S410, that is, the de-identified training data, and the features received from the teacher recognition network in S420 (S430).

For example, as shown in FIG. 5, the teacher network may transfer, to a recognition network (student network), features from a deep learning model trained with existing model training datasets (images which have not been subjected de-identification processing) and the existing model training labels (object/space recognition information). The recognition network (student network), which trains a model with the de-identified training datasets, may train the model with the features received from the teacher network together, having no significant influence on the object/space recognition performance even though some identification information related to pedestrian faces and vehicle license plates has disappeared from in the image data.

In the instant case, the recognition network (student network) may be trained to mimic the teacher network after a difference between the results of the teacher network and the recognition network (student network) is included in the loss function.

When the recognition network (student network) is trained with the training data including the features of the teacher network and the de-identified image data in S430, a recognition device may recognize spaces and objects from the de-identified image data input to the recognition network by use of the trained recognition network S440.

In the instant case, S440 may be a phase performed only with the recognition network trained by the process of S410 to S430 described above, in which the recognition network recognizes objects and spaces from the de-identified image data in real time by inputting the de-identified image data to the recognition network, which has been trained in advance, after the image data obtained by the camera of an autonomous vehicle has been de-identified through the process of FIG. 1.

In other words, in S440, objects and spaces may be to accurately recognized from the de-identified image data in real time by providing de-identification areas as an input to the recognition network after the de-identification areas have been de-identified in real time through the process of S110 to S130 of FIG. 1.

Thus, the method of de-identifying image data according to an exemplary embodiment of the present disclosure and the recognition method thereof may include de-identifying data which requires personal information protection from image data collected by a camera of an autonomous vehicle, and training the recognition network based on the de-identified image data and Knowledge Distillation to recognize objects and spaces from the de-identified image data.

Furthermore, the method of de-identifying image data according to an exemplary embodiment of the present disclosure may include performing de-identification processing on data requiring personal information protection, such as pedestrian faces and vehicle license plates, which are detected through an AI model for detecting de-identification areas, thus preventing violation of personal information protection laws when collecting image data.

Furthermore, a method of recognizing de-identified image data according to an exemplary embodiment of the present disclosure may include recognizing objects and spaces from the de-identified image data obtained in real time using a recognition network trained based on a training dataset of the de-identified image data and knowledge distillation using a teacher network after image data obtained by the camera of the autonomous vehicle has been de-identified.

The method according to various exemplary embodiments of the present disclosure is not limited or restricted to de-identifying image data obtained from a camera of a foreign autonomous vehicle, and may also de-identify image data obtained from a camera of a domestic vehicle. Of course, when the recognition network is applied to the vehicle, the autonomous vehicle may combine the de-identification method according to an exemplary embodiment of the present disclosure with the recognition network to de-identify image data obtained from the camera of the autonomous vehicle in real time, and then recognize objects and spaces from the de-identified image data in real time by use of the recognition network.

When image data obtained by the camera of a foreign autonomous vehicle is de-identified and then provided to a domestic server, the method of the present disclosure may include the following processor. For example, when camera image data is first obtained from the autonomous vehicle that has obtained the data, data curation is performed based on an acquisition scenario, the curated data is de-identified by the method of the present disclosure, and the de-identified image data is uploaded to the server after de-identification. As described with reference to FIG. 1, in the de-identification processing, input image may be scaled to fit the input size of the AI learning model, de-identification areas such as pedestrian faces and vehicle license plates may be detected through the AI learning model. Thereafter, the detected de-identification areas, that is, bounding boxes, may be post-processed to determine one bounding box for each de-identified object, and de-identified through blurring, DeepFake or the like to de-identify pedestrian faces and vehicle license plates that require privacy protection. The method of de-identifying image data according to an exemplary embodiment of the present disclosure may include finally outputting the de-identified image data that has been processed as described above.

Furthermore, the method of de-identifying image data described with reference to FIG. 1 may include immediately detecting a de-identification area from image data using an AI learning model and then de-identifying the detected de-identification area. Alternatively, the method of de-identifying image data may include detecting the de-identification area by combining a conventional object detection learning model that detects objects such as pedestrians and vehicles and a de-identification area detection learning model that detects only a de-identification area from an object detected by the object detection learning model. For example, the method of de-identifying image data according to another exemplary embodiment of the present disclosure may include detecting, by a detector, objects included in image data using an object detection AI model, and selecting by a selector, at least one preset object, such as a pedestrian and a vehicle, among the detected objects, recognizing, by a recognition device, at least a partial area for personal information protection among an area for the selected at least one object, a pedestrian face or vehicle license plate as a de-identification area, and performing, by a de-identifier, de-identification on a de-identification area included in the image data by de-identifying the recognized de-identification area.

In an exemplary embodiment of the present disclosure, the selector, the recognition device and the de-identifier may be implemented with a processor.

FIG. 6 is a block diagram of a device for de-identifying image data according to another exemplary embodiment of the present disclosure, and is a block diagram of a configuration of a device performing the method of FIG. 1, FIG. 2A, FIG. 2B and FIG. 2C, FIG. 3A and FIG. 3B, FIG. 4, and FIG. 5.

Referring to FIG. 6, a device for de-identifying image data 600 according to another exemplary embodiment of the present disclosure may include a receiver 610, a detector 620, a determiner 630, a de-identifier 640, and storage 650.

The storage 650 is a means for storing various data related to the technology of the present disclosure, and may store information such as image data captured by a camera, an artificial intelligence learning model for detecting de-identification areas, a recognition network or recognition model for recognizing an object and a space, and de-identified image data. It should be noted that the storage may store any information related to the technology of the present disclosure. The storage 650 may be referred to memory.

The receiver 610 may receive image data obtained by a vehicle camera.

The detector 620 may detect de-identification areas of an object included in the image data, such as a pedestrian face and a vehicle license plate, using an AI learning model for detecting de-identification areas. The detector 620 may include an AI learning model trained to detect de-identification areas. The detector 620 may be implemented with a processor.

The determiner 630 may be a component required when a plurality of de-identification areas of each of objects detected by the detector 620 are detected and may be configured to determine a de-identification area of each object by determining a bounding box for the de-identification areas of each object using non-maximum suppression (NMS) when a plurality of de-identification areas are detected for each object.

According to an exemplary embodiment of the present disclosure, the determiner 630 may be configured to determine a bounding method of one of the plurality of bounding boxes for the same de-identification area by removing remaining bounding boxes, except only one bounding box with the highest score or confidence score among multiple bounding boxes for the same de-identification area and also removing bounding boxes that do not satisfy an intersection over union (IOU) threshold, for coordinate values of a plurality of bounding boxes output by the AI learning model.

The de-identifier 640 may de-identify de-identification areas detected by the detector 620 or the de-identification area determined by the determiner 630.

According to an exemplary embodiment of the present disclosure, the de-identifier 640 may de-identify the de-identification area by blurring the de-identification area or replacing the de-identification area with a composite image, through a deep fake technique.

According to an exemplary embodiment of the present disclosure, the de-identifier 640 may transmit or upload the de-identified image data to a server. The de-identified image data may be transmitted to a server. The server may receive the de-identified image data from the image data de-identification device. The server may store the received de-identified image data. The server may transmit the received de-identified image data to another electronic device.