FIG. 3, the image feature correlation between the first screen image and the second screen image indicates that the image contents of the two screen images are similar. Based on this, the screen module can be determined to have no display defect through the technical solution of the present disclosure. That is, the defect area in the first screen image can be caused by dust, which is similar as the first case. Based on this, the situation that dust on the display screen leads to the display defect by mistake can be eliminated through the technical solution of the present disclosure.

Similarly, for the second case, if the defect area of the first screen image is caused by the display defect, and the display screen also has dust, the second screen image collected in the black screen state can also have the dust area. Then, the first screen image can include the dust area and the defect area, and the second screen image can only include the dust area. Thus, the first screen image and the second screen image can be different.

As shown in FIG. 4, a first screen image 401 on the left includes a defect area 402 formed by a plurality of circular points that are distributed clearly and not clearly. A second screen image 403 on the right includes a dust area 404. The defect area 402 is different from the dust area 404. Based on this, after determining that the first screen image has a defect area, the image feature correlation between the first screen image and the second screen image determined through the technical solution of the present disclosure can indicate that the image features of the first screen image and the second screen image are not similar. Then, the screen module can be determined to have a display defect, which is consistent with the second case. Based on this, in the technical solution of the present disclosure, the situation that the screen module is directly false determined to not have the defect based on the dust of the second screen image when the display screen is in the black screen state to cause the missed detection can be reduced.

The above illustrates a possible implementation of determining the display defect result according to an example of the information indicated by the second screen image when the display screen is in the second test state. However, in some embodiments, another method of determining the display defect result can be included when the second test state is different, which is not limited here.

The first screen image and the second screen image are the image of the whole display screen. The dust and the possible defect area can only occupy a small portion of the display screen. Assuming that the dust on the display screen leads to the false detection of the first screen image having the defect area. The test states of the display screen corresponding to the first screen image and the second screen image can be different. In some embodiments, some dust areas represented in the first screen image may not be displayed clearly in the first screen image. Thus, the image feature correlation of the first screen image and the second screen image can be determined to indicate that the two screen images are not similar. Thus, the screen module can be falsely determined to have the display defect.

Based on this, to further improve the accuracy of display defect detection, after determining that the first screen image has the defect area, whether the images in the position areas corresponding to the defect area of the first screen image and the defect area of the second screen image are similar can be compared.

FIG. 5 illustrates a schematic flowchart of another defect detection method according to some embodiments of the present disclosure. The method can include the following steps.

At S501, the first screen image and the second screen image of the screen module are obtained.

The first screen image can be an image displayed by the display screen collected when the display screen of the screen module is in the first test state. The second screen image can be an image displayed by the display screen collected when the display screen of the screen module is in the second test state.

For example, the first screen image can be an image displayed by the display screen when the display screen is in a mode of outputting a predetermined color, and the second screen image can be an image displayed by the display screen in a black screen state.

At S502, if the first screen image is determined to have the defect area, the position information of the defect area in the first screen image is determined.

The defect area can indicate that the screen module has a display defect.

For the method of determining that the first screen image has the defect area, reference can be made to the above embodiments, which are not repeated here.

The position information of the defect area in the first screen image can be used to position the defect area in the first screen image. For example, the position information can include a coordinate range of the defect area in the first screen image.

In the process of performing the defect detection on the first screen image, if the first screen image is detected to have the defect area, the position information of the defect area can be determined.

For example, the first screen image can be input into a defect detection model. If the defect detection model detects a defect area in the first screen image, the position information of the defect area can be marked in the output first screen image.

The position information of the defect area can be determined in other methods, which are not limited.

At S503, based on the position information, the first defect image area is determined from the first screen image, and the second defect image area is determined from the second screen image.

The first defect image area can include at least the defect area. The second defect image area can include at least an image area in the second screen image matching the position information.

The first screen image and the second screen image can be the image displayed by the display screen. Thus, the position information of the defect area in the first screen image can indicate the corresponding position of the defect area on the display screen. Based on this, in connection with the position information of the defect area, the image area in the second screen image corresponding to the position information can be determined. Thus, the second defect image area can be determined.

In some embodiments, the image of the defect area can be extracted from the first screen image to obtain the first defect image area. Correspondingly, the image area corresponding to the position information of the defect area can be extracted from the second screen image to obtain the second defect image area.

At S504, the image feature correlation between the first defect image area and the second defect image area is determined.

As shown in FIG. 2 to FIG. 4, the first screen image is the first defect image area extracted from the first screen image, and the second screen image is the second defect image extracted from the second screen image.

The method of determining the image feature correlation between the first defect image area and the second defect image area can be the same as the method of determining the image feature correlation between the first screen image and the second screen image.

For example, in some embodiments, the image feature correlation can be the image feature similarity between the first defect image area and the second defect image area.

In some other embodiments, determining the image feature correlation can include determining a category feature between the first defect image area and the second defect image area. The category feature can be used to indicate whether the content in the first defect image area is feature-wise similar to the content in the second defect image area. Based on the category feature, a target category to which the first defect image area and the second defect image area belong can be determined. The target category can be one of a first category and a second category. The first category can indicate that the features of the contents of the two images are the same. The second category can indicate the features of the contents of the two images are not the same.

At S505, if the image feature correlation indicates that the image features of the first defect image area and the second defect image area are not similar, the display defect is determined in the screen module.

At S506, if the image feature correlation indicates that the image features of the first defect image area and the second defect image area are similar, the display defect is not determined in the screen module.

For example, if the image feature correlation is the image feature similarity between the first defect image area and the second defect image area, the first defect image area and the second defect image area can be determined to be similar when the image feature similarity exceeds the predetermined threshold. Otherwise, if the image feature similarity does not exceed the predetermined threshold, the first image defect area and the second image defect area can be determined to be not similar.

For another example, if the image feature correlation is the target category to which the first defect image area and the second defect image area belong, the image features of the two defect image areas can be indicated to be similar when the target category is the first category. Otherwise, if the target category is the second category, the image features of the two defect image areas can be indicated to be not similar.

To facilitate the understanding, the description can be made by taking the first test state being the display screen in the mode of displaying a gray color and the second test state being the display screen in the black screen state. If the first screen image includes the defect area, based on the position of the defect area in the display screen, the dust image same as the defect area can be determined at the corresponding position in the second screen image. Thus, the defect area of the first screen image can be determined to be caused by the dust on the display screen, and no defect can exist in the screen module.

In the embodiment, since only image areas corresponding to the defect areas of the first screen image and the second screen image are compared, even dust appears at a position outside the defect area of the second screen image, and no image of the corresponding dust is shown in the other areas outside the defect area of the first screen image, the first screen image and the second screen image may not be determined to be no similar. Thus, the defect area of the first screen image caused by the dust may not be falsely determined as the defect caused by the display defect.

In summary, in embodiments of the present disclosure, if the first screen image includes the defect area, the first defect image area at the position corresponding to the defect area can be determined from the first screen image. The second defect image area at the position corresponding to the defect area can be determined from the second screen image. Based on this, in the present disclosure, only the image feature correlation between the first defect image area including the defect area and the second defect image area corresponding to the defect area in the second screen image. The situation of falsely determining the display defect caused by the interference by the image outside the defect area can be reduced. Thus, the accuracy of the display defect detection can be further improved.

In embodiments of the present disclosure, the image feature similarity between the two images can be determined in a plurality of methods, which are not limited.

To facilitate understanding, the method of determining the image feature similarity in connection with a similarity matching model is described below.

The similarity matching model can be obtained through the training based on a plurality of pairs of images marked with similarity matching results. Each pair of image samples can include two image samples. The similarity matching result of a pair of image samples can be used to indicate whether the image features of the pair of image samples are similar. Correspondingly, the similarity matching result can include two types of results, i.e., similar or not.

The similarity matching model can be trained in any supervised training method, which is not limited.

After the similarity matching model is trained, the first defect image area and the second defect image area can be input into the similarity matching model to obtain the image feature similarity between the first defect image area and the second defect image area output by the similarity matching model.

A network structure of the similarity matching model can be set as needed and is not limited.

FIG. 6 illustrates a schematic structural diagram of a similarity matching model according to some embodiments of the present disclosure.

As shown in FIG. 6, the similarity matching model includes deep neural networks (DNN) and a decision network layer.

The DNN model can be configured to extract the image features from two images, and the decision network layer can be configured to calculate the similarity between the image features of the two images. In some embodiments, a feature distance between the image features of the two images can be calculated. When the feature distance is smaller, the similarity can be higher. Correspondingly, whether the two images are similar can be determined based on the similarity between the two images.

In the process of training the similarity matching model, a pair of image samples with a similarity matching result indicating similarity can be used as a positive sample pair, while a pair of image samples with a similarity matching result indicating dissimilarity can be used as a negative sample pair.

In the present disclosure, a training purpose for training the similarity matching model can include that the feature distance between the features of the two image samples is as small as possible in the positive sample pair, and the feature distance between the features of the two image samples is as big as possible in the negative sample pair. Based on the training purpose, the similarity matching model configured to determine whether the first defect image area and the second defect image area are similar can be trained.

Based on this, as shown in FIG. 6, after inputting the first defect image area and the second defect image area into the DNN model of the similarity matching model, the DNN model can determine the image features of the two images and input the image features to the decision network layer. Correspondingly, if the defect area in the first defect image area is caused by the dust, the feature distance between the image feature of the first defect image area determined by the DNN model and the image feature of the second defect image area determined by the DNN model can be smaller than a target threshold. Thus, the image feature similarity between the two defect image areas can be determined to exceed the predetermined threshold.

As shown in FIG. 6, the decision network layer is able to directly output the feature similarity between the first defect image area and the second defect image area through training, which has a similar principle and is not repeated here.

The above description is made by taking the image feature similarity between the first defect image area and the second defect image area as an example, which is also suitable for determining the image feature similarity between the first screen image and the second screen image.

In the present disclosure, the image feature correlation between the two images can also be determined in connection with a pre-trained model. To facilitate understanding, an example of determining the target category to which the first defect image area and the second defect image area belong can be described.

In the present disclosure, a category comparison model and a classification model can be pre-trained. The category comparison model and the classification model can be obtained through training based on the plurality of pairs of image samples marked with the belonging category. The category comparison model and the classification model can be trained synchronously and do not need to be trained separately.

Accordingly, in the present disclosure, the first defect image area and the second defect image area can be input into the category comparison model as a set of images to obtain the category feature output by the category comparison model. Based on this, the category feature can be input into the classification model to obtain the target category output by the classification model.

Description can be made to an example of the category comparison model and the classification model. FIG. 7 illustrates a schematic diagram of determining the image feature correlation between the first defect image area and the second defect image area based on the category comparison model and the classification model according to some embodiments of the present disclosure.

In FIG. 7, a DNN model is used as the category comparison model. A classification model is connected to the DNN model. The classification model can include but is not limited to a fully connected network layer with a normalization function layer.

In the present disclosure, the purpose for training the category comparison model and the classification model can include that the image sample of the display defect caused by the dust and the image sample in the second test state (e.g., the display screen in the black screen state) are identified to belong to the same category.

In FIG. 7, after splicing the first defect image area and the second defect image area as one set of images and inputting the one set of images into the DNN model, the DNN model outputs the category feature for the set of images. After the category feature is input to the classification model, the classification model outputs the target category that indicates whether the first defect image area and the second defect image area are similar. For example, “1” represents the first category, and “0” represents the second category. Based on this, if the classification model outputs “1,” the first defect image area and the second defect image area can be indicated to belong to the same category. That is, the two defect image areas are similar.

The above description is made by taking the target category between the first defect image area and the second defect image area as an example. The principle of determining the target category between the first screen image and the second screen image can be similar, which is not limited.

Corresponding to the defect detection method of the present disclosure, the present disclosure further provides a defect detection device.

FIG. 8 illustrates a schematic structural diagram of the defect detection device according to some embodiments of the present disclosure. The device of embodiments of the present disclosure includes an image acquisition unit 801, a correlation determination unit 802, and a result determination unit 803.

The image acquisition unit 801 can be configured to obtain the first screen image and the second screen image of the screen module. The first screen image can be an image displayed by the display screen that is collected when the display screen of the screen module is in the first test state. The second screen image can be the image displayed by the display screen that is collected when the display screen of the screen module is in the second test state.

The correlation determination unit 802 can be configured to determine the image feature correlation between the first screen image and the second screen image if the defect area is determined in the first screen image. The defect area can indicate that the screen module has the display defect.

The result determination unit 803 can be configured to determine the defect detection result of the screen module based on the image feature correlation.

In some embodiments, the result determination unit can include a first result determination unit and a second result determination unit.

The first result determination unit can be configured to determine that the screen module has the display defect if the image feature correlation indicates that the image features of the first screen image and the second screen image are not similar.

The second result determination unit can be configured to determine that the screen module does not have the display defect if the image feature correlation indicates that the image features of the first screen image and the second screen image are similar.

In some other embodiments, the correlation determination unit can include a position determination unit, an image area determination unit, and a feature correlation determination unit.

The position determination unit can be configured to determine the position information of the defect area in the first screen image if the defect area is determined in the first screen image.

The image area determination unit can be configured to determine the first defect image area from the first screen image and the second defect image area from the second screen image based on the position information. The first defect image area can include at least the defect area, and the second defect image area can include at least the image area in the second screen image that matches the position information.

The feature correlation determination unit can be configured to determine the image feature correlation between the first defect image area and the second defect image area.

In some other embodiments, the feature correlation determination unit can include a similarity determination unit and a second result determination unit.

The similarity determination unit can be configured to determine the image feature similarity between the first defect image area and the second defect image area.

The second result determination unit can be configured to, when the image feature correlation indicates that the image features of the first screen image and the second screen image are similar, determine the image features of the first screen image and the second screen image to be similar if the image feature similarity exceeds the predetermined threshold.