303 as a working storage area.

A network interface card (NIC) 305 executes processing for mutual communication with the terminal device 201 via the communication network 210. The CPU 301, the ROM 302, the RAM 303, the HDD 304, and the NIC 305 are connected to an internal bus 306 and communicate with each other via the internal bus 306.

When directly connected to the prediction score calculation device 202, the external storage device 203 receives access from the CPU 301 via the internal bus 306. In addition, when connected to the communication network 210, the external storage device 203 receives access from the CPU 301 via the communication network 210 and the NIC 305.

FIG. 4 is a block diagram illustrating a configuration of functions implemented by the prediction score calculation device 202 executing the prediction score calculation program. As illustrated in FIG. 4, the prediction score calculation device 202 includes the prediction score calculation unit 410, the machine learning unit 420, a learning data input unit 430, and the threshold value determination unit 440. In addition, FIGS. 5 and 7 are flowcharts describing the operation of the prediction score calculation device 202.

The prediction score calculation unit 410 operates in cooperation with a prediction score output unit 411 and a process inspection data input unit 412. The terminal device 201 inputs the inspection data of the process inspections 103 and 105 to the process inspection data input unit 412 and requests a prediction score. Thus, the process inspection data input unit 412 receives the input of the inspection data from the terminal device 201 (S501 in FIG. 5A).

The inspection data of the process inspection is input from the terminal device 201 for each combination with an identifier for identifying the inspection item of the process inspection as indicated by the inspection data table illustrated in FIG. 6A. In addition, in accordance with the inspection data, an identifier (such as a serial number of a product) of an in-process product, which is a process inspection target, is also input from the terminal device 201 to the prediction score calculation device 202.

When the process inspection data input unit 412 receives the input of the inspection data, the prediction score calculation unit 410 reads a learning parameter of a machine learning model from the external storage device 203 (S502 in FIG. 5A). The prediction score calculation unit 410 sets the learning parameter in the machine learning model (S503 in FIG. 5A).

By setting the learning parameter, the machine learning model can output the prediction score obtained by predicting the result of the quality determination of the final inspection 107 according to the inspection data of the process inspections 103 and 105. The prediction score calculation unit 410 inputs the inspection data to the machine learning model in which the learning parameter is set and calculates the prediction score (S504 in FIG. 5A).

The prediction score output unit 411 outputs the prediction score calculated by the prediction score calculation unit 410 to the terminal device 201 (S505 in FIG. 5A). The terminal device 201 compares a threshold value of the prediction score to be described below with the prediction score, and displays a predicted quality determination result of the final inspection 107. The inspection worker refers to the displayed prediction of the quality determination result and performs exclusion or the like of the in-process product.

The machine learning unit 420 executes machine learning in response to an instruction or the like from an administrator of the prediction score calculation device 202. Specifically, first, as the learning data, the inspection data of the process inspection and the quality determination result of the final inspection corresponding to the inspection data are read from the external storage device 203 (S511 in FIG. 5B). Here, the quality determination result of the final inspection is a quality determination result of the final inspection regarding the finished product produced using the in-process product, which is a process inspection target. The in-process product may be a component incorporated into a finished product, or may be a jig, a material, or the like used for producing a finished product.

In addition, the inspection data of the process inspection has a data structure similar to that of the inspection data of the process inspection input to the prediction score calculation unit 410 (FIG. 6A). In addition, similar to the inspection data of the process inspection, the quality determination result of the final inspection is also a combination of the identifier for identifying the quality determination result of the final inspection and the quality determination result of the final inspection. In the present embodiment, the value of the quality determination result is set to “1” when the finished product is determined to be non-defective in the final inspection, and the value of the quality determination result is set to “0” when the finished product is determined to be defective.

Next, using the quality determination result of the final inspection as training data, machine learning is performed such that the prediction score obtained by inputting the inspection data to the machine learning model asymptotically approaches the training data, and a learning parameter is generated (S512 in FIG. 5B). When the machine learning is completed, the generated learning parameter is stored in the external storage device 203 (S513 in FIG. 5B). In addition, the prediction score calculated using the learning parameter is stored in the external storage device 203 in association with the learning data input to the machine learning model for calculating the prediction score (S514 in FIG. 5B).

Note that, in the present embodiment, a gradient boosting decision tree is used as the machine learning model. However, it goes without saying that the present disclosure is not limited to this, and instead of the gradient boosting decision tree, a machine learning model such as decision tree analysis, logistic regression, random forest, or neural network may be used.

Since machine learning models have different features, it is desirable to cause a plurality of machine learning models to perform machine learning using learning data, then compare the accuracy of prediction scores calculated using verification data, and adopt a machine learning model most suitable for a production line, which is a target of prediction.

The learning data input unit 430 receives the inspection data of the process inspection such as the inspection data table illustrated in FIG. 6A and the quality determination result of the final inspection such as the quality determination result table illustrated in FIG. 6B from the terminal device 201 as the learning data used when the machine learning unit 420 performs the machine learning, and stores them in the external storage device 203. Note that in a case where the terminal device 201 can directly access the external storage device 203 via the communication network 210, the learning data may be directly stored in the external storage device 203 without going through the learning data input unit 430.

The threshold value determination unit 440 determines a threshold value to be used in predicting the quality determination result of the final inspection after using the prediction score. Specifically, the learning data is read from the external storage device 203 (S701 in FIG. 7) and the prediction score corresponding to the learning data is read (S702 in FIG. 7) in response to an instruction of an administrator of the prediction score calculation device 202 to perform machine learning to generate a learning parameter, a request from the terminal device 201, or the like.

Next, the quality determination results of the final inspections are sorted among the learning data corresponding to the prediction score using the prediction score as a key (S703 in FIG. 7). In the present embodiment, the prediction score also takes a value within a range of 0 or more and 1 or less, corresponding to the fact that the determination of non-defective is “1” and the determination of defective is “0” in the final inspection.

FIG. 8 is a table illustrating a sorting result. As illustrated in Table 801 of FIG. 8, the serial numbers of the in-process products, the quality determination results of the final inspection with respect to the finished products using the in-process products, and the prediction scores output from the machine learning model by inputting the inspection data of the process inspection with respect to the in-process products are sorted in descending order of the prediction scores.

In FIG. 8, the prediction score is the probability that the finished product is determined to be non-defective in the final inspection. Therefore, there is a higher probability that an in-process product having a higher prediction score is determined to be non-defective in the final inspection, and conversely, there is a higher probability that an in-process product having a lower prediction score is determined to be defective in the final inspection.

Therefore, for example, when the threshold value of the prediction score is set to a high value of 0.701433 and when it is predicted that a finished product using an in-process product having a prediction score equal to or higher than the threshold value is determined to be non-defective, as shown in Table 811, the number of in-process products with which the finished product is predicted to be erroneously determined to be defective in the final inspection among in-process products constituting the finished product determined to be non-defective in the final inspection (the number of error reports) is 323, which is excessively large. When the number of error reports is too large, although it is highly probable that the finished product using the in-process products is determined to be non-defective in the final inspection, the number of in-process products to be predicted to be defective and excluded or the like becomes too large, so that unnecessary cost is generated.

On the other hand, when the threshold value of the prediction score is set to a low value of 0.399902 and when it is predicted that a finished product using an in-process product having a prediction score equal to or higher than the threshold value is determined to be non-defective, as shown in Table 813, the number of in-process products with which the finished product is predicted to be correctly determined to be defective in the final inspection among in-process products constituting the finished product determined to be defective in the final inspection (the number of defectives captured) is 2, which is excessively small. When the number of defectives captured can be increased, it is not necessary to send the in-process products constituting the finished product having a high probability of being determined to be defective in the final inspection to the next process, so that the occurrence of unnecessary cost can be reduced.

Therefore, when the threshold value of the prediction score is set to 0.517755, which is the same as the prediction value of the serial number XA1043147, as shown in Table 812, it is possible to predict the quality determination result of the final inspection so that the number of error reports does not become too large and the number of defectives captured does not become too small.

With respect to the series of learning data used for determining the threshold value, the cost merit that can be obtained by predicting the quality determination result of the final inspection can be calculated from the number of error reports, an error report loss cost (a negative value in the present embodiment) that is a loss that can be caused by one error report, the number of defectives captured, and a defective capture merit (a positive value in the present embodiment) that is a profit that can be obtained by one defective capture by using a cost function such as Formula (1) described below.

(Cost merit)=(number of error reports)×(error report loss cost)+(number of defectives captured)×(defective capture merit) (1)

Therefore, it is desirable that the threshold value for predicting the quality determination result of the finished product in the final inspection from the prediction score is determined so that the cost merit calculated by using the cost function of Formula (1) described above is maximized

Therefore, the threshold value determination unit 440 further reads the error report loss cost and the defective capture merit from the external storage device 203 (S704 in FIG. 7), specifies the number of error reports and the number of defectives captured in order of sorted prediction scores by using the prediction score as the threshold value, and calculates the cost merit using the cost function of Formula (1) described above (S705 in FIG. 7). Thereafter, it is sufficient if a threshold value corresponding to the maximum cost merit within the calculated cost merit is specified (S706 in FIG. 7).

A threshold value output unit 441 outputs the threshold value specified by the threshold value determination unit 440 to the terminal device 201. The inspection worker operates the terminal device 201 and predicts the quality determination result in the final inspection regarding the finished product using the in-process product according to whether or not the prediction score calculated using the machine learning model from the inspection data obtained in the process inspection for each in-process product is equal to or larger than the threshold value.

When it is predicted to be determined to be non-defective in the final inspection, the in-process product is sent to the next process, and when it is predicted to be determined to be defective in the final inspection, the in-process product is excluded or corrected, such that the number of finished products determined to be defective in the final inspection can be reduced as compared with a case where the quality determination result in the final inspection is not predicted, so that the production cost of the finished product can be improved.

[4] Modifications

Although the present disclosure has been described on the basis of the embodiment, it is needless to say that the present disclosure is not limited to the above-described embodiment, and the modifications described below can be performed.

(4-1) In the above embodiment, the case where the quality determination result in the final inspection regarding the finished product using the in-process product is predicted from the inspection data of the process inspections 103 and 105 has been described as an example, but it is needless to say that the present invention is not limited to this, and those described below may be performed instead.

For example, the quality determination result in the final inspection may be predicted using only the inspection data of one of the process inspections 103 and 105, or the quality determination result in the final inspection may be predicted using inspection data of yet another process inspection in addition to the inspection data of the process inspections 103 and 105.

In a case where the quality determination result of the final inspection is predicted only from the result data of a single process inspection, the processing load for calculating the prediction score can be reduced. On the other hand, when the quality determination result of the final inspection is predicted only from the result data of a plurality of process inspections, the multilateral inspection data can be referred to, so that the accuracy of prediction of the quality determination result can be improved.

In addition, as illustrated in FIG. 9, when a plurality of process inspections 903 and 913 is performed in parallel, the quality determination result of the final inspection may be predicted from their inspection data after waiting for completion of all of the plurality of process inspections 903 and 913. The prediction may be performed using the inspection data of the reception inspection in addition to the process inspections, or the prediction may be performed by combining the inspection data of the process inspections and the inspection data of the reception inspection.

In addition, it is needless to say that the prediction target is not limited to the quality determination result of the final inspection, and the inspection result of the downstream process inspection may be predicted from one or more pieces of upstream process inspection data in the production line 1. In the inspection before the unit price of the in-process product increases, when the inspection result after the unit price of the in-process product increases is predicted and the in-process product predicted to be determined to be defective is excluded or the like, the loss of production cost can be reduced as compared with the case where the in-process product determined to be defective in the inspection after the unit price of the in-process product increases is excluded or the like.

The machine learning and the prediction of the score by the machine learning model can further use the inspection data described below. For example, among the inspection data in the process inspection, inspection data regarding a part or function of the inspection target related to the item of the final inspection may be used for the machine learning of the machine learning model or the prediction of the score using the machine learning model. In the machine learning, the accuracy of the machine learning is easily improved by increasing the information diversity of the feature amount, but the accuracy of the machine learning is more easily improved by using the relevant data.

In addition, when the inspection data in a plurality of process inspections is used as the machine learning data of the machine learning model or the like, it is desirable to use the inspection data before and after processing of the same portion of the inspection target. In this way, since a change in the quality determination result of the final inspection caused by the processing can be reflected in the machine learning, it is possible to exclude the in-process product predicted to be determined to be defective due to the processing without waiting for the final inspection.

In addition, when the inspection data in a plurality of process inspections is used, measurement data for different parts of the inspection target in the process inspections may be combined and used as the inspection data. As the information diversity of the feature amount increases, the accuracy of the machine learning easily increases.

In addition, when the inspection data in a plurality of process inspections is used, measurement data of physical amounts different from each other between process inspections for the same part of the inspection target may be used in combination as the inspection data. As the information diversity of the feature amount increases, the accuracy of the machine learning easily increases.

(4-2) In the above embodiment, the case where raw inspection data obtained in the process inspection is used for predicting the quality determination result of the final inspection has been described as an example, but it is needless to say that the present disclosure is not limited to this, and the quality determination result of the final inspection may be predicted using a statistic amount that can be calculated from the raw inspection data instead of the raw inspection data or in addition to the raw inspection data.

The statistic amount such as the minimum value, the maximum value, the average value, the median value, the standard deviation, and the variance that can be calculated from the inspection data is also deeply related to each inspection data, and thus is useful in predicting the quality determination result in the final inspection.

The calculation of the statistic amount may be performed as preprocessing for the prediction score calculation device 202 to calculate the prediction score, or the terminal device 201 may execute preprocessing for calculating the statistic amount and input the obtained statistic amount to the prediction score calculation device 202 as inspection data. In addition, in the preprocessing, a feature amount other than the statistic amount may be obtained. In this way, it is not necessary to cause the machine learning to search for the feature amount or cause the machine learning model to perform the processing of calculating the feature amount, and thus, it is possible to reduce the processing load related to the machine learning model.

(4-3) Although not specifically mentioned in the above embodiment, learning and prediction using the machine learning model can be implemented by using an open source library such as LightGBM for a boosting decision tree, and scikit-learn for logistic regression, decision tree analysis, and random forest, and neural network, for example, as described below.

For example, when the boosting decision tree is used as the machine learning model, first, the inspection data of the process inspection and the final inspection is recorded in a Comma Separated Values (CSV) file, and data is read from the CSV file (for example, the file name is “inspectiondata csv”) using LightGBM.

- df=pd.read #csv(“inspectiondata.csv”, encoding=‘SHIFT-JIS’)

Note that, in order to distinguish between the case of performing the machine learning and the case of verifying the result of the machine learning, labeling may be performed as described below.

- df #train=df[df[‘verify’]==0]
- df #test=df[df[‘verify’]==1]

Next, an explanatory variable is set.

- x #train=df #train.loc[:,‘inspection data #1’:‘inspection data #2’]
- x #test=df #test.loc[:,‘inspection data #1’:‘inspection data #2’]
- In addition, an objective variable is set.
- y #train=df #train [′pass/fail determination (fail: 1)′]
- y #test=df #test [′pass/fail determination (fail: 1)′]

Then, the LightGBM library (boosting decision tree) is imported.

- import lightgbm as lgb
- Learning data is set in the LightGBM library
- train=lgb.Dataset(x #train, label=y #train), and
- data for verification is also set.
- valid=train.create #valid(x #test, label=y #test)

Further, a hyperparameter is set.

param = [

‘boosting’:‘gbdt’,

‘objective’: ‘binary’,

‘metric’/binary’,

‘num#iterations’:20000,

‘num#leaves’:5,

‘min#data#in#leaf’:12,

‘learning#rate’:0.05099340688202502,

‘sub#feature’:0.9,

‘sub#row’:0.8,

‘bagging#freq’:1,

‘lambda#11’:0.054599059167475525,

‘lambda#12’:0.02858778312976605,

‘num#threads’:4,

‘seed’:seed # random number seed value setting

]

The hyperparameter is a parameter for designating a machine learning execution condition. It is sufficient if the number of learning steps to be executed, in other words, the number of decision trees is, for example,

num #round=20000. After such setting, the machine learning is executed to create a machine learning model.

- m #lgb=lgb.train(param, train, 20000, valid #sets=[valid], early #stopping #rounds=1500)

When performing the prediction using the created machine learning model (gradient boosting decision tree),

- pred #test=m #lgb.predict(x #test, num #iteration=m #lgb.best #iteration) is sufficient.

When logistic regression is used as a machine learning model, a scikit-learn library can be used. First, a package of logistic regression is imported

- from sklearnlinear #model import LogisticRegression, and
- First, an instance of a machine learning model is created
- lr=LogisticRegression( ), and
- machine learning is executed.
- lr.fit(x #train, y #train)

In a case where the prediction is performed using a learned machine learning model,

- pred #test=lr.predict #proba(x #test)
- is set.

Similarly, when decision tree analysis, random forest, and neural network are used as the machine learning model, the scikit-learn library can be used. Similarly, in the decision tree analysis, a package of the decision tree analysis is imported

- from sklearn.tree import DecisionTreeClassifier,
- an instance is created
- dt=DecisionTreeClassifier( ), and
- after executing machine learning
- dt.fit(x #train, y #train),
- the prediction is performed.
- pred #test=dt.predict(x #test)
- When random forest is used as the machine learning model, a package of the random forest is imported
- from sklearn.ensemble import RandomForestClassifier,
- an instance is created
- rf=RandomForestClassifier(random #state=777), and
- after executing machine learning
- rf.fit(x #train, y #train),
- the prediction is performed.
- pred #test=rf.predict(x #test)
- Similarly for the neural network, a package of scikit-learn is imported
- from sklearn.neural #network import MLPClassifier,
- an instance is created
- nn=MLPClassifier(solver=“sgd”, random #state=0, max #iter=10000), and
- after executing machine learning
- nn.fit(x #tmin, y #train),