100_i (i=0, 1, . . . , N−1) uses its own data to obtain a different local update parameter Δ_i(i=0, 1, . . . , N−1) indicating a training result. In addition, each of the first to Nth federated learning apparatuses 100_i (i=0, 1, . . . , N−1) holds a different amount of data.

Thus, in order to aggregate the shares of the local update parameters and obtain a share of a global update parameter, each of the first to Nth federated learning apparatuses 100_i (i=0, 1, . . . , N−1) needs to compute a weighted average of the local update parameters Δ_i(i=0, 1, . . . , N−1) weighted by data volumes V_iof the first to Nth federated learning apparatuses 100_i (i=0, 1, . . . , N−1).

The aggregation and secure computation part 104_i computes a weighted average of the local update parameters Δ_i(i=0, 1, . . . , N−1) weighted by the data volumes V_i(i=0, 1, . . . , N−1) by performing addition of the shares of the local update parameters and multiplication of the shares by cleartext constants. The data volumes V_i(i=0, 1, . . . , N−1), which have been acquired and stored in advance, may be used. Alternatively, the data volumes V_i(i=0, 1, . . . , N−1) may be transmitted when the shares of the local update parameters are transmitted.

In normal secure computations, accumulation requires a lot of communication costs. However, the addition of the shares and the multiplication of the shares by cleartext constants can be computed without communications. Concretely, the addition and the multiplication can be computed as follows.

The addition of the shares [x]+[y]=[z] can be computed by [z]_i=(z_i, z_i+1, . . . , z_i+N−1) as z_i=x_i+y_imod n. The multiplication of the shares by cleartext constants [x]·c=[x·c] can be computed by [x·c]_i=(x′_i, x′_i+1, . . . , x′_i+N−1) as x′_i=c·x_imod n.

The global training part 105_i is a processing part for reconstructing the shares of the global update parameter and updating the shared model. The global training part 105_i receives the shares of the global update parameter stored in the first to Nth federated learning apparatuses 100_i (i=0, 1, . . . , N−1) in a secret sharing manner, and reconstructs the global update parameter. Next, by applying the reconstructed global update parameter to the shared model, which is the target of the federated learning, the global training part 105_i trains the shared model.

As described above, in the federated learning system according to the first example embodiment, since each of the first to Nth federated learning apparatuses 100_i (i=0, 1, . . . , N−1) has the above configuration, it is possible to perform the federated learning while concealing the local update parameters. In the federated learning system according to the first example embodiment, since the processing can be performed only by performing a secure computation with addition and multiplication by cleartext constants (without multiplications), the communication cost is low. In addition, since the federated learning system according to the first example embodiment uses the N-out-of-N additive secret sharing of which the replication is 1, the federated learning system has a high security against collusion attacks.

Method

Next, a federated learning method according to the first example embodiment will be described. FIG. 3 is a flowchart illustrating an outline of a procedure of the federated learning method according to the first example embodiment.

As illustrated in FIG. 3, the federated learning method according to the first example embodiment includes a model replication step (S11), a local training step (S12), an update parameter secret sharing step (S13), an update parameter aggregation step (S14), and a global training step (S15).

In the model replication step (S11), each of the first to Nth federated learning apparatuses 100_i (i=0, 1, . . . , N−1) replicates and stores a shared model, which is the target of the federated learning, as a local model. Next, in the local training step (S12), each of the first to Nth federated learning apparatuses 100_i (i=0, 1, . . . , N−1) trains the local model by using the data held thereby.

In the update parameter secret sharing step (S13), each of the first to Nth federated learning apparatuses 100_i (i=0, 1, . . . , N−1) decomposes a local update parameter obtained in the local training step (S12) into shares based on an additive secret sharing scheme and transmits shares to the other federated learning apparatuses 100_i. In this step, the above N-out-of-N additive secret sharing is used as the additive secret sharing scheme. That is, the replication of the additive secret sharing scheme according to the first example embodiment is 1, and a different share is given to each of the first to Nth federated learning apparatuses 100_i (i=0, 1, . . . , N−1). That is, this additive secret sharing achieves the highest security against collusion attacks in a sense that the original value x cannot be reconstructed even if, of all the N participants, t participants collude with each other (t<N).

In the update parameter aggregation step (S14), each of the first to Nth federated learning apparatuses 100_i (i=0, 1, . . . , N−1) performs a secure computation for the shares of a global update parameter by performing addition of the received shares of the local update parameters and multiplication of the shares by cleartext constants. Concretely, each of the first to Nth federated learning apparatuses 100_i (i=0, 1, . . . , N−1) computes a weighted average of the local update parameters Δ_i(i=0, 1, . . . , N−1) weighted by their respective data volumes V_i(i=0, 1, . . . , N−1).

In the global training step (S15), each of the first to Nth federated learning apparatuses 100_i (i=0, 1, . . . , N−1) reconstructs the shares of the global update parameter and updates the shared model by the global update parameter. Concretely, each of the first to Nth federated learning apparatuses 100_i (i=0, 1, . . . , N−1) receives the shares of the global update parameter stored in a secret sharing manner from the other federated learning apparatuses 100_i and reconstructs the global update parameter. Next, each of the first to Nth federated learning apparatuses 100_i (i=0, 1, . . . , N−1) trains the shared model by applying the reconstructed global update parameter to the shared model, which is the target of the federated learning.

As described above, in the federated learning method according to the first example embodiment, since each of the first to Nth federated learning apparatuses 100_i (i=0, 1, . . . , N−1) has the above configuration, it is possible to perform the federated learning while concealing the local update parameters. In the federated learning method according to the first example embodiment, since the processing can be performed only by performing a secure computation with addition and multiplication by cleartext constants (without multiplications), the communication cost is low. In addition, since the federated learning system according to the first example embodiment uses the N-out-of-N additive secret sharing of which the replication is 1, the federated learning system has a high security against collusion attacks.

Second Example Embodiment

Hereinafter, a federated learning system and federated learning apparatuses according to a second example embodiment will be described with reference to FIGS. 4 and 5.

FIG. 4 is a block diagram illustrating a functional configuration example of the federated learning system according to the second example embodiment. As illustrated in FIG. 4, a federated learning system 200 according to the second example embodiment includes first to Nth federated learning apparatuses 200_i (i=0, 1, . . . , N−1). These first to Nth federated learning apparatuses 200_i (i=0, 1, . . . , N−1) are connected to each other via a network such that these apparatuses can communicate with each other. N is a natural number of 3 or more.

As in the other example embodiments, the first to Nth federated learning apparatuses 200_i (i=0, 1, . . . , N−1) are each operated by an independent organization (user). Although the first to Nth federated learning apparatuses 200_i (i=0, 1, . . . , N−1) do not wish to disclose their own data to the outside, these federated learning apparatuses 200_i wish to mutually utilize their data and to perform machine learning on a shared model.

FIG. 5 is a block diagram illustrating a functional configuration example of a federated learning apparatus according to the second example embodiment. As illustrated in FIG. 5, each of the first to Nth federated learning apparatuses 200_i (i=0, 1, . . . , N−1) includes a model storage part 201_i, a local training part 202_i, a secret sharing part 203_i, an aggregation and secure computation part 204_i, a global training part 205_i, a first stop-type fraud detection part 206_i, and a second stop-type fraud detection part 207_i.

The model storage part 201_i is a storage device for replicating and storing a shared model, which is the target of the federated learning, as a local model. In addition, the local training part 202_i is a processing part for training the local model by using data held by the individual one of the first to Nth federated learning apparatuses 200_i (i=0, 1, . . . , N−1).

The secret sharing part 203_i is a processing part for decomposing a local update parameter indicating the training result of the local model into shares based on an additive secret sharing scheme and for transmitting shares to the other federated learning apparatuses 200_i.

The additive secret sharing scheme used in the second example embodiment is (N−t)-out-of-N replicated secret sharing [(N−t, N)-RSS], which is a particular form of the additive secret sharing. Since the (N−t)-out-of-N replicated secret sharing is a particular form of the additive secret sharing, a value x is decomposed into a sum on a residue class ring Z_Lof modulo L as follows.

x=x₀+ . . . +x_N−1mod L [Equation 3]

In the (N−t)-out-of-N replicated secret sharing, the value x decomposed into a sum on the residue class ring Z_Lof modulo L is dispersedly held as follows such that the replication is t+1. Note that t(t+1)<N. That is, for example, when N=3, t=1.

[x]_{L,(t+1,N,t+1)R}=[x]_{L(t+1,N,t+1)R,0}, . . . ,[x]_{L(t+1,N,t+1)R,N−1})

[x]_{L,(t+1,N,t+1)R,i}=(x_i, . . . ,x_i+t) where x_(N−1)+1=x₀ [Equations 4]

When the value x is held in a secret sharing manner as described above, even if a number t of participants of (among) all the N participants collude with each other to reconstruct the value, the security is ensured. In addition, as will be described below, since the replication is t+1, even if t participants collude with each other and falsify a value, the falsification can be detected, and the processing can be stopped.

The first stop-type fraud detection part 206_i is a processing part that requests the other federated learning apparatuses to transmit the shares of their local update parameters and stops the processing if the shares of the local update parameters transmitted in response to the request do not match.

As described above, in the (N−t)-out-of-N replicated secret sharing, t+1 participants hold replications of the same sub-shares. Thus, even if t participants collude with each other and falsify a value, it is impossible to falsify all the sub-shares to the same values. Thus, the first stop-type fraud detection part 206_i requests other t federated learning apparatuses 200_i, which are supposed to hold replications of the same sub-shares as its own sub-shares, to transmit the shares of their local update parameters. If the shares of the local update parameters transmitted in response to the request do not match, the first stop-type fraud detection part 206_i determines that there is a dishonest participant and stops the processing.

In contrast, if the shares of the local update parameters transmitted in response to the request match, the first stop-type fraud detection part 206_i continues the processing and adopts the shares as the accurate shares of the local update parameters. When whether the shares of the local update parameters match is determined, the other federated learning apparatuses 200_i may first convert the shares into hash values by using a hash function and may next transmit the hash values to the first stop-type fraud detection part 206_i, instead of directly transmitting the shares. By transmitting the hash values, the communication amount can be reduced.

The aggregation and secure computation part 204_i performs a secure computation for shares of a global update parameter by performing addition of the shares of the local update parameters adopted as the accurate shares of the local update parameters by the first stop-type fraud detection part 206_i and by performing multiplication of the shares by cleartext constants. Concretely, the aggregation and secure computation part 204_i computes a weighted average of local update parameters Δ_i(i=0, 1, . . . , N−1) weighted by data volumes V_i(i=0, 1, . . . , N−1) by performing addition of the shares of the local update parameters and multiplication of the shares by cleartext constants.

The second stop-type fraud detection part 207_i is a processing part that stops the processing if the shares of the global update parameter that are received to reconstruct the global update parameter do not match. Because the shares of the global update parameter are held in a secret sharing manner in the first to Nth federated learning apparatuses 200_i (i=0, 1, . . . , N−1), it is necessary to aggregate the shares of the global update parameter held in a secret sharing manner in the first to Nth federated learning apparatuses 200_i (i=0, 1, . . . , N−1) when the global update parameter is reconstructed. If any of the shares of the global update parameter has been falsified, the global update parameter cannot be reconstructed accurately.

The second stop-type fraud detection part 207_i requests other t+1 federated learning apparatuses 200_i, which are supposed to hold replications of the same sub-shares of the global update parameter, to transmit the shares of their global update parameter. If the shares of the global update parameter transmitted in response to the request do not match, the second stop-type fraud detection part 207_i determines that there is a dishonest participant and stops the processing. In contrast, if the shares of the global update parameter transmitted in response to the request match, the second stop-type fraud detection part 207_i adopts the shares as the accurate shares of the global update parameter.

When whether the shares of the global update parameter match is determined, the other federated learning apparatuses 200_i may first convert the shares into hash values by using a hash function and may next transmit the hash values to the second stop-type fraud detection part 207_i, instead of directly transmitting the shares. By transmitting the hash values, the communication amount can be reduced.

The global training part 205_i is a processing part that reconstructs the shares of the global update parameter and that updates the shared model by the global update parameter. The global training part 205_i reconstructs the global update parameter by using the shares adopted as the accurate shares of the global update parameter by the second stop-type fraud detection part 207_i. Next, the global training part 205_i trains the shared model by applying the reconstructed global update parameter to the shared model, which is the target of the federated learning.

As described above, in the federated learning system according to the second example embodiment, since each of the first to Nth federated learning apparatuses 200_i (i=0, 1, . . . , N−1) has the above configuration, it is possible to perform the federated learning while concealing the local update parameters. In the federated learning system according to the second example embodiment, since the processing can be performed only by performing a secure computation with addition and multiplication by cleartext constants (without multiplications), the communication cost is low. In addition, since the federated learning system according to the second example embodiment uses the (N−t)-out-of-N replicated secret sharing of which the replication is t+1, even if t participants collude with each other and falsify a value, the federated learning system can detect the falsification and stop the processing.

Method

Next, a federated learning method according to the second example embodiment will be described. FIG. 6 is a flowchart illustrating an outline of a procedure of the federated learning method according to the second example embodiment.

As illustrated in FIG. 6, the federated learning method according to the second example embodiment includes a model replication step (S21), a local training step (S22), an update parameter secret sharing step (S23), a step of first stop-type fraud detection step (S24), an update parameter aggregation step (S25), a second stop-type fraud detection step (S26), and a global training step (S27).

In the model replication step (S21), each of the first to Nth federated learning apparatuses 200_i (i=0, 1, . . . , N−1) replicates and stores a shared model, which is the target of the federated learning, as a local model. Next, in the local training step (S22), each of the first to Nth federated learning apparatuses 200_i (i=0, 1, . . . , N−1) trains the local model by using the data held thereby.

In the update parameter secret sharing step (S23), each of the first to Nth federated learning apparatuses 200_i (i=0, 1, . . . , N−1) decomposes a local update parameter obtained in the local training step (S22) into shares based on an additive secret sharing scheme and transmits shares to the other federated learning apparatuses 200_i (i=0, 1, . . . , N−1). In this step, the above (N−t)-out-of-N replicated secret sharing is used as the additive secret sharing scheme. That is, in the additive secret sharing scheme according to the second example embodiment, the value x decomposed into a sum on the residue class ring Z_Lof modulo L is distributed in a secret sharing manner such that the replication is t+1.

In the (N−t)-out-of-N replicated secret sharing, t+1 participants hold replications of the same sub-shares. Thus, even if t participants collude with each other and falsify a value, it is impossible to falsify all the sub-shares to the same values. Thus, in the first stop-type fraud detection step (S24), the first stop-type fraud detection part 206_i requests other t federated learning apparatuses 200_i, which are supposed to hold replications of the same sub-shares as its own sub-shares, to transmit the shares of their local update parameters. If the shares of the local update parameters transmitted in response to the request do not match, the first stop-type fraud detection part 206_i determines that there is a dishonest participant.

In the first stop-type fraud detection step (S24), each of the first to Nth federated learning apparatuses 200_i (i=0, 1, . . . , N−1) requests the other federated learning apparatuses to transmit the shares of their local update parameters. If the shares of the local update parameters transmitted in response to the request do not match (S24; N), the first stop-type fraud detection part 206_i determines that there is a dishonest participant and stops the processing.

In contrast, in the first stop-type fraud detection step (S24), if each of the first to Nth federated learning apparatuses 200_i (i=0, 1, . . . , N−1) determines that the shares of the local update parameters transmitted in response to the request match (S24; Y), its first stop-type fraud detection part 206_i continues the processing and adopts the shares as the accurate shares of the local update parameters.

In the update parameter aggregation step (S25), each of the first to Nth federated learning apparatuses 200_i (i=0, 1, . . . , N−1) performs a secure computation for shares of a global update parameter by performing addition of the shares of the local update parameters adopted as the accurate shares of the local update parameters in the first stop-type fraud detection step (S24) and by performing multiplication of the shares by cleartext constants. In the update parameter aggregation step (S25), each of the first to Nth federated learning apparatuses 200_i (i=0, 1, . . . , N−1) computes a weighted average of the local update parameters Δ_i(i=0, 1, . . . , N−1) weighted by the data volumes V_i(i=0, 1, . . . , N−1) by performing addition of the shares of the local update parameters and multiplication of the shares by cleartext constants.

In the second stop-type fraud detection step (S26), if each of the first to Nth federated learning apparatuses 200_i (i=0, 1, . . . , N−1) determines that the shares of the global update parameter that are received to reconstruct the global update parameter do not match, its second stop-type fraud detection part 207_i stops the processing.

In the second stop-type fraud detection step (S26), the second stop-type fraud detection part 207_i requests other t+1 federated learning apparatuses 200_i, which are supposed to hold replications of the same sub-shares of the global update parameter, to transmit their shares. If the shares of the global update parameter transmitted in response to the request do not match (S26; N), the second stop-type fraud detection part 207_i determines that there is a dishonest participant and stops the processing. In contrast, if the shares of the global update parameter transmitted in response to the request match (S26; Y), the second stop-type fraud detection part 207_i adopts the shares as the accurate shares of the global update parameter.

In the global training step (S27), each of the first to Nth federated learning apparatuses 200_i (i=0, 1, . . . , N−1) reconstructs the shares of the global update parameter and updates the shared model by the global update parameter. Each of the first to Nth federated learning apparatuses 200_i (i=0, 1, . . . , N−1) reconstructs the global update parameter by using the shares adopted as the accurate shares of the global update parameter in the second stop-type fraud detection step (S26). Next, each of the first to Nth federated learning apparatuses 200_i (i=0, 1, . . . , N−1) trains the shared model by applying the reconstructed global update parameter to the shared model, which is the target of the federated learning.

As described above, in the federated learning method according to the second example embodiment, since each of the first to Nth federated learning apparatuses 200_i (i=0, 1, . . . , N−1) has the above configuration, it is possible to perform the federated learning while concealing the local update parameters. In the federated learning method according to the second example embodiment, since the processing can be performed only by performing a secure computation with addition and multiplication by cleartext constants (without multiplications), the communication cost is low. In addition, since the federated learning system according to the second example embodiment uses the (N−t)-out-of-N replicated secret sharing of which the replication is t+1, even if t participants collude with each other and falsify a value, the federated learning system can detect the falsification and stop the processing.

Third Example Embodiment

Hereinafter, a federated learning system and federated learning apparatuses according to a third example embodiment will be described with reference to FIGS. 7 and 8.

FIG. 7 is a block diagram illustrating a functional configuration example of the federated learning system according to the third example embodiment. As illustrated in FIG. 7, a federated learning system 300 according to the third example embodiment includes first to Nth federated learning apparatuses 300_i (i=0, 1, . . . , N−1). These first to Nth federated learning apparatuses 300_i (i=0, 1, . . . , N−1) are connected to each other via a network such that these apparatuses can communicate with each other. N is a natural number of 4 or more.

As in the other example embodiments, the first to Nth federated learning apparatuses 300_i (i=0, 1, . . . , N−1) are each operated by an independent organization (user). Although the first to Nth federated learning apparatuses 300_i (i=0, 1, . . . , N−1) do not wish to disclose their own data to the outside, these federated learning apparatuses 300_i wish to mutually utilize their data and to perform machine learning on a shared model.

FIG. 8 is a block diagram illustrating a functional configuration example of a federated learning apparatus according to the third example embodiment. As illustrated in FIG. 8, each of the first to Nth federated learning apparatuses 300_i (i=0, 1, . . . , N−1) includes a model storage part 301_i, a local training part 302_i, a secret sharing part 303_i, an aggregation and secure computation part 304_i, a global training part 305_i, a first nonstop-type fraud detection part 306_i, and a second nonstop-type fraud detection part 307_i.

The model storage part 301_i is a storage device for replicating and storing a shared model, which is the target of the federated learning, as a local model. In addition, the local training part 302_i is a processing part for training the local model by using data held by the individual one of the first to Nth federated learning apparatuses 300_i (i=0, 1, . . . , N−1).

The secret sharing part 303_i is a processing part for decomposing a local update parameter indicating the training result of the local model to shares based on an additive secret sharing scheme and for transmitting shares to the other federated learning apparatuses 300_i.

The additive secret sharing scheme used in the third example embodiment is (N−2t)-out-of-N replicated secret sharing [(N−2t, N)-RSS], which is a particular form of the additive secret sharing. Since the (N−2t)-out-of-N replicated secret sharing is a particular form of the additive secret sharing, a value x is decomposed into a sum on a residue class ring Z_Lof modulo L as follows.

x=x₀+ . . . +x_N−1mod L [Equation 5]

In the (N−2t)-out-of-N replicated secret sharing, the value x decomposed into a sum on the residue class ring Z_Lof modulo L is dispersedly held as follows such that the replication is 2t+1. Note that t(2t+1)<N. That is, for example, when N=4, t=1.

[x]_{L,(t+1,N,2t+1)R}=[x]_{L(t+1,N,2t+1)R,0}, . . . ,[x]_{L(t+1,N,2t+1)R,N−1})

[x]_{L,(t+1,N,2t+1)R,i}=x_i, . . . ,x_i+2t) where x_(N−1)+1=x₀ [Equations 6]

When the value x is held in a secret sharing manner as described above, even if a number t of participants among all the N participants collude with each other to reconstruct the value, the security is ensured. In addition, as will be described below, since the replication is 2t+1, even if t participants collude with each other and falsify a value, the accurate values can be selected, and the processing can be continued.

The first nonstop-type fraud detection part 306_i is a processing part that requests the other federated learning apparatuses to transmit the shares of their local update parameters and adopts, of all the shares of the local update parameters transmitted in response to the request, a majority of shares that match, as the accurate shares of the local update parameters.

As described above, in the (N−2t)-out-of-N replicated secret sharing, 2t+1 participants hold replications of the same sub-shares. Thus, even if t participants collude with each other and falsify a value, it is impossible to falsify all the sub-shares to the same values, and t+1 participants hold the accurate values. That is, a majority of the values are accurate values. The first nonstop-type fraud detection part 306_i requests other 2t federated learning apparatuses 300_i, which are supposed to hold replications of the same sub-share as its own sub-shares, to transmit the shares of their local update parameters and adopts, of all the shares of the local update parameters transmitted in response to the request, a majority of shares that match, as accurate shares.

If federated learning apparatuses 300_i that provide local update parameters are operated by attackers and if falsification has already been made at the time when the federated learning apparatuses 300_i have distributed their local update parameters in a secret sharing manner, a majority of shares of the local update parameters could not match. In this case, the first nonstop-type fraud detection part 306_i does not adopt these local update parameters and continues the processing by recognizing the parameters as if the parameters were not present.

When whether the shares of the local update parameters match is determined, the other federated learning apparatuses 200_i may first convert the shares into hash values by using a hash function and may next transmit the hash values to the first nonstop-type fraud detection part 306_i, instead of directly transmitting the shares. By transmitting the hash values, the communication amount can be reduced.

The aggregation and secure computation part 304_i performs a secure computation for shares of a global update parameter by performing addition of the shares of the local update parameters adopted as the accurate shares of the local update parameters by the first nonstop-type fraud detection part 306_i and by performing multiplication of the shares by cleartext constants. Concretely, the aggregation and secure computation part 304_i computes a weighted average of local update parameters Δ_i(i=0, 1, . . . , N−1) weighted by data volumes V_i(i=0, 1, . . . , N−1) by performing addition of the shares of the local update parameters and multiplication of the shares by cleartext constants.

The second nonstop-type fraud detection part 307_i adopts, of all the shares of the global update parameter received to reconstruct the global update parameter, a majority of shares that match, as the accurate shares of the global update parameter. The second nonstop-type fraud detection part 307_i requests other 2t federated learning apparatuses 300_i, which are supposed to hold replications of the same sub-shares of the global update parameter, to transmit the shares of their global update parameter, and adopts, of all the shares of the global update parameter transmitted in response to the request, a majority of shares that match, as accurate values.

When whether the shares of the global update parameter match is determined, the other federated learning apparatuses 300_i may first convert the shares into hash values by using a hash function and may next transmit the hash values to the second nonstop-type fraud detection part 307_i, instead of directly transmitting the shares. By transmitting the hash values, the communication amount can be reduced.

The global training part 305_i is a processing part that reconstructs the shares of the global update parameter and that updates the shared model by the global update parameter. The global training part 305_i reconstructs the global update parameter by using the shares adopted as the accurate shares of the global update parameter by the second nonstop-type fraud detection part 307_i. Next, the global training part 305_i trains the shared model by applying the reconstructed global update parameter to the shared model, which is the target of the federated learning.

As described above, in the federated learning system according to the third example embodiment, since each of the first to Nth federated learning apparatuses 300_i (i=0, 1, . . . , N−1) has the above configuration, it is possible to perform the federated learning while concealing the local update parameters. In the federated learning system according to the third example embodiment, since the processing can be performed only by performing a secure computation with addition and multiplication by cleartext constants (without multiplications), the communication cost is low. In addition, since the federated learning system according to the third example embodiment uses the (N−2t)-out-of-N replicated secret sharing of which the replication is 2t+1, even if t participants collude with each other and falsify a value, the federated learning system can continue the processing by selecting the accurate values.

Method

Next, a federated learning method according to the third example embodiment will be described. FIG. 9 is a flowchart illustrating an outline of a procedure of the federated learning method according to the third example embodiment.

As illustrated in FIG. 9, the federated learning method according to the third example embodiment includes a model replication step (S31), a local training step (S32), an update parameter secret sharing step (S33), a first nonstop-type fraud detection step (S34), an update parameter aggregation step (S35), a second nonstop-type fraud detection step (S36), and a global training step (S37).

In the model replication step (S31), each of the first to Nth federated learning apparatuses 300_i (i=0, 1, . . . , N−1) replicates and stores a shared model, which is the target of the federated learning, as a local model. Next, in the local training step (S32), each of the first to Nth federated learning apparatuses 300_i (i=0, 1, . . . , N−1) trains the local model by using the data held thereby.

In the update parameter secret sharing step (S33), each of the first to Nth federated learning apparatuses 300_i (i=0, 1, . . . , N−1) decomposes a local update parameter obtained in the local training step (S32) to shares based on an additive secret sharing scheme and transmits shares to the other federated learning apparatuses 300_i. In this step, the above (N−2t)-out-of-N replicated secret sharing is used as the additive secret sharing scheme. That is, in the additive secret sharing scheme according to the third example embodiment, the value x decomposed into a sum on the residue class ring Z_Lof modulo L is distributed in a secret sharing manner such that the replication is 2t+1.

In the (N−2t)-out-of-N replicated secret sharing, 2t+1 participants hold replications of the same sub-shares. Thus, even if t participants collude with each other and falsify a value, it is impossible to falsify all the sub-shares to the same values. In other words, t+1 participants hold the accurate values. That is, a majority of the values are accurate values. In the first nonstop-type fraud detection step (S34), the first nonstop-type fraud detection part 306_i requests other 2t federated learning apparatuses 300_i, which are supposed to hold replications of the same sub-shares as its own sub-shares, to transmit the shares of their local update parameters and adopts, of all the shares of the local update parameters transmitted in response to the request, a majority of shares that match, as accurate values.

In the update parameter aggregation step (S35), each of the first to Nth federated learning apparatuses 300_i (i=0, 1, . . . , N−1) performs a secure computation for shares of a global update parameter by performing addition of the shares of the local update parameters adopted as the accurate shares of the local update parameters in the first nonstop-type fraud detection step (S34) and by performing multiplication of the shares by cleartext constants. In the update parameter aggregation step (S35), the aggregation and secure computation part 304_i computes a weighted average of local update parameters Δ_i(i=0, 1, . . . , N−1) weighted by the data volumes V_i(i=0, 1, . . . , N−1) by performing addition of the shares of the local update parameters and multiplication of the shares by cleartext constants.

In the second nonstop-type fraud detection step (S36), each of the first to Nth federated learning apparatuses 300_i (i=0, 1, . . . , N−1) adopts, of all the shares of the global update parameter received to reconstruct the global update parameter, a majority of shares that match, as the accurate shares of the global update parameter.

In the second nonstop-type fraud detection step (S36), the second nonstop-type fraud detection part 307_i requests other 2t federated learning apparatuses 300_i, which are supposed to hold replications of the same sub-shares of the global update parameter, to transmit the shares of their global update parameter, and adopts, of all the shares of the global update parameter transmitted in response to the request, a majority of shares that match, as the accurate shares of the global update parameter.

In the global training step (S37), each of the first to Nth federated learning apparatuses 300_i (i=0, 1, . . . , N−1) reconstructs the shares of the global update parameter and updates the shared model by the global update parameter. In the second nonstop-type fraud detection step (S36), each of the first to Nth federated learning apparatuses 300_i (i=0, 1, . . . , N−1) reconstructs the global update parameter by using the shares adopted as the accurate shares of the global update parameter. Next, each of the first to Nth federated learning apparatuses 300_i (i=0, 1, . . . , N−1) trains the shared model by applying the reconstructed global update parameter to the shared model, which is the target of the federated learning.

As described above, in the federated learning method according to the third example embodiment, since each of the first to Nth federated learning apparatuses 300_i (i=0, 1, . . . , N−1) has the above configuration, it is possible to perform the federated learning while concealing the local update parameters. In the federated learning method according to the third example embodiment, since the processing can be performed only by performing a secure computation with addition and multiplication by cleartext constants (without multiplications), the communication cost is low. In addition, since the federated learning system according to the third example embodiment uses the (N−2t)-out-of-N replicated secret sharing of which the replication is 2t+1, even if t participants collude with each other and falsify a value, the federated learning system can continue the processing by selecting the accurate values.

Fourth Example Embodiment

Next, a federated learning system, federated learning server apparatuses, and federated learning client apparatuses according to a fourth example embodiment will be described with reference to FIGS. 10 to 12.

FIG. 10 is a block diagram illustrating a functional configuration example of the federated learning system according to the fourth example embodiment. As illustrated in FIG. 10, a federated learning system 400 according to the fourth example embodiment includes first to Nth federated learning client apparatuses 410_i (i=0, 1, . . . , N−1) and first to Nth federated learning server apparatuses 420_i (i=0, 1, . . . , N−1). The first to Nth federated learning client apparatuses 410_i (i=0, 1, . . . , N−1) and the first to Nth federated learning server apparatuses 420_i (i=0, 1, . . . , N−1) are connected to each other via a network such that these apparatuses can communicate with each other. N is a natural number of 2 or more.

The first to Nth federated learning client apparatuses 410_i (i=0, 1, . . . , N−1) are each operated by an independent organization (user). In addition, the first to Nth federated learning client apparatuses 410_i (i=0, 1, . . . , N−1) store data that their respective organizations do not wish to disclose to the outside. Although the organizations does not wish to disclose their own data to the outside, these organizations wish to mutually utilize their data and to perform machine learning on a shared model. In addition, the first to Nth federated learning server apparatuses 420_i (i=0, 1, . . . , N−1) are each operated by an independent organization (user).

That is, the federated learning system 400 according to the fourth example embodiment is a server-client federated learning system. More specifically, as will be described below, the federated learning system 400 according to the fourth example embodiment is a federated learning system obtained by dividing the apparatuses included in the federated learning system 100 according to the first example embodiment into a group of apparatuses serving as servers and a group of apparatuses serving as clients. Thus, although the first to Nth federated learning client apparatuses 410_i (i=0, 1, . . . , N−1) and the first to Nth federated learning server apparatuses 420_i (i=0, 1, . . . , N−1) are distinguished from each other for convenience of description, these apparatuses can adopt the same hardware configuration. If the same hardware configuration is adopted, for example, a program that limits one or more functions may be executed.

FIG. 11 is a block diagram illustrating a functional configuration example of a federated learning client apparatus according to the fourth example embodiment. As illustrated in FIG. 11, each of the first to Nth federated learning client apparatuses 410_i (i=0, 1, . . . , N−1) includes a model storage part 401_i, a local training part 402_i, a secret sharing part 403_i, and a global training part 405_i. FIG. 12 is a block diagram illustrating a functional configuration example of a federated learning server apparatus according to the fourth example embodiment. As illustrated in FIG. 12, each of the first to Nth federated learning server apparatuses 420_i (i=0, 1, . . . , N−1) includes an aggregation and secure computation part 404_i.

The model storage part 401_i of each of the first to Nth federated learning client apparatuses 410_i (i=0, 1, . . . , N−1) is a storage device for replicating and storing a shared model, which is the target of the federated learning, as a local model. In addition, the local training part 402_i is a processing part for training the local model by using data held by the individual one of the first to Nth federated learning client apparatuses 410_i (i=0, 1, . . . , N−1).

The secret sharing part 403_i of each of the first to Nth federated learning client apparatuses 410_i (i=0, 1, . . . , N−1) is a processing part for decomposing a local update parameter indicating the training result of the local model into shares based on an additive secret sharing scheme and for transmitting the shares to the first to Nth federated learning client apparatuses 410_i (i=0, 1, . . . , N−1).

The additive secret sharing scheme used in the fourth example embodiment is N-out-of-N additive secret sharing. That is, the secret sharing part 403_i decomposes the local update parameter indicating the training result of the local model into a sum on a residue class ring Z_Lof modulo L and transmits the values obtained by the decomposition to the first to Nth federated learning server apparatuses 420_i (i=0, 1, . . . , N−1) such that the replication is 1.

The aggregation and secure computation part 404_i of each of the first to Nth federated learning server apparatuses 420_i (i=0, 1, . . . , N−1) performs a secure computation for the shares of a global update parameter by performing addition of the shares of the local update parameters received from the first to Nth federated learning client apparatuses 410_i (i=0, 1, . . . , N−1) and by performing multiplication of the shares by cleartext constants. Concretely, the aggregation and secure computation part 404_i computes a weighted average of local update parameters Δ_i(i=0, 1, . . . , N−1) weighted by data volumes V_i(i=0, 1 N−1) by performing addition of the shares of the local update parameters and multiplication of the shares by cleartext constants.

The global training part 405_i of each of the first to Nth federated learning client apparatuses 410_i (i=0, 1, . . . , N−1) is a processing part for reconstructing the shares of a global update parameter and updating the shared model. The global training part 405_i reconstructs a global update parameter by using the shares adopted as the shares of the global update parameter received from the first to Nth federated learning server apparatuses 420_i (i=0, 1, . . . , N−1). Next, the global training part 405_i trains the shared model by applying the reconstructed global update parameter to the shared model, which is the target of the federated learning.

As described above, in the federated learning system according to the fourth example embodiment, since the first to Nth federated learning client apparatuses 410_i (i=0, 1, N−1) and the first to Nth federated learning server apparatuses 420_i (i=0, 1, . . . , N−1) have their respective configurations described above, it is possible to perform the federated learning while concealing the local update parameters. In the federated learning system according to the fourth example embodiment, since the processing can be performed only by performing a secure computation with addition and multiplication by cleartext constants (without multiplications), the communication cost is low. Since the federated learning system according to the fourth example embodiment uses the N-out-of-N additive secret sharing of which the replication is 1, the federated learning system has a high security against collusion attacks.

Method

Next, a federated learning method according to the fourth example embodiment will be described. FIG. 13 is a system flowchart illustrating an outline of a procedure of the federated learning method according to the fourth example embodiment.

As illustrated in FIG. 13, the federated learning method according to the fourth example embodiment includes the processing performed by the first to Nth federated learning client apparatuses 410_i (i=0, 1, . . . , N−1) and the processing performed by the first to Nth federated learning server apparatuses 420_i (i=0, 1, . . . , N−1). The processing performed by the first to Nth federated learning client apparatuses 410_i (i=0, 1, . . . , N−1) includes a model replication step (S41), a local training step (S42), an update parameter secret sharing step (S43), and a global training step (S45). The processing performed by the first to Nth federated learning server apparatuses 420_i (i=0, 1, . . . , N−1) includes an update parameter aggregation step (S44).

In the model replication step (S41), each of the first to Nth federated learning client apparatuses 410_i (i=0, 1, . . . , N−1) replicates and stores a shared model, which is the target of the federated learning, as a local model. In the local training step (S42), each of the first to Nth federated learning client apparatuses 410_i (i=0, 1, . . . , N−1) trains the local model by using the data held thereby.

In the update parameter secret sharing step (S43), each of the first to Nth federated learning client apparatuses 410_i (i=0, 1, . . . , N−1) decomposes a local update parameter obtained in the local training step (S42) to shares based on an additive secret sharing scheme and transmits the shares to the first to Nth federated learning server apparatuses 420_i (i=0, 1, . . . , N−1). In this step, the above N-out-of-N additive secret sharing is used as the additive secret sharing scheme. That is, the replication of the additive secret sharing scheme according to the fourth example embodiment is 1, and a different share is transmitted to each of the first to Nth federated learning server apparatuses 420_i (i=0, 1, . . . , N−1).

In the update parameter aggregation step (S44), each of the first to Nth federated learning server apparatuses 420_i (i=0, 1, . . . , N−1) performs a secure computation for the shares of a global update parameter by performing addition of the received shares of the local update parameters and multiplication of the shares by cleartext constants. Concretely, each of the first to Nth federated learning server apparatuses 420_i (i=0, 1, . . . , N−1) computes a weighted average of the local update parameters Δ_i(i=0, 1, . . . , N−1) weighted by their respective data volumes V_i(i=0, 1, . . . , N−1).

In the global training step (S45), each of the first to Nth federated learning client apparatuses 410_i (i=0, 1, . . . , N−1) reconstructs the shares of the global update parameter and updates the shared model by the global update parameter. Concretely, each of the first to Nth federated learning client apparatuses 410_i (i=0, 1, . . . , N−1) receives the shares of a global update parameter distributed in a secret sharing manner from the first to Nth federated learning server apparatuses 420_i (i=0, 1, . . . , N−1) and reconstructs the global update parameter. Next, each of the first to Nth federated learning client apparatuses 410_i (i=0, 1, . . . , N−1) trains the shared model by applying the reconstructed global update parameter to the shared model, which is the target of the federated learning.

As described above, in the federated learning method according to the fourth example embodiment, the first to Nth federated learning client apparatuses 410_i (i=0, 1, . . . , N−1) and the first to Nth federated learning server apparatuses 420_i (i=0, 1, . . . , N−1) can perform the federated learning while concealing the local update parameters by coordinating with each other. In the federated learning system according to the fourth example embodiment, since the processing can be performed only by performing a secure computation with addition and multiplication by cleartext constants (without multiplications), the communication cost is low. In addition, since the federated learning system according to the fourth example embodiment uses the N-out-of-N additive secret sharing of which the replication is 1, the federated learning system has a high security against collusion attacks.

Fifth Example Embodiment

Next, a federated learning system, federated learning server apparatuses, and federated learning client apparatuses according to a fifth example embodiment will be described with reference to FIGS. 14 to 16.

FIG. 14 is a block diagram illustrating a functional configuration example of the federated learning system according to the fifth example embodiment. As illustrated in FIG. 14, a federated learning system 500 according to the fifth example embodiment includes first to Nth federated learning client apparatuses 510_i (i=0, 1, . . . , N−1) and first to Nth federated learning server apparatuses 520_i (i=0, 1, . . . , N−1). The first to Nth federated learning client apparatuses 510_i (i=0, 1, . . . , N−1) and the first to Nth federated learning server apparatuses 520_i (i=0, 1, . . . , N−1) are connected to each other via a network such that these apparatuses can communicate with each other. N is a natural number of 3 or more.

The first to Nth federated learning client apparatuses 510_i (i=0, 1, . . . , N−1) are each operated by an independent organization (user). In addition, the first to Nth federated learning client apparatuses 510_i (i=0, 1, . . . , N−1) store data that their respective organizations do not wish to disclose to the outside. Although the organizations do not wish to disclose their own data to the outside, these organizations wish to mutually utilize their data and to perform machine learning on a shared model. In addition, the first to Nth federated learning server apparatuses 520_i (i=0, 1, . . . , N−1) are each operated by an independent organization (user).

That is, the federated learning system 500 according to the fifth example embodiment is a server-client federated learning system. More specifically, as will be described below, the federated learning system 500 according to the fifth example embodiment is a federated learning system obtained by dividing the apparatuses included in the federated learning system 200 according to the second example embodiment into a group of apparatuses serving as servers and a group of apparatuses serving as clients. Thus, although the first to Nth federated learning client apparatuses 510_i (i=0, 1, . . . , N−1) and the first to Nth federated learning server apparatuses 520_i (i=0, 1, . . . , N−1) are distinguished from each other for convenience of description, these apparatuses can adopt the same hardware configuration. If the same hardware configuration is adopted, for example, a program that limits one or more functions may be executed.

FIG. 15 is a block diagram illustrating a functional configuration example of a federated learning client apparatus according to the fifth example embodiment. As illustrated in FIG. 15, each of the first to Nth federated learning client apparatuses 510_i (i=0, 1, . . . , N−1) includes a model storage part 501_i, a local training part 502_i, a secret sharing part 503_i, a global training part 505_i, and a second stop-type fraud detection part 507_i. FIG. 16 is a block diagram illustrating a functional configuration example of a federated learning server apparatus according to the fifth example embodiment. As illustrated in FIG. 16, each of the first to Nth federated learning server apparatuses 520_i (i=0, 1, . . . , N−1) includes an aggregation and secure computation part 504_i and a first stop-type fraud detection part 506_i.

The model storage part 501_i of each of the first to Nth federated learning client apparatuses 510_i (i=0, 1, . . . , N−1) is a storage device for replicating and storing a shared model, which is the target of the federated learning, as a local model. In addition, the local training part 502_i is a processing part for training the local model by using data held by the individual one of the first to Nth federated learning client apparatuses 510_i (i=0, 1, . . . , N−1).

The secret sharing part 503_i of each of the first to Nth federated learning client apparatuses 510_i (i=0, 1, . . . , N−1) is a processing part for decomposing a local update parameter indicating the training result of the local model to shares based on an additive secret sharing scheme and for transmitting the shares to the first to Nth federated learning server apparatuses 520_i (i=0, 1, . . . , N−1).

The additive secret sharing scheme used in the fifth example embodiment is (N−t)-out-of-N replicated secret sharing. That is, the secret sharing part 503_i decomposes the local update parameter indicating the training result of the local model into a sum on a residue class ring Z_Lof modulo L and transmits the values obtained by the decomposition to the first to Nth federated learning server apparatuses 520_i (i=0, 1, . . . , N−1) such that the replication is t+1.

The first stop-type fraud detection part 506_i of each of the first to Nth federated learning server apparatuses 520_i (i=0, 1, . . . , N−1) is a processing part that requests the other federated learning server apparatuses to transmit the shares of their local update parameters and stops the processing if the shares of the local update parameters transmitted in response to the request do not match.

As described above, in the (N−t)-out-of-N replicated secret sharing, t+1 participants hold replications of the same sub-shares. Thus, even if t participants collude with each other and falsify a value, it is impossible to falsify all the sub-shares to the same values. Thus, the first stop-type fraud detection part 506_i requests other t federated learning server apparatuses 520_i, which are supposed to hold replications of the same sub-shares as its own sub-shares, to transmit the shares of their local update parameters. If the shares of the local update parameters transmitted in response to the request do not match, the first stop-type fraud detection part 506_i determines that there is a dishonest participant and stops the processing.

In contrast, if the shares of the local update parameters transmitted in response to the request match, the first stop-type fraud detection part 506_i continues the processing and adopts the shares as the accurate shares of the local update parameters. When whether the shares of the local update parameters match is determined, the other federated learning server apparatuses 520_i may first convert the shares into hash values by using a hash function and may next transmit the hash values to the first stop-type fraud detection part 506_i, instead of directly transmitting the shares. By transmitting the hash values, the communication amount can be reduced.

The aggregation and secure computation part 504_i of each of the first to Nth federated learning server apparatuses 520_i (i=0, 1, . . . , N−1) performs a secure computation for shares of a global update parameter by performing addition of the shares of the local update parameters adopted as the accurate shares of the local update parameters by the first stop-type fraud detection part 506_i and by performing multiplication of the shares by cleartext constants. Concretely, the aggregation and secure computation part 504_i computes a weighted average of local update parameters Δ_i(i=0, 1, . . . , N−1) weighted by data volumes V_i(i=0, 1, . . . , N−1) by performing addition of the shares of the local update parameters and multiplication of the shares by cleartext constants.

The second stop-type fraud detection part 507_i of each of the first to Nth federated learning client apparatuses 510_i (i=0, 1, . . . , N−1) is a processing part that stops the processing if the shares of the global update parameter received to reconstruct the global update parameter do not match. The second stop-type fraud detection part 507_i requests other t+1 federated learning server apparatuses 520_i (i=0, 1, . . . , N−1), which are supposed to hold replications of the same sub-shares of the global update parameter, to transmit the shares of their global update parameter. If the shares of the global update parameter transmitted in response to the request do not match, the second stop-type fraud detection part 507_i determines there is a dishonest participant and stops the processing. In contrast, if the shares of the global update parameter transmitted in response to the request match, the second stop-type fraud detection part 507_i adopts the shares as the accurate shares of the global update parameter.

When whether the shares of the global update parameter match is determined, the t+1 other federated learning server apparatuses 520_i may first convert the shares into hash values by using a hash function and may next transmit the hash values to the second stop-type fraud detection part 507_i, instead of directly transmitting the shares. By transmitting the hash values, the communication amount can be reduced.

The global training part 505_i of each of the first to Nth federated learning client apparatuses 510_i (i=0, 1, . . . , N−1) is a processing part that reconstructs the shares of the global update parameter and that updates the shared model by the global update parameter. The global training part 505_i reconstructs the global update parameter by using the shares adopted as the accurate shares of the global update parameter by the second stop-type fraud detection part 507_i. Next, the global training part 505_i trains the shared model by applying the reconstructed global update parameter to the shared model, which is the target of the federated learning.

As described above, in the federated learning system according to the fifth example embodiment, since the first to Nth federated learning client apparatuses 510_i (i=0, 1, . . . , N−1) and the first to Nth federated learning server apparatuses 520_i (i=0, 1, . . . , N−1) have their respective configurations described above, it is possible to perform the federated learning while concealing the local update parameters. In the federated learning system according to the fifth example embodiment, since the processing can be performed only by performing a secure computation with addition and multiplication by cleartext constants (without multiplications), the communication cost is low. In addition, since the federated learning system according to the fifth example embodiment uses the (N−t)-out-of-N replicated secret sharing of which the replication is t+1, even if t participants collude with each other and falsify a value, the federated learning system can detect the falsification and stop the processing.

Method

Next, a federated learning method according to the fifth example embodiment will be described. FIG. 17 is a system flowchart illustrating an outline of a procedure of the federated learning method according to the fifth example embodiment.

As illustrated in FIG. 17, the federated learning method according to the fifth example embodiment includes the processing performed by the first to Nth federated learning client apparatuses 510_i (i=0, 1, . . . , N−1) and the processing performed by the first to Nth federated learning server apparatuses 520_i (i=0, 1, . . . , N−1). The processing performed by the first to Nth federated learning client apparatuses 510_i (i=0, 1, . . . , N−1) includes a model replication step (S51), a local training step (S52), an update parameter secret sharing step (S53), a second stop-type fraud detection step (S56), and a global training step (S57). The processing performed by the first to Nth federated learning server apparatuses 520_i (i=0, 1, . . . , N−1) includes a first stop-type fraud detection step (S54) and an update parameter aggregation step (S55).

In the model replication step (S51), each of the first to Nth federated learning client apparatuses 510_i (i=0, 1, . . . , N−1) replicates and stores a shared model, which is the target of the federated learning, as a local model. Next, in the local training step (S52), each of the first to Nth federated learning client apparatuses 510_i (i=0, 1, . . . , N−1) trains the local model by using the data held thereby.

In the update parameter secret sharing step (S53), each of the first to Nth federated learning client apparatuses 510_i (i=0, 1, . . . , N−1) decomposes a local update parameter obtained in the local training step (S52) to shares based on an additive secret sharing scheme and transmits the shares to the first to Nth federated learning server apparatuses 520_i (i=0, 1, . . . , N−1). In this step, the above (N−t)-out-of-N replicated secret sharing is used as the additive secret sharing scheme. That is, the replication of the additive secret sharing scheme according to the fifth example embodiment is t+1.

In the first stop-type fraud detection step (S54), each of the first to Nth federated learning server apparatuses 520_i (i=0, 1, . . . , N−1) requests the other federated learning server apparatuses to transmit the shares of their local update parameters. If the shares of the local update parameters transmitted in response to the request do not match (S54; N), the first stop-type fraud detection part 506_i determines that there is a dishonest participant and stops the processing. In contrast, if the shares of the local update parameters transmitted in response to the request match (S54; Y), the first stop-type fraud detection part 506_i continues the processing and adopts the shares as the accurate shares of the local update parameters.

In the update parameter aggregation step (S55), each of the first to Nth federated learning server apparatuses 520_i (i=0, 1, . . . , N−1) performs a secure computation for shares of a global update parameter by performing addition of the shares of the local update parameters adopted as the accurate shares of the local update parameters in the first stop-type fraud detection step (S54) and by performing multiplication of the shares by cleartext constants. In the update parameter aggregation step (S55), the aggregation and secure computation part 504_i computes a weighted average of local update parameters Δ_i(i=0, 1, . . . , N−1) weighted by data volumes V_i(i=0, 1, . . . , N−1) by performing addition of the shares of the local update parameters and multiplication of the shares by cleartext constants.

In the second stop-type fraud detection step (S56), each of the first to Nth federated learning client apparatuses 510_i (i=0, 1, . . . , N−1) stops the processing if the shares of the global update parameter received to reconstruct the global update parameter do not match. In the second stop-type fraud detection step (S56), the second stop-type fraud detection part 507_i requests t+1 federated learning server apparatuses 520_i, which are supposed to hold replications of the same sub-shares of the global update parameter, to transmit the shares of the global update parameter. If the shares of the global update parameter transmitted in response to the request do not match (S56; N), the second stop-type fraud detection part 507_i determines that there is a dishonest participant and stops the processing. In contrast, if the shares of the global update parameter transmitted in response to the request match (S56; Y), the second stop-type fraud detection part 507_i adopts the shares as the accurate shares of the global update parameter.

In the global training step (S57), each of the first to Nth federated learning client apparatuses 510_i (i=0, 1, . . . , N−1) reconstructs the shares of the global update parameter and updates the shared model by the global update parameter. Each of the first to Nth federated learning client apparatuses 510_i (i=0, 1, . . . , N−1) reconstructs the global update parameter by using the shares adopted as the accurate shares of the global update parameter in the second stop-type fraud detection step (S56). Next, each of the first to Nth federated learning client apparatuses 510_i (i=0, 1, . . . , N−1) trains the shared model by applying the reconstructed global update parameter to the shared model, which is the target of the federated learning.

As described above, in the federated learning method according to the fifth example embodiment, the first to Nth federated learning client apparatuses 510_i (i=0, 1, . . . , N−1) and the first to Nth federated learning server apparatuses 520_i (i=0, 1, . . . , N−1) can perform the federated learning while concealing the local update parameters by coordinating with each other. In the federated learning system according to the fifth example embodiment, since the processing can be performed only by performing a secure computation with addition and multiplication by cleartext constants (without multiplications), the communication cost is low. In addition, since the federated learning system according to the fifth example embodiment uses the (N−t)-out-of-N replicated secret sharing of which the replication is t+1, even if t participants collude with each other and falsify a value, the federated learning system can detect the falsification and stop the processing.

Sixth Example Embodiment

Next, a federated learning system, federated learning server apparatuses, and federated learning client apparatuses according to a sixth example embodiment will be described with reference to FIGS. 18 to 20.

FIG. 18 is a block diagram illustrating a functional configuration example of the federated learning system according to the sixth example embodiment. As illustrated in FIG. 18, a federated learning system 600 according to the sixth example embodiment includes first to Nth federated learning client apparatuses 610_i (i=0, 1, . . . , N−1) and first to Nth federated learning server apparatuses 620_i (i=0, 1, . . . , N−1). The first to Nth federated learning client apparatuses 610_i (i=0, 1, . . . , N−1) and the first to Nth federated learning server apparatuses 620_i (i=0, 1, . . . , N−1) are connected to each other via a network such that these apparatuses can communicate with each other. N is a natural number of 4 or more.

The first to Nth federated learning client apparatuses 610_i (i=0, 1, . . . , N−1) are each operated by an independent organization (user). In addition, the first to Nth federated learning client apparatuses 610_i (i=0, 1, . . . , N−1) store data that their respective organizations do not wish to disclose to the outside. Although the organizations do not wish to disclose their own data to the outside, these organizations wish to mutually utilize their data and to perform machine learning on a shared model. In addition, the first to Nth federated learning server apparatuses 620_i (i=0, 1, . . . , N−1) are each operated by an independent organization (user).

That is, the federated learning system 600 according to the sixth example embodiment is a server-client federated learning system. More specifically, as will be described below, the federated learning system 600 according to the sixth example embodiment is a federated learning system obtained by dividing the apparatuses included in the federated learning system 300 according to the third example embodiment into a group of apparatuses serving as servers and a group of apparatuses serving as clients. Thus, although the first to Nth federated learning client apparatuses 610_i (i=0, 1, . . . , N−1) and the first to Nth federated learning server apparatuses 620_i (i=0, 1, . . . , N−1) are distinguished from each other for convenience of description, these apparatuses can adopt the same hardware configuration. If the same hardware configuration is adopted, for example, a program that limits one or more functions may be executed.

FIG. 19 is a block diagram illustrating a functional configuration example of a federated learning client apparatus according to the sixth example embodiment. As illustrated in FIG. 19, each of the first to Nth federated learning client apparatuses 610_i (i=0, 1, . . . , N−1) includes a model storage part 601_i, a local training part 602_i, a secret sharing part 603_i, a global training part 605_i, and a second nonstop-type fraud detection part 607_i. FIG. 20 is a block diagram illustrating a functional configuration example of a federated learning server apparatus according to the sixth example embodiment. As illustrated in FIG. 20, each of the first to Nth federated learning server apparatuses 620_i (i=0, 1, . . . , N−1) includes an aggregation and secure computation part 604_i and a first nonstop-type fraud detection part 606_i.

The model storage part 601_i of each of the first to Nth federated learning client apparatuses 610_i (i=0, 1, . . . , N−1) is a storage device for replicating and storing a shared model, which is the target of the federated learning, as a local model. In addition, the local training part 602_i is a processing part for training the local model by using data held by the individual one of the first to Nth client apparatuses 610_i (i=0, 1, . . . , N−1).

The secret sharing part 603_i of each of the first to Nth federated learning client apparatuses 610_i (i=0, 1, . . . , N−1) is a processing part for decomposing a local update parameter indicating the training result of the local model to shares based on an additive secret sharing scheme and for transmitting the shares to the federated learning server apparatuses 620_i (i=0, 1, . . . , N−1).

The additive secret sharing scheme used in the sixth example embodiment is (N−2t)-out-of-N replicated secret sharing. That is, the secret sharing part 603_i decomposes the local update parameter indicating the training result of the local model into a sum on a residue class ring Z_Lof modulo L and transmits the values obtained by the decomposition to the first to Nth federated learning server apparatuses 620_i (i=0, 1, . . . , N−1) such that the replication is 2t+1.

The first nonstop-type fraud detection part 606_i of each of the first to Nth federated learning server apparatuses 620_i (i=0, 1, . . . , N−1) is a processing part that requests transmission of the shares of the local update parameters and adopts, of all the shares of the local update parameters transmitted in response to the request, a majority of shares that match, as the accurate shares of the local update parameters.

As described above, in the (N−2t)-out-of-N replicated secret sharing, 2t+1 participants hold replications of the same sub-shares. Thus, even if t participants collude with each other and falsify a value, it is impossible to falsify all the sub-shares to the same values. Thus, t+1 participants hold accurate values. That is, a majority of the values are accurate values. The first nonstop-type fraud detection part 606_i requests other 2t federated learning server apparatuses 620_i, which are supposed to hold replications of the same sub-shares as its own sub-shares, to transmit the shares of their local update parameters and adopts, of all the shares of the local update parameters transmitted in response to the request, a majority of shares that match, as accurate values.

If some of the first to Nth federated learning client apparatuses 610_i (i=0, 1, . . . , N−1) that provide local update parameters are operated by attackers and if falsification has already been made at the time when these federated learning client apparatuses 610_i have distributed their local update parameters in a secret sharing manner, a majority of shares of the local update parameters could not match. In this case, the first nonstop-type fraud detection part 606_i does not adopt these local update parameters and continues the processing by recognizing the parameters as if the parameters were not present.

When whether the shares of the local update parameters match is determined, the first to Nth federated learning client apparatuses 610_i (i=0, 1, . . . , N−1) may first convert the shares into hash values by using a hash function and may next transmit the hash values to the first to Nth federated learning server apparatuses 620_i (i=0, 1, . . . , N−1), instead of directly transmitting the shares. By transmitting the hash values, the communication amount can be reduced.

The aggregation and secure computation part 604_i of each of the first to Nth federated learning server apparatuses 620_i (i=0, 1, . . . , N−1) performs a secure computation for shares of a global update parameter by performing addition of the shares of the local update parameters adopted as the accurate shares of the local update parameters by the first nonstop-type fraud detection part 606_i and by performing multiplication of the shares by cleartext constants. Concretely, the aggregation and secure computation part 604_i computes a weighted average of local update parameters Δ_i(i=0, 1, . . . , N−1) weighted by data volumes V_i(i=0, 1, . . . , N−1) by performing addition of the shares of the local update parameters and multiplication of the shares by cleartext constants.

The second nonstop-type fraud detection part 607_i of each of the first to Nth federated learning client apparatuses 610_i (i=0, 1, . . . , N−1) adopts, of all the shares of the global update parameter received to reconstruct the global update parameter, a majority of shares that match, as the accurate shares of the global update parameter. The second nonstop-type fraud detection part 607_i requests 2t+1 federated learning server apparatuses 620_i, which are supposed to hold replications of the same sub-shares of the global update parameter, to transmit the shares of their global update parameter, and adopts, of all the shares of the global update parameter transmitted in response to the request, a majority of shares that match, as accurate values.

When whether the shares of the global update parameter match is determined, the 2t+1 federated learning server apparatuses 620_i may first convert the shares into hash values by using a hash function and may next transmit the hash values to the second nonstop-type fraud detection part 607_i, instead of directly transmitting the shares. By transmitting the hash values, the communication amount can be reduced.

The global training part 605_i of each of the first to Nth federated learning client apparatuses 610_i (i=0, 1, . . . , N−1) is a processing part that reconstructs the shares of the global update parameter and that updates the shared model by the global update parameter. The global training part 605_i reconstructs the global update parameter by using the shares adopted as the accurate shares of the global update parameter by the second nonstop-type fraud detection part 607_i. Next, the global training part 605_i trains the shared model by applying the reconstructed global update parameter to the shared model, which is the target of the federated learning.

As described above, in the federated learning system according to the sixth example embodiment, since the first to Nth federated learning client apparatuses 610_i (i=0, 1, . . . , N−1) and the first to Nth federated learning server apparatuses 620_i (i=0, 1, . . . , N−1) have their respective configurations described above, it is possible to perform the federated learning while concealing the local update parameters. In the federated learning system according to the sixth example embodiment, since the processing can be performed only by performing a secure computation with addition and multiplication by cleartext constants (without multiplications), the communication cost is low. In addition, since the federated learning system according to the sixth example embodiment uses the (N−2t)-out-of-N replicated secret sharing of which the replication is 2t+1, even if t participants collude with each other and falsify a value, the federated learning system can continue the processing by selecting the accurate values.

Method

Next, a federated learning method according to the sixth example embodiment will be described. FIG. 21 is a system flowchart illustrating an outline of a procedure of the federated learning method according to the sixth example embodiment.

As illustrated in FIG. 21, the federated learning method according to the sixth example embodiment includes the processing performed by the first to Nth federated learning client apparatuses 610_i (i=0, 1, . . . , N−1) and the processing performed by the first to Nth federated learning server apparatuses 620_i (i=0, 1, . . . , N−1). The processing performed by each of the first to Nth federated learning client apparatuses 610_i (i=0, 1, . . . , N−1) includes a model replication step (S61), a local training step (S62), an update parameter secret sharing step (S63), a second nonstop-type fraud detection step (S66), and a global training step (S67). The processing performed by each of the first to Nth federated learning server apparatuses 620_i (i=0, 1, . . . , N−1) includes a first nonstop-type fraud detection step (S64) and an update parameter aggregation step (S65).

In the model replication step (S61), each of the first to Nth federated learning client apparatuses 610_i (i=0, 1, . . . , N−1) replicates and stores a shared model, which is the target of the federated learning, as a local model. Next, in the local training step (S62), each of the first to Nth federated learning client apparatuses 610_i (i=0, 1, . . . , N−1) trains the local model by using the data held thereby.

In the update parameter secret sharing step (S63), each of the first to Nth federated learning client apparatuses 610_i (i=0, 1, . . . , N−1) decomposes a local update parameter obtained in the local training step (S62) to shares based on an additive secret sharing scheme and transmits the shares to the first to Nth federated learning server apparatuses 620_i (i=0, 1, . . . , N−1). In this step, the above-described (N−2t)-out-of-N replicated secret sharing is used as the additive secret sharing scheme. That is, the replication of the additive secret sharing scheme according to the sixth example embodiment is 2t+1.

In the (N−2t)-out-of-N replicated secret sharing, 2t+1 participants hold replications of the same sub-shares. Thus, even if t participants collude with each other and falsify a value, it is impossible to falsify all the sub-shares to the same values. In other words, t+1 participants hold the accurate values. That is, a majority of the values are accurate values. In the first nonstop-type fraud detection step (S64), the second nonstop-type fraud detection part 607_i requests other 2t federated learning server apparatuses 620_i, which are supposed to hold replications of the same sub-shares as its own sub-shares, to transmit the shares of their local update parameters, and adopts, of all the shares of the local update parameters transmitted in response to the request, a majority of shares that match, as accurate values.

In the update parameter aggregation step (S65), each of the first to Nth federated learning server apparatuses 620_i (i=0, 1, . . . , N−1) performs a secure computation for shares of a global update parameter by performing addition of the shares of the local update parameters adopted as the accurate shares of the local update parameters in the first nonstop-type fraud detection step (S64) and by performing multiplication of the shares by cleartext constants. In the update parameter aggregation step (S65), each of the first to Nth federated learning server apparatuses 620_i (i=0, 1, . . . , N−1) computes a weighted average of the local update parameters Δ_i(i=0, 1, . . . , N−1) weighted by the data volumes V_i(i=0, 1, . . . , N−1) by performing addition of the shares of the local update parameters and multiplication of the shares by cleartext constants.

In the second nonstop-type fraud detection step (S66), each of the first to Nth federated learning client apparatuses 610_i (i=0, 1, . . . , N−1) adopts, of all the shares of the global update parameter received to reconstruct the global update parameter, a majority of shares that match, as the accurate shares of the global update parameter. In the second nonstop-type fraud detection step (S66), the second nonstop-type fraud detection part 607_i requests 2t+1 federated learning server apparatuses 620_i, which are supposed to hold replications of the same sub-shares of the global update parameter, to transmit the shares of their global update parameter, and adopts, of all the shares of the global update parameter transmitted in response to the request, a majority of shares that match, as the accurate shares of the global update parameter.

In the global training step (S67), each of the first to Nth federated learning client apparatuses 610_i (i=0, 1, . . . , N−1) reconstructs the shares of the global update parameter and updates the shared model by the global update parameter. Each of the first to Nth federated learning client apparatuses 610_i (i=0, 1, . . . , N−1) reconstructs the global update parameter by using the shares adopted as the accurate shares of the global update parameter in the second nonstop-type fraud detection step (S66). Next, each of the first to Nth federated learning client apparatuses 610_i (i=0, 1, . . . , N−1) trains the shared model by applying the reconstructed global update parameter to the shared model, which is the target of the federated learning.

As described above, in the federated learning method according to the sixth example embodiment, the first to Nth federated learning client apparatuses 610_i (i=0, 1, . . . , N−1) and the first to Nth federated learning server apparatuses 620_i (i=0, 1, . . . , N−1) can perform the federated learning while concealing the local update parameters by coordinating with each other. In the federated learning system according to the sixth example embodiment, since the processing can be performed only by performing a secure computation with addition and multiplication by cleartext constants (without multiplications), the communication cost is low. In addition, since the federated learning system according to the sixth example embodiment uses the (N−2t)-out-of-N replicated secret sharing of which the replication is 2t+1, even if t participants collude with each other and falsify a value, the federated learning system can continue the processing by selecting the accurate values.

Hardware Configuration Example

FIG. 22 is a diagram illustrating a hardware configuration example of a federated learning apparatus. That is, the hardware configuration example illustrated in FIG. 22 is a hardware configuration example of any one of the federated learning apparatuses 100_i, 200_i, and 300_i, the federated learning client apparatuses 410_i, 510_i, and 610_i, and the federated learning server apparatuses 420_i, 520_i, and 620_i (i=0, 1, . . . , N−1). An information processing apparatus (a computer) that adopts the hardware configuration illustrated in FIG. 22 can realize the individual functions of any one of the federated learning apparatuses 100_i, 200_i, and 300_i, the federated learning client apparatuses 410_i, 510_i, and 610_i, and the federated learning server apparatuses 420_i, 520_i, and 620_i (i=0, 1, . . . , N−1) by executing the corresponding one of the above secure computation methods as a program.

The hardware configuration example illustrated in FIG. 22 is an example of the hardware configuration that realizes the individual functions of any one of the federated learning apparatuses 100_i, 200_i, and 300_i, the federated learning client apparatuses 410_i, 510_i, and 610_i, and the federated learning server apparatuses 420_i, 520_i, and 620_i (i=0, 1, . . . , N−1), and does not limit the hardware configuration of any one of the federated learning apparatuses 100_i, 200_i, and 300_i, the federated learning client apparatuses 410_i, 510_i, and 610_i, and the federated learning server apparatuses 420_i, 520_i, and 620_i (i=0, 1, . . . , N−1). The federated learning apparatuses 100_i, 200_i, and 300_i, the federated learning client apparatuses 410_i, 510_i, and 610_i, and the federated learning server apparatuses 420_i, 520_i, and 620_i (i=0, 1, . . . , N−1) may include hardware not illustrated in FIG. 22.

As illustrated in FIG. 22, a hardware configuration 10 that can be adopted by any one of the federated learning apparatuses 100_i, 200_i, and 300_i, the federated learning client apparatuses 410_i, 510_i, and 610_i, and the federated learning server apparatuses 420_i, 520_i, and 620_i (i=0, 1, . . . , N−1) includes, for example, a CPU (Central Processing Unit) 11, a main storage device 12, an auxiliary storage device 13, and an IF (Interface) part 14, which are connected to each other via an internal bus.

The CPU 11 executes various commands included in the federated learning program executed by the corresponding one of the federated learning apparatuses 100_i, 200_i, and 300_i, the federated learning client apparatuses 410_i, 510_i, and 610_i, and the federated learning server apparatuses 420_i, 520_i, and 620_i (i=0, 1, . . . , N−1). The main storage device 12 is, for example, a RAM (Random Access Memory) and temporarily stores various kinds of programs such as the secure computation program executed by the corresponding one of the federated learning apparatuses 100_i, 200_i, and 300_i, the federated learning client apparatuses 410_i, 510_i, and 610_i, and the federated learning server apparatuses 420_i, 520_i, and 620_i (i=0, 1, . . . , N−1) such that the CPU 11 can process these programs.

The auxiliary storage device 13 is, for example, an HDD (Hard Disk Drive) and can store, in the mid-to-long term, various kinds of programs such as the secure computation program executed by the corresponding one of the federated learning apparatuses 100_i, 200_i, and 300_i, the federated learning client apparatuses 410_i, 510_i, and 610_i, and the federated learning server apparatuses 420_i, 520_i, and 620_i (i=0, 1, . . . , N−1). These various kinds of programs such as the secure computation program can be recorded in a non-transitory computer-readable storage medium and can be provided as a program product. The auxiliary storage device 13 can be used to store, in the mid-to-long term, various kinds of programs such as the secure computation program recorded in a non-transitory computer-readable storage medium. The IF part 14 provides an interface regarding the input and output among at least one of the federated learning apparatuses 100_i, 200_i, and 300_i, the federated learning client apparatuses 410_i, 510_i, and 610_i, and the federated learning server apparatuses 420_i, 520_i, and 620_i (i=0, 1, . . . , N−1).

An information processing apparatus that adopts the hardware configuration 10 as described above realizes the functions of any one of the federated learning apparatuses 100_i, 200_i, and 300_i, the federated learning client apparatuses 410_i, 510_i, and 610_i, and the federated learning server apparatuses 420_i, 520_i, and 620_i (i=0, 1, . . . , N−1) by executing the corresponding one of the above-described federated learning methods as a program.

The above example embodiments can partially or entirely be described, but not limited to, as the following notes.

[Note 1]

A federated learning system, which includes at least two federated learning apparatuses connected to each other via a network so as to train a shared model, an individual one of the federated learning apparatuses comprising:

- a model storage part that replicates and stores the shared model as a local model;
- a local training part that trains the local model by using data held by the individual one of the federated learning apparatuses;
- a secret sharing part that decomposes a local update parameter indicating a training result of the local model into shares based on an additive secret sharing scheme and distributes the shares to the other federated learning apparatuses;
- an aggregation and secure computation part that performs a secure computation for shares of a global update parameter by performing addition of the shares of the local update parameters and multiplication of the shares by cleartext constants; and
- a global training part that reconstructs the shares of the global update parameter and updates the shared model by the global update parameter.
  
  [Note 2]
  
  The federated learning system according to note 1, comprising a first stop-type fraud detection part that requests the other federated learning apparatuses to transmit the shares of the local update parameters and stops processing if the shares of the local update parameters transmitted in response to the request do not match.
  
  [Note 3]
  
  The federated learning system according to note 1 or 2, comprising a second stop-type fraud detection part that stops processing if the shares of the global update parameter received when the reconstruction is performed do not match.
  
  [Note 4]
  
  The federated learning system according to note 1, comprising a first nonstop-type fraud detection part that requests the other federated learning apparatuses to transmit the shares of the local update parameters and adopts, of all the shares of the local update parameters transmitted in response to the request, a majority of shares that match, as accurate shares of the local update parameters.