3, and they do not share any parameters of their predictive model. This reduces the amount of data that needs to be communicated for the collaborative learning, while allowing great freedom for the design of the predictive model. In this way, the predictive model can be a neural net of a substantial size, or any other differentiable predictive model. On the other hand, the controller neural net can be of small size, enabling feasible communication of parameters of the neural net. Unlike the standard neural nets where the neural weights play the role of memory, in this design, the memory is external in the shared memory. Hence, there is no need for the controller neural net to be large in itself.

Still, this architecture allows the central memory 22 to be large. The central node 3 does not need to communicate the entire shared memory 22 with the agent devices; only readings from the shared memory 22 being vectors of smaller size are sent to the agent devices. Since the shared memory 22 can be quite large, it is capable of storing long-term representations of data (meta data) that can help improve the accuracy in such cases.

FIG. 2 is a schematic diagram illustrating functional components and interaction between the central component and agent devices of FIG. 1.

Each one of the agent devices 2a-b comprises a respective controller neural net 10a-b (also referred to as the first model) and a predictive model 11a-b (also referred to as the second model).

The central device comprises (or has access to) a shared memory 22. An aggregator 20 collects and aggregates data from the agent devices 2a-b and a write head 21 is used to write to the shared memory 22. Respective read heads 23a-b are used to read data from the shared memory 22, tailored to the respective agent device 2a-b. Even only two agent devices 2a-b are shown here, the same structure applies if more than two agent devices are provided.

The parameters presented in FIG. 2 are as follows:

- M_t−1previously learned memory matrix
- M_tcurrent memory matrix

${r_{t - 1}^{A, i}}_{i = 1 : R}$

previous readings from memory for the first agent device 2a

${r_{t - 1}^{B, i}}_{i = 1 : R}$

previous readings from memory for the second agent device 2b

${r_{t}^{A, i}}_{i = 1 : R}$

current readings from memory for the first agent device 2a

${r_{t}^{B, i}}_{i = 1 : R}$

current readings from memory for the second agent device 2b

$x_{t}^{A}$

input local data for the first agent device 2a

$x_{t}^{B}$

input local data for the second agent device 2b

$v_{t}^{A}$

controller output for the first agent device 2a

$v_{t}^{B}$

controller output for the second agent device 2b

$Θ_{t}^{A, r}$

read interface parameters for the first agent device 2a

$Θ_{t}^{B, r}$

read interface parameters for the second agent device 2b

$Θ_{t}^{A, w} .$

write interface parameters for the first agent device 2a

$Θ_{t}^{B, w}$

write interface parameters for the second agent device 2b

${\overline{Θ}}_{t}^{w}$

aggregated write interface parameters

$y_{t}^{A}$

output response for the first agent device 2a

$y_{t}^{B}$

output response for the second agent device 2b

It is to be noted that any or all of the parameters can be compressed to reduce network footprint on the communication channel prior to transmitting the data. Some examples of such compression include quantization, lossless compression, lossy compression in the case of secret sharing, auto encoder, employing generative models (GAN), etc. The receiver would then need to apply corresponding decompression prior to applying the parameter(s).

The memory matrix M is of size N by W. It is a row matrix implying that each row specifies a memory location of length W. The mentioned structure can be transposed between rows and columns and still support embodiments presented herein.

The controller neural net output v is a vector of length W.

The read interface parameters Θ^rinclude a number of parameters that are used to read from the memory. Each reading r from memory can be vector of length W or a collection of R vectors of length W. The agent device can choose to take a single reading from the memory or multiple readings, as specified in the read interface parameters of the agent device.

An example of read interface parameters includes a read lookup key and a read key strength. The read lookup key is a vector or a number of vectors of length W. The read lookup key is a vector when the agent device wants a single reading from the memory. The read lookup key is a matrix of R rows with length W, in the case of R readings from the memory. The read key strength is a scalar for the case of a single read lookup key, and a vector of size R for the case of R readings from memory.

The write interface parameters Θ^winclude a number of parameters that are used to write into the memory. The write interface parameters of all agent devices have the same dimensionality, since the write interface parameters from the different agent devices are aggregated by the central device. The writings in the memory contributed by the agent devices are in the form of vectors. At any point in time, a single vector of length W is written to the memory.

An example of write interface parameters of an agent device includes a write lookup key (a vector of length W), a write key strength (a scalar) and/or an erase vector (a vector of length W).

The overview procedure will now be described with reference to FIG. 2.

The controller neural nets 10a-b are located in the respective agent devices 2a-b. Each agent device 2a-b has its own controller neural net 10a-b, and all agent devices 2a-b share the same architecture for the controller neural nets 10a-b. They may be initialized similarly.

The predictive models 11a-b are also located in the agent devices 2a-b. Each devices 2a-b has its own predictive model 11a-b. The different agent devices 2a-b do not need to share the same architecture for their predictive models 11a-b. The predictive model can be a neural net or any other differentiable model, such as random forest, decision tree, linear models, etc. The agent devices 2a-b do not need to reveal the architecture of their predictive model to the central device or other agent devices 2a-b.

By separating the controller neural net and the predictive model, the agent devices can contribute to and benefit from the collective learning in the controller neural nets 10a-b without the need to disclose the architecture of their predictive model. The agent devices do not need to share any of its predictive model parameters with any other agent devices or the central device. Only the parameters for the controller neural net are shared with the central device. This makes the agent devices robust to attacks such as deep leakage if combined with secure aggregation. Moreover, the predictive models can be used to contain private information that is not meant to be shared with the central device or other agent devices in the federation.

The central device 3 controls and maintains the shared memory 22, which can e.g. be in the form of a memory matrix. The memory matrix is learned using data from all agent devices 2a-b. In other words, the shared memory 22 is a manifestation of the collaborative learning from the agent devices 2a-b. The memory matrix contains a number of rows, also known as memory locations. Each row (memory location) is a location vector of a certain size. The size of the location vector depends on the size of the output layer of the controller neural net. The mentioned structure can be transposed between rows and columns and still support embodiments presented herein.

The aggregator 20 takes the write interface parameters sent from the controller neural nets 10a-b of the agent devices 2a-b. The aggregator 20 produces an aggregated (e.g. averaged quantity either by taking simple averaging or weighted averaging) set of write interface parameters. When the averaging is weighted, Simply, the weights can e.g. be a function of the number of data points that an agent device has. Alternatively, the weights can be computed by comparing similarities between the interface parameters of the agent devices. An agent device whose write interface parameters are very different from other agent devices can be assigned, accordingly, higher or lower weights than the other agent devices whose write interface parameters are similar. Other procedures for calculating the weights could also be applied.

Hence, the agent devices 2a-b do not have the right to directly write in the shared memory 22. Only the central device 3 can write in the shared memory 22 using its write head 21, which, in turn, is based on aggregation of the write interface parameters. The write head 21 receives the aggregated write interface parameters from the aggregator 20, and writes to the shared memory 22, resulting in an updated version of the shared memory 22. There are no explicit constraints on the size of the shared memory 22; in practice, it can be arbitrarily large.

There is a read head 23a-b for each agent device 21a-b. Each read head is only accessible for its associated agent device 2a-b. In this way, the central device 3 sends personalized readings of the shared memory 22 to the agent devices 2a-b using their respective read heads 23a-b. A read head 23a-b of a given agent device 2a-b takes as its input the updated shared memory 22 and the read interface parameters of the agent device in question 2a-b. This results in a reading from the shared memory 22 that is personalized for each agent device 2a-b.

FIGS. 3A-B are flow charts illustrating embodiments of methods performed by the agent device for enabling collaborative machine learning, a result of which is stored in a shared memory controlled by a central device. Furthermore, these embodiments enable performing inference by a plurality of agent devices 2, 2a-b comprising respective machine learning models configured for inference based on the shared memory.

Before the method starts, the central device and the agent devices are initialised. Specifically, the central device initializes the memory matrix and sends the initial readings from the memory to the agent devices.

In an obtain local data step 40, the agent device obtains local input data.

In a generate read interface parameters step 41, the agent device generates read interface parameters based on the local input data using a controller neural net 10a-b being a first model. The read interface parameters are also based on previous readings from the shared memory.

In a generate write interface parameters step 42, the agent device generates write interface parameters based on the local input data using the controller neural net 10a-b. The write interface parameters are also based on previous readings from the shared memory.

In a transmit central reading request step 43, the agent device transmits a central reading request to the central device 3. The central reading request comprises the read interface parameters and the write interface parameters. The central reading request requests the central device 3 to update a memory matrix based on the write interface parameters and to read the memory matrix based on the read interface parameters.

In a receive central reading step 44, the agent device receives a central reading from the central device 3.

In an update controller neutral net step 45, the agent device updates the controller neural net 10a-b of the agent device based on the central reading.

In a provide predictor output step 46, the agent device provides a predictor output of local input data based on the controller neural net and a second model 11a-b of the agent device 2, 2a-b. The second model 11a-b has as an input an output of the controller neural net 10a-b. The predictor output is thus obtained from the second model 11a-b.

Looking now to FIG. 3B, only new or modified steps will be described.

In an optional compute loss step 47, the agent device computes a loss by comparing an estimation of the predictor output with an actual outcome. The agent devices use the current readings from the shared memory and the output from the controller neural net to produce an estimation of the predictor output. Each agent device then computes the loss between the true predictor output (e.g., labels) and the estimated predictor output, as [Image Omitted] =loss(y,ŷ), where y is the true predictor output and ŷ is the estimated predictor output. The function loss( ) is any suitable loss function that indicates discrepancies between y and ŷ.

In an optional compute controller gradients step 48, the agent device computes a set of controller gradients of the loss with respect to the read interface parameters and the write interface parameters, respectively.

In an optional compute prediction gradients step 49, the agent device computes a set of prediction gradients of the loss with respect to the predictor output.

In an optional transmit controller gradients step 50, the agent device transmits the set of controller gradients to the central device 3.

In an optional receive central read gradients step 51, the agent device receives from the central device 3 a set of central read gradients of the read interface parameters with respect to a read head 23a-b in the central device 3 for the agent device 2, 2a-b.

In an optional receive central write gradients step 52, the agent device receives from the central device 3 a set of central write gradients of the write interface parameters with respect to a read head 23a-b in the central device 3 for the agent device 2, 2a-b.

In an optional update controller neutral net step 53, the agent device updates neural net parameters for the controller neural net 10a-b based on the received central read gradients and the received central write gradients.

In an optional update second model step 54, the agent device updates parameters for the second model 11a-b based on the received central read gradients and the received central write gradients.

In an optional conditional done step 55, the agent device repeats the method determines when a stopping criterion is true. In one embodiment, the stopping criterion is true when a variation metric of average loss for the agent device and other agent devices is less than a threshold. In one embodiment, the stopping criterion is true after a threshold number of iterations of the method. Then at least one stopping criterion is true, the method ends. Otherwise, the method returns to the obtain local data step 40.

FIGS. 4A-B are flow charts illustrating embodiments of methods performed by the central device for enabling collaborative machine learning, a result of which is stored in a shared memory controlled by a central device. Furthermore, these embodiments enable performing inference by a plurality of agent devices 2, 2a-b comprising respective machine learning models configured for inference based on the shared memory.

In a receive request step 140, the central device receives a plurality of central reading requests from respective agent devices 2, 2a-b. The central reading request comprises read interface parameters and write interface parameters.

In an aggregate step 142, the central device aggregates the write interface parameters of the plurality of central reading requests. The aggregation can e.g. be performed by calculating an average of the write interface parameters from the agent devices. The averaging can be a simple average or a weighted average.

In a provide write head parameters step 143, the central device provides parameters for a write head 21 based on the aggregated write interface parameters. The parameters for the write head 21 can be provided based on the aggregated write interface parameters and optionally a current state of the memory matrix 22.

In an update memory matrix step 144, the central device updates a memory matrix 22 of the shared memory based on the write head 21.

In a provide read head parameters step 145, the central device provides, for each agent device 2, 2a-b, parameters for a read head 23a-b based on the respective read interface parameters. The parameters for the read head 23a-b can be provided based on the respective read interface parameters and optionally a current state of the memory matrix 22.

In a read step 146, the central device reads, for each agent device, a central data reading being a selection of data from the memory matrix in the shared memory based on the respective read head 23a-b.

In a transmit reading step 147, the central device transmits the central data readings, respectively, to each one of the agent devices 2, 2a-b.

Looking now to FIG. 4B, only new or modified steps will be described.

In an optional receive controller gradients step 148, the central device receives respective sets of controller gradients from each one of the plurality of agent devices