FIG. 1), so as to jointly form a 3D structure.

In some embodiments, the 3D memory array 110 may be implemented with a volatile memory (e.g., a dynamic random access memory (DRAM), a static random access memory (SRAM)), a non-volatile memory (e.g., a magnetoresistive random access memory (MRAM), a ferroelectric random access memory (FeRAM)) or a combination of the above components.

In some embodiments, each of the 2D memory arrays 111_1-111_p comprises a plurality of memory cells (e.g., a plurality of transistors illustrated in FIG. 1). The internal structure of these memory cells will be described in detail in subsequent paragraphs.

The encoding circuit 120 is coupled to the 2D memory arrays 111_1-111_p of the 3D memory array 110 and the processing circuit 140, and is configured to receive neural network data A1-An and B1-Bm from the processing circuit 140, and transmit the corresponding input voltages V, based on the neural network data A1-An and B1-Bm, to the 2D memory arrays 111_1-111_p.

The sensing circuit 130 is coupled to the 2D memory arrays 111_1-111_p of the 3D memory array 110 and the processing circuit 140, and is configured to receive the output currents I from the 2D memory arrays 111_1-111_p, calculate corresponding neural network data C1-Cm and D1-Dn based on the output currents I, and transmit the neural network data C1-Cm and D1-Dn to the processing circuit 140.

The processing circuit 140 is coupled to the encoding circuit 120 and the sensing circuit 130, and is configured to transmit the neural network data A1-An and B1-Bm to the encoding circuit 120 and receive the neural network data C1-Cm and D1-Dn from the sensing circuit 130.

Since the 2D memory arrays 111_1-111_p are coupled to the encoding circuit 120 and the sensing circuit 130 in a similar manner, for the sake of brevity, only the connection relationship between the memory array 111_1, the encoding circuit 120 and the sensing circuit 130 will be described in subsequent paragraphs. FIG. 2A is a schematic diagram of the encoding circuit 120, the sensing circuit 130 and the 2D memory array 111_1 in accordance with some examples. In some embodiments, the 2D memory array 111_1 comprises memory cells G11-G1m, G21-G2m, . . . , Gn1-Gnm, and the memory cells G11-G1m, G21-G2m, . . . , Gn1-Gnm are arranged in an array with m column(s) and n row(s), wherein m and n are positive integers.

As shown in FIG. 2A, after receiving the neural network data A1-An, the encoding circuit 120 will generate corresponding input voltages V1-Vn to the 2D memory array 111_1 according to the neural network data A1˜An. Next, the 2D memory array 111_1 generates output currents I1-Im to the sensing circuit 130. After receiving the output currents I1-Im, the sensing circuit 130 will generate the corresponding neural network data C1-Cm based on the output currents I1-Im.

For the internal structure and current path of the 2D memory array 111_1, please refer to FIG. 2B. FIG. 2B is a schematic diagram of the internal structure and current path of the 2D memory array 111_1 in accordance with some embodiments of the present disclosure.

In some embodiments, the memory cells G11-G1m, G21-G2m, . . . , Gn1-Gnm respectively have impedances W11-W1m, W21-W2m, . . . , Wn1-Wnm. Operationally, when the input voltages V1-Vn are inputted to the 2D memory array 111_1 through the word lines WL1-WLn respectively, the 2D memory array 111_1 will generate a part of the output current I1 on the bit line BL1 based on the input voltage V1 (corresponding to the neural network data A1) and the impedance W11 of the memory cell G11, generate another part of the output current I1 on the bit line BL1 based on the input voltage V2 (corresponding to the neural network data A2) and the impedance W21 of the memory cell G21, and so on. Consequently, the inputs of the input voltages V1-Vn (corresponding to the neural network data A1-An) will generate n parts of the output current I1 on the bit line BL1, and the sum of these n parts is the output current I1 (corresponding to the neural network data C1).

Similarly, the 2D memory array 111_1 will generate a part of the output current I2 on the bit line BL2 based on the input voltage V1 (corresponding to the neural network data A1) and the impedance W12 of the memory cell G12, generate another part of the output current I2 on the bit line BL2 based on the input voltage V2 (corresponding to the neural network data A2) and the impedance W22 of the memory cell G22, and so on. Consequently, the inputs of the input voltages V1-Vn (corresponding to the neural network data A1-An) will generate n parts of the output current I2 on the bit line BL2, and the sum of these n parts is the output current I2 (corresponding to the neural network data C2). The output currents I3-Im (corresponding to the neural network data C3-Cm) are generated in a manner similar to the output currents I1 and I2. For the sake of brevity, they will not be repeated here.

The relationship between the input voltages V1-Vn, the impedances W11-W1m, W21-W2m, . . . , Wn1-Wnm and the output currents I1-Im can be used to implement the calculation between two adjacent neural network layers of a neural network model. Please refer to FIG. 2B and FIG. 2C together. FIG. 2C is a schematic diagram of a neural network model in accordance with some embodiments of the present disclosure.

In the embodiment of FIG. 2C, the neural network data A1-An are stored in the neurons of the K^thneural network layer, and the neural network data C1-Cm are stored in the neurons of the (K+1)^thneural network layer. The neural network data of a neuron in the current layer is the sum of the neural network data of all neurons in the previous layer multiplied by corresponding weights. For example, in the embodiment of FIG. 2C, the neural network data C1 is the sum of the neural network data A1-An multiplied by the weights W11, W21, . . . , Wn1 respectively; the neural network data C2 is the sum of the neural network data A1-An multiplied by the weights W12, W22, . . . , Wn2 (not shown for the sake of brevity) respectively, and so on. Therefore, the calculation of the neural network data C1-Cm can be expressed as the following Formula 1:

Cj
=

∑

i
=
1

n

⁢

(

Ai
*
Wij

)

.

Formula
⁢

1

Since the calculation methods for the neural network data C1-Cm in FIG. 2B and FIG. 2C are the same, the impedances W11, W21, . . . , Wn1 in FIG. 2B can be regarded as implementing the weights W11, W21, . . . , Wn1 of the neural network model in FIG. 2C, thereby implementing the function of storing the neural network data of the K^thneural network layer.

In some embodiments, in addition to receiving neural network data through the word lines WL1-WLn and outputting neural network data through the bit lines BL1-BLm, the 2D memory array 111_1 can also receive neural network data through the bit lines BL1-BLm and output neural network data through word lines the WL1-WLn. Please refer to FIG. 2D. FIG. 2D is a schematic diagram of the internal structure and current path of the 2D memory array 111_1 in accordance with some embodiments of the present disclosure.

In the embodiment of FIG. 2D, the 2D memory array 111_1 receives the neural network data B1 through the bit line BL1, and generates a part of the neural network data D1 on the word line WL1 based on the neural network data B1 and the impedance W11 of the memory cell G11; receives the neural network data B2 through the bit line BL2, and generates another part of the neural network data D1 on the word line WL1 based on the neural network data B2 and the impedance W12 of the memory cell G12, and so on. Consequently, the inputs of the neural network data B1-Bm will generate m parts of the neural network data D1 on the word line WL1, and the sum of these m parts is the neural network data D1, and so on. Therefore, the neural network data Dn can be calculated based on the neural network data B1-Bm received by the bit lines BL1-BLm and the impedance of each neural network data along the path of word line WLn.

Therefore, similar to the embodiment of FIG. 2B, the relationship between the neural network data B1-Bm, the impedances W11-W1m, W21-W2m, . . . , Wn1-Wnm and the neural network data D1-Dn can also be used to implement the calculation between two adjacent neural network layers of a neural network model (e.g., the neural network model in FIG. 2C). The calculation of the neural network data D1-Dn can be expressed as the following Formula 2:

Di
=

∑

j
=
1

m

⁢

(

Bj
*
Wij

)

.

Formula
⁢

2

In conclusion, the 2D memory array 111_1 can store two different pieces of data in the same 2D memory array by receiving and outputting neural network data through word lines and bit lines respectively, and receiving and outputting neural network data through bit lines and word lines respectively.

For the implementation of the memory cells G11-G1m, G21-G2m, . . . , Gn1-Gnm, please refer to FIGS. 3A-3C. FIGS. 3A-3C are circuit diagrams of the 2D memory array 111_1 in accordance with various embodiments of the present disclosure.

In some embodiments, the memory cells G11-G1m, G21-G2m, . . . . Gn1-Gnm can be connected by horizontal and vertical wires (e.g., word lines and bit lines), so as to form a cross-point type array. For example, in the embodiment of FIG. 3A, the memory cells G11-G13 and the memory cells G21-G23 and G31-G33 (not labeled for the sake of brevity) are respectively coupled to adjacent memory cells through the horizontal and vertical wires, and each of the memory cells (i.e., each cross point of the cross-point type array) is implemented with a circuit comprising a resistor.

In other embodiments, the memory cells G11-G1m, G21-G2m, Gn1-Gnm can also be connected by horizontal and vertical wires, and can further control their conduction status through additional wires, so as to form a NOR type array. For example, in the embodiment of FIG. 3B, the memory cells G11-G13 and the memory cells G21-G23 and G31-G33 (not labeled for the sake of brevity) are respectively coupled to adjacent memory cells through the horizontal and vertical wires, and each of the memory cells is implemented with a circuit comprising a resistor and a capacitor. In addition, the control terminals of memory cells on the same row of the memory array are connected to an additional line to control whether they are turned on or not.

Similar to FIG. 3B, in the embodiment of FIG. 3C, the memory cells G11-G13, G21-G23 and G31-G33 also form a NOR type array. The difference is that each memory cell in FIG. 3C is implemented with a circuit comprising an inductor and a capacitor.

It should be noted that the implementations of the memory cells G11-G13, G21-G23 and G31-G33 in FIGS. 2B-2D are only examples, and are not intended to limit the present disclosure. As long as the circuit structures of the memory cells meet the conditions of cross-point type array or NOR type array, other implementations of memory cells are within the scope of the present disclosure.

FIG. 4A is a schematic diagram of the 2D memory arrays 111_1 and 111_2 storing neural network data in accordance with some embodiments of the present disclosure. It should be noted that for the sake of brevity of the figure, the word lines and bit lines connected to each 2D memory array are omitted in FIGS. 4A-4C and FIGS. 6A-6D. When the encoding circuit 120 is connected to the left or right side of a 2D memory array in the figure, it represents that the 2D memory array receives the neural network data through the word lines; when the encoding circuit 120 is connected to the upper or lower side of a 2D memory array in the figure, it represents that the 2D memory array receives the neural network data through the bit lines. When the sensing circuit 130 is connected to the left or right side of a 2D memory array in the figure, it represents that the 2D memory array transmits the neural network data through the word lines; when the sensing circuit 130 is connected to the upper or lower side of a 2D memory array in the figure, it represents that the 2D memory array transmits the neural network data through the bit lines.

In the embodiment of FIG. 4A, the 2D memory array 111_1 is coupled to the 2D memory array 111_2 through a set of encoding circuit 120, sensing circuit 130 and processing circuit 140, so as to store the neural network data related to the K^thneural network layer of a first neural network model. Specifically, after the encoding circuit 120 inputs the neural network data related to the K^thneural network layer of the first neural network model into the 2D memory array 111_1, with the weighting and summing by the 2D memory array 111_1, the sensing circuit 130 can calculate the neural network data related to the (K+1)^thneural network layer of the first neural network model, and the processing circuit 140 uses it as the neural network data input to the 2D memory array 111_2 for subsequent calculations.

Similarly, the 2D memory array 111_2 is also coupled to the 2D memory array 111_3 through another set of encoding circuit 120, sensing circuit 130 and processing circuit 140, and calculates the neural network data related to the (K+2)^thneural network layer of the first neural network model in a manner similar to the 2D memory array 111_1, thereby storing the neural network data related to the (K+1)^thneural network layer of the first neural network model.

In addition, the 2D memory array 111_1 is further coupled to the 2D memory array 111_2 through another set of encoding circuit 120, sensing circuit 130 and processing circuit 140, so as to store the neural network data related to the K^thneural network layer of a second neural network model; the 2D memory array 111_2 is further coupled to the 2D memory array 111_3 through yet another set of encoding circuit 120, sensing circuit 130 and processing circuit 140, so as to store the neural network data related to the (K+1)^thneural network layer of the second neural network model.

Therefore, each of the 2D memory arrays can store two sets of data through two connection methods. The connection method and data transmission method of the 2D memory arrays 111_3-111_p are similar to the 2D memory arrays 111_1 and 111_2, and will not be repeated here.

In some embodiments, the 2D memory arrays 111_1-111_p receive the neural network data related to the first neural network model through the word lines, and transmit the neural network data related to the first neural network model through the bit lines (as shown in the upper part of FIG. 4A), so as to store the first dataset. In addition, the 2D memory arrays 111_1-111_p further receive the neural network data related to the second neural network model through the bit lines, and transmit the neural network data related to the second neural network model through the word lines (as shown in the lower part of FIG. 4A), so as to store the second dataset.

FIG. 4B is a schematic diagram of the 2D memory arrays 111_1 and 111_2 storing neural network data in accordance with other embodiments of the present disclosure. Similar to FIG. 4A, the 2D memory array 111_1 in FIG. 4B is also configured to store the neural network data related to the K^thneural network layer of the first neural network model and the neural network data related to the K^thneural network layer of the second neural network model, and the 2D memory array 111_2 is also configured to store the neural network data related to the (K+1)^thneural network layer of the first neural network model and the neural network data related to the (K+1)^thneural network layer of the second neural network model.

Different from FIG. 4A, in the embodiment of FIG. 4B, a part of the 2D memory arrays 111_1-111_p can receive the neural network data related to the first neural network model through the word lines, and then transmit the neural network data related to the first neural network model through the bit lines. The other part of the 2D memory arrays 111_1-111_p can receive the neural network data related to the first neural network model through the bit lines, and then transmit the neural network data related to the first neural network model through the word lines. In this way, the 2D memory arrays 111_1-111_p can store the first dataset.

For example, as shown in the upper part of FIG. 4B, the 2D memory array 111_1 receives the neural network data related to the K^thneural network layer of the first neural network model through the word lines, and transmits the neural network data related to the (K+1)^thneural network layer of the first neural network model through the bit lines, while the 2D memory array 111_2 receives the neural network data related to the (K+1)^thneural network layer of the first neural network model through the bit lines, and transmits the neural network data related to the (K+2)^thneural network layer of the first neural network model through the word lines.

Therefore, when the 2D memory arrays 111_1-111_p store the neural network data related to the second neural network model (i.e., the second dataset), a part of the 2D memory arrays can receive the neural network data through the bit lines and transmit the neural network data through the word lines, and the other part of the 2D memory arrays can receive the neural network data through the word lines and transmit the neural network data through the bit lines.

Continuing with the embodiment shown in FIG. 4B, the 2D memory array 111_1 receives the neural network data related to the K^thneural network layer of the second neural network model through the bit lines, and then transmits the neural network data related to the (K+1)