⁢
im

,

k
⁢
1

,

k
⁢
2
⁢
…
⁢
kn

=

∑

j
⁢
1

,

j
⁢
2
⁢
…
⁢
jn

A

i
⁢
1

,

i
⁢
2
⁢
…
⁢
im

,

j
⁢
1

,

j
⁢
2
⁢
…
⁢
jn

⁢

B

j
⁢
1

,

j
⁢
2
⁢
…
⁢
jn

,

k
⁢
1

,

k
⁢
2
⁢
…
⁢
kp

where i1, i2 . . . im, k1, k2 . . . kp each define a specific index for each respective dimension I1, I2 . . . Im, K1, K2 . . . Kp. In order to perform this operation, one must identify matching dimensions, which in this case are J1, J2 . . . Jn. A tensor-tensor multiplication or tensor contraction can have multiple different outputs depending on which dimensions are contracted over, so part of defining it includes defining which matching dimensions are contracted over. It is to be understood as well that if two tensors do not have matching dimensions, they cannot be multiplied or contracted. Matrix-matrix multiplication is a special case of a tensor contraction: given two matrices A and B, where A has dimensions N by M and B has dimensions M by D, the output C can be defined as a tensor contraction between two 2D tensors over the dimension M. Note that the right-hand side of the equation is equivalent to a summation, or aggregation, of dot products. When iterating over the j_ndimension for example, we can fix all dimensions of A except for the j_ndimension and fix all dimensions of B except for it's j_ndimension—now we have a dot product between two vectors. Therefore, when we define a “transistor tensor contraction” similar to how TMMs, TMVs were defined, we can see that a transistor tensor contraction is a result of many aggregations of TVV operations contracting over a set of shared dimensions.

We also can define “transistor convolution operation” in similar ways. For a convolution tensor (W^NMK) and input tensor (X^RSK), the transistor convolution operation on the starting parameter X_m,n,1of the input tensor is σ [Σ_i^NΣ_j^MΣ_k^Kf_i,j,k(W_i,j,k, X_{m+i−1,n+j−1,k})], which will be referred to as a “transistor Hadamard operation” in the following discussions. A tensor convolution operation is a special case of a transistor tensor convolution operation where all scalar functions are multiplications defined as f_i,j,k(a, b)=a×b and the output functions are all identity functions.

Similarly, a “transistor tensor convolution operation” extends convolutions of matrices to convolutions of tensors. A convolution using a d×d kernel matrix over a n×n input matrix is a special case of a transistor tensor convolution operation. It is to be understood that the dimensions of the kernel tensor in a transistor tensor convolution operation must have matching or contracting dimensions with the input tensor that it convolves.

We now define “transistor tensor operations” to be the superset of transistor vector-vector operations, transistor matrix-vector operations, transistor matrix-matrix operations, transistor tensor contractions, and transistor tensor convolution operations. A transistor tensor operation can refer to any of these operations.

Using the notation defined above, one level of neural-network computation is typically a combination of calculations and transformations over the input that result in an output vector

$(A_{i}^{N}) = σ [(W_{i, j}^{N, M}) (V_{j}^{M}) + (B_{i}^{N})],$

where (V_j^M) is the input vector, each component W_i,jof the matrix (W_i,j^{N, M}) is called a “weight”, (B_i^N) is a vector called “bias vector”, σ is a vector function such as but not limited to the sigmoid function or the “Rectified Linear activation function” (ReLU) function. The a output function is also commonly called an activation function. A deep neural-network typically consists of multiple layers, where the output vector of each layer is the input vector of the next layer until the final output vector A^(L)is generated.

$\begin{matrix} A^{(L)} = σ^{(L)} [W^{(L)} A^{(L - 1)} + B^{(L)}]; \\ A^{(L - 1)} = σ^{(L - 1)} [W^{(L - 1)} A^{(L - 2)} + B^{(L - 1)}]; \\ \dots; \\ A^{(k)} = σ^{(k)} [W^{(k)} A^{(k - 1)} + B^{(k)}]; \\ \dots; \\ A^{(1)} = σ^{(1)} [W^{(1)} A^{(0)} + B^{(1)}]; \end{matrix}$

Here, integer k is greater than or equal to 1 and less than or equal to positive integer L; A⁽⁰⁾is the initial input vector; A^(L)is the final output vector of this neural network with L levels. This final output vector can be further transformed to compute the final output prediction of the neural network. In classification applications, for example, the size of the final output vector may be equal to the number of classes in the application, where each index in the output vector maps to a particular class. A SoftMax function is applied over the output vector, and the greatest element in the vector is selected as the final classification.

The configurations of neural networks are typically defined by software, which call software modules to define the number of levels, the number of neurons at each level, the types of operators used, the activation functions used, and other customizable operators. This configuration can be referred to as the “architecture” of a model or neural network. Software provides excellent flexibility in configuring neural networks and designing custom architectures. Note that the example in paragraph 8 represents the most classic neural network architecture known as the multi-layered perceptron (MLP). Other architectures may use more operations other than matrix-vector operations. For example, tensor convolutions may be used to transform the input when the input is a rank 3 or greater tensor instead of a vector. Inputs from one level may be concatenated with other outputs from previous levels. While the specific operations in a level can be anything, a level in general transforms an input tensor into an output tensor.

Neural networks are only valuable if trained properly. Training can be defined as the process of finding a set of model parameters that can adequately approximate the relationship between a set of input variables and outputs. This is typically achieved by the following procedures: (1) Gathering a large dataset. Each sample in the dataset can be denoted as (X, y), where X is a tensor of inputs and y is the “label”, which can be defined as the actual output or result given the input X; (2) Defining a “loss function” that adequately quantifies the difference between the model's predictions and the labels. The loss of an epoch is a function of the model's outputs and the labels. A typical example of a loss function is mean squared error: L=Σ_i(ŷ_i−y_i)², where y_iis the label of the ith sample and ŷ_irepresents the model's output prediction; and (3) Finding a set of model parameters that minimize the loss function over multiple passes through the dataset. A single pass through the dataset is known as an “epoch”. This is typically achieved through an algorithm known as “gradient descent”. After each forward pass, the derivative of the loss function is taken with respect to each weight, or parameter, in the model. Weights are then updated by adding a proportion of the negative of the gradient to them, and the process is repeated until the model achieves sufficient performance.

Training neural networks is extremely computationally expensive as a single forward pass can involve billions of operations. This has led to optimizations of the process—the most notable being the use of back-propagation to more efficiently calculate gradients, as well as techniques like mini-batching and stochastic gradient descent that decrease the number of computations needed (which theoretically sacrifices some amount of model performance) before updating model parameters. While backpropagation is used in nearly all cases of training to update parameters, since backpropagation relies on the chain rule in calculus and therefore the differentiability of the model, a major restriction it imposes is that the derivative of every operation in the neural network must have a closed form solution and must be well-defined. This includes the loss function and all activation functions used. While one can still obtain the derivative of the loss with respect to each weight δL/δW_i,jfor non-differentiable operations, this process would involve changing the value of the weight W_i,jslightly, then measuring the corresponding change of the loss L. The derivative can then be computed by dividing the change in loss over the change in weight. Measuring the change of the loss L however requires an entire forward pass while fixing all other parameters. A large neural network with billions of parameters could require billions of forward passes just to complete an iteration of weight or parameter updates, which is intractable using prior art digital computation systems.

Prior art machine learning computations are typically executed by digital integrated circuits such as center processing units (CPUs), graphic processing units (GPUs), or specialized artificial intelligence (AI) integrated circuits such as “tensor processing units” (TPUs) that are equipped with digital computation circuits such as binary multipliers and binary adders. Prior art machine learning computations typically require large computational resources—a forward pass of a neural network with one billion parameters involves the execution of around one billion multiplications and one billion summations. A single CPU with a few multipliers and few adders would take billions of instruction cycles to execute one forward pass. A GPU or specialized AI chip can comprise thousands of computation circuits that can execute thousands of multiply-summation computations in parallel, achieving faster computation speeds than general purpose CPUs. However, millions of computation cycles are still required to execute one forward pass. In addition, the system needs to continuously move billions of parameters to and from storage memory devices and the digital computation circuits. Such data movements consume significant amounts of energy and they can result in performance bottlenecks.

In the above equations, the “synapse” of the j'th input neuron V_jto the i'th output neuron A_iis assumed to be linearly proportional to V_jas W_i,j*V_j, where W_i,jis the weight. This linear relationship is not necessarily the best model for all cases. Actual biological neuron cells typically activate only when the input signal is greater than a threshold value and do not respond when the input is lower than such threshold. Biological synapse functions are more similar to nonlinear functions such as the ReLU function instead of a linear function like multiplication. The influence of the j'th input neuron V_jto the i'th output neuron Y_ican be modeled as a general function f(Q_i,j, V_j), where Q_i,jis a field-programmable parameter. Using aforementioned definition of a transistor matrix-vector operation, the output vector can then be written as:

$(A_{i}) = σ {AGG [f_{j} (Q_{i, j}, V_{j})]} .$

For a neural network of L levels, we have:

$\begin{matrix} A^{(L)} = σ^{(L)} {{AGG}^{(L)} [f^{(L)} (Q^{(L)}, A^{(L - 1)})]}; \\ A^{(L - 1)} = σ^{(L - 1)} {{AGG}^{(L - 1)} [f^{(L - 1)} (Q^{(L - 1)}, A^{(L - 2)})]} \\ \dots; \\ A^{(k)} = σ^{(k)} {{AGG}^{(k)} [f^{(k)} (Q^{(k)}, A^{(k - 1)})]}; \\ \dots; \\ A^{(1)} = σ^{(1)} {{AGG}^{(1)} [f^{(1)} (Q^{(1)}, A^{(0)})]} . \end{matrix}$

Prior art digital integrated circuits do not possess circuits that can execute thousands of f_j(Q_i,j,V_j)s in parallel and are therefore not well-equipped to execute computations other than weighted summations. It is therefore desirable to develop algorithms that possess a large number of circuits that can be configured for transistor matrix-vector operations in addition to weighted summations.

Prior art digital neural network devices executed by binary computations still achieve successes in applications such as image classification, object detection, text generation, image generation, and have also found success in the field of reinforcement learning. On the other hand, the cost and power required by prior art digital neural network devices have always limited the progress and success of the models they perform computations for—the popularity of deep learning for example was never feasible until the widespread adoption of GPUs, which still possess significant limitations. A prior art large-scale deep level neural network can cost millions in dollars and months in time to train and also consumes significant amounts of power that in the long run will exacerbate the energy consumption crisis. While prior art digital neural network devices have undoubtedly unlocked solutions to previously unsolvable problems, the prices in cost and energy are much higher relative to biological brains. It is therefore highly desirable to develop systems that require less power, cost less, and execute faster.

FIG. 1(a) shows a schematic symbol of a metal-oxide-semiconductor (MOS) transistor. An MOS transistor has gate, source, drain, and substrate terminals, as shown in FIG. 1(a). For simplicity, the substrate terminal is not typically drawn in schematic diagrams of MOS transistors. The substrate terminal is however an important terminal that can influence the behavior of the transistor. For example, if the substrate terminal is at a voltage that is lower than both the source voltage and drain voltage of an n-channel MOS transistor, the effective threshold voltage of the transistor will increase, and the channel leakage current will decrease; such effects caused by substrate voltages are called “body effects”. P-channel MOS transistors also have body effects except that the polarity of voltages is opposite to that of n-channel MOS transistors.

FIG. 1(b) shows a schematic symbol of a floating-gate transistor. A floating-gate transistor is a type of MOS transistor that also has gate, source, drain, and substrate terminals, as shown in FIG. 1(b). For simplicity, the substrate terminal of a floating-gate transistor is not typically drawn in schematic diagrams. The major difference between a floating-gate transistor and a common MOS transistor is that its threshold voltage VT is field-programmable. The floating-gate is conductor surrounded by insulators, as illustrated symbolically in FIG. 1(b). Electrical charges can be trapped in the floating-gate to change its threshold voltage. Typically, the threshold voltage of a floating-gate transistor can be increased by putting a high drain-to-source voltage and gate voltage that puts the transistor in a saturation region to generate strong electrical fields in its channel, which can inject electrons into the floating-gate transistor by hot electron effects, causing an increase of threshold voltage. This process is called “programming” of the floating-gate transistor. For a floating-gate transistor, its threshold voltage is a function of the storage charge (Q) trapped in its floating-gate, modeled as

$V_{T} = V_{T 0} + C_{e} * Q,$

where V_T0is the intrinsic threshold voltage when Q is zero and C_eis the equivalent coupling capacitor of the floating-gate transistor. V_T0and C_eare determined by transistor properties. Current art integrated circuit technologies are able to manufacture large quantities of floating-gate transistors with matched V_T0and C_e. It is possible to control the change of V_Tin analog values using short pulses of programming and subsequently measuring changes in channel current until V_Treaches a desired value.

Typically, the threshold voltage of a floating-gate transistor can be decreased by putting a high voltage on its drain/source terminals, and a low voltage on its gate terminal, invoking a high electric field between the gate and drain/source which pulls electrons out of the floating-gate through tunneling. This process is called an “erase” of the floating-gate transistor. The amount of decrease in VT is typically difficult to accurately control during the erase process. For some cases, the erase procedure can be executed on an individual floating-gate transistor, but is more often executed on the whole array of floating-gate transistors. This is known as a “flash erase”.

The floating-gate of the floating-gate transistor is surrounded by insulators that trap injected charges in the floating-gate with little loss over a long time. This property makes it useful as a non-volatile memory device. Currently, floating-gate transistors are widely used for applications such as solid-state hard discs, USB memory sticks, mobile phone memory cards, firmware storage memory, and many others. Floating-gate transistors function as memory devices that store data in these examples.

The current-voltage (I-V) relationships of MOS transistors are:

In the cut-off region where V_GS<V_T,

$I_{DS} = 0,$

where I_DSis the channel current flowing from the drain terminal to the source terminal of the transistor, V_GSis the gate-to-source voltage, and V_Tis the threshold voltage. In the saturation region where V_GS>V_T, and V_DS>V_GS−V_T,

$I_{DS} = 1 / 2 K_{n} {(V_{GS} - V_{T})}^{2},$

where K_n=U_nC_o(W/L), U_nis carrier mobility, C_ois equivalent capacitance, W is effective channel width, and L is effective channel length. In the triode region, where V_GS>V_T, and V_DS<V_GS−V_T,

$I_{DS} = K_{n} [(V_{GS} - V_{T}) V_{DS} - 1 / 2 V_{DS}^{2}] .$

When the term (½ V_DS²) is negligible, we have

$I_{DS} = K_{n} (V_{GS} - V_{T}) V_{DS} .$

The current-voltage (I-V) relationships of floating-gate transistors are similar to that of common MOS transistors. The difference is that its threshold voltage V_T=V_T0−C_eQ is field-programmable.

FIG. 1(c) shows an exemplary current-voltage relationship with fixed V_GSand variable V_DS. The channel current (I_DS) increases linearly with V_DSin the triode region (111), but is virtually independent of V_DSin the saturation region (112). In the triode region (111), I_DS=K_nV_DS(V_GS−V_T) and the transistor behaves as an analog multiplier with V_DSand K_n(V_GS−V_T) as the two inputs of the analog multiplier. Thus, we will call this computation condition “multiplier mode”. The linear relationship in multiplier mode provides a method to implement weighted multiplication operations in neural networks with one transistor; the value of input neuron would be V_DS, and weight would be K_n(V_GS−V_T). For a floating-gate transistor, the weight (V_GS−V_T) is field-programmable by changing storage charge Q, as (V_GS−V_T0−C_eQ).

FIG. 1(d) shows an exemplary current-voltage relationship with fixed V_DSand variable V_GS. The channel current (I_DS) is near zero when V_GSis less than V_T, and increases approximately linearly when V_GSis greater than V_Tin the triode region. We will call this condition “ReLU mode” because this relationship provides a way to implement the “Rectified Linear activation function” (ReLU) with one transistor. If the device is operating in the saturation region, the current can increase super-linearly and the channel current is not sensitive to drain voltage. We will call this condition “saturation mode”, where the I-V relationship is similar to ReLU mode but current increases super-linearly after the device is turned on.

FIG. 1(e) shows an exemplary current-voltage relationship when V_DS=V_GSare input variables. The channel current (I_DS) is close to zero when V_GSis less than V_Tand increases rapidly when V_GSis greater than V_T. We will call this condition “rectifier mode” because this I-V relationship is similar to that of a rectifier.

These I-V relationships show that a floating-gate transistor not only can be used as a memory device, but can also be used to execute computations. An MOS transistor can execute the function of a multiplier when in the triode region where V_DSand (V_GS−V_T) are inputs. It can execute the ReLU function when V_GSis used as the input. While in the saturation region, it can execute the function of a super-linear ReLU function using V_GSas the input. When V_GSand V_DSare connected together as the input, it can execute the function of a rectifier. An MOS transistor can be a “current-mode computation” (CMC) cell with channel current as its computation output. A programmable-threshold voltage MOS transistor can be a current-mode computation memory cell with threshold voltage as the storage data and channel current as the computation output. It is also a type of “field-programmable current-mode computation memory cell” because the parameter stored in the transistor is field-programmable.

The above equations are valid for long channel MOS transistors under ideal conditions. The I-V relationship can be more complex for practical situations, but the general trend still holds. Thus, we will use these equations in the following discussions. Almost all the prior art floating-gate transistors are n-channel transistors; we will therefore assume they are n-channel in the following discussions. The results are also applicable to p-channel transistors if the polarities of voltages and currents are reversed. We will use floating-gate transistors, which are an example of programmable-threshold voltage transistors, as example embodiments in most of our discussions, though the same principles are applicable to other types of transistors.

Prior art digital computations are not efficient at summations involving a large number of elements. Take the example of summing over 128 binary numbers. Prior art binary computations first must use 64 binary adders in parallel, resulting in 64 summation results. Next, 32 adders are used in parallel to output 32 summation results; 16 are used next, and so on until the final adder is used with two summation outputs to yield the final result. The sequential nature makes this operation slow, and worsens as the number of summations increases. While it is possible to use “partial adders”, which sum three binary numbers and output two binary numbers, to increase speed, the sequential nature of the summation still remains.

In contrast, FIG. 1(f) illustrates the principle of current-mode summation using a conductor line (141). Multiple electrical currents (I₁, I₂, . . . , I_j, . . . , I_M) flow into a conductor (141) and an electrical current (I_sum) flows out of one end of the conductor (141) to a current summation circuit (ISC), as illustrated in FIG. 1(f). According to the principle of conservation of electrical charges, we have

$I_{sum} = (I_{1} + I_{2} + \dots + I_{j} + \dots . + I_{M}) = \sum_{j}^{M} I_{j} .$

A piece of electrical conductor (141) is therefore able to sum over a large number of electrical currents flowing into it and sends the resulting summation current to the output circuit (ISC). We will call such conductor (141) a “summation line” and call this method “Current-Mode Summation” (CMS) in the following discussions. When parasitic leakage currents are negligible, at steady states, current-mode summation results are accurate. In reality, errors maybe introduced due to none ideal effects such as parasitic leakage currents. CMS is significantly more efficient since all additions can be done in parallel, and performs the function of multiple binary adders in parallel at extremely high speed. By definition, a current-mode summation is a summation of electrical currents using an electrical conductor based on the principle of conservation of electrical charges. A “summation line” is typically shaped as a line, but the electrical conductor can have wide varieties of shapes.

While current mode computations are excellent at parallel computations, especially summations, using direct current (DC) to represent numbers consumes power even when the hardware is not executing computations. Prior art CMOS integrated circuits in contrast consume little power while idle. This “idle state power consumption” is therefore a disadvantage of current mode computations relative to digital computations.

FIG. 2(a) is a simplified schematic diagram for a prior art two-dimensional array of N columns and M rows of floating-gate transistors that are configured to support neural network computations, where N and M are positive integers. In this example, the drain terminals of the floating-gate transistors (M_1,j, M_2,j, . . . , M_i−1,j, M_i,j, . . . . M_N−1,j, M_N,j) on the same row are connected by a conductor line to an input at voltage Vd_jas shown in FIG. 2(a), where i is an integer greater than or equal to 1 and less than or equal to N. A conductor line that provides the same input to a plurality of computation memory cells will be called an “input line” in the following discussions. The source terminals of the floating-gate transistors (M_i,1, M_i,2, . . . , M_i,j−1, M_i,j, . . . M_i,M−1, M_i,M) on the same column and a current summation circuit (201) are connected by a conductor line (Is_i), as shown in FIG. 2(a), where j is an integer greater than or equal to 1 and less than or equal to M. A conductor that carries the output currents (in this case, the channel currents of floating-gate transistors) of a plurality of computation cells connected to the conductor to an output circuit (201) will be called a “branch summation line” in the following discussions. The current summation circuit (201) fixes the voltage on the branch summation line (Is_i) to a predefined reference voltage V_sand provides an output voltage (Vo_i) that is a function of the summation current on the branch summation line (Is_i). In this embodiment, the gate terminals of all floating-gate transistors in the array are biased to the same voltage (V_g) while executing computations, as illustrated in FIG. 2(a). This gate voltage is selected to bias all the floating-gate transistors in the array in multiplier mode. Under ideal conditions where parasitic parameters are negligible, the branch output current (Is_i) in this example is the summation of all channel currents of the floating-gate transistors connected to it as:

${Is}_{i} = \sum_{j}^{} (K_{n} (V_{g} - V_{s} - V_{Ti, j}) * {Vd}_{j}) = \sum_{j}^{} (K_{n} (V_{g} - V_{s} - V_{T 0} - C_{e} Q_{i, j}) * {Vd}_{j})$

Is_iis the branch output current by the i'th branch summation line, Vd_jis input voltage at the j'th input line, V_Ti,jis the threshold voltage, and Q_i,jis the storage charge of the floating-gate transistor at the i'th column and the j'th row. Because IC technologies are able to manufacture large numbers of transistors in nearly equal properties, it is valid to assume K_n, V_T0, and C_eof all the transistors in the array are equal.

The above discussions show that the floating-gate transistor array in FIG. 2(a) is able to execute a weighted summation Σ_j(W_i,j*V_j) by setting the weight W_i,j=K_n(V_g−V_s−V_T0−C_eQ_i,j). This can be done by programming the floating-gate storage charge Q_i,j. One floating-gate transistor is therefore able to serve the functions of an analog multiplier and the same floating-gate transistor can serve as a non-volatile storage memory of the weight without the need to move the parameter from external memory to a computation circuit. Using the natural properties of electrical currents, the summation of all synapses for an output neuron is performed by one conductor line instead of a large number of binary adders. A matrix-vector operation is executed by an array of floating-gate transistors in parallel. In theory, the floating-gate transistor array in FIG. 3(a) is able to execute computations that are fully compatible with prior art digital neural network devices with dramatic advantages to cost, power, and performance. Consider a floating-gate transistor array that supports 1000 inputs and 1000 outputs, and it is able to finish one calculation in 100 nanoseconds. Such a floating-gate transistor array can achieve the performance of 10¹³multiply-summations per second.

In theory, the floating-gate transistor array in FIG. 2(a) appears to be ideal for neural network computations, but significant limitations arise in practice. Prior art software-controlled configurations can flexibly configure the number of levels, the operations at each level, and the output sizes at each level. However, the prior art floating-gate transistor array in FIG. 2(a) has a fixed number of rows and columns that cannot be altered after it is manufactured. It can support a matrix computation when the number of inputs is less than or equal to M and when the number of outputs is less than or equal to N, but does not have the flexibility to support a matrix computation that has a greater number of inputs or greater number of outputs. In addition, the configuration in FIG. 2(a) only can support one level of matrix computations; it cannot support multiple levels using the same array. It is therefore desirable to make the sizes of the computation array field-programmable and to provide the capabilities to support multiple levels of matrix function computations.

Configurability is not the only problem. Because the conductor materials used to manufacture branch summation lines (Is_i) are not superconductors, the branch summation line's parasitic resistance is non-zero; when electrical currents flow through it, there is a current times resistance (I*R) induced voltage difference. At worst-case conditions, this voltage difference increases with the square of the number of transistors (M²) attached to the branch summation line. Feedback mechanisms provided by the output circuit (201) can fix the voltage of the branch summation lines (Is_i) at a desired source voltage (Vs) at one end of the line, but the source voltage will change at positions away from the output end due to the I*R induced voltage difference. This means the gate-to-source voltage along the conductor line is less than (V_g−V_s), which can cause significant error when M is large. Furthermore, the computation assumes the total current (Is_i) detected by the output circuit (201) is the summation of the output currents of all the floating-gate transistors connected to the branch summation line. In reality, there are error sources caused by parasitic leakage currents such as junction leakage currents or channel leakage currents. This parasitic leakage current induced error is proportional to the number of transistors (M) connected to the branch summation line. In addition, while a conductor line in theory can take summation of currents at light speed, in practice the conductor line has parasitic capacitance; there are resistance-times-capacitance (R*C) induced delay times and capacitance-divided-by-current (C/I) induced delay times. Similarly, there are I*R induced voltage differences along input conductor lines (Vd_i) proportional to N². The parasitic resistance of the input conductor line can cause an I*R voltage difference on actual drain voltages applied to transistors along the line, which is proportional to N²in the worst-case. The parasitic capacitance of the input line also causes R*C and C/I induced delay time along input conductor lines. These non-ideal effects are summarized in the following table (Table 1).

TABLE 1

Relation to

Error sources
array size

1: I*R induced voltage difference on branch
M²

summation line

2: Parasitic leakage current induced error on
M

branch summation line

3: R*C delay on branch summation line
M²

4: C/I delay on branch summation line
M

5: I*R induced voltage difference on input line
N²

6: Parasitic leakage current induced I*R
N²

voltage difference on input line

7: R*C delay on input line
N²

8: C/I delay on input line
N

These error sources increase rapidly with the size of the array. Floating-gate transistor arrays configured as in FIG. 2(a) are therefore not practical when array sizes are large either in the number of inputs or the number of outputs, rendering them inapplicable for large production models used in modern day applications that can contain up to trillions of parameters. It is therefore desirable to develop practical devices that can support large-scale deep neural network computations.

Changing a parameter for a prior art floating-gate transistor neural network requires program and erase procedures—increasing the threshold voltage of a floating-gate transistor requires time-consuming pulsed programming procedures. Reducing the threshold voltage involves first erasing followed by programming to the correct value. Erase operations must erase the entire array. This means an entire array must be reprogrammed to change a single parameter. Methods for updating parameters in prior art floating-gate neural networks are therefore very complex and inconvenient relative to prior art digital neural network devices. Given that the training process of neural networks involves considerable parameter updates, this makes it difficult and expensive to train a neural network using floating-gate transistor arrays. It is therefore highly desirable to develop more efficient methods of changing parameters.

The prior art programmable-threshold voltage transistor array in FIG. 2(a) executes multiply-summation computations in parallel, achieving high performance. Synapses that carry large currents however can experience “current overload”, a problem that arises when the circuit is damaged by a large volume of current concentrating in a small area. Current overload is related to the size of the transistor array: when large numbers of transistors are connected to each branch summation line, the parasitic leakage current is large. The designer must as a result increase the channel currents of the floating-gate transistors to have enough signal-to-noise ratio. This not only increases operation power but also increases the chance of current overload. It is essential to prevent current overload before using floating-gate transistors for large-scale matrix function computations.

Another problem that the prior art array in FIG. 2(a) exhibits is the “idle state power consumption” (ISPC) problem. DC currents continuously flow in the array and consume power even when computations complete. It is therefore desirable to design current mode computation circuits that consume little power when they are idle. The prior art array in FIG. 2(a) is also limited to matrix-vector computation; it is not able to support other applications such as vector-vector or convolutions.

FIG. 2(b) is a simplified schematic diagram for another prior art two-dimensional array configured to support neural network computations. In this example, gate terminals of the floating-gate transistors (M_1,j, M_2,j, . . . , M_i−1,j, M_i,j, . . . . M_N−1,j, M_N,j) on the same row are connected by an input line at input voltage V_j, as shown in FIG. 2(b), where i is an integer greater than or equal to 1 and less than or equal to N. The drain terminals of the floating-gate transistors (M_i,1, M_i,2, . . . , M_i,j−1, M_i,j, . . . M_i,M−1, M_i,M) on the same column are connected by a branch summation line (Id_i) to a current summation circuit (211), as shown in FIG. 2(b), where j is an integer greater than or equal to 1 and less than or equal to M. The current summation circuit (211) fixes the voltage on the conductor line (Id_i) to a predefined voltage V_dand provides an output (Vo_i) that is proportional to the current on the branch summation line (Id_i). While executing matrix computations, the source terminals of all floating-gate transistors in the array are connected to ground, as illustrated in FIG. 2(b). The configurations of the branch summation lines (Id_i) in FIG. 2(b) are the same as those (Is_i) in FIG. 2(a). The differences are that (1) the voltage on the branch summation line (Vd) is higher than the source voltages of the transistors connected to it, making the connected terminals drain terminals rather than source terminals, and (2) the channel currents flow in opposite polarity for the case in FIG. 2(b) versus the case in FIG. 2(a). Under this configuration, the transistors are biased in ReLU mode; the channel current (Id_i,j) of the transistor at the i'th column and j'th row is zero when input voltage is less than its threshold voltage (V_Ti,j). When input voltage is greater than its threshold voltage (V_Ti,j) and the transistor is operating in the triode region, channel current is

${Id}_{i, j} = K_{n} (V_{j} - V_{Ti, j}) * V_{DS} = K_{n} (V_{j} - V_{T 0} - C_{e} Q_{i, j}) * V_{DS}$

In this configuration, the synapse of the j'th input neuron V_jto the i'th output neuron represented by Id_i,jis a ReLU function, as illustrated in FIG. 1(c). The branch output current on the i'th branch summation line (Id_i) is

${Id}_{i} = \sum_{j}^{} K_{n} * V_{DS} * (V_{j} - V_{T 0} - C_{e} Q_{i, j}),$

which is a summation of ReLU functions.

Prior art digital neural network device computations often use ReLU functions as output vector functions, but do not apply such activation functions to individual synapses. For this embodiment in FIG. 2(b), the ReLU function is applied to every synapse, making it incompatible with many prior art digital neural network devices. Additionally, the configuration in FIG. 2(b) still has the parasitic parameters induced problems listed in Table 1, difficulties in training, current overload problems, the ISPC problem, and configurability problem from fixed array sizes. It is also limited to matrix-vector computations; it is not able to support other applications such as vector-vector computations or convolutions.

FIG. 2(c) is a simplified schematic diagram for another prior art two-dimensional array configured to support neural network computations. In this example, the gate terminals and drain terminals of the floating-gate transistors (M_1,j, M_2,j, . . . , M_i−1,j, M_i,j, . . . . M_N−1,j, M_N,j) on the same row are connected by an input line at input voltage V_j, as shown in FIG. 2(c). The source terminals of the floating-gate transistors (M_i,1, M_i,2, . . . , M_i,j−1, M_i,j, . . . M_i,M−1, M_i,M) on the same column are connected by a branch summation line (Is_i) to a current summation circuit (221), as shown in FIG. 2(c), where j is an integer greater than or equal to 1 and less than or equal to M. The current summation circuit (221) fixes the voltage on the conductor line (Is_i) to a predefined reference voltage V_sand provides an output (Vo_i) proportional to the current on the branch summation line (Is_i). In this configuration, the “synapse” of the j'th input neuron V_jto the i'th output neuron represented by Id_i,jis a super-linear rectifier function, as illustrated in FIG. 1(d).

For the embodiment in FIG. 2(c), the rectifier function is used in every synapse, making it incompatible with prior art digital neural network device computations. The configuration in FIG. 2(c) also possesses the problems listed in Table 1, difficulties in training, current overload problems, the ISPC problem, and the configurability problem due to fixed array sizes. It is also limited to matrix-vector computations; it is not able to support other applications such as vector-vector computations or convolutions.

FIG. 2(d) is a simplified schematic diagram for another prior art two-dimensional array configured to support neural network computations. In this example, the gate terminals of the floating-gate transistors (M_1,j, M_2,j, . . . , M_i−1,j, M_i,j, . . . M_N−1,j, M_N,j) on the same row are connected by an input line at input voltage V_j, as shown in FIG. 2(d). The source terminals of the floating-gate transistors (M_i,1, M_i,2, . . . , M_i,j−1, M_i,j, . . . M_i,M−1, M_i,M) on the same column are connected by a branch summation line (Is_i) to a current summation circuit (231), as shown in FIG. 2(d). The drain terminals of the floating-gate transistors (M_i,1, M_i,2, . . . , M_i,j−1, M_i,j, . . . M_i,M−1, M_i,M) on the same column are connected by a control line set at a control voltage Vc_i, as shown in FIG. 2(d). The current summation circuit (221) fixes the voltage on the conductor line (Is_i) to a predefined reference voltage V_sand provides an output (Vo_i) proportional to the current on the branch summation line (Is_i). Under this configuration, the transistors on the same column are turned off if Vc_i=V_s. If Vc_iis set at a constant voltage greater than V_s, the transistors will be biased in ReLU mode with the I-V relationship illustrated in FIG. 1(c).

The embodiment in FIG. 2(d) serves similar functions as the embodiment in FIG. 2(b) except that it can selectively enable or disable columns of the array using the drain voltage control lines (Vc_i). These control lines can work as multiplex select signals that can reduce the number of output circuits (231) to achieve better array efficiency. The configuration in FIG. 2(d) has the same advantages and disadvantages as the configuration in FIG. 2(b). The drain voltage control lines (Vc_i) are able to field program which columns are activated, but the computation mode of transistors is not field-programmable. The configuration in FIG. 2(c) also possesses the problems listed in Table 1, difficulties in training, current overload problems, the ISPC problem, and the configurability problem due to fixed array sizes. It is also limited to matrix-vector computations; it is not able to support other applications such as vector-vector computations or convolutions.

Park et al. in U.S. Pat. No. 4,999,525 disclosed computation memory cells that have four floating-gate transistors to support positive and negative synapse values. The floating-gate transistors are connected in ReLU mode.

Canepa et al. in U.S. Pat. No. 5,055,897 disclosed a neural cell using a special floating-gate transistor; the floating-gate of an n-channel floating-gate device is used as the gate terminal of a nearby p-channel transistor. Holler et al. in U.S. Pat. No. 5,256,911 disclosed a floating-gate transistor array for performing weighted sum computation in ReLU mode, and a special computation memory cell that comprises one floating-gate transistor and one common MOS transistor; pairs of summing currents are connected to differential amplifier to provide the possibility to have negative weighing factor. Sethi in U.S. Pat. No. 5,284,786 disclosed a split floating-gate transistor combined with a nearby common transistor to form a synapse cell. Yariv et al. in U.S. Pat. No. 5,353,382 disclosed a synapse cell that uses one floating-gate transistor plus one MOS transistor used for neural networks. Yariv's cell array was in multiplier mode. Li et al. in U.S. Pat. No. 5,444,821 disclosed a computation memory cell that has two floating-gate transistors with different polarities of drain connection to allow both positive and negative synapse values. Li's cells are connected in ReLU mode. Castro et al. in U.S. Pat. No. 5,028,810 disclosed synapse cells that used two floating-gate transistors and two common transistors to discharge a capacitor as a way to implement 4 quadrant computations. Fick et al. in U.S. Pat. No. 9,760,533 disclosed floating-gate transistor array for performing weighted sum computation in configurations that matches the prior art configuration shown in FIG. 2(d). Leobandug in U.S. Pat. No. 11,055,607 disclosed integrated circuit configuration using a floating-gate transistor array to implement neural network computations. Leobandug's synapse cells comprise floating-gate transistors with gate and source terminals connected together as input and drain used as output, similar to the configuration shown in FIG. 2(c). Holler et al. in U.S. Pat. No. 5,268,320 disclosed methods for intentionally baking flash memory devices at high temperatures as a method to calibrate weighing parameters. Such methods are not methods used to mitigate the influences of unintentional heating on results. Gong et al. in US patent application number 16,785,797 (Gong) disclosed a temperature assisted programming of flash memory for neuromorphic computing. Gong's methods intentionally control temperature to assist programming and reading of flash memory. Such methods are not methods used to mitigate the influences of unintentional heating on results. Breitwisch et al. in U.S. Pat. No. 8,589,320 disclosed floating-gate transistors connected gate terminal and drain terminal as diode configuration to support neural network applications similar to the prior art configuration discussed in FIG. 2(c). Strukov et al. in US patent application number 16,608,006 disclosed prior art current summation lines and used a special synapse cell that comprises two floating-gate transistors and three common MOS transistors.

None of the above and other prior art publications addressed the critical limitations caused by parasitic parameters. While such approaches may be applicable for small neural networks, none provide solutions needed to implement multiple levels of large-scale transistor tensor operations. None of the above prior art publications provide solutions to address flexibility problems caused by fixed array sizes; none mention solutions for preventing current overload problems; none mention solutions for preventing the ISPC problem. None are able to support other applications such as vector-vector computations or convolutions. All of them only use floating-gate transistors as the computation transistors.

SUMMARY

One embodiment of the present invention is directed to an integrated circuit comprising a plurality of current-mode computation (CMC) branches, each CMC branch comprising a plurality of CMC cells and a branch summation line, and a plurality of layer-one current summation circuits. An individual CMC cell may comprise at least a computation transistor or a computation resistor, and may produce a CMC cell output current that is a function of the channel current of the computation transistor or the electrical current flowing through the computation resistor. A branch summation line may comprise an electrical conductor that may be electrically coupled to, and may receive the CMC cell output currents produced by, a plurality of CMC cells in an individual CMC branch. A branch summation line may produce a branch output current that is a current-mode summation of the received CMC cell output currents. An individual layer-one current summation circuit may be electrically coupled to, and may receive the branch output current produced by, at least one branch summation line. An individual layer-one current summation circuit may comprise a feedback circuit that can limit at least an amplitude of the received branch output current, also may comprise a current mirror circuit. A layer-one current summation circuit may produce a layer-one current summation circuit output current that is a function of the received branch output currents.

The integrated circuit may also comprise a plurality of layer-two summation lines and a plurality of layer-two current summation circuits. An individual layer-two summation line may comprise an electrical conductor that may be electrically coupled to, and may receive the layer-one current summation circuit output currents produced by, a plurality of layer-one current summation circuits. A layer-two summation line may produce a layer-two summation line output current that is a current-mode summation of the received layer-one current summation circuit output currents. An individual layer-two current summation circuit may be electrically coupled to, and may receive the layer-two summation line output current produced by, at least one layer-two summation line. A layer-two current summation circuit may produce a layer-two current summation circuit output current that is a function of the received layer-two summation line output currents.

The integrated circuit may also comprise a plurality of layer-three summation lines and a plurality of layer-three current summation circuits. An individual layer-three summation line may comprise an electrical conductor that may be electrically coupled to, and may receive the layer-two current summation circuit output currents produced by, a plurality of layer-two current summation circuits. A layer-three summation line may produce a layer-three summation line output current that is a current-mode summation of the received layer-two current summation circuit output currents. An individual layer-three current summation circuit may be electrically coupled to, and may receive the layer-three summation line output current produced by, at least one layer-three summation line. A layer-three current summation circuit may produce a layer-three current summation circuit output current that is a function of all received layer-three summation line output currents.

A computation transistor of at least one of the CMC cells may comprise an MOS transistor, a native transistor, or a programmable-threshold-voltage transistor, which may be a floating-gate transistor. A computation resistor of at least one of the CMC cells may comprise a resistor of fixed value, or a variable resistor with field-programmable value. The CMC cells may be integrated with inter-layer dielectric embedded components. The integrated circuit may also comprise a select transistor having a source or drain terminal that is electrically coupled to the gate terminal of a computation transistor of one of the CMC cells, or to the gate terminals of computation transistors of a plurality of CMC cells. The integrated circuit may also comprise a storage-control capacitor electrically coupled to the gate terminal of a computation transistor of each of said CMC cells while the other terminal of the storage-control capacitor is electrically coupled to a field-programmable CMC cell gate voltage control signal. The integrated circuit may also comprise hybrid-analog-digital CMC cells, wherein a hybrid-analog-digital CMC cell is a CMC cell that can execute computations between one digital input and one analog input. A hybrid-analog-digital CMC cell may comprise memory devices that can store binary numbers, such as DRAM or SRAM memory cells.

A plurality of the CMC cells may be field-programmable to operate in multiple modes, which may include a multiplier mode, a ReLU mode, a rectifier mode, and/or a saturation mode. The integrated circuit may be field-programmable to disable parts of the circuits. At least one of the layer-one current summation circuits may be field-programmable, and the integrated circuit may perform transistor tensor operations by programming one or more of the layer-one current summation circuits to combine all branch output currents received by those layer-one current summation circuits. At least one of the layer-two current summation circuits may be field-programmable, and the integrated circuit may perform transistor tensor operations by programming one or more of the layer-two current summation circuits to combine all layer-two summation line output currents received by those layer-two current summation circuits. At least one of the layer-three current summation circuits may be field-programmable, and the integrated circuit may perform transistor tensor operations by programming one or more of the layer-three current summation circuits to combine all layer-three summation line output currents received by those layer-three current summation circuits.

The integrated circuit may comprise fixed voltage current sensing circuits that may be used in layer-one current summation circuits, layer-two current summation circuits, layer-three current summation circuits, inter-dice signal transfer circuits, or external signal transfer circuits. The integrated circuit may comprise multiple semiconductor dice that are connected by inter-dice connections. Using embodiments of the present invention, millions, billions, trillions or more of the CMC cells can be field-programmed to execute computations in parallel to support transistor tensor operations.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1(a) is a typical schematic symbol of a metal-oxide-semiconductor (MOS) transistor;

FIG. 1(b) is a typical schematic symbol of a floating-gate transistor;

FIGS. 1(c-e) show examples of the current vs voltage (I-V) relationship of a typical MOS transistor;

FIG. 1(f) is a symbolic diagram illustrating the principle of current-mode summation;

FIGS. 2(a-d) are simplified symbolic diagrams of examples of prior art floating-gate transistor arrays configured to support neural network computations;

FIG. 3(a) is a simplified symbolic block diagram of one embodiment of a layer-two block of the present invention;

FIG. 3(b) is a simplified symbolic block diagram of one embodiment of a layer-one block in FIG. 3(a) that uses current-mode computation memory cells;

FIG. 3(c) is a simplified schematic diagram of one embodiment of a layer-one input circuit (VI1) in FIG. 3(b);

FIG. 3(d) is a simplified schematic diagram of one embodiment of a layer-one current summation circuit (IS1) in FIG. 3(b);

FIG. 3(e) is a simplified schematic diagram of one embodiment of a layer-two current summation circuit (IS2);

FIG. 3(f) is a simplified schematic diagram of one embodiment of an interface output circuit (VOUT) in FIG. 3(e);

FIG. 3(g) is a simplified schematic diagram of one embodiment of a layer-one gate voltage input circuit (VI1g) in FIG. 3(b);

FIG. 3(h) is a simplified symbolic block diagram of one embodiment of a layer-one block in FIG. 3(a);

FIGS. 3(i-s) are simplified schematic diagram of embodiments of current mode computation cells;

FIG. 3(t) is a simplified schematic diagram of one embodiment of a layer-one current summation circuit (1040) with a voltage clamping circuit;

FIGS. 3(u-w) are simplified schematic diagram of embodiments of voltage clamping circuits that can be used for the layer-one current summation circuit in FIG. 3(s);

FIG. 3(x) is a simplified schematic diagram of one embodiment of a layer-two current summation circuit (1060) with a voltage clamping circuit;

FIG. 4(a) is a simplified symbolic block diagram of one embodiment of a layer-three block of the present invention;

FIG. 4(b) a simplified schematic diagram of one embodiment of a layer-three current summation circuit (IS3);

FIG. 4(c) a simplified schematic diagram of one embodiment of a layer-one current summation circuit (IS1) that can be used when the layer-one array is configured at saturation mode;

FIG. 5(a) is a simplified symbolic block diagram of one embodiment of a multiple-layer current summation configuration of the present invention;

FIG. 5(b) is a simplified symbolic block diagram of one embodiment of a multiple-layer input configuration of the present invention;

FIG. 5(c) is a simplified symbolic block diagram of one embodiment of a system configuration of the present invention;

FIGS. 5(d-e) show simplified embodiments of inter-dice connections;

FIG. 6(a) is a simplified schematic diagram for a two-transistor CMC cell that can have positive or negative values;

FIG. 6(b) is a simplified schematic diagram for a current subtraction circuit that allows synapses to have positive or negative values;

FIG. 6(c) is a simplified schematic diagram for a dual polarity current mirror circuit;

FIG. 6(d) is a simplified schematic diagram for a four-transistor synapse cell that can have positive or negative values in both inputs and outputs;

FIG. 6(e) is a simplified flow chart for implementing double-precision parameters; and

FIG. 7(a-c) are simplified cross-section diagrams for embodiments of CMC cells comprise computation resistors.

DETAILED DESCRIPTION

As discussed in previous sections, prior art floating-gate transistor arrays cannot support large-scale transistor matrix-vector operations due to the non-ideal effects listed in Table 1. These non-ideal effects increase rapidly with the size of the transistor arrays. Embodiments of the present invention overcome these problems by distribute computation cells into a large number of small units and combine the computations executed in different units to execute full large-scale computations. This architecture also reduces power consumption, solves the current overload problem, and provides configurability.

FIG. 3(a) is a simplified symbolic block diagram for an embodiment of a layer-two block (B⁽²⁾) of an embodiment of the present invention. This layer-two block (B⁽²⁾) comprises P columns and U rows of layer-one blocks (B_m,n), as shown in FIG. 3(a), where P and U are positive integers, n is an integer greater than or equal to 1 and less than or equal to U, and m is an integer greater than or equal to 1 and less than or equal to P.

FIG. 3(b) is a simplified schematic diagram for one embodiment of a layer-one block in FIG. 3(a). This layer-one block (B_m,n) comprises a two-dimensional array of N columns and M rows of CMC cells that are configured to support transistor matrix-vector operations, where N and M are positive integers, i is an integer greater than or equal to 1 and less than or equal to N, and j is an integer greater than or equal to 1 and less than or equal to M. In this embodiment, each CMC cell comprises one programmable-threshold voltage transistor (M_i,j), which is a floating-gate transistor that is used as a computation transistor and as a memory device. The drain terminals of a subset of the floating-gate transistors (M_1,j, M_2,j, . . . , M_i−1,j, M_i,j, . . . M_N−1,j, M_N,j) are connected by a conductor line (Vd_j) to a drain voltage input circuit (VI1)(312), as shown in FIG. 3(b), where j is an integer greater than or equal to 1 and less than or equal to M. A conductor line (e.g. Vd₁, Vd₂, . . . , Vd_j, . . . , Vd_M) that controls the drain voltages of a subset of the computation transistors in the layer-one block (B_m,n) will be called a “drain voltage input line” in the following discussions. The number (N) of transistors connected to each drain voltage input line is designed to be low enough such that the aforementioned parasitic parameter induced problems are negligible, allowing the drain voltage input line to be considered as an equal-potential electrical conductor line. FIG. 3(c) shows a simplified schematic diagram for an exemplary “sample-and-hold” (S&H) circuit that can serve the function of the drain voltage input circuit (VI1) that drives a drain voltage input line. In “sample” mode, when the field-programmable control signal Smp is activated, transistor Me4 is turned on such that the voltage (Vhi) on the storage capacitor (Ch) is substantially equal to the input voltage (Vdp), as shown in FIG. 3(c). A unit gain amplifier (321) senses and drives the voltage on its input node (Vhi) to its output (Vh) at a voltage equal to Vdp during sampling mode. During “hold” mode, when the field-programmable control signal Smp is deactivated, transistor Me4 is turned off such that the voltage on the storage capacitor (Ch) is held at the previously sampled value even when the input voltage Vdp changes. When the field-programmable enable signal (ENI) is activated, and the field-programmable reset signal (Rst) is deactivated, transistor Me5 is turned on, transistor Mrst is turned off, and the output voltage on the drain voltage input line (Vdj) is driven by the unit gain amplifier (321) to be equal to its output voltage at Vh. When the field-programmable enable signal (ENI) is deactivated, and the field-programmable reset signal (Rst) is activated, transistor Me5 is turned off, transistor Mrst is turned on, and the output voltage on the drain voltage input line (Vdj) is reset to voltage Vs. The number of transistors (N) connected to the drain voltage input line (Vdj) is designed to be low enough such that the R*C delay time caused by the parasitic parameters of the input conductor line (Vdj) is no more than a fraction of the intrinsic delay time of the input circuit (VI1).

The gate terminals of the same subset of the floating-gate transistors (M_1,j, M_2,j, . . . , M_i−1,j, M_i,j, . . . M_N−1,j, M_N,j) that share the same drain voltage input line (Vd_j) in the layer-one block (B_m,n) are connected by another conductor line (Vg_j) to another gate voltage input circuit (VI1g) (313), as shown in FIG. 3(g). A conductor line (e.g Vg₁, Vg₂, . . . , Vg_j, . . . , Vg_M) that controls the gate voltages of a subset of the computation transistors in the layer-one block (B_m,n) will be called a “gate voltage input line” in the following discussions. In this embodiment, the sample-and-hold (S&H) circuit (VI1g) in FIG. 3(g) is nearly identical to the S&H circuit in FIG. 3(c), except that one terminal of its storage capacitor (Ch) is connected to a field-programmable reference voltage (Vsg) instead of a fixed voltage (Vs), as shown in the design in FIG. 3(g). When Vsg=Vs, VI1g is identical to VI1 and serves the same functions of a voltage sample-and-hold circuit based on the same operation principles exhibited by the circuit in FIG. 3(c). In “hold” mode, a change in Vsg causes a change in output voltage (Vg_i), providing a convenient way to execute training algorithms of an embodiment of the present invention. The number of transistors (N) connected to the gate voltage input line (Vg_j) is designed to be low enough such that the R*C delay time caused by the parasitic parameters of the Vg input conductor line (Vg_j) is no greater than a fraction of the intrinsic delay time of the gate voltage input circuit (VI1g).

The source terminals of a subset of the floating-gate transistors (M_i,1, M_i,2, . . . , M_i,j−1, M_i,j, . . . M_i,M−1, M_i,M) in the layer-one block (B_m,n) are connected by a branch summation line (315) to a layer-one current summation circuit (IS1) (311), as shown in FIG. 3(b), where j is an integer greater than or equal to 1 and less than or equal to M. During computation, feedback circuits in the layer-one current summation circuits (IS1) fix the voltages on the branch summation lines to a predefined voltage (Vs). Under this configuration, the channel current (Ids_i,j) of the transistor (M_i,j) at the i'th column and j'th row is a function of the threshold voltage (V_Ti,j), drain voltage (Vd_j), source voltage (Vs), and gate voltage (Vg_j) of the transistor (M_i,j) is Ids_i,j((Vd_j−Vs), (Vg_j−Vs−V_Ti,j)). The number of transistors (M) connected to each branch summation line (Is_i) is designed to be low enough so that the total parasitic leakage currents on the conductor line is negligible relative to the transistor channel currents. Under these conditions, the branch output current (Is_i) carried by the branch summation line (315) to the layer-one current summation circuit (311) is equal to the current-mode summation of the channel currents (Ids_i,j) of all the transistors ((M_i,1, M_i,2, . . . , M_i,j−1, M_i,j, . . . M_i,M−1, M_i,M)) connected to the branch summation line:

${Is}_{i} = \sum_{j}^{} {Ids}_{i, j} (({Vd}_{j} - Vs), ({Vg}_{j} - Vs - V_{Ti, j})) .$

In this embodiment, the transistor M_i,jserves the functions of a current mode computation (CMC) cell that executes a scalar function of two scalar inputs, (Vd_j−Vs) and (Vg_j−Vs−V_Ti,j), to generate a CMC cell output current (Ids_i,j) that is a function of the channel current (Ids_i,j) of the computation transistor (M_i,j). A plurality of CMC cells (M_i,1, M_i,2, . . . , M_i,j−1, M_i,j, . . . M_i,M−1, M_i,M) are coupled to a branch summation line (315) to form a current mode computation (CMC) branch (CB_i). This CMC branch (CB_i) executes transistor vector-vector operation on two input vectors (Vd_j−Vs)^Mand (Vg_j−Vs−V_Ti,j)^M, and produces a branch out current (Is_i). The branch summation line (315) is an electrical conductor that is electrically coupled to, and receives the CMC cell output currents (Ids_i,1, Ids_i,2, . . . , Ids_i,j−1, Ids_i,j, . . . , Ids_i,M−1, Ids_i,M) produced by, a plurality of CMC cells (M_i,1, M_i,2, . . . , M_i,j−1, M_i,j, . . . M_i,M−1, M_i,M) in the CMC branch (CB_i), and produces a branch output current (Is_i) that is a current-mode summation of all received CMC cell output currents. Under ideal conditions, the current-mode summation of a plurality of electrical currents (Ids_i,1, Ids_i,2, . . . , Ids_i,j−1, Ids_i,j, . . . , Ids_i,M−1, Ids_i,M) equals to the summation of those currents (Σ_jIds_i,j), while in reality none ideal effects such as parasitic leakage currents or timing delay may introduce inaccuracies. The layer-one current summation circuit (IS1) is electrically coupled to, and receives the branch output current (Is_i) produced by, at least one branch summation line (315), and produces a layer-one current summation circuit output current (Io_i) that is a function of the received branch output currents (Is_i). Combining the functions of a plurality of the CMC branches (CB₁, CB₂, . . . , CB_i, . . . , CB_N), the layer-one block in FIG. 3(b) executes transistor matrix-vector operations. Combining the functions of large number of layer-one blocks (B_m,n) in a multiple layer architecture, millions, a billion, a trillion or more of the CMC cells can be field-programmable to execute computations in parallel using their computation transistors, together with the branch summation lines that are coupled to those CMC cells, and the layer-one current summation circuits that are coupled to those branch summation lines, to execute large scale multiple-level transistor tensor operations that can have millions, billions, trillions or more scalar computations.

The layer-one block in FIG. 3(b) has two input lines for each row of transistors—one drain voltage input line and one gate voltage input line. This configuration provides the flexibility to field-program the computation mode of each computation transistor, as listed in the following table (Table 2).

TABLE 2

gate voltage
drain voltage
Transistor

input line
input line
computation mode

Constant
input
Multiplier

input
Constant
ReLU

input
input
Rectifier

input
Vs
Saturation

Vs
Don't care
Turned off

Source also at Vs
Vs
Turned off

When the voltage on the gate voltage input line is set to a constant voltage and the drain voltage input line is used to provide input signals, the computation transistors on the row operate in multiplier mode. When the voltage on the drain voltage input line is fixed and the gate voltage input line is used to provide input signals, the computation transistors on the row operate in ReLU mode. When both the drain voltage input line and the gate voltage input line are set to provide the same input voltage, the computation transistors on the row operate in rectifier mode. When the gate voltage input line is set to source voltage Vs, the computation transistors on the row are all turned off. When the drain voltage input line is set to source voltage Vs and the branch summation line is also set to voltage Vs, the computation transistors on the row are all turned off. When the drain voltage input line is set to source voltage Vs, and the branch summation lines are coupled to the layer-one saturation mode current summation circuit (IS1′) in FIG. 4(c), the floating-gate transistors on the row operate in saturation mode. In other words, this configuration provides the flexibility to program a selection of the scalar functions for computation transistors.

While the preferred embodiments have been illustrated and described herein, other embodiments of the invention may incorporate modifications and changes to the embodiments described herein. In the above embodiments, while each current-mode computation (CMC) cell comprises one floating-gate transistor, the CMC cell can also use other types of programmable-threshold voltage transistors. For example, instead of trapping electrical charges in an isolated conductor, it is also possible to trap electrical charges in the insulator layer that forms the gate insulator, creating a programmable-threshold voltage transistor. Each CMC cell can have multiple transistors. CMC cells can be arranged in different geometries instead of simple two-dimensional arrays. The layer-one blocks can be arranged in different geometries instead of two-dimensional arrays. The inputs and outputs can be configured in various directions. Instead of having two sets of input lines, the layer-one blocks can have one set of input lines or more than two sets of input lines. The layer-one block in the above embodiment does not have multiplexers at its inputs and outputs, but multiplexers can be used for embodiments of the present invention to improve array efficiency. A branch summation line does not have to be a line, it can be of any shape; it also can be a combination of electrical conductors of different shapes or materials. It is to be understood that there are many other possible modifications and implementations so that the scope of the invention is not limited by the specific embodiments discussed herein.

FIG. 3(d) is a simplified schematic diagram for one embodiment of the layer-one current summation circuit (IS1) in FIG. 3(b). A high gain amplifier (331) fixes the voltage on the branch summation line (315) to a predefined reference voltage V_sand provides an output (Vgm) that controls the gate voltages of three matched transistors (Mm1, Mm2, Mm3) whose source terminals are connected together to the same voltage at Vss. The voltage Vs also can be field-programmable. When the field-programmable input enable signal (Ens) is activated, select transistor Me1 is turned on and the amplifier (331) adjusts the current flowing through Mm1 to be equal to the branch output current (Is_i) by adjusting the gate voltage (Vgm) of matched transistors. Matched transistor Mm2 has the same gate voltage, source voltage, and properties as transistor Mm1. Therefore, when transistor Me2 is turned on by field-programmable enable signal EN2, the channel current flowing through Mm2 is designed to be equal to (Wm2/Wm1)*Is_i, where Wm2 is the effective channel width of transistor Mm2 and Wm1 is the effective channel width of transistor Mm1. Similarly, when transistor Me3 is turned on by field-programmable enable signal EN3, the channel current flowing through Mm3 is designed to be equal to (Wm3/Wm1)*Is_i, where Wm3 is the effective channel width of transistor Mm3. Matched transistors (Mm1, Mm2, Mm3) are arranged in a configuration known as “current mirrors” in the art of circuit design. The layer-one current summation circuit output current (Io_i) is therefore proportional to the branch output current (Is_i):

${Io}_{i} = K_{i} * \ln_{i}$

K_iis a scale factor depending on the design of the circuit IS1 and the status of the field-programmable enable signals (Ens, EN2, EN3). For example, when EN2 is activated and EN3 is deactivated, K_i=(Wm2/Wm1); when EN2 is deactivated and EN3 is activated, K_i=(Wm3/Wm1). Having multiple matched transistors (Mm2, Mm3) allows the scale factor K_ito be field-programmable, and we can use more matched transistors to increase the range of selection for the scale factor. The layer-one current summation circuit output current (Io_i) is coupled to the layer-two summation line. If all the field-programmable enable signals (EN2, EN3) are deactivated, this output is isolated from the layer-two summation line.

A “fixed voltage current sensing” (FVCS) circuit is defined as an electrical circuit comprises (1) an FVCS current-mode input terminal that receives a FVCS electrical current as a current input signal to the FVCS circuit, (2) an FVCS feedback circuit that is designed to fix the steady-state voltage of the FVCS current-mode input terminal at a predetermined voltage, and (3) an FVCS output circuit that produces an output signal that is a function of the FVCS electrical input current received at the FVCS current-mode input terminal. The electrical circuit (IS1) illustrated in FIG. 3(d) comprises a current-mode input terminal (315) that receives an electrical current (Is_i) as an input signal to the electrical circuit (IS1). The high gain amplifier (311) and transistor Mm1 form a feedback circuit that can fix the steady-state voltage of the current-mode input terminal (315) at a predetermined voltage (Vs). The current mirrors formed by matched transistors (Mm1, Mm2, Mm3) produce an output signal (Io_i) that is a function of the input current (Is_i) at the current-mode input terminal (315). Therefore, the electrical circuit illustrated in FIG. 3(d) is an embodiment of FVCS circuit. FIG. 3(t) illustrates an electrical circuit (1050) that is almost the same as the circuit in FIG. 3(d) except it has a “voltage clipper” (1051). There are two types of voltage clipper circuits: an “upper voltage clipper” that is defined as an electrical circuit designed to prevent the voltage of an electrical signal from exceeding a predetermined reference voltage, and a “lower voltage clipper” that is defined as an electrical circuit designed to prevent the voltage of an electrical signal from being lower than a predetermined reference voltage. The voltage clipper (1051) in FIG. 3(t) is used to fix the range of transient voltage (SVs) on the input terminal (315) of the FVCS circuit (1050), which is helpful to improve the speed and stability of the FVCS circuit.

FIG. 3(u-w) are simplified schematic diagram for embodiments of voltage clippers that can be used as the voltage clipper (1051) in FIG. 3(t). FIG. 3(u) shows a voltage clipper that comprise an upper voltage clipper formed by a diode (1052) and a reference voltage (VCPb). This voltage clipper prevents the voltage of SVs from exceeding the voltage level of VCPb plus the forward bias voltage of the diode (1052). FIG. 3(v) shows a voltage clipper that comprise an upper voltage clipper formed by a Zener diode (1054) and a reference voltage (VCPb). This voltage clipper prevents the voltage of SVs from exceeding the voltage level of VCPb plus the breakdown voltage of the Zener diode (1054). FIG. 3(w) shows a voltage clipper that comprise an upper voltage clipper formed by a transistor (1056) configured as a rectifier and a reference voltage (VCPb). This voltage clipper prevents the voltage of SVs from exceeding the voltage level of VCPb plus the threshold voltage of the transistor connected in rectifier mode (1056).

While the preferred embodiments have been illustrated and described herein, other embodiments of the invention may incorporate modifications and changes to the embodiments described herein. For instance, lower voltage clippers, instead of upper volage clippers (1052, 1054, 1056), or both types of voltage clippers can be used. In addition, there are other ways to implement voltage clippers or current mirrors. P-channel transistors, instead of n-channel transistors, can be used to form current mirror circuits. In the above embodiments, FVCS circuits (IS1, 1050) are used as the layer-one current summation circuits while FVCS circuits can be used for layer-two, layer-three, or other layers. The FVCS current-mode input terminals of the FVCS circuits may be coupled to an electrical signal provided by another integrated circuit on a different semiconductor substrate. FVCS circuits are very useful to support signal transfers between different IC chips or between different IC dice, achieving ultra-high bandwidth current mode signal transfer at low power. For example, FVCS circuits can support the inter-dice signal transfers using the inter-dice connections (537, 547, 548) illustrated in FIG. 5(d), or the signals transfers between different packaged integrated circuits such as those (525, 526, 527) illustrated in FIG. 5(c). It is to be understood that there are many other possible modifications and implementations so that the scope of the invention is not limited by the specific embodiments discussed herein.

A small enough layer-one block yields negligible parasitic leakage current on branch summation lines. The CMC cell output currents can be adjusted to be small while meeting the required signal-to-noise ratio. This not only saves operation power, but also mitigates the current overload problem. The layer-one current summation circuit (IS1) is also designed to prevent the “current overload” problem. Firstly, M is chosen to be low enough such that when all computation memory cells along the branch summation line are turned on at their maximum currents, the layer-one block will still not exhibit current overload. In addition, the output range of the amplifier (331) is designed to work, meaning to output a current proportional to the input summation current (Is_i), only when the branch current (Is_i) is less than a pre-calibrated maximum value (Is_max). When (Is_i) is close to Is_max, the amplifier (331) no longer can hold the voltage of the branch summation line at Vs, reducing the gate-to-source voltage of all transistors connected to the branch summation line and therefore the branch output current. Meanwhile, the current mirror output current (Io_i) is no longer proportional to the input current so that the maximum value of the output current is also limited, which prevents current overload problem at upper layers. In this way, the amplifier (331) and transistor Mm1 form a feedback circuit that can limit the amplitude of the received branch output current (Is_i). Transistor Mm1 and the maximum output voltage of the high gain amplifier (331) defines Is_max. Adjusting the size of transistor Mm1 can adjust Is_max. It may be desirable to make the size of Mm1 field-programmable. Such design effectively prevents the current overload problem, and provides a way to limit operation power. The computation remains accurate as well because when Is_iis close to Is_max, the output current is limited by the output function.

When the input functions are operating in saturation mode, the polarity of channel currents is changed and the layer-one saturation mode current summation circuit (IS1′) shown in the simplified schematic diagram in FIG. 4(c) can be used. IS1′ has the same n-channel current mirrors as those in FIG. 3(d). The difference is that its input stage is a p-channel current mirror formed by two matched transistors (Mp7, Mp8) and two enable transistors (Mpe7, Mpe8) as shown in FIG. 4(c). When field-programmable enable signals Enp7 and ENp8 are both activated, the branch output current (Is_i) is duplicated to the input of the n-channel current mirrors, providing a layer-one saturation mode current summation circuit output current (Io_i) that is compatible with the circuit in FIG. 3(d). At saturation mode, the channel currents of MOS transistors are not sensitive to the value of the drain voltage. IS1′ therefore does not need a feedback mechanism to fix the voltage of the branch summation line in this case. Because a current mirror is significantly faster than a high gain amplifier (331), saturation mode can achieve better performance than other modes. The circuit in FIG. 4(c) is also able to prevent the current overload problem because current mirror circuits by nature have upper limitations on the amplitudes of output currents.

While the preferred embodiments have been illustrated and described herein, other embodiments of the invention may incorporate modifications and changes to the embodiments described herein. For instance, in the above embodiments, the output currents produced by the layer-one current summation circuits (IS1) or layer-one saturation mode current summation circuit (IS1′) is proportional to the branch output current (Is_i). For more general cases, loi can be other functions of Is_iinstead of a simple proportional relationship. IS1, IS1′, VI1, VI1g can be designed in multiple ways. It is to be understood that there are many other possible modifications and implementations so that the scope of the invention is not limited by the specific embodiments discussed herein.

To avoid the non-ideal parasitic parameter induced problems listed in Table 1, the layer-one block (B_m,n) for embodiments of the present invention typically has a small transistor array (for example: 128 rows by 64 columns) that when combined with other blocks can support large-scale transistor tensor operations. FIG. 3(a) is a simplified symbolic block diagram showing an embodiment of a layer-two block (B⁽²⁾) for embodiments of the present invention. The layer-two summation lines (301) are coupled to a plurality of layer-one blocks (B_m,n) as shown in FIG. 3(a). An individual layer-two summation line comprises an electrical conductor that is electrically coupled to, and receives the layer-one current summation circuit output currents produced by, a plurality of layer-one current summation circuits (IS1 or IS1′) in the layer-one blocks (B_m,n), and that produces a layer-two summation line output current that is a current-mode summation of all received layer-one current summation circuit output currents. FIG. 3(e) is a simplified schematic diagram for one embodiment of the layer-two current summation circuit (IS2) in FIG. 3(a). In this embodiment, four matched transistors (Mp1, Mp2, Mp3, Mp4) and four enable transistors (Mpe1, Mpe2, Mpe3, Mpe4) are configured as current mirrors, as shown in FIG. 3(e). P-channel transistors are used in IS2 because, in this embodiment, the polarity of the input current (Is⁽²⁾) is opposite to that of the layer-one current summation circuits (IS1). When the field-programmable enable signals Enp1 and Enp2 are activated and Enp3 is deactivated, the layer-two summation circuit output current Io⁽²⁾=(Wp2/Wp1)Is⁽²⁾, where Wp2 is the effective channel width of transistor Mp2 and Wp1 is the effective channel width of transistor Mp1. When the field-programmable enable signals Enp1 and Enp3 are activated and Enp2 is deactivated, the layer-two summation circuit output current Io⁽²⁾=(Wp3/Wp1)Is⁽²⁾, where Wp3 is the effective channel width of transistor Mp3. When the field-programmable enable signals Enp1 and Enp4 are activated, a layer-two output circuit input current Iso=(Wp4/Wp1)Is⁽²⁾is produced as the input current to the interface output circuit (Vout) illustrated in FIG. 3(f), where Wp4 is the effective channel width of transistor Mp4. IS2 is also able to prevent the current overload problem because current mirror circuits by nature have an upper limit on the output currents. An individual layer-two current summation circuit (IS2) is electrically coupled to, and receives the layer-two summation line output current (Is⁽²⁾) produced by, at least one layer-two summation line (301), and produces a layer-two current summation circuit output current (Io⁽²⁾) that is a function of all received layer-two summation line output currents.

FIG. 3(f) is a simplified schematic diagram for one embodiment of the interface output circuit (Vout) in FIG. 3(e). The layer-two output circuit input current (Iso) in FIG. 3(e) passes through a variable impedance formed by a diode (D1) and 4 field-programmable variable resistors (VR1, VR2, VR3, VR4) to generate a voltage (Vi4) connected to the input of a programmable gain amplifier (351) which drives a layer-two output voltage (Avo) that is proportional to the voltage (Vi4) at the input of the amplifier (351). The gain of this programmable gain amplifier (351) is designed to be adjustable by a reference transistor (fgTa), as shown in FIG. 3(f). This reference transistor (fgTa) matches the computation transistors used in CMC cells such that it has matched temperature dependence in channel current. Using this reference transistor (fgTa) as a thermometer to adjust the gain of the amplifier (351) can mitigate temperature induced variations. The field-programmable gate voltage (Vga) allows field-programmable gain control of the amplifier (351). Avo can be used as input voltage for the next level of neural network computation; it is also connected to an analog-to-digital converter (ADC) to provide digital outputs (DVo) to digital interfaces. The field-programmable variable resistors (VR1, VR2, VR3, VR4) can typically be implemented by transistors with field-programmable gate voltages. These variable resistors (VR1, VR2, VR3, VR4) allow the interface output circuit (Vout) to support different output functions. For example, when VR2 and VR3 are set to zero, Vi4 is a sigmoid function with minimum level controlled by the field-programmable voltage Vref; when VR1 and VR4 are set to zero, the output is a ReLU function; when VR3 and VR4 are set to zero, the output is a linear function. This design allows field-programmable individual selection of output functions for transistor tensor operations.

The layer-two summation lines (301) and layer-two current summation circuits (SI2) provide effective ways to expand the number of inputs for transistor tensor operations. For example, if a layer-one block can support 128 inputs, and a layer-two summation line (301) is coupled to 64 layer-one current summation circuits (IS1), then the layer-two block with two-layer current summation architecture can support transistor tensor operations with up to 8192 inputs. If less inputs are needed, unused rows or layer-one blocks are disabled. If more inputs are needed, then more columns of layer-one blocks or layer-three summation lines can be used. Biases can be implemented as one layer-one row with constant inputs.

Layer-two summation lines (301) also can suffer parasitic parameter induced non-ideal effects similar to those in Table 1. It is important to choose a proper number (U) of layer-one current summation circuits (IS1) coupled to each layer-two summation line to avoid such parasitic induced errors. Because current mirrors are not sensitive to the voltage on input lines, parasitic voltage differences are less important. The width and the length of the layer-two summation line should be designed properly to avoid significant R*C delay. The current amplification factor in the layer-one current summation circuit (IS1) should be chosen properly to have a good signal-to-noise ratio and optimum power consumption.

The layer-two block in FIG. 3(a) also has layer-two input circuits (VI2) that drive layer-two input lines (303). These layer-two input circuits (VI2) also can be implemented by S&H circuits similar to that (VI1) of FIG. 3(c). The layer-two input lines (303) are coupled to the inputs of layer-one input circuits (VI1, VI1g) in layer-one blocks (B_m,n). Therefore, the voltages on the layer-one input lines (Vd_j, Vg_j) in each layer-one block can be controlled by layer-two input circuits (VI2). This multiple-layer input configuration supports field-programmable input configurations that approach the flexibility of prior art software configurations. Such flexible multiple-layer input configurations can be used to duplicate the same sets of inputs into different layer-one blocks that have different branch summation lines in order to expand the number of outputs. For example, if a layer-one block can support 64 outputs, and each layer-two input line (303) is coupled to 32 sets of layer-one input circuits (VI1, VI1g), then the layer-two block with two-layer input architecture can support transistor tensor operations with up to 2048 outputs. If less outputs are needed, unused columns or layer-one blocks are disabled. If more outputs are needed, more columns of layer-one blocks or layer-three input lines can be used to handle more outputs.

While the preferred embodiments have been illustrated and described herein, other embodiments of the invention may incorporate modifications and changes to the embodiments described herein. For example, the layer-two, layer-three, and other upper-layer summation lines and input lines can be in different shapes (do not need to be a line) or travel along different directions. These summation lines can be a combination of different layers of metals, vias, contacts, and other components. Blocks in different layers do not need to have the same dimensions or structures, and they can be arranged in different geometries instead of two-dimensional arrays. IS2, IS3, and VOUT can be designed in multiple ways. The input lines of the layer-one block in FIG. 3(b) are always horizontal to the branch summation lines; such configuration can support transistor matrix-vector (TMV) operations, while layer-one blocks for embodiments of the present invention can be configured in other ways to support other tensor operations such as TVV or TMM operations. In the above embodiments, current mirror circuits (IS2) are used as the layer-two current summation circuits, while we also can use FVCSC circuits (1060), such as the embodiment illustrated by the simplified schematic diagram in FIG. 3(x), as the layer-two current summation circuits. The embodiment in FIG. 3(x) is nearly the same as IS2 in FIG. 3(e) except that a high gain amplifier (1061), is used as a feedback circuit to fix the steady state voltage on the input terminal (SVps) to be equal to a reference voltage (VPs), and to provide the gate voltage (Vgmp) of the current mirrors formed by the matched transistors (Mp1, Mp2, Mp3, Mp4). In addition, a voltage clipper (1062) is also used to prevent overshoots of the voltage on SVps. It is to be understood that there are many other possible modifications and implementations so that the scope of the invention is not limited by the specific embodiments discussed herein.

FIG. 3(h) shows another layer-one block (370) that comprises a plurality of “current-mode computation” (CMC) branches (CM₁, CM₂, . . . , CM_i−1, CM_i, . . . , CM_N), where i is an integer larger or equal to 1 and less than or equal to N. An individual CMC branch (CM_i) comprises (a) a plurality of CMC cells (C_i,1, C_i,2, . . . , C_i,j−1, C_i,j, . . . , C_i,M), where an individual CMC cell comprises at least one computation transistor or a computation resistor, and produces a CMC cell output current that is a function of the channel current of the computation transistor or the electrical current flowing through the computation resistor, such as the exemplary CMC cells illustrated in FIG. 3(i-m); and (b) a branch summation line (371) that comprises an electrical conductor that is electrically couples to, and receives the CMC cell output currents produced by, a plurality of the CMC cells (C_i,1, C_i,2, . . . , C_i,j−1, C_i,j, . . . , C_i,M) in the CMC branch (CM_i), and that produces a branch output current (Is_i) that is a current-mode summation of all received CMC cell output currents. The branch summation line (371) is coupled to a layer-one current summation circuit (IS1) that receives the branch output current (Is_i) produced by the branch summation line (371), and produces a layer-one current summation circuit output current (Io_i) that is a function of the received branch output current. In this embodiment, each layer-one current summation circuit (IS1) is coupled to one CMC branch (CM_i), while a layer-one current summation circuit can also be coupled to multiple CMC branches. Multiplexers can sometimes be used between the CMC branches and the layer-one current summation circuits. Field-programmable selection circuits can be used to selectively activate a subset of or all of the layer-one and layer-two current summation circuits in order to combine the computation capabilities of different CMC branches to support parts of or all of a large-scale transistor tensor operation.

The input connections to the CMC branches (CM₁, CM₂, . . . , CM_i−1, CM_i, . . . , CM_N) are not shown in FIG. 3(h) because there are many possible configurations to arrange the input connections; different input configurations can support different types of transistor tensor operations. For example, the layer-one block (B_m,n) shown in FIG. 3(b) is a special case when the input lines (Vd₁, Vd₂, . . . , Vd_j, . . . , Vd_M; Vg₁, Vg₂, . . . , Vg_j, . . . , Vg_M) connected to its CMC branches (CB₁, CB₂, . . . , CB_i−1, CB_i, . . . , CB_N−1, CB_N) all travel in a horizontal direction and are shared by every CMC branch in the layer-one block; such configuration is suitable to support transistor matrix-vector operations.

FIG. 3(i) shows the schematic symbol for an embodiment of a CMC cell (991) that can be used for the layer-one block (370) illustrated in FIG. 3(h). In this embodiment, the CMC cell (991) comprises one computation transistor (MM_i,j) that is controlled by the gate-to-source voltage (VG_i,j) and drain-to-source voltage (VD_i,j) and outputs its channel current (I_i,j) as the CMC cell output current. The channel current (I_i,j) is a function of VG_i,jand VD_i,jaccording to the I-V relationship of the computation transistor (MM_i,j). If the computation transistor is operating in triode region, we have

$I_{i, j} = K_{n} * {VD}_{i, j} * ({VG}_{i, j} - V_{T})$

where V_Tis the threshold voltage of the computation transistor (MM_i,j). In this embodiment, The branch output current (Is_i) of CMC branch CM_iis the current mode summation of CMC cell output currents (I_i,1, I_i,2, . . . , I_i,j, . . . , I_i,M) of all the CMC cells (C_i,1, C_i,2, . . . , C_i,j, . . . , C_i,M) coupled to the branch summation line (371) as:

${Is}_{i} = \sum_{j}^{M} I_{i, j} = \sum_{j}^{M} [K_{n} {VD}_{i, j} * ({VG}_{i, j} - V_{T})],$

where we assume the V_Tof all computation transistors are the same because they are matched transistors manufactured by IC technology. The above equation shows that the summation current is proportional to the dot product of two input vectors (VD_i,1, . . . , VD_i,j, . . . , VD_i,M) and ((VG_i,1−V_T), . . . , (VG_i,j−V_T), . . . , (VG_i,M−V_T)). In this embodiment, a CMC branch executes transistor vector-vector operation of two input vectors and produces a branch output current to represent the transistor vector-vector computation results. The computation transistor (MM_i,j) can use body effects as a way to reducer parasitic leakage currents. One convenient approach is to use a native transistor as the computation transistor (MM_i,j). A native transistor is an MOS transistor with a threshold voltage close to zero. Native transistors can be manufactured by proper adjustment of threshold voltage implant dosage, by adjusting the storage charge of a field-programmable-threshold voltage transistor, or by other methods. Using native transistors, we have Is_i=Σ_j^MI_i,j=Σ_j^M[K_nVD_i,j*VG_i,j].

Each CMC branch (CM_i) in FIG. 3(h) with the CMC cells illustrated in FIG. 3(i) is able to execute the transistor vector-vector operation of two vectors of size M. The input vectors of an CMC branch can be a set of various kinds of parameters. For example, the vector can be parts or all of external input vectors, parts of all of internally vectors generated during deep level transistor tensor operation, a set of data stored in CMC memory cells, parts of or all of Hadamard parameters used for transistor convolution operations, parts of signed data, parts of multiple-precision data, or many other types of parameters. If the size of the input vector is larger than M, we can use layer-two summation lines to combine CMC branches in multiple layer-one blocks to execute dot products of large vectors. The number of layers can be increased to meet larger vector size. If vector components have both positive and negative numbers, the methods described in FIGS. 6(a-d) can be used to support negative components. If the computations require greater accuracy, the methods described in FIG. 6(e) can be used. With field-programmable configurations, CMC branches for embodiments of the present invention are able to support a wide variety of dot product computations. Since all tensor operations are based on vector dot products, CMC branches for embodiments of the present invention are therefore able to support vector-vector, matrix-vector, matrix-matrix, and other kinds of tensor operations of various size, level, and rank while achieving unprecedented performance by parallel computations.

In addition, the CMC cell in FIG. 3(i) not only can operate in the triode region to support multiplier mode, but can also operate in other modes such as ReLU, saturation, or rectifier modes. In other words, CMC branches for embodiments of the present invention can support transistor vector-vector operations using scalar functions other than multiplication, such as ReLU, saturation, or rectifier functions. Since all transistor tensor operations are based on transistor vector-to-vector operations, CMC branches for embodiments of the present invention are therefore able to support transistor vector-vector, transistor matrix-vector, transistor matrix-matrix, and other kinds of transistor tensor operations of various size, depth, and rank while achieving unprecedented performance by parallel computations executed by CMC cells.

FIG. 3(j) shows the schematic symbol for another embodiment of a CMC cell (992) that can be used in the layer-one block (B_m,n) illustrated in FIG. 3(h). In this embodiment, the CMC cell (992) comprises one field-programmable-threshold voltage transistor (FM_i,j) that is controlled by the gate-to-source voltage (VG_i,j), drain-to-source voltage (VD_i,j), and the storage charge stored in its floating-gate (Q_i,j). Its channel current (I_i,j) is the CMC cell output current. The channel current (I_i,j) is a function of (VG_i,j−V_T0−C_eQ_i,j) and VD_i,jaccording to the I-V relationship of the transistor. A CMC branch equipped with such CMC cells (992) is therefore able to support transistor vector-vector operations of two input vectors (VD_i,1, . . . , VD_i,j, . . . , VD_i,M) and ((VG_i,1−V_T0−C_eQ_i,1), . . . , (VG_i,j−V_T0−C_eQ_i,j), . . . , (VG_i,M−V_T0−C_eQ_i,M)). The most common embodiment of field-programmable-threshold voltage transistors are floating-gate MOS transistors. The floating-gate transistor (FM_i,j) not only can execute scalar functions supported by common MOS transistors as a computation transistor, but can also serve as a memory device for storing operation parameters.

FIG. 3(k) shows the schematic symbol for another embodiment of a CMC cell (993) that can be used in FIG. 3(h). This CMC cell comprises one computation transistor (MD_i,j) that executes scalar computations and provides its channel current (I_i,j) as the cell output current of the CMC cell. One select transistor (MW_i,j) provides the gate voltage (VG_i,j) of the computation transistor from a cell gate voltage input line (BL) when the select transistor is activated by its gate select signal (WL) and isolates the gate terminal of the computation transistor from the cell gate voltage input line (BL) when the select transistor is deactivated by WL. This select transistor (MW_i,j) can be an MOS transistor (including a native transistor), a field effect transistor, or other types of transistors; it also can be a variable resistor controlled by WL. This CMC cell (993) is able to support all transistor tensor operations that the CMC cell (991) in FIG. 3(i) can support. In addition, this CMC cell (993) also functions as a memory device by holding the value of the gate voltage of the computation transistor (MD_i,j). However, it cannot fix the voltage indefinitely due to non-ideal effects such as leakage currents. To maintain the value of the gate voltage, it can “refresh” the value by rewriting the same value periodically, similar to how memory is updated in dynamic random-access memory (DRAM). This CMC cell (993) is therefore a current-mode computation dynamic memory cell.

FIG. 3(l) shows the schematic symbol for another embodiment of a CMC cell (994) that is nearly identical to the CMC dynamic memory cell (993) in FIG. 3(k). This cell however has a storage-control capacitor (C_i,j); one terminal of (C_i,j) is coupled to the gate terminal of the computation transistor (MD_i,j) and the other terminal is coupled to a field-programmable CMC cell gate voltage control signal (GC). When the voltage on GC is a constant, the storage-control capacitor (C_i,j) behaves as a storage capacitor that helps to reduce the influence of leakage currents and voltage coupling problems, similar to the functions of the storage capacitors in DRAM memory cells. When GC is controlled as an input signal, then the CMC cell in FIG. 3(l) functions as a floating gate device that can support the functions of the CMC cell in FIG. 3(j), where VG_i,jbehaves as the floating gate and GC as the gate of the floating gate device, except that the storage charge of this device may need to be refreshed periodically.

FIG. 3(m) shows the schematic symbol for an embodiment of a plurality of CMC cells (995, 996, 997) that share one select transistor (MW_s) that provides the gate voltage (VG_s) of the computation transistors in those CMC cells (995, 996, 997) from an cell gate voltage input line (BL) when the select transistor is activated by its gate select signal (WL) and isolates the gate terminals of those computation transistors from the cell gate voltage input line (BL) when the select transistor is deactivated by WL. The CMC cells (995, 996, 997) can be in different CMC branches. The circuit in FIG. 3(m) is also a type of CMC dynamic memory cell because it can store and maintain datum with dynamic refresh procedures. Such design is useful when the same input is shared by multiple CMC cells like when performing transistor convolution tensor operations. For CMC dynamic memory cells of embodiments of the present invention, reduction of transistor leakage currents is more important than transistor performance. Use of body effects by proper selection of substrate voltages can reduce leakage currents.

FIG. 3(n) shows the schematic symbol for another embodiment of a CMC cell (1011) that can be used in the layer-one block (B_m,n) illustrated in FIG. 3(h). In this embodiment, the CMC cell (1011) comprises one computation resistor (RM_i,j). A computation resistor is a resistor in a CMC cell that is used to support computation function for the CMC cell. One terminal of the computation resistor (RM_i,j) is coupled to an input line at voltage VR_jwhile the other terminal is connected to a summation line at voltage VRs_i, as shown in FIG. 3(p). The CMC cell output current I_i,j=(VR_j−VRs_i)/(RRM_i,j), where RRM_i,jis the resistance value of the computation resistor RM_i,j. The branch output current (Is_i) of CMC branch CM_iis the current mode summation of all the CMC cells coupled to the branch summation line (371) as Is_i=Σ_j^M(VR_j−VRs_i)/(RRM_i,j). A CMC branch equipped with such CMC cells (1011) is therefore able to support transistor vector-vector operations of two input vectors (1/RRM_i,1, . . . , 1/RRM_i,j, . . . , 1/RRM_i,M) and ((VR₁−VRs_i), . . . , (VR_i−VRs_i), . . . , (VR_M−VRs_i)). Therefore, this CMC cell (1011) is able to execute tensor computations. The current-voltage relationships of ideal resistors are linear, so that this CMC cell (1011) operates in multiplier mode.

FIG. 3(o) shows the schematic symbol for another embodiment of a CMC cell (1012) that is almost the same as the CMC cell (1011) in FIG. 3(n) except that it comprises a variable computation resistor (RV_i,j) instead of a resistor with a fixed value. The resistance value of this variable computation resistor (RV_i,j) can be controlled by a variable computation resistor control signal (VRc_j), which also can be used as a field-programmable input signal. Such variable computation resistor can be implemented using MOS transistor, field effect transistor (FET), floating-gate transistor, or other types of devices.

Resistors built by current art integrated circuit technologies are larger and more difficult to be precise than transistors so that the CMC cells in FIGS. 3(n, o) do not have advantages over the CMC cells in FIG. 3(i-m) if they are built on the active areas used to build transistors. However, resistors can be built without using active areas, as illustrated in FIG. 7(a). In this embodiment, transistors (1071) are built on a semiconductor substrate (1070), while computation resistors (1075) are built on top of those transistors, as shown by the simplified cross-section picture in FIG. 7(a). One terminal of the computation resistor (1075) is coupled to a metal input line (1076), while the other terminal of the computation resistor (1075) is coupled to a piece of metal (1077) which is connected to a current summation line (1074) using a via (1078). This current summation line (1074) is connected to the diffusion area (1072) of a transistor (1071) using a contact (1073). Each computation resistor (1075) and nearby metal structures (1076, 1077) form a CMC cell that does not occupy active semiconductor areas that can be used to build transistors (1071). Such CMC cells are therefore able to achieve higher density at lower cost. FIG. 7(b) shows another example where computation resistors (1079) are built inside via holes between an input line (1078) and a summation line (1074). Current art integrated circuit technologies can have 20 layers of metals so that it is also possible to stack computation resistors in large number of layers as illustrated in FIG. 7(c). In this cross-section diagram, 10 layers of computation resistors (1081) are stacked on top of transistors (1071). Each computation resistor (1081) and nearby metal structures (1082, 1083) form a CMC cell illustrated in FIG. 3(n) or FIG. 3(o). A summation line (1074) is connected to computation resistors in different layers through vias (1084). Tensor operations with billions, trillions or more parameters can be executed by a small semiconductor dice using such structures. We will call electrical components that are embedded in the inter-layer dielectric (ILD) layers of integrated circuits as “ILD embedded components”. These components can include transistors, resistors, and other electrical components that are embedded in the ILD layers to reduce the overall size of the integrated circuit.

While the preferred embodiments have been illustrated and described herein, other embodiments of the invention may incorporate modifications and changes to the embodiments described herein. The transistor in the CMC cell shown in FIG. 3(i) is an n-channel MOS transistor, while other types of transistors such as p-channel transistors, field-effect transistors (FET), bipolar transistors, and combinations of other types of electrical devices also can be used in CMC cells. The CMC cell output current does not have to be the drain-to-source current; a source-to-drain current also can be used as the output current. Combinations of channel currents and/or resistor currents from multiple sources also can be used as the CMC cell output current. Current art IC technologies typically manufacture transistors that are optimized for speed. For transistors used in CMC cells of embodiments of the present invention, low leakage and low power are of greater priority. Body effects can be used to reduce leakage currents of the transistors in the CMC cells. The computation resistors shown in FIG. 7(a-c) can be fixed value resistors, variable resistors, poly silicon transistors, or other types of ILD embedded components. It is to be understood that there are many other possible modifications and implementations so that the scope of the invention is not limited by the specific embodiments discussed herein.

FIG. 3(p) shows a simplified schematic diagram for an embodiment of a hybrid-analog-digital (HAD) CMC cell. A HAD CMC cell is defined as a CMC cell that is designed to support computations between a digital input and an analog input. In this example, the CMC cell (1005) comprises 4 sub-cells (1000-1003), where each sub-cell comprises one binary memory device (MC0_i,j, MC1_i,j, MC2_i,j, MC3_i,j), one enable transistor (ME0_i,j, ME1_i,j, ME2_i,j, MC3_i,j), and one computation resistor (RM0_i,j, RM1_i,j, RM2_i,j, RM3_i,j) of predefined resistor value (RM0, RM1, RM2, RM3). All four sub-cells (1000-1003) are connected to an input line at voltage VR_j, and provide sub-cell output currents (I0_i,j, I1_i,j, I2_i,j, I3_i,j) combined to be a CMC cell (1005) output current (I_i,j), where I_i,j=(I0_i,j+I1_i,j+I2_i,j+I3_i,j), as illustrated in FIG. 3(p). This CMC cell output current (I_i,j) flows into a current summation line at voltage VRs_i, as shown in FIG. 3(p). The binary memory devices (MC0_i,j, MC1_i,j, MC2_i,j, MC3_i,j) are memory devices that stores one binary number (0 or 1). Examples of such binary memory devices are DRAM cells, static-random-access-memory (SRAM) cells, floating-gate transistors, or other types of memory devices. The data stored in these memory devices (MC0_i,j, MC1_i,j, MC2_i,j, MC3_i,j) control gate voltages (Vg0_i,j, Vg1_i,j, Vg2_i,j, Vg3_i,j) of the enable transistors (ME0_i,j, ME1_i,j, ME2_i,j, MC3_i,j) to activate or turn off those transistors. For example, when MC0_i,jstores binary number 1, ME0_i,jis activated, and the output current of the sub-cell (1000) I0_i,j=(VR_j−VRs_i)/RM0; when MC0_i,jstores binary number 0, ME0_i,jis turned off, and I0_i,jis approximately zero; when MC1_i,jstores binary number 1, ME1_i,jis activated, and the output current of the sub-cell (1001) I1_i,j=(VR_j−VRs_i)/RM1; when MC1_i,jstores binary number 0, ME1_i,jis turned off, and I1_i,jis approximately zero; when MC2_i,jstores binary number 1, ME2_i,jis activated, and the output current of the sub-cell (1002) I2_i,j=(VR_j−VRs_i)/RM2; when MC2_i,jstores binary number 0, ME2_i,jis turned off, and I2_i,jis approximately zero; when MC3_i,jstores binary number 1, ME3_i,jis activated, and I3_i,j=(VR_j−VRs_i)/RM3; and when MC3_i,jstores binary number 0, I3_i,jis approximately zero. In this example, if we control the values of the computation resistors so that RM3=RM0/8, RM2=RM0/4, and the RM1=RM0/2, then the data stored in the 4 memory devices (MC0_i,j, MC1_i,j, MC2_i,j, MC3_i,j) form a 4-bit binary number, and the CMC cell output current (I_i,j) equals the binary number times (VR_j−VRs_i)/RM0. The CMC cell illustrated in FIG. 3(p) is therefore an embodiment of a hybrid-analog-digital (HAD) CMC cell that is designed to support computations between a digital input and an analog input.

FIG. 3(q) is a simplified schematic diagram of another embodiment of a HAD CMC cell (1025). The structures of this CMC cell (1025) are nearly the same as those of the HAD CMC cell (1005) in FIG. 3(p) except that the computation resistors (RV0_i,j, RV1_i,j, RV2_i,j, RV3_i,j) in the sub-cells (1020-1023) are variable resistors controlled by an HAD cell variable computation resistor control voltage (VVR_j), as illustrated in FIG. 3(q). This HAD CMC cell (1025) is therefore able to execute computations between digital data stored in memory devices (MC0_i,j, MC1_i,j, MC2_i,j, MC3_i,j) and analog inputs represented by (VR_j−VRs_i) or represented by WR_j.

FIG. 3(r) is a simplified schematic diagram of another embodiment of a HAD CMC cell (1035). The structures of this CMC cell (1035) are nearly the same as those of the HAD CMC cell (1025) in FIG. 3(q), except that the variable computation resistors are replaced by transistors (MVR0_i,j, MVR1_i,j, MVR2_i,j, MVR3_i,j) in the sub-cells (1030-1033). The gate terminals of these transistors (MVR0_i,j, MVR1_i,j, MVR2_i,j, MVR3_i,j) are controlled by an HAD cell gate voltage control signal (VVG_j), as illustrated in FIG. 3(r). When those transistors (MVR0_i,j, MVR1_i,j, MVR2_i,j, MVR3_i,j) are operating in triode region, they behave as variable computation resistors so that this CMC cell (1035) can support similar functions as the CMC cell (1025) in FIG. 3(q). In addition, the transistors (MVR0_i,j, MVR1_i,j, MVR2_i,j, MVR3_i,j) can operate in saturation regions to function as current mirrors. This enables execution of computations between digital data stored in memory devices (MC0_i,j, MC1_i,j, MC2_i,j, MC3_i,j) and analog inputs represented by (VR_j−VRs_i) or by WG_j.

FIG. 3(s) is a simplified schematic diagram of another embodiment of a HAD CMC cell (1045). The structures of this CMC cell (1045) are nearly the same as those of the HAD CMC cell (1035) in FIG. 3(r), except that enable transistors (ME0_i,j, ME1_i,j, ME2_i,j, MC3_i,j) and the binary memory devices (MC0_i,j, MC1_i,j, MC2_i,j, MC3_i,j) in FIG. 3(r) are replaced by floating-gate transistors (MCF0_i,j, MCF1_i,j, MCF2_i,j, MCF3_i,j). The gate terminals of these floating-gate transistors (MCF0_i,j, MCF1_i,j, MCF2_i,j, MCF3_i,j) are controlled by an HAD cell floating gate control signal (VVFG_j), as illustrated in FIG. 3(s). These floating-gate transistors (MCF0_i,j, MCF1_i,j, MCF2_i,j, MCF3_i,j) serve both the functions of binary memory device as well as the functions of the enable transistors. If a These floating-gate transistors (MCF0_i,j, MCF1_i,j, MCF2_i,j, MCF3_i,j) is programmed to store binary number 0, the transistor is always off; and if it is programmed to store binary number 1, then the transistor can be activated by WFG_j. This enables execution of computations between digital data stored in memory devices (MCF0_i,j, MCF1_i,j, MCF2_i,j, MCF3_i,j) and analog inputs represented by (VR_j−VRs_i) or by WG_j.

While the preferred embodiments have been illustrated and described herein, other embodiments of the invention may incorporate modifications and changes to the embodiments described herein. For example, the digital data stored in HAD CMC cells do not need to be 4-bit binary data—larger or smaller digital data also can be used as inputs to HAD CMC cells. The analog input signals do not have to be voltages—analog signals represented by electrical currents or other formats also can be used. It is to be understood that there are many other possible modifications and implementations so that the scope of the invention is not limited by the specific embodiments discussed herein.

FIG. 4(a) is a simplified symbolic block diagram of an embodiment of a layer-three block (B⁽³⁾) of the present invention. This layer-three block (B⁽³⁾) comprises R columns and T rows of layer-two blocks (B⁽²⁾_r,t), where R and T are positive integers, r is an integer greater than or equal to 1 and less than or equal to R, and t is an integer greater than or equal to 1 and less than or equal to T. The layer-three summation lines (401) are coupled to the layer-two current summation circuit (IS2) inside those layer-two blocks (B⁽²⁾_r,t). Embodiments of layer-two blocks are illustrated in FIG. 3(a, h), while an embodiment of layer-two current summation circuit (IS2) is illustrated in FIG. 3(e). An individual layer-three summation line (401) comprises an electrical conductor that is electrically coupled to, and receives the layer-two current summation circuit output currents produced by, a plurality of layer-two current summation circuits (IS2), and that produces a layer-three summation line output current (Is⁽³⁾) that is a current-mode summation of the received layer-two current summation circuit output currents (ideally, Is⁽³⁾=Σ_sIo⁽²⁾_s). This layer-three block (B⁽³⁾) comprises a plurality of layer-three current summation circuits (IS3), wherein an individual layer-three current summation circuit is electrically coupled to, and receives the layer-three summation line output current (Is⁽³⁾) produced by, at least one layer-three summation line (401), and produces a layer-three current summation circuit output current (Io⁽³⁾) that is a function of all received layer-three summation line output currents (Is⁽³⁾). FIG. 4(b) is a simplified schematic diagram of one embodiment of the layer-three current summation circuit (IS3) in FIG. 4(a). In this embodiment, four matched transistors (Mn1, Mn2, Mn3, Mn4) and four enable transistors (Mne1, Mne2, Mne3, Mne4) are connected as current mirrors, as shown in FIG. 4(b). N-channel transistors are used in IS3 because the polarity of the input current (Is⁽³⁾) is opposite of that of the layer-two current summation circuits (IS2). In addition, two matched p-channel transistors (Mp5, Mp6) and two enable transistors (Mpe5, Mpe6) are connected as a current mirror to duplicate the current amplitude and invert the polarity of the current flows through Mn4, as shown in FIG. 4(b). When the field-programmable enable signals ENn1 and ENn2 are activated and ENn3 is deactivated, the layer-three summation circuit output current Io⁽³⁾=(Wn2/Wn1)Is⁽³⁾, where Wn2 is the effective channel width of transistor Mn2 and Wn1 is the effective channel width of transistor Mn1. When the field-programmable enable signals ENn1 and ENn3 are activated and ENn2 is deactivated, the output current Io⁽³⁾=(Wn3/Wn1)Is⁽³⁾, where Wn3 is the effective channel width of transistor Mn3. When the field-programmable enable signals ENn1, ENn4, ENp5, and ENp6 are activated, a current Iso3=(Wn4/Wn1)Is⁽³⁾is sent to the interface output circuit (Vout), as illustrated in FIG. 4(b), where Wn4 is the effective channel width of transistor Mn4. The function of the interface output circuit (Vout) in this embodiment can be the same as that in FIG. 3(f). IS3 is also able to prevent current overload because current mirror circuits by nature have an upper limitation on maximum value of output currents.

The layer-three block (B⁽³⁾) in FIG. 4(a) also has layer-three input circuits (VI3) that drive layer-three input lines (403). These layer-three input circuits (VI3) also can be implemented by S&H circuits similar to that of FIG. 3(c). The layer-three input lines (403) and layer three input circuits (VI3) provide field-programmable controls on the layer-two input circuits (VI2) in all the layer-two blocks (B⁽²⁾_r,t). Therefore, the voltages on the layer-one input lines (Vd_j, Vg_j) in all layer-one blocks included in this layer-three block (B⁽³⁾) can be controlled by those layer-three input circuits (VI3). This multiple-layer input configuration supports field-programmable input configurations that approaches the flexibility of prior art software configurations. Such flexible multiple-layer input configurations can be used to duplicate the same sets of inputs into different layer-one blocks that have different branch summation lines in order to expand the number of outputs. If the number of inputs is greater than the capacity of one layer-two block, we can use the layer-three current summation circuits (IS3) to form a three-layer current summation structure to support more inputs without significant increases in delay time. If insufficient, more layers can be added until the requirement is fulfilled. Similarly, layer-three input circuits (VI3) can be used to expand the number of output neurons.

FIG. 5(a) is a simplified block diagram for an embodiment of a multiple-layer summation line architecture of the present invention described in FIGS. 3(a-h) and in FIG. 4(a-c). A subset of CMC cells (501) represented symbolically by small circles in FIG. 5(a) are coupled to a branch summation line (503). All the cell output currents (502) from those CMC cells (501) flow into the branch summation line (503) that brings current mode summation of all the cell output currents to a layer-one current summation circuit (IS1). This circuit produces a layer-one current summation circuit output current (504) that is a function of the summation of all CMC cell output currents (502) from the CMC cells (501) coupled to the branch summation line (503). All layer-one current summation circuit output currents (504) of the layer-one current summation circuits (IS1) are received by a layer-two summation line (505) and are inputted to a layer-two current summation circuit (IS2) that produces a layer-two summation circuit output current (507) that is a function of the current-mode summation of the layer-one current summation circuit output currents received by the layer-two summation line (505). Similarly, the layer-two current summation circuit output currents (507) are received by a layer-three summation line (508) such that the summation of the IS2 output currents is input to a layer-three current summation circuit (IS3). This multiple layer architecture can be expanded to an arbitrary number of layers. If the number of inputs can be supported in layer-two, IS2 can send the result of current summation to an interface output circuit (506) that produces the inputs for the next level transistor tensor operation or produces digitized outputs to a digital interface. If a third layer is needed, then outputs are produced at the layer-three interface output circuit (509). One can continue to add additional layers as needed—outputs will be produced at the final layer. If one semiconductor dice cannot provide the needed computation memory cells, upper layer summation lines or input lines can be implemented by inter-dice connections to expand parallel transistor computations beyond dice boundaries, as illustrated in FIGS. 5(d, e). The architecture illustrated in FIG. 5(a) provides flexibility to configure CMC cells to support large-scale transistor tensor operations. Prudent designs of the summation lines in each layer can mitigate non-ideal parasitic parameters induced problems such that the delay time introduced by each layer is only the delay time of current mirrors. With this architecture, large-scale transistor tensor operations can be executed within a few gate delays.

FIG. 5(b) is a simplified block diagram for an embodiment of input control architecture of the present invention described in FIGS. 3(a-h) and in FIG. 4(a-c). Layer-three input circuits (VI3) drive layer-three input lines (513) under field-programmable controls to reach layer-two input circuits (VI2). Layer-two input circuits (VI2) drive layer-two input lines (512) under field-programmable controls to reach layer-one input circuits (VI1, VI1g). Layer-one drain voltage and gate voltage input circuits (VI1, VI1g) drive layer-one input lines inside layer-one blocks (B_i,j) to configure CMC cells. All input circuits (VI3, VI2, VI1, VIg) in various layers only drive a limited number of active devices, mitigating non-ideal parasitic parameter induced problems. If the number of outputs is greater than the number of outputs in layer-one blocks, the inputs are duplicated to other layer-one blocks that have different current summation trees, effectively expanding the overall number of outputs. Careful designs for the input lines in each layer can mitigate non-ideal parasitic parameter induced problems such that the delay time introduced by each layer is the delay time of input circuits (VI3, VI2, VI1, VI1g): one analog gate delay per layer. The architecture illustrated in FIG. 5(b) provides flexibility to configure CMC cells to support transistor tensor operations of various sizes. With this architecture, large-scale transistor tensor operations can be executed in parallel within a few gate delays.

A transistor tensor operation of embodiments of the present invention comprises many field-programmable options to allow flexibility in configuration. These options are typically selected by register write, scan chain, or memory write operations to memory devices distributed around an integrated circuit (IC) of embodiments of the present invention. The procedures to define those available options require detailed knowledge of actual designs, and such details can differ between IC designs. Therefore, a packaged IC chip of embodiments of the present invention typically comprises integrated circuits of different functions packaged in the same chip, as shown by the symbolic block diagram in FIG. 5(c). In this embodiment, multiple layer current-mode computation integrated circuits of embodiments of the present invention (521) are packaged in the same package with CPU or GPU (522), a flash EPROM (523), a system memory (524) which is typically a dynamic random-access memory (DRAM) IC, and an image sensor or image display IC (528). The multiple layer CMC IC (521) of embodiments of the present invention can have multiple stacked ICs as described in above sections. FIG. 5(d) shows a simplified embodiment of horizontal inter-dice communication lines (537) between different IC dice (531, 532, 533, 534). FIG. 5(e) shows a simplified embodiment of through-die inter-dice communication lines (547) between vertically stacked IC dice (541, 542, 543, 544). The microprocessor (522) executes firmware stored in the flash EPROM (523) to configure CMC IC (521) through a digital interface (525). The system memory (524) is used for temporary storages to achieve better performance. The Flash EPROM (523) also can store information such as which parts in the transistor tensor operation (521) ICs are defective, which blocks are used by which application, and the parameters needed to configure the CMC IC (521). Libraries of firmware circuits are also stored in the flash EPROM (523) to support functions similar to those of an operating system. The CMC ICs (521) also can send data to the microprocessor (522) through a digital interface (525) such that the microprocessor (522) can assist in executing computations that are not built in the ICs computing the transistor tensor operations. This digital interface (525) also provides the capability to communicate with external systems through an external digital interface (526). The CMC ICs (521) can also have a direct communication interface (527) with image sensor and/or image display ICs (528). This direct communication interface (527) can be a digital interface or an analog interface.

The architectures illustrated in FIGS. 3(a-h), FIG. 4(a-c), and FIG. 5(a-c) provide flexibility to configure CMC cells to support transistor tensor operations of wide varieties of sizes. For example, if a layer-one array comprises 128 inputs and 64 outputs, while each upper layer summation line connects 32 lower layer current summation circuits and each upper layer input line connects 32 lower layer input circuits, then a layer-two block can support a transistor tensor operation of up to 8 million parameters; a layer-three block can support up to 8 billion parameters; a layer-four block would be able to support up to 8 trillion parameters. If all the devices in an integrated circuit are insufficient to support the parameters of the transistor tensor operation being executed, the analog or digital output circuits allow convenient interfaces to other integrated circuits to expand into multiple-dice operations. For smaller computations, the unused layer-one blocks can be configured to support computations at different levels, or computations of other applications. Under this architecture, multi-leveled large-scale transistor tensor operations can be implemented with a flexible field-programmable configuration. Each level of transistor tensor operations can be executed in parallel and finished in a few gate delays, and multiple-level transistor tensor operations can be executed in parallel for a few gate delays in each level. Using the sample-and-hold circuits, one can choose to pipeline multiple-level computations to achieve higher throughputs or disable unused circuits to save power.