164 may be stored in computer registers, in a relational database as a table having a plurality of different fields and records, XML documents, or flat files. The data 164 may also be formatted in a computer-readable format such as, but not limited to, binary values, ASCII or Unicode. Moreover, the data 164 may comprise information sufficient to identify the relevant information, such as numbers, descriptive text, proprietary codes, pointers, references to data stored in other memories (including other network locations) or information that is used by a function to calculate the relevant data. Data 164 may also contain instructions or instruction sets which are received by ML-ASIC 110 to process.

The instructions 163 may be algorithms which are selectively invoked or executed by the ML-ASIC 110. For example, the instructions may be algorithms which outline how certain commands are executed on ML-ASIC 110. Instructions 163 may also be integrated into various components of the ASIC to perform functions described within this disclosure.

Data port 170 may be a physical port which allows for input and/or output of data from the ML-ASIC 110. For example, data port 170 may send and receive commands or computations for ML-ASIC 110 to respond to.

Also illustrated in FIG. 1B in dotted lines are informational pathways through which components within an ML-ASIC communicate, such as pathways 170a-170d. For example, chip manager 140 and sequencer 180 may communicate with one another via pathway 170a, chip manager 140 and data port 170 may communicate via pathway 170b, chip manager 140 and sequencer 180 may communicate via pathway 170c, and sequencer 180 and processors 161 may communicate via pathways 170d. Although only four pathways are shown in FIG. 1B, other pathways between components may be included in the ML-ASIC 110. A person of skill in the art would understand that these pathways may be integrated into the ASIC or achieved through dedicated interconnects and connectors.

Example Methods

In addition to the operations described above and illustrated in the figures, various operations will now be described. It should be understood that the following operations do not have to be performed in the precise order described below. Rather, various steps can be handled in a different order or simultaneously, and steps may also be added or omitted.

FIG. 2 is a flowchart 200 illustrating the dynamic throttling of an ML-ASIC, such as ML-ASIC 110. At block 205, a voltage regulator, such as voltage regulator 191 may monitor sense line 195 for sense signals 194 indicative of a threshold condition being satisfied. For example, a threshold condition may be excessive current or voltage being drawn by the ML-ASIC 110.

In response to detecting a threshold condition, the voltage regulator 191 may send a signal, such as control signal 192 to ML-ASIC 110, as shown in block 210. In some instances, prior to sending a control signal, the voltage regulator 191 may generate an OCP_FLAG, as discussed here. Setting the OCP_FLAG may trigger a control signal 192 to be sent to the ML-ASIC 110. The control signal 192 may be a digital signal, which allows for more granularity and precision in the instructions being sent to the ML-ASIC 110 than an analog signal. The control signal may be preconfigured to be understood by ML-ASIC 110 and may be sent repeatedly in fixed time intervals or constantly asserted by the voltage regulator 191. For example, the asserted signal may be programmed signal, such as a “STOP” or “BRAKE” signal, which may be configured by the ML-ASIC 110 to stop further processing. In some examples, it is possible for the voltage regulator to send different signals based on the specific conditions being sensed in the sense line. In other examples, control signal 192 may be chosen from a set of pre-determined signals based on the sensed condition.

In response to the control signal 192, the ML-ASIC 110 may apply a throttle mask, as shown in block 215. As described herein, the throttle mask may be a series of steps taken by the ML-ASIC 110 to limit the processing being performed by processors 161, thereby limiting the power draw of the ML-ASIC.

FIG. 3 illustrates a flowchart 300 outlining the steps performed by a ML-ASIC 110 upon receiving a control signal 192. In this regard, upon receiving a control signal 192, synchronizer 130 may synchronize the control signal 192 which is external to ML-ASIC 110, to a core clock domain of ML-ASIC 110. An external analog signal, such as control signal 192, may arrive between or at the edge of two clock periods. The external analog signal needs to be synchronized to internal clock signals so that it can be translated into a digital signal for processing by the ML-ASIC 110. It may also be necessary to synchronize the external analog input signal to avoid an undefined output due to sampling the external analog input signal at an incorrect point relative to the input time window. By synchronizing the control signal 192, throttling of the ML-ASIC may occur on the scale of nanoseconds, and thus achieve high efficiency from an ML-ASIC while still protecting the ASIC from over current or overpower events.

At block 310, the synchronizer 130 may communicate the received control signal 192, or a signal generated by the synchronizer responsive to the received control signal 192, to chip manager 140.

At block 315, a throttle mask may be generated by the chip manager 140. The throttle mask may throttle or otherwise reduce the instruction flow of the processors 161 of the ML-ASIC. The generated throttle mask may maintain the ML-ASIC 110 within predefined parameters to ensure that power usage is within predefined threshold. For example, the throttle mask may ensure that the voltage received by ML-ASIC 110 is always within an upper and lower bound, and that a minimum amount of power is always supplied to the ML-ASIC by the voltage regulator 191.

Chip manager 140 may contain various algorithms to generate the throttle mask. In some examples, the throttle mask may be a series of steps, with each step defining a limit to the number of instructions processors 161 may execute in a particular period. In some examples, distinct masks could be generated for or applied to specific functional units of ML-ASIC 110, such as a mask of floating point units, integer units, or vector units, so that only some of the elements or components within ML-ASIC 110 would be disabled or throttled on a given mask cycle. In some examples, a throttle mask may be a throttle percentage, where the percent counts down cycles to stall the operation of the ML-ASIC without specifically generating an encoded mask. In some examples, a throttle mask may be a binary string, such as “11110011”, where a “1” would indicate that chip run normally, while the “0” indicates that the chip run at a “no operation” or idle. Each “1” or “0” within the throttle mask can indicate a duration equivalent to one clock cycle of the ML-ASIC, so that, a “1” would indicate one cycle where the chip is running while the “0” indicates one cycle where the chip is idling. In this example binary string as there are two “0”s and six “1”s, the string indicates an overall throttling rate of 2/8, or 25%. In other examples, the throttle mask can be a binary string which is a longer or shorter string length.

Chip manager 140 may also aggregate other digital signals which are internal to ML-ASIC 110. For example, ML-ASIC 110 may also generate a digital signal or indicator flag to indicate that the ASIC is running too hot. In one example, the ML-ASIC 110 may generate a digital signal, such as “PROC_HOT” to indicate that the processor in ML-ASIC 110 is running hot. As this signal is generated within ML-ASIC 110, the synchronizer 130 need not synchronize it to the clock of ML-ASIC 110. In some examples, the synchronizer may first receive all the signals before they are relayed to chip manager 140.

Multiple external signals may be received by multiple synchronizers. The synchronizes may then be relayed to chip manager 140. This enables synchronizer 130 to synchronize one or more external signals and allow throttle masks to be generated by the chip manager 140 which better correspond with the real-time conditions in which the ML-ASIC is operating. The throttle mask generated by the chip manager 140 may incorporate information from multiple sources or throttling requests, including hardware and software requests, such as control signal 192, an internal ASIC signal, such as PROC_HOT, or other software requests which may be received through data port 170. The other software requests may be different than external signal 196 and may indicate information which is separate or additional to the power control of the ML-ASIC. Chip manager may send the generated mask to a sequencer to apply the mask.

As shown at block 320, the generated throttle mask may be applied by the sequencer 180. In this regard, the sequencer 180 may regulate the instruction flow to the processors 161 in accordance with the throttle mask. As discussed, the generated throttle mask may include a series of steps which limit the instruction rate of the processors 161. The sequencer 180 may be any functional block or component which handles instruction flow to a processor, such as a CPU, a GPU, or a TPU.

As shown by arrow 325, steps 305 to 320 may be repeated. For example, if a particular condition to stop the throttling is not met, the throttling may continue or be re-calculated with another mask. In some examples, the particular condition may be a “release” signal received by the ML-ASIC 110 from the power and control unit 190. In other examples, it may be another external signal received by the ML-ASIC 110. By applying a throttling mask, the ML-ASIC 110 may continue operation without being shut down, while allowing the workflow being processed by the ML-ASIC to continue without overheating or damaging the ML-ASIC 110 or voltage regulator 191.

Referring to block 220 of FIG. 2, the ML-ASIC 110 may be restored to full function as full power is restored. This may occur in a step-wise manner after the throttle mask is de-asserted. For example, a percentage of the instruction rate may be restored in a series of steps. A pre-determined or dynamic waiting period may be set after each step, to ensure that the same over current condition is not recreated. For example, a series of 8, 16, or 32 steps may be taken with a fixed waiting period between the steps. During this time, the sense line may continue to be monitored by the voltage regulator 191 to confirm voltage, current, or power draw by the ML-ASIC 110 does not match a threshold condition. If a step does cause a threshold condition to be met, the instruction rate of the previous step may be reestablished until the power or current condition stabilizes. This process is illustrated further with reference to FIG. 6, described herein.

FIG. 4 illustrates a flowchart 400 outlining the return of an ML-ASIC to its previous state of operation without a throttle mask applied. At block 405, the voltage regulator 191 may evaluate the sense line for sense signals indicating that the threshold condition is no longer met. In some examples, block 405 will not take place until a pre-determined time period after the first step. If it is determined that the threshold condition causing the control signal 192 to be generated and sent from voltage regulator 191 is no longer existing, the voltage regulator may now stop the generation and/or sending of the control signal 192.

At block, 410, the control signal 192, may be de-asserted by the voltage regulator if a control signal is determined to not be required as shown in block 405. At this step, the control signal sent by the voltage regulator 191 may no longer be sent.

At block 415, a resume signal may be sent by voltage regulator 191 to ML-ASIC 110 to restore a previous operational state of the ML-ASIC. The resume signal may be a signal sent over data line 199. The resume signal may be similar to control signal 192. In some embodiments, not receiving a control signal 192 may be interpreted by sequencer 130 as being equivalent to a resume signal. In other examples, it is possible to send a different digital signal to inform the synchronizer that the condition causing the throttle mask to be generated no longer exists. Thus, the throttle mask may stop being applied, or gradually be de-asserted. In other examples, the signal may be de-asserted only when voltage regulator 191 detects that the power demanded by ML-ASIC 110 has sufficiently decreased. In some examples, control signal 192 may also be de-asserted or stopped being sent by the voltage regulator upon the voltage regulator determining that the particular criterion which caused the signal to be sent is no longer present for at least a pre-determined period of time. In some examples, control signal 192 may stopped being sent once voltage regulator detects that the ASIC current demand is below a threshold for at least a pre-determined period of time, such as 100 ns.

At block 420, power may be restored by the voltage regulator 191. This may occur in a step-wise manner after the throttle mask is de-asserted. In some examples, the throttle mask is de-asserted after the ML-ASIC 110 receives a resume signal. In other examples, after not receiving a control signal 192 for a pre-determined period of time, the synchronizer begins to take steps to de-assert the throttle mask. In some examples, the throttle mask may be de-asserted as a percentage of the instruction rate may be restored in a series of steps. A pre-determined or dynamic waiting period may be set after each step, to ensure that the same over current condition is not recreated. For example, a series of 8, 16, or 32 steps may be taken with a fixed waiting period between the steps. During this time, the power or current conditions may be monitored. If a step does cause a power or current condition to react unexpectedly, the instruction rate of the previous step may be reestablished until the power or current condition stabilizes.

The steps described in methods 200-400 may also be combined with or used in combination with other current control techniques. For example, other techniques may be implemented at block 320, such as gating clock cycles or adjust phased-locked loop (PLL) settings. Although these other techniques have advantages, they may create a mismatch in the timing of instructions being processed by the ASIC and/or downstream effects of introducing noise on the power rail. Yet, these techniques may be optionally combined with the application of a throttle mask.