220 handles the request by communication with the off-die memory 275, by the atomic and merge operations 250, or by both. Data read by the off-die memory controller 220 can be stored in the cache 210 by the cache controller 215 for later use.

The atomic and merge operations 250 are coupled to receive (as input) the output of the off-die memory controller 220, and to provide output to the cache 210, the network communication interface 225, or directly to the chiplet network 280. The memory hazard clear (reset) unit 260, write merge unit 265 and the built-in (e.g., predetermined) atomic operations unit 265 can each be implemented as state machines with other combinational logic circuitry (such as adders, shifters, comparators, AND gates, OR gates, XOR gates, or any suitable combination thereof) or other logic circuitry. These components can also include one or more registers or buffers to store operand or other data. The PAU 270 can be implemented as one or more processor cores or control circuitry, and various state machines with other combinational logic circuitry or other logic circuitry, and can also include one or more registers, buffers, or memories to store addresses, executable instructions, operand and other data, or can be implemented as a processor.

The write merge unit 255 receives read data and request data, and merges the request data and read data to create a single unit having the read data and the source address to be used in the response or return data packet). The write merge unit 255 provides the merged data to the write port of the cache 210 (or, equivalently, to the cache controller 215 to write to the cache 210). Optionally, the write merge unit 255 provides the merged data to the network communication interface 225 to encode and prepare a response or return data packet for transmission on the chiplet network 280.

When the request data is for a built-in atomic operation, the built-in atomic operations unit 265 receives the request and read data, either from the write merge unit 265 or directly from the off-die memory controller 220. The atomic operation is performed, and using the write merge unit 255, the resulting data is written to the cache 210, or provided to the network communication interface 225 to encode and prepare a response or return data packet for transmission on the chiplet network 280.

The built-in atomic operations unit 265 handles predefined atomic operations such as fetch-and-increment or compare-and-swap. In an example, these operations perform a simple read-modify-write operation to a single memory location of 32-bytes or less in size. Atomic memory operations are initiated from a request packet transmitted over the chiplet network 280. The request packet has a physical address, atomic operator type, operand size, and optionally up to 32-bytes of data. The atomic operation performs the read-modify-write to a cache memory line of the cache 210, filling the cache memory if necessary. The atomic operator response can be a simple completion response, or a response with up to 32-bytes of data. Example atomic memory operators include fetch-and-AND, fetch-and-OR, fetch-and-XOR, fetch-and-add, fetch-and-subtract, fetch-and-increment, fetch-and-decrement, fetch-and-minimum, fetch-and-maximum, fetch-and-swap, and compare-and-swap. In various example embodiments, 32-bit and 64-bit operations are supported, along with operations on 16 or 32 bytes of data. Methods disclosed herein are also compatible with hardware supporting larger or smaller operations and more or less data.

Built-in atomic operations can also involve requests for a “standard” atomic operation on the requested data, such as a comparatively simple, single cycle, integer atomics (e.g., fetch-and-increment or compare-and-swap), which will occur with the same throughput as a regular memory read or write operation not involving an atomic operation. For these operations, the cache controller 215 generally reserves a cache line in the cache 210 by setting a hazard bit (in hardware), so that the cache line cannot be read by another process while it is in transition. The data is obtained from either the off-die memory 275 or the cache 210, and is provided to the built-in atomic operation unit 265 to perform the requested atomic operation. Following the atomic operation, in addition to providing the resulting data to the data packet encoder 240 to encode outgoing data packets for transmission on the chiplet network 280, the built-in atomic operation unit 265 provides the resulting data to the write merge unit 255, which will also write the resulting data to the cache circuit 210. Following the writing of the resulting data to the cache 210, any corresponding hazard bit which was set will be cleared by the memory hazard clear unit 260.

The PAU 270 enables high performance (high throughput and low latency) for programmable atomic operations (also referred to as “custom atomic operations”), comparable to the performance of built-in atomic operations. Rather than executing multiple memory accesses, in response to an atomic operation request designating a programmable atomic operation and a memory address, circuitry in the memory controller chiplet 205 transfers the atomic operation request to PAU 270 and sets a hazard bit stored in a memory hazard register corresponding to the memory address of the memory line used in the atomic operation, to ensure that no other operation (read, write, or atomic) is performed on that memory line. The hazard bit is then cleared upon completion of the atomic operation. Additional direct data paths provided for the PAU 270 to execute the programmable atomic operations allow for additional write operations without any limitations imposed by the bandwidth of the communication networks and without increasing any congestion of the communication networks.

The PAU 270 include a RISC-V instruction set architecture (RISC-V ISA) based multi-threaded processor having one or more processor cores, and may further have an extended instruction set for executing programmable atomic operations. When provided with the extended instruction set for executing programmable atomic operations, the PAU 270 can be embodied as one or more hybrid threading processors. In some example embodiments, the PAU 270 provides barrel-style, round-robin instantaneous thread switching to maintain a high instruction-per-clock rate.

Programmable atomic operations can be performed by the PAU 270 that involve requests for a programmable atomic operation on the requested data. A user can prepare programming code to provide such programmable atomic operations. For example, the programmable atomic operations can be comparatively simple, multi-cycle operations such as floating-point addition, or comparatively complex, multi-instruction operations such as a Bloom filter insert. The programmable atomic operations can be the same as or different than the predetermined atomic operations, insofar as they are defined by the user rather than a system vendor. For these operations, the cache controller 215 can reserve a cache line in the cache 210, by setting a hazard bit (in hardware), so that cache line cannot be read by another process while it is in transition. The data is obtained from either the off-die memory 275 or the cache 210, and is provided to the PAU 270 to perform the requested programmable atomic operation. Following the atomic operation, the PAU 270 will provide the resulting data to the network communication interface 225 to directly encode outgoing data packets having the resulting data for transmission on the chiplet network 280. In addition, the PAU 270 will provide the resulting data to the cache controller 215, which will also write the resulting data to the cache 210. Following the writing of the resulting data to the cache 210, any corresponding hazard bit which was set will be cleared by the cache control circuit 215.

The approach taken for programmable atomic operations is to provide multiple, generic, custom atomic request types that can be sent through the chiplet network 280 to the memory controller chiplet 205 from an originating source such as a processor or other system component. The cache controller 215 and off-die memory controller 220 identify the request as a custom atomic request and forward the request to the PAU 270. In a representative embodiment, the PAU 270: (1) is a programmable processing element capable of efficiently performing a user defined atomic operation; (2) can perform load and stores to memory, arithmetic and logical operations and control flow decisions; and (3) can leverage the RISC-V ISA with a set of new, specialized instructions to facilitate interacting with the cache and off-die memory controllers 215, 220 to atomically perform the user-defined operation. It should be noted that the RISC-V ISA contains a full set of instructions that support high level language operators and data types. The PAU 270 can leverage the RISC-V ISA, but generally supports a more limited set of instructions and limited register file size to reduce the die size of the unit when included within the memory controller chiplet 205.

As mentioned above, any hazard bit which is set will be cleared by the memory hazard clear unit 260. Prior to the writing of the read data to the cache 210, a set hazard bit for the reserved cache line is to be cleared by the memory hazard clear unit 260. Accordingly, when the request and read data is received by the write merge unit 255, a reset or clear signal can be transmitted by the memory hazard clear unit 260 to the cache 210 to reset the set memory hazard bit for the reserved cache line. Also, resetting this hazard bit will also release a pending read or write request involving the designated (or reserved) cache line, providing the pending read or write request to an inbound request multiplexer for selection and processing.

A chiplet system such as the chiplet system 110 of FIG. 1A may include many chiplets and the chiplets may be arranged in a tightly packed matrix to create a higher performance functional block in a minimum footprint. To facilitate communication among the chiplets, the chiplets include multiple input-output (I/O) channels. An I/O channel can be implemented in any of multiple protocols and structures (for example, an AIB channel) to provide a communication channel between chiplets. In an example, the I/O channel could be used by a computing element of one chiplet (for example, a memory controller chiplet) to access memory of a second chiplet (which may store, for example, data in one or more memory die implementing one or more memory storage technologies). Memory requests would flow from the computing element of the first chiplet to the second chiplet and responses to the memory requests would flow in the reverse direction from the second chiplet back to the first chiplet. The I/O channels are arranged in columns of channels at the periphery of the chiplets, and extend to contacts (such as pads) configured to couple to conductive interconnects on an interposer to form a die-to-die physical layer interface. A set of hardware registers is included in the chiplets to support configuring by software of one or both of the channels and the columns of channels of the chiplets for interoperability.

The chiplets may be configured using an initialization process. The initialization process can be performed using hardware or software. In some examples, the initialization is performed after system reset using a dedicated interface such as an SPI. An SPI bus includes four signals: a serial clock (SCLK), a principal output subordinate input (POSI), a principal input subordinate output (PISO), and a subordinate select (SS). During each clock cycle of SCLK data may transmitted to shift data into registers of the chiplets using the POSI and PISO lines.

In some examples, one of the chiplets may serve as an SPI principal chiplet or SPI governing chiplet. The SPI principal chiplet may be a chiplet that includes a PCIe interface. The SPI principal chiplet connects to the other chiplets through the SPI bus to write the hardware registers to configure the columns of I/O channels. Once the SPI SCLK is up and running and the associated reset or resets are de-asserted, the SPI principal chiplet has access to the registers of the other chiplets.

In some examples, the configuration data that is written into the hardware registers of the chiplets may come from a host processor of a device separate from the chiplet system. The software running on the host processor may read and write memory-mapped input-output (MMIO) space that maps to the PCIe of the chiplet system. The host processor may provide the configuration data using a PCIe transaction that maps to the MMIO address space to the interface of the SPI principal chiplet. In some examples, the interface to the SPI principal chiplet is an external interface and the configuration data can be written directly to the SPI principal chiplet.

FIG. 3 is a block diagram illustrating an example of portions of a chiplet-based system 310 including a central chiplet 312 bordered by two periphery chiplets 314A, 314B. Each of the chiplets may use multiple channels of I/O. Several channels may be “bonded” together or combined to form a wider interface or wider channel. Additionally, several channels can be “ganged” or grouped together into multiple columns of I/O channels. The central chiplet 312 shows a column on each side of the chiplet with channels labeled (AUX, CH0, CH1 . . . CHn). Each of the periphery chiplets 314A, 314B shows a column with the same designated channels as the columns of the central chiplet. The references in the description to the “central” chiplet and “peripheral” chiplets are meant to provide clarity in the description of where I/O channels of the interface are re-programmed; and contain no other implication as to configuration, function, or physical placement of such chiplets in a chiplet-based system. In a chiplet-based system, any—or all—chiplets may be configured with registers and/or other circuitry providing programmable configuration of I/O Channels on the chiplet. For example, in accordance with the discussion below, the central chiplet may be implemented to provide programmability of I/O channels on one or more sides; while a peripheral chiplet in communication with the central chiplet may be implemented to provide programmability at an interface on an additional side of the central chiplet.

FIG. 3 shows that the periphery chiplets may need to be rotated with respect to the central chiplet or chiplets to allow use of all four sides of the central chiplet. One or both of the columns and the channels of a peripheral chiplet may need to be reversed from the original orientation to align to the channels of the central chiplet 312. In FIG. 3, it can be seen that the periphery chiplet 314A on the left-hand side of the diagram should be reversed from the original orientation so that the channels match the channels of the central chiplet 312. When an interconnect becomes reversed, it would be desirable for there to be a means for software to correctly re-orient the interface in hardware. FIG. 3 is a simplified block diagram and an actual chiplet system may include many chiplets. As noted above, a memory controller chiplet is one example of a chiplet that may have I/O channel interfaces on multiple (or all) sides (as depicted for chiplets 312, 412A, and 412B). In the example of a memory controller chiplet, some example configurations will have an industry-standard memory interface on one or more sides; and a programmable channel I/O interface, as described above, on one or more sides.

FIG. 4 is a block diagram illustrating another example embodiment of a chiplet system 410. The example shows two central chiplets 412A, 412B and two periphery chiplets 414A and 414B. Each of the central and periphery chiplets have two columns of channels on one side of the chiplet. In the diagram, it can be seen that the two channels of the columns of chiplet 414B on the right need to be reversed for interconnection to the columns on the right-hand edge of central chiplet 310B.

A single channel (e.g., CHn) of I/O of a column forms the smallest interface component or unit and a single channel includes multiple receive (RX) and transmit (TX) lines carrying RX and TX signals. A single column includes multiple channels of I/O. Within a column, multiple channels can be bonded together to form a wider channel interface. For example, CH0 and CH1 of a column can be bonded together to form a channel twice as wide as one of the channels.

To facilitate reversing the columns and bonding multiple unit-sized channels together into wider channels, the chiplets include a set of hardware-based, software-accessible registers. The capabilities of the channel hardware can be communicated to software using the registers and the channels and columns can be configured by the software using the registers. The registers can be included in the chiplet Column and Channel Swap Logic circuitry 316 shown in FIG. 3. In the column and channel swap logic circuitry of a chiplet, there is a set of capability and configuration registers associated with the columns of I/O channels, and a set of capability and configuration registers associated with the channels of the columns. FIG. 5-8 illustrate some examples of capability and configuration registers.

FIG. 5 is a diagram of an example of a column capability register 570. Each column of a chiplet may include a column capability register 570. The column capability register includes a column reversal capable (RevC) field. The RevC field can written with a value indicating that the order of the channels of the column is reversible (e.g., because the column is connected to swap logic circuitry that supports channel reversal). The RevC field is then readable by software when configuring the columns of the chiplet, and the RevC field indicates that column reversal is available.

FIG. 6 is a diagram of an example of a column configuration register 672. Each column of a chiplet may include a column configuration register 672. The column configuration register 672 includes a column reversal enable field (RevEn). Enabling the column reversal enable field enables the reversal of the channels of the column. In the example of FIG. 3, enabling the column reversal enable field of the column of periphery chiplet 314A reverses the order of the channels in the column to match the channels of the left-hand column of the central chiplet 312.

Returning to FIG. 5, the column capability register 570 may also include a multi-column reverse capable (Multi-Column RevC) field. The multi-column reverse capable field indicates if the column is connected to swap logic circuitry to support multi-column reversal of a number of columns. In some examples, the multi-column reverse capable field is a bit vector. If a bit of a bit vector is set, this indicates that the column supports multi-column reversal across a number of columns equal with the number of columns equal to the bit position set plus one. For example, if the bit of bit position two of the multi-column reverse capable field is set, this indicates that the column supports reversal across three columns.

Returning to FIG. 6, the column configuration register 672 includes a number of columns to reverse enable (No. Col. RevEn) field to enable reversal of multiple columns of a chiplet. In the example of FIG. 4, the number of columns to reverse enable can be used to reverse the order of the channels of the column in right-hand periphery chiplet 414B to match the channels of the columns of the right-hand edge of central chiplet 412B. Note that the columns of the right-hand edge of central chiplet 412B includes interface A (IPA) and interface B (IPB). Both columns of the right-hand periphery chiplet 414B need to be reversed together for the interfaces to align correctly. Reversing the order of the columns individually rather than both together would not result in the correct interface alignment.

Returning to FIG. 6, in some examples writing a value N to the number of columns to reverse enable reverses N columns of the chiplet starting with the column corresponding to the column configuration register and the N−1 columns above the column, where N is a positive integer. For example, writing the value 2 into the number of columns to reverse enable field of column 1 of periphery chiplet 414B reverses both the column 2 and column 1 together. This can be useful when the channels of more than one column are grouped together.

The approach to channel reversal can be extended to reversal of less than a full column of channels. The reverse capability registers and reversal enable registers can be extended to include fields in the registers to selectively reverse a subset of the channels within a column instead of the whole column. This may be useful to reverse one or more portions of one column that extends over multiple chiplets.

As explained previously herein, the logic circuitry of a chiplet includes capability and configuration registers associated with the channels of the columns. These capability and configuration registers are used to bond multiple unit-sized channels together into wider channels.

FIG. 7 is a diagram of an example of a programmable channel capability register 774. In certain examples, each channel of a chiplet includes a channel capability register 774. The channel capability register 774 includes a channel bonding capable field. The channel bonding capable field may be a bit vector. Setting a bit of the bit vector indicates that the channel supports bonding across a number of channels equal to the bit position of the bit vector plus one.

FIG. 8 is a diagram of an example of a programmable channel configuration register 876. In certain examples, each channel of a chiplet includes a channel configuration register 876. The channel configuration register 876 includes a number of channels to bond field that indicates the number of channels to bond together into a channel interface. In certain examples, writing a value X into the number of channels to bond field bonds the channel corresponding to the channel configuration register 876 and the X−1 channels designated above it into one channel interface of X unit-sized channels, where X is a positive integer greater than one. In certain examples, the number of channels bonded may extend beyond the channels of one column. In this way, channels of channels of multiple columns can be bonded into the wider channel interface.

FIG. 9 is a flow diagram of an example of a method 900 performed by a circuit or system to configure I/O channels of chiplets of a chiplet-based system. The I/O channels of a chiplet are arranged as columns of the chiplet. At 905, the order of channels of a column of a first chiplet of the system is set. The order of the channels may be set as a default configuration. A second chiplet of the system is programmed according to the arrangement of the channels of the first chiplet. At 910, a column capability register of a column of the second chiplet is read to determine if the order of the channels of the column is reversible to change the order of channels of a column of the second chiplet to match the order of channels of the column of the first chiplet. At 915, a column configuration register of the column of the second chiplet is written to reverse the order of the channels of the column to match the order of the channels of the column of the first chiplet.

In some examples, the order of channels of the column of the second chiplet is reversed to match the order of channels of the column of the first chiplet. In some examples, multiple channels of the first chiplet are programmed to bond the multiple channels together as a wider channel interface. Multiple channels of the second chiplet are bonded together to match the wider channel interface of the first chiplet.

FIG. 10 illustrates a block diagram of an example machine 1000 upon which any one or more of the techniques (e.g., methodologies) discussed herein may be implemented. Examples, as described herein, may include, or may operate by, logic or a number of components, or mechanisms in the machine 1000. Circuitry (e.g., processing circuitry) is a collection of circuits implemented in tangible entities of the machine 1000 that include hardware (e.g., simple circuits, gates, logic, etc.). Circuitry membership may be flexible over time. Circuitries include members that may, alone or in combination, perform specified operations when operating. In an example, hardware of the circuitry may be immutably designed to carry out a specific operation (e.g., hardwired). In an example, the hardware of the circuitry may include variably connected physical components (e.g., execution units, transistors, simple circuits, etc.) including a machine readable medium physically modified (e.g., magnetically, electrically, moveable placement of invariant massed particles, etc.) to encode instructions of the specific operation. In connecting the physical components, the underlying electrical properties of a hardware constituent are changed, for example, from an insulator to a conductor or vice versa. The instructions enable embedded hardware (e.g., the execution units or a loading mechanism) to create members of the circuitry in hardware via the variable connections to carry out portions of the specific operation when in operation. Accordingly, in an example, the machine-readable medium elements are part of the circuitry or are communicatively coupled to the other components of the circuitry when the device is operating. In an example, any of the physical components may be used in more than one member of more than one circuitry. For example, under operation, execution units may be used in a first circuit of a first circuitry at one point in time and reused by a second circuit in the first circuitry, or by a third circuit in a second circuitry at a different time. Additional examples of these components with respect to the machine 1000 follow.

In alternative embodiments, the machine 1000 may operate as a standalone device or may be connected (e.g., networked) to other machines. In a networked deployment, the machine 1000 may operate in the capacity of a server machine, a client machine, or both in server-client network environments. In an example, the machine 1000 may act as a peer machine in peer-to-peer (P2P) (or other distributed) network environment. The machine 1000 may be a personal computer (PC), a tablet PC, a set-top box (STB), a personal digital assistant (PDA), a mobile telephone, a web appliance, a network router, switch or bridge, or any machine capable of executing instructions (sequential or otherwise) that specify actions to be taken by that machine. Further, while only a single machine is illustrated, the term “machine” shall also be taken to include any collection of machines that individually or jointly execute a set (or multiple sets) of instructions to perform any one or more of the methodologies discussed herein, such as cloud computing, software as a service (Saas), other computer cluster configurations.

The machine (e.g., computer system) 1000 may include a hardware processor 1002 (e.g., a central processing unit (CPU), a graphics processing unit (GPU), a hardware processor core, or any combination thereof), a main memory 1004, a static memory (e.g., memory or storage for firmware, microcode, a basic-input-output (BIOS), unified extensible firmware interface (UEFI), etc.) 1006, and mass storage 1008 (e.g., hard drives, tape drives, flash storage, or other block devices) some or all of which may communicate with each other via an interlink (e.g., bus) 1030. The machine 1000 may further include a display unit 1010, an alphanumeric input device 1012 (e.g., a keyboard), and a user interface (UI) navigation device 1014 (e.g., a mouse). In an example, the display unit 1010, input device 1012 and UI navigation device 1014 may be a touch screen display. The machine 1000 may additionally include a storage device (e.g., drive unit) 1008, a signal generation device 1018 (e.g., a speaker), a network interface device 1020, and one or more sensors 1016, such as a global positioning system (GPS) sensor, compass, accelerometer, or other sensor. The machine 1000 may include an output controller 1028, such as a serial (e.g., universal serial bus (USB), parallel, or other wired or wireless (e.g., infrared (IR), near field communication (NFC), etc.) connection to communicate or control one or more peripheral devices (e.g., a printer, card reader, etc.).

Registers of the processor 1002, the main memory 1004, the static memory 1006, or the mass storage 1008 may be, or include, a machine readable medium 1022 on which is stored one or more sets of data structures or instructions 1024 (e.g., software) embodying or utilized by any one or more of the techniques or functions described herein. The instructions 1024 may also reside, completely or at least partially, within any of registers of the processor 1002, the main memory 1004, the static memory 1006, or the mass storage 1008 during execution thereof by the machine 1000. In an example, one or any combination of the hardware processor 1002, the main memory 1004, the static memory 1006, or the mass storage 1008 may constitute the machine-readable media 1022. While the machine-readable medium 1022 is illustrated as a single medium, the term “machine readable medium” may include a single medium or multiple media (e.g., a centralized or distributed database, and/or associated caches and servers) configured to store the one or more instructions 1024.

The term “machine-readable medium” may include any medium that is capable of storing, encoding, or carrying instructions for execution by the machine