106. Examples of processing unit 104 and core 108 configurations include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), a field programmable gate array (FPGA), an accelerated processing unit (APU), and a digital signal processor (DSP). For example, the core 108 is a processing unit that reads and executes instructions (e.g., of a program), examples of which include to add, to move data, and to branch. Although one core 108 is depicted in the illustrated example, in variations, the device 102 includes more than one core 108 (e.g., the device 102 is a multi-core processor). The memory module 106 is implemented as a printed circuit board, on which, memory 116 (e.g., physical memory) and a processing-in-memory component 118 are disposed (e.g., physically and communicatively coupled using one or more sockets).

The processing unit 104 includes a cache system 110 having a plurality of cache levels 112, examples of which are illustrated as a level 1 cache 114(1) through a level “N” cache 114(N). The cache system 110 is configured in hardware (e.g., as an integrated circuit) communicatively disposed between the processing unit 104 and the memory 116 of the memory module 106. The cache system 110 is configurable as integral with the core 108 as part of the processing unit 104, as a dedicated hardware device as part of the processing unit 104, and so forth. Configuration of the cache levels 112 as hardware is utilized to take advantage of a variety of locality factors. Spatial locality is used to improve operation in situations in which data is requested that is stored physically close to data that is a subject of a previous request. Temporal locality is used to address scenarios in which data that has already been requested will be requested again.

In cache operations, a “hit” occurs to a cache level when data that is subject of a load operation is available via the cache level, and a “miss” occurs when the desired data is not available via the cache level. When employing multiple cache levels, requests proceed through successive cache levels 112 until the data is located (e.g., from an L1 cache, to an L2 cache, to an L3 cache, and so forth). The cache system 110 is configurable in a variety of ways (e.g., in hardware) to address a variety of processor unit 104 configurations, such as a central processing unit cache, graphics processing unit cache, parallel processor unit cache, digital signal processor cache, and so forth.

In one or more implementations, the memory module 106 is a circuit board (e.g., a printed circuit board), on which memory 116 (e.g., physical memory such as dynamic random-access memory) is mounted and includes a processing-in-memory component 118 (e.g., implemented in hardware using one or more integrated circuits). In some variations, one or more integrated circuits of the memory 116 are mounted on the circuit board of the memory module 106, and the memory module 106 includes one or more processing-in-memory components 118. Examples of the memory module 106 include, but are not limited to, a TransFlash memory module, a single in-line memory module (SIMM), and a dual in-line memory module (DIMM). In one or more implementations, the memory module 106 is a single integrated circuit device that incorporates the memory 116 and the processing-in-memory component 118 on a single chip. In some examples, the memory module 106 is formed using multiple chips that implement the memory 116 and the processing-in-memory component 118 that are vertically (“3D”) stacked together, are placed side-by-side on an interposer or substrate or are assembled via a combination of vertical stacking or side-by-side placement.

The memory 116 is a device or system that is used to store data, such as for immediate use in a device (e.g., by the core 108 and/or by the processing-in-memory component 118). In one or more implementations, the memory 116 corresponds to semiconductor memory where data is stored within memory cells on one or more integrated circuits. In at least one example, the memory 116 corresponds to or includes volatile memory, examples of which include random-access memory (RAM), dynamic random-access memory (DRAM), synchronous dynamic random-access memory (SDRAM), and static random-access memory (SRAM). Alternatively or in addition, the memory 116 corresponds to or includes non-volatile memory, examples of which include solid state disks (SSD), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), and electronically erasable programmable read-only memory (EEPROM).

The processing-in-memory component 118 is implemented in hardware (e.g., as an integrated circuit) configured to perform operations responsive to processing-in-memory instructions (e.g., received from the core 108). The processing-in-memory component 118 is representative of a processor with example processing capabilities ranging from relatively simple (e.g., an adding machine) to relatively complex (e.g., a CPU/GPU compute core). In an example, the processing-in-memory component 118 processes the instructions using data stored in the memory 116.

Processing-in-memory contrasts with standard computer architectures which obtain data from memory, communicate the data to a remote processing unit (e.g., the core 108), and process the data using the remote processing unit (e.g., using the core 108 rather than the processing-in-memory component 118). In various scenarios, the data produced by the remote processing unit as a result of processing the obtained data is written back to memory, which involves communicating the produced data from the remote processing unit to memory.

In terms of data communication pathways, the remote processing unit (e.g., the core 108) is further away from the memory 116 than the processing-in-memory component 118. As a result, these standard computer architectures suffer from increased data transfer latency, reduced data communication bandwidth, and increased data communication energy, particularly when the volume of data transferred between the memory and the remote processing unit is large, which can also decrease overall computer performance. Thus, the processing-in-memory component 118 enables increased computer performance while reducing data transfer energy as compared to standard computer architectures that implement remote processing hardware. Further, the processing-in-memory component 118 alleviates memory performance and energy bottlenecks by moving one or more memory-intensive computations closer to the memory 116.

Although the processing-in-memory component 118 is illustrated as being disposed within the memory module 106 (e.g., within a same integrated circuit or on a same printed circuit board), other examples are also contemplated. The processing-in-memory component 118, for instance, is also configurable to incorporate processing capability near memory modules so that tasks are also processed without costly round-trip transmission to host processors or other distant computing units. Access to the memory module 106 for the processing unit 104 is controlled through use of a memory controller 120.

The memory controller 120 is a digital circuit (e.g., implemented in hardware) that manages the flow of data to and from the memory 116 of the memory module 106. By way of example, the memory controller 120 includes logic to read and write to the memory 116. In one or more implementations, the memory controller 120 also includes logic to read and write to registers (e.g., temporary data storage) maintained by the processing-in-memory component 118, and to interface with the processing-in-memory component 118 (e.g., to provide instructions for processing by the processing-in-memory component 118).

The memory controller 120 also interfaces with the core 108. For instance, the memory controller 120 receives instructions from the core 108, via the cache coherence controller 122. The instructions involve accessing the memory 116 and/or the registers of the processing-in-memory component 118 and provide data to the core 108 (e.g., for processing by the core 108). In one or more implementations, the memory controller 120 is communicatively located between the core 108 and the memory module 106, and the memory controller 120 interfaces with the core 108, the memory module 106, and the cache coherence controller 122.

The core 108 is configured to initiate processing-in-memory (PIM) operations by the processing-in-memory component 118 using processing-in-memory requests that each include one or more processing-in-memory instructions. To ensure that the processing-in-memory component 118 operations on a valid version of data in the memory 116, a cache coherence controller 122 is employed. The cache coherence controller 122 is configurable in hardware (e.g., as one or more integrated circuits), support execution of instructions (e.g., by a microcontroller), and so forth. Validity of the data in the memory 116 refers to a scenario in which a version of data that is to be subject of a processing-in-memory operation is valid (e.g., is accurate in that the data has not been subsequently processed elsewhere) in the processing-in-memory component and the cache system 110. The cache coherence controller 122 is configured to query a cache directory 124 in what is referred to as a “cache directory lookup.” The cache directory 124 describes which memory addresses of the memory 116 are maintained in the cache system 110 and a status of data at those memory addresses. A cache directory lookup, for instance, is used to determine whether the data at the memory address is “clean” and unchanged with respect to the data for that memory address maintained in the memory 116 or “dirty” and is changed. Therefore, a cache directory lookup as performed by the cache coherence controller 122 is usable to determine “what” data is stored in the cache system 110 and a status of that data.

Ensuring data validity is performable by the cache coherence controller 122 as a flush in a “dirty” scenario in which the data is caused to be “flushed” from the cache system 110 for storage in the memory 116, which then makes the data stored in the memory 116 valid for a processing-in-memory operation by the processing-in-memory component 118. In a “clean” scenario, the cache coherence controller 122 generates a cache request to cause the cache system 110 to invalidate the clean data in the cache system 110 such that subsequent accesses to the memory address are performed using the memory 116 and not the cache system 110, and as such is also valid for use as part of a processing-in-memory operation in that the accesses are performed to the memory 116 and not the cache system 110. In this manner, subsequent memory requests (e.g., as part of corresponding PIM operations) retrieve the data from memory 116 (e.g., that has been processed as part of the PIM operation) instead of using stale data from the cache system 110.

In conventional systems, processing-in-memory requests issued by the core 108 for execution by the processing-in-memory component 118 bypass the cache system 110 (e.g., due to the processing-in-memory component 118 being unable to directly access data from the cache system 110) and are routed to the cache directory 124. Upon receipt of a processing-in-memory request, the cache directory 124 ensures validity of data elements (e.g., in memory 116) involved in executing the processing-in-memory request. However, such conventional systems inherently induce latency into a pipeline of executing processing-in-memory requests by stalling the processing-in-memory requests at the cache directory 124 while waiting for the cache coherence controller 122 to ensure validity (e.g., via flushing or invalidating) data elements involved in executing the processing-in-memory requests.

To avoid such delays and latency experienced by conventional system architectures, the device 102 is depicted as employing speculative invalidation logic 126. Although depicted in the illustrated example of FIG. 1 as being implemented at the cache system 110 to represent how invalidation and/or flushing of data elements involved in executing a processing-in-memory request is performed locally at the cache system 110, in some implementations the speculative invalidation logic 126 is employed at the core 108. The speculative invalidation logic 126 represents functionality of the device 102 to route a processing-in-memory request from the core 108 to the cache system 110 before forwarding to the cache directory 124.

In implementations, the speculative invalidation logic 126 causes the cache system 110 to query a cache tag table to determine whether one or more data elements involved in executing a processing-in-memory request are stored in one or more cache levels 112 of the cache system 110. In response to detecting that one or more data elements involved in executing the processing-in-memory request are stored in the cache system 110, the cache system 110 generates local invalidations/flushes for the one or more data elements, such that the one or more data elements are “valid” in memory 116 upon arrival of the processing-in-memory request at the cache directory 124.

To further optimize system performance, the speculative invalidation logic 126 further represents functionality (e.g., of the core 108) to configure a processing-in-memory request with a speculation bit that serves as a binary indicator as to whether the processing-in-memory request was first forwarded to the cache system 110 for local invalidation/flushing of data elements. As a specific example, a processing-in-memory request having a speculation bit equal to one “1” indicates that the processing-in-memory request was routed to the cache system 110 for local invalidation/flushing of data before routing to the cache directory 124. Alternatively, a processing-in-memory request having a speculation bit equal to zero “0” indicates that the processing-in-memory request was not first routed to the cache system 110 and that the cache directory 124 thus needs to ensure validity of data elements that may still be cached at the cache system 110. Consequently, in response to receiving a processing-in-memory request having a speculation bit indicating that local invalidation/flushing was performed at the cache system 110, the cache coherence controller 122 avoids generating local invalidations (e.g., avoids transmitting requests to the cache system 110 for the cache system 110 to flush data elements involved in the processing-in-memory request) before dispatching the processing-in-memory request (e.g., to the memory controller 120), and thus avoids the delay associated with stalling the processing-in-memory request at the cache directory 124, as necessitated by conventional systems.

In this way, the techniques described herein improve operational efficiency of the processing-in-memory component 118 to operate on valid data, reduce latency, reduce power consumption, and reduce bottlenecks caused by conventional techniques that stall processing-in-memory requests as a result of the cache directory lookup.

FIG. 2 is a block diagram of a non-limiting example system 200 showing operation of a cache system 110 of FIG. 1 in greater detail as performing local invalidation/flushing of data elements for a processing-in-memory request before the request arrives at a cache directory based on speculative invalidation logic. In the illustrated example of FIG. 2, the memory 116 is implemented using a plurality of memory banks, examples of which are illustrated as memory bank 202(1), memory bank 202(2), . . . , memory bank 202(M), where M represents any suitable integer (e.g., in one implementations M=16). Likewise, the processing-in-memory component 118 is illustrated as including respective processing-in-memory (PIM) compute units, examples of which are illustrated as PIM compute unit 204(1), PIM compute unit 204(2), . . . , PIM compute unit 204(X), where X represents any suitable integer.

The PIM compute units 204(1)-204(X) are configurable with a variety of processing capabilities in hardware (e.g., using one or more integrated circuits) ranging from relatively simple (e.g., an adding machine) to relatively complex (e.g., a CPU/GPU compute core). The processing unit 104 is configured to offload memory bound computations to the one or more in-memory processors of the processing-in-memory component 118. To do so, the core 108 generates PIM instructions and transmits the PIM instructions, via the memory controller 120, to the memory module 106. The processing-in-memory component 118 receives the PIM instructions and processes the instructions as PIM operations using the PIM compute units 204(1)-204(X) and data stored in the memory 116.

Processing-in-memory using PIM compute units 204(1)-204(X) contrasts with standard computer architectures which obtain data from memory 116, communicate the data to the core 108 of the processing unit 104, and process the data using the core 108 rather than the processing-in-memory component 118. In various scenarios, the data produced by the core 108 as a result of processing the obtained data is written back to the memory 116, which involves communicating the produced data over the pathway from the core 108 to the memory 116. In terms of data communication pathways, the core 108 is further away from the memory 116 than the processing-in-memory component 118. As a result, conventional (e.g., non-processing-in-memory enabled) computer architectures suffer from increased data transfer latency, reduced data communication bandwidth, and increased data communication energy, particularly when the volume of data transferred between the memory 116 and the processing unit 104 is large, which can also decrease overall device 102 performance.

In one or more implementations, the core 108 retrieves data from the memory 116 and stores the data in one or more caches levels 112 of a cache system 110 associated with the core 108. By way of example, the cache levels 112 of the core 108 include a level 1 cache 114(1), . . . , through a level “N” cache 114(N). In implementations in which the core 108 is a multi-core processor, for instance, the cache levels 112 include a level 3 cache is shared by each of the multiple cores 108. Thus, in these implementations, each core 108 of the multi-core processor stores data in a level 1 cache, a level 2 cache, and a shared level 3 cache. In terms of data communication pathways, the cache levels 112 are closer to the core 108 than the memory 116, and as such, data stored in the cache system 110 is accessible in less amount of time by the core 108 than an amount of time taken to access the data stored in the memory 116. It is to be appreciated that the one or more cores 108 of the processing unit 104 are configurable to include cache subsystems with differing numbers of caches and different hierarchical structures without departing from the spirit or scope of the described techniques.

In various examples, the core 108 retrieves a cache line in response to receiving an instruction to access a particular memory address. As used herein, a “cache line” is a unit of data transfer between the memory 116 and the cache system 110. In one example, the cache line is four bytes and the core 108 retrieves a contiguous four-byte block of data from the memory 116 that includes the data of the particular memory address. Further, the core 108 stores the four-byte block of data as a cache line in the cache system 110. If the core 108 receives a subsequent instruction to access a memory address that is a part of the cache line, the core 108 accesses the data of the memory address from the cache system 110, rather than the memory 116.

In one or more examples, the cache system 110 and the memory 116 store different versions of a corresponding cache line. For instance, the core 108 modifies a cache line that is stored in a cache level 112 of the cache system 110, and as such, the data corresponding to the cache line that is stored in the memory 116 is stale and therefore not valid for operations. Accordingly, the cache coherence controller 122 is employed to enforce cache coherence among the cache system 110 and the memory 116. Notably, cache coherence is the uniformity of data that is storable in multiple different memory resources in a system (e.g., the cache system 110 and the memory 116). As part of enforcing cache coherence, the cache coherence controller 122 employs a cache directory 124, which includes cache directory entries 206 for cache lines that are stored in one or more of the cache levels 112 of the cache system 110. In response to cache lines being added to the cache system 110, the cache coherence controller 122 creates cache directory entries 206 in the cache directory 124 that includes a range of memory addresses corresponding to the cache line.

In one example, the cache coherence controller 122 receives a memory request 208 to access data of a memory address from the memory 116. The memory request 208 is representative of a processing-in-memory request, a non-processing-in-memory request, or combinations thereof. In response to receiving the memory request 208, the cache coherence controller 122 performs a cache directory lookup 210 in the cache directory 124. The cache directory lookup 210 is used to determine whether one of the cache directory entries 206 represents a cache line that includes the memory address referenced by the memory request 208.

Based on a result of the cache directory lookup 210, the cache coherence controller 122 performs a corresponding coherence protocol. By way of example, a cache directory 124 miss occurs when the cache directory entries do not include the memory address (e.g., address range) specified by the memory request 208, and therefore the data as maintained for that memory address is valid in memory 116. In contrast, a cache directory 124 hit occurs when there is a cache directory entry 206 included in the cache directory 124 having an address range that includes the memory address of the memory request 208, and therefore the data as maintained for that memory address is not valid in memory 116.

If the cache directory lookup 210 indicates that data specified for a memory address in the memory request 208 is not valid for execution of a processing-in-memory instruction, the cache coherence controller 122 transmits a cache request 212 to the cache system 110. This causes the cache system 110 to transmit a cache response 214 such that the data in the memory 116 is subsequently valid for computation by the processing-in-memory component. This is performable, for instance, by leveraging the cache request to cause the cache system to write the data 216 back to memory (e.g., “flush” the data) and/or invalidate 218 the data in the cache system 110, which is acknowledged by the cache response 214. The cache coherence controller 122 then releases the memory request 208 to the processing-in-memory component 118 for processing (e.g., via a memory controller 120 for performance as part of a processing-in-memory operation).

Thus, the determination of whether a hit “has” or “has not” occurred serves as a basis to determine whether data in the memory specified by the memory request 208 is valid (e.g., for execution of a PIM operation by a respective processing-in-memory component 118). As previously described above, scenarios in which the data is not valid involve additional latency, either to cause the data to be flushed from the cache system 110 to the memory 116 or set to invalidate the data in the cache system 110. This challenge is increased when confronted with parallel execution scenarios.

As illustrated in FIG. 2, the memory 116 includes a plurality of memory banks 202(1)-202(M) that are organized into one or more memory arrays (e.g., grids), which include rows and columns such that data is stored in individual cells of the memory arrays. The memory banks 202(1)-202(M) are representative of a grouping of banks in relation to which the processing-in-memory component 118 is configured to perform various in-memory processing operations. By way of example, PIM compute units 204(1)-204(X) of the processing-in-memory component 118 are included as part of a memory channel along with respective ones of the memory banks 202(1)-202(M). The processing-in-memory component 118, through use of the PIM compute units 204(1)-204(X), performs in-memory processing operations on the data that is stored in the memory banks 202(1)-202(M). In the illustrated example, a plurality of memory channels includes a respective one of the PIM compute units 204(1)-204(X) and a respective one of the memory banks 202(1)-202(M), and a cache coherence controller 122 to enforce cache coherence among the memory banks 202(1)-202(M) within the memory channel and the cache levels 112 of the cache system 110.

The processing-in-memory component 118 is configurable to operate on each of the memory banks 202(1)-202(M) in parallel to execute a single PIM instruction. In the illustrated example, the processing-in-memory component 118 is configured to operate on M memory banks 202(1)-202(M) and receives a PIM instruction to read data from a particular row and column address. To execute the instruction, the processing-in-memory component 118 reads the data of the particular row and column address from each of the memory banks 202(1)-202(M) in parallel.

Therefore, a single PIM instruction of a conventionally configured system triggers a plurality of cache directory lookups 210 in the cache directory 124 (e.g., one lookup for memory addresses in each one of the multiple memory banks 202(1)-202(M)). This is performed to ensure that the requested data stored in each of the memory banks 202(1)-202(M) is “valid” as being coherent with other instances of the requested data stored in the cache system 110.

Continuing with the previous example in which the processing-in-memory component 118 is configured to operate on M memory banks 202(1)-202(M), a standard cache coherence controller 122 performs M cache directory 124 cache directory lookups 210 for a single PIM instruction. A cache directory lookup 210, however, is a computationally expensive task, particularly when a significant number (e.g., sixteen) of cache directory lookups are performed sequentially. Moreover, this significant number of cache directory lookups when performed even for a single PIM instruction often create a bottleneck in the cache directory 124 that affects both PIM workloads and non-PIM workloads. These problems are exacerbated by the notion that PIM instructions are often issued together as a series of sequential PIM instructions, rather than interspersed with non-PIM instructions. Due to this, the number of cache directory lookups to be performed multiplies with each sequential PIM instruction, thereby worsening the bottleneck and increasing cache directory lookup 210 latency and latency of operations that depend on these lookups (e.g., for processing by the processing-in-memory component 118).

To overcome these drawbacks of conventional techniques, speculative invalidation logic 126 is employed to route a processing-in-memory request (e.g., a memory request 208 to be executed by the processing-in-memory component 118) to the cache system 110 before routing to the cache coherence controller 122. The speculative invalidation logic 126 sets a value of a speculation bit 220 for a processing-in-memory request to indicate whether the processing-in-memory request is routed to the cache system 110 before arriving at the cache directory 124. For instance, in implementations where the speculation bit 220 specifies a value of one, the speculation bit 220 informs the cache directory 124 that the PIM request was first routed to the cache system 110 for local invalidation/flushing of data elements involved in executing the PIM request.

In implementations where the speculative invalidation logic 126 routes a processing-in-memory request to the cache system 110, the speculative invalidation logic 126 sets a value of the speculation bit 220 to a predefined value (e.g., one) and causes the cache system 110 to check whether a cache system 110 tag table 222 indicates that one or more data elements involved in executing the PIM request are stored in the cache system 110. As described herein, the tag table 222 represents a portion of memory in the cache system 110 (e.g., a portion of memory in the level “N” cache 114(N) that describes what data elements are stored in the cache levels 112. For instance, the tag table 222 represents metadata describing an address of data as stored in memory 116 and an indication of whether a requested data element is stored in the cache system 110 (e.g., is a cache hit) or not (e.g., is a cache miss). Cache tags in a lower-level cache (e.g., cache 114(N)), as described herein, refer to indications of whether a given data element or set of data elements is associated with a cache hit in the lower-level cache as well as other cache levels 112 in the hierarchy of the cache system 110.

Upon routing of a PIM request (e.g., by the speculative invalidation logic 126) to the cache system 110, the cache system 110 consults tag table 222 to identify whether there is a tag indicating a cache hit for one or more data elements involved in executing the PIM request (e.g., data 216), before the PIM request is sent to the cache coherence controller 122. In response to identifying a cache hit based on the tag table 222, an invalidation 218 for the requested data is generated locally at the cache system, which causes the requested data element(s) to be invalidated or flushed to memory 116 before the PIM request arrives at the cache directory 124 (e.g., before forwarding to the memory controller 120 for scheduling execution of the PIM request by the processing-in-memory component 118).

In addition to causing local invalidation/flushing (e.g., triggering invalidation/flushing at the cache system 110) of data elements involved in executing a PIM request having a speculation bit 220 set to a predefined value (e.g., one), in some implementations the speculative invalidation logic 126 causes the cache system 110 to invalidate/flush one or more additional data elements that are not involved in executing the PIM request. For instance, in such implementations where additional data elements are invalidated/flushed locally at the cache system 110, the speculative invalidation logic 126 identifies that there is a high probability that one or more subsequent PIM requests involve the additional data elements, due to memory layout configurations. A single PIM operation, for instance, is configurable to execute a same operation (i.e., instruction) at a row and column specified in a memory request for each of a plurality of memory banks in memory. To reduce an overhead of opening new rows in memory, PIM operations are configurable to leverage spatial locality and issue a subsequent memory request to the same rows. Techniques used to perform memory address interleaving, for instance, are usable to place multiple blocks of adjacent data in a same row in memory. Typically, PIM operations are employed in real-world scenarios for large data structures that cover entire rows in the memory banks, and potentially several rows. Therefore, if a PIM operation operates using a particular row and column in a memory bank, a subsequent PIM operation is likely to involve access to other columns in the same row activations.

In this manner, in certain situations the speculative invalidation logic 126 proactively causes the cache system 110 to flush and/or invalidate additional data elements to improve performance of the device 102 (e.g., by avoiding the latency associated with stalling one or more subsequent PIM requests at the cache directory 124 while waiting for the cache coherence controller 122 to ensure validity of the additional data elements in memory 116). Thus, when the speculative invalidation logic 126 is triggered for a PIM request, the speculative invalidation logic 126 causes the cache system 110 to proactively flush and/or invalidate data elements involved in executing the PIM request. Upon receiving a PIM request with a speculation bit 220 indicating that the cache system 110 locally performed invalidation/flushing of data elements involved in executing the PIM request, the cache directory 124 avoids performing cache directory lookups 210 for data elements involved in executing the PIM request, thus avoiding computational delays and energy consumption requirements of conventional systems.

In the following discussion, operation of the example system depicted in FIG. 3 is described in parallel to the procedures depicted in FIGS. 4 and 5. FIG. 4 is a block diagram of a non-limiting example procedure 400 describing performance of a speculative cache invalidation for a processing-in-memory request. FIG. 5 is a block diagram of a non-limiting example procedure 500 describing performance of a cache directory dispatching a processing-in-memory request based on an indication of whether a speculative cache invalidation is performed for the processing-in-memory request.

FIG. 3 is a block diagram of a non-limiting example system 300 illustrated using a first stage 302, a second stage 304, a third stage 306, and a fourth stage 308 as showing receipt of a memory request specifying a memory address, querying a tag table maintained at a cache system for the memory address, querying the tag table for an additional memory address based on the memory address, and forwarding the memory request to a cache directory after flushing and/or invalidating data at the cache system.

At the first stage 302, a processing-in-memory request 310 that includes at least one memory address 312 is received (block 402; block 502) and processed by a device component implementing the speculative invalidation logic 126. The processing-in-memory request 310, for instance, originates through software execution (e.g., an application, operating system) at the core 108 and is received at the cache system 110 via a communicative coupling (e.g., a bus structure, a network-on-chip, or the like).

At the second stage 304, a lookup is performed in a cache tag table 222 based on the memory address 312 (block 404). The speculative invalidation logic 126 causes the cache system 110 to consult a tag table 222 maintained in a level “N” cache 114(N) of the cache system 110 to determine whether the at least one memory address 312 for data element(s) involved in executing the processing-in-memory request 310 is stored in the cache system 110.

At the third stage 306, an additional memory address 314 is selected based on the at least one memory address 312. In an example, the speculative invalidation logic 126 identifies the additional memory address 314 based on the at least one memory address 312. Spatial locality is used in one example to select the additional memory address 314 as being stored physically close to the memory address 312. Temporal locality is used in another example to address scenarios in which an additional memory address 314 which was already requested and a likelihood that the additional memory address 314 will be requested again (e.g., as associated with the memory address 312).

As depicted in the third stage 306, the speculative invalidation logic 126 causes the cache system 110 to locally generate a cache query 316 for data elements associated with the at least one memory address 312, and optionally the additional memory address