FIG. 6 illustrates, by way of example and not limitation, an example of operations performed using the distributed hybrid buffer in the memory system when another write command to the same memory location is received. In the illustrated example, both the data and parity are being buffered. In a first step in the illustrated example, the buffer 613 is storing data D17_1 and parity P17, and the RAID controller 612 receives from a host a write command 629 to the RAID controller 612 to write data to the data memory location in stripe 17 block 1 (D17_1). As both of the data (D17_1) and parity (P17) are in the buffer, the memory controller is capable of performing the write command without any back-end access to the memory device(s) via the DRAM channels 618. In a second step in response to the received Write D17_1 command, the RAID controller 612 may perform a cached read 630 of the previously stored data (D17_1) within the buffer(s) 613. In a third step, the RAID controller 612 may perform a cached read 631 of the previous parity block for stripe 17 (P17) within the buffer(s) 613. In a fourth step, the RAID controller 612 may compute a parity update 632 (e.g., XORs new data for D17_1 with old data for D17_1). In a fifth step, the RAID controller 612 may write new data (D17_1) 633 back to the buffer(s) 613 and in a sixth step may write a new parity block for stripe 17 634 back to the buffer(s) 613. Accesses to the SRAM buffer are faster and more power efficient than accesses to the DRAM device(s). Therefore, the memory system is capable of performing the requested write in a power efficient and fast manner using the distributed hybrid buffer.

FIG. 7 illustrates, by way of example and not limitation, an example of operations performed using the distributed hybrid buffer in the memory system when another write command to a different data block in the same stripe is received. In the illustrated example, the buffer 713 is storing data D17_1 and parity P17. In a first step, the RAID controller 712 may receive from a host a write command 735 to the RAID controller 712 to write data to the data memory location in stripe 17 block 2 (D17_2). The RAID controller 712 may determine that the data and/or parity should be buffered and send a corresponding BUFFER FLAG. In the illustrated example, the RAID controller 712 determines that both data and parity are to be buffered. Thus, the requested write is in the same stripe, but a different data block in the stripe. In a second step, as the data (D17_2) is not in the buffer 713, the RAID controller 712 may use a third memory channel (e.g., MEMC2) within the DRAM channels 718 to perform a back end read 736 of previous data D17_2. However, as the parity (P17) is in the buffer 713, the RAID controller 712 may, as illustrated in a third step, read the previous parity (P17) 737 from the SRAM buffer 713 rather than having to perform a back end read to the memory device(s). In fourth step, the memory controller may compute the parity update for stripe #17 738, may write the new data D17_2 739 to another location in the buffer as illustrated in a fifth step, and write the new or updated parity 740 to the buffer 713 as illustrated in a sixth step. Thus, the memory controller is capable of performing the write command to a new data block without any back-end accesses for reading/writing parity.

FIG. 8 illustrates, by way of example and not limitation, an eviction process performed to write buffered data or buffered parity through the DRAM memory channels 818 back to the memory and clear buffer space. The buffer(s) may become full such that a new read or new write may trigger an eviction. A read or write 841 may be received by the RAID controller 812. The RAID controller 812 may determine that the read or write should include writes to the buffer(s) 813 and send a corresponding BUFFER FLAG, but the buffer may be full (e.g., buffer locations 842 associated with channel 0 (MEMC0) may be full). An eviction procedure 843 may be performed according to an eviction policy in preparation to write at least one of new data or new parity to the buffer. The eviction procedure may include writing buffered information from at least one of the buffer locations to the memory and clearing the buffered information from the at least one of the buffer locations. For example, at least one of the buffer locations for channel 0 may be written back to the DRAM memory devices using channel 0, and then channel 0 may be used to buffer data associated with the newly received read or write 841. Victim(s) may be chosen from the buffered data and/or the buffered parity depending on the policy. The eviction policy may be changed at boot up of the memory controller. By way of example and not limitation, the eviction policy may favor keeping buffered parity in the buffer over buffered data. For example, it may be determined that it is more likely that the buffered parity will be used than a particular data block within a stripe. The eviction policy may provide a weighted eviction for maintaining a range of percentages of parity blocks and a range of percentages of date blocks. By way of example and not limitation, the policy may require that 60 to 80 percent of the available space in the buffer is used to store parity and 20 to 40 percent of the available space in the buffer is used to store data. By way of a nonlimiting example, the system may try to maintain 75% parity blocks and 25% user blocks based on some heuristic. The weighted eviction policy may be used to bias another policy such as LRU. The eviction may be based on a least recently used (LRU) buffer location or based on received temperature metadata indicative of a prediction of memory location use. Other nonlimiting examples of an eviction policy may be a not most recently used (NMRU) buffer location which may provide a faster approximation of LRU, a random eviction such as may be used if the system does not have space to maintain line usage metadata.

FIG. 9 illustrates, by way of example and not limitation, a flow diagram representative of operations performed in FIG. 5. The operations may be implemented using a memory and a memory controller configured to communicate with the memory over two or more memory channels where the memory controller includes a buffer having a plurality of buffer locations distributed across the two or more memory channels. At 944, the memory controller may receive a first write command (e.g., Write D17_1) for writing current data to a memory location within a memory that includes stripes. Each of the stripes includes memory blocks and a parity block. The memory location corresponds to one of the memory blocks in one of the of stripes. At 945, it is determined whether to store data for the memory location in the buffer and/or parity for a stripe containing the memory location in the buffer. At 946, previous data may be read at the memory location (e.g., Read D17_1). A previous parity which depends on the previous data may be read (e.g., Read Parity P17) from the parity block in the stripe containing the memory location, as illustrated at 947. As illustrated at 948, parity for the stripe containing the memory location may be updated to provide a current parity (e.g., P17) that depends on the current data. At 949, the current data may be written using a first memory channel to a first buffer location in the buffer (e.g., Cache D17_1) when it is determined to store the data in the buffer. At 950, the current parity may be written using a second memory channel into a second buffer location in the buffer when it is determined to store the parity in the buffer.

FIG. 10 illustrates, by way of example and not limitation, a flow diagram representative of operations performed in FIG. 7. The operations may be implemented using the memory and the memory controller used in FIGS. 5 and 10, and the memory location is a first memory location and the current parity in the second buffer location is a first parity. The operations may include receiving at the memory controller a second write command 1051 for writing second current data to a second memory location within the memory (e.g., Write D17_2). The second memory location may correspond to a second one of the memory blocks, and the stripe containing the first memory location also includes the second memory location. The operations may include determining, for the second memory location, whether to store in the buffer at least one of: data for the second memory location; or parity for a stripe containing the first and second memory locations 1052. Second data may be read (e.g., Read D17_2) 1053 at the second memory location, and the first parity (e.g., P17) may be read 1054 from the second buffer location (see 950 in FIG. 9 where the second buffer location is storing the current value for the parity P17). The operations may include, as illustrated at 1055, updating parity (e.g., parity P17) for the stripe containing both the first memory location and the second memory location to provide a second parity. The operations may include writing the second data (D17_2) using a third memory channel to a third buffer location 1056, and writing the second parity (e.g., P17) using the second memory channel into the second buffer location. The second parity replaces the first parity in the second buffer location 1057.

FIG. 11 illustrates, by way of example and not limitation, a flow diagram for performing buffered or unbuffered operations in response to receiving a write command. The memory controller may receive a write command to write to a memory location (e.g., DX_Y, which represents data block Y in the stripe X) 1158. The RAID controller may determine whether to buffer data and/or parity and provide a corresponding BUFFER FLAG to the buffer. The buffer may then check to see if a BUFFER FLAG is present 1159. If the flag is not present the memory controller may perform unbuffered operations 1160. If the flag is present the memory controller may perform buffered operations 1161 to buffer data and/or parity. The unbuffered operations, as generally illustrated in FIG. 4, may include reading the previous data DX_Y from memory 1162 in the DRAM memory device(s), reading previous parity PX from memory for stripe X 1163 from memory 1162 in the DRAM memory device(s), determining new parity PX for stripe X 1164, writing new data DX_Y to the memory 1165 in the DRAM memory device(s), and writing new parity PX to the memory 1166 in the DRAM memory device(s). Thus, the illustrated unbuffered operation may include four back-end operations. If CRC information is also read and updated, the unbuffered operation may include six back-end requests. However, if flag(s) are present at 1159, the memory may implement a process to buffer the data (DX_Y) as illustrated at 1167 and FIG. 12 and/or a process to buffer parity (PX_Y) as illustrated at 1168 and FIG. 13.

FIG. 12 illustrates, by way of example and not limitation, a flow diagram for performing buffered data operations (see FIG. 11 at 1167). The operations may check to see if the data DX_Y is already in the buffer 1267. If it is in the buffer, then the operations may include reading the current data DX_Y from the current buffer location 1268 and writing new data DX_Y to the current buffer location 1269. That is, the same buffer location may be updated with the new data. However, the new data may be entered into another buffer location and identified as the current data. If at 1267 it is determined that the data DX_Y is not in the buffer, then the operations may include performing a back-end request to read data DX_Y from the memory stripe X block Y 1270. A determination may be made whether the buffer is full 1271. If the buffer is full, then the memory controller may be configured to implement an eviction process to free up space in the buffer. The eviction process may include back-end request(s) to write the buffered information (e.g., evicted data and/or parity pursuant to the eviction policy) into the DRAM and then clearing from the buffer. If the buffer is not full, then the new data DX_Y may be written to open the buffer location 1273.

FIG. 13 illustrates, by way of example and not limitation, a flow diagram for performing buffered parity operations (see FIG. 11 at 1168). The operations may check to see if the parity PX is already in the buffer 1374. If it is in the buffer, then the operations may include reading the current parity PX from the current buffer location 1375, determining new parity PX for stripe X 1376, and writing new data PX to the current buffer location 1377. That is, the same buffer location may be updated with the new parity. However, the new parity may be entered into another buffer location and identified as the current parity. If at 1374 it is determined that the parity is not in the buffer, then the operations may include performing a back-end request to read parity PX 1378 for the memory stripe X from the memory device(s). At 1379, a new parity may be determined. A determination may be made whether the buffer is full 1380. If the buffer is full, then the memory controller may be configured to implement an eviction process to free up space in the buffer 1381. The eviction process may include back-end request(s) to write the buffered information (e.g., evicted data and/or parity pursuant to the eviction policy) into the DRAM and then clear from the buffer. If the buffer is not full, then the new parity PX may be written to open the buffer location 1382.

FIG. 14 illustrates, by way of example and not limitation, an example of operations that may be performed in response to a host read using a distributed hybrid buffer in the memory system. The RAID controller 1412 may be configured to determine that a particular read command should have its corresponding data and/or parity buffered, and to send the buffer(s) 1413 a buffer flag (“BUFFER FLAG”) when it determines that the data and/or parity should be buffered. For example, the RAID controller may receive a signal indicating that there is an anticipated write to the same location that is being read. The signal may come from a hot page detector, which may be a circuit (hardware, firmware, software or combinations thereof) that is configured to identify or predict a page temperature. For example, certain applications may tend to write to certain addresses shortly after they have been read, or there may be a pattern of read/writes that may be identified for a specific application or memory processes. Similarly, there may be pattern of read/writes to data blocks within the same stripe. The buffer may respond to this information by storing data on a read request. For example, if the memory controller receives a read request and the hot page detector predicts that this data will be updated soon, the read data may be buffered. Furthermore, the memory controller may fetch the parity from memory and buffer it in anticipation of updating this data soon. Effectively the read request may act as a kind of prefetch hint for later accesses.