FIG. 4.

The packetization unit 260 may receive the flag from the clock edge detection unit 250 and generate a clock packet (Clock Packet) indicating or characterizing the flag. Examples of such clock packet are described in detail below with reference to FIG. 6. The clock packet generated as described above may be transferred to the sub-FPGA (or to the receiver of the sub-FPGA) through a transmitter 290.

In addition, the generated clock packet may be transferred to a receiver 280 of the main FPGA through the transmitter 290 of the main FPGA. In this case, the transmitter 290 and the receiver 280 of the main FPGA may be connected directly or connected through any circuit (not illustrated). Under this configuration, the main FPGA may receive the generated clock packet by performing loopback on the generated clock packet. The clock packet transferred to the main FPGA through loopback as described above may be transferred to the packetization unit 260.

The packetization unit 260 may detect the transferred clock packet and transfer the detected clock packet or information on the detection of the clock packet to the clock gating unit 220. Under this configuration, the main FPGA processes the data based on the clock packet received through loopback, and the sub-FPGA processes the data based on the clock packet received from the main FPGA, so that the clocks used in each FPGA are the same as each other or take substantially the same packetization time and transfer delay time, thereby minimizing or eliminating a delay between the time of start of data processing in the main FPGA 200 and the time of start of data processing in the sub-FPGA.

The clock gating unit 220 may generate a gated user clock for the main FPGA based on the received clock packet and transfer the generated clock to each of the user design logic 230 and the packetization unit 260. The clock gating unit 220 may be any module or unit that can, after the clock packet is received, control (e.g., gate) the user clock received from the clock control unit 210, and provide the gated user clock to each of the user design logic 230 and the packetization unit 260. For example, the clock gating unit 220 may include any clock gating primitive cell widely used in the FPGA field. The information or signals exchanged between the user design logic 230 and the packetization unit 260 may be controlled by the gated user clock.

The user design logic 230 may generate a plurality of data items through at least some of the logics included in the user design logic 230 based on the gated user clock, and transfer the generated plurality of data items to the packetization unit 260. For example, the user design logic 230 may correspond to one or more of any blocks (BLK A, BLK B, BLK C, BLK DO, BLK D1, BLK D2, . . . , BLK D8) illustrated in FIG. 1.

The packetization unit 260 may receive a sampling clock from a communication unit 370 and generate one or more data packets including a plurality of data items provided from the user design logic 230. For example, one or more data packets generated as described above may be provided to the sub-FPGA through the transmitter 290 of the communication unit 270 using the sampling clock. At this time, the number of one or more data packets may be determined based on the size of the plurality of data items provided from the packetization unit 260 (e.g., data bit width, etc.) and the size of the data included in one data packet. For example, the number of one or more data packets may be determined by Equation 1 below.

⌈

(

W

TOT
DATA

W
DATA

)

⌉

<

Equation
⁢

1

>

That is, the number of one or more data packets may be the number of data packets calculated by dividing the size (W_TOT_DATA)) of all data items by the data size (W_DATA) included in one data packet and rounding up. For example, if the size of the total data item is 6500 bits and the size of data included in one data packet through the communication unit is 800 bits, the number of data packets is determined to be 9, and the entire data items may be divided and generated into 9 packets by a packetization unit 360. The packets generated as described above may be provided to another FPGA (e.g., sub-FPGA, etc.) using the sampling clock of the communication unit 270.

In another example, if a plurality of groups (not illustrated) including the main FPGA and one or more sub-FPGAs are included in the FPGA system, one or more of the generated data packets may be provided to the main FPGA of another group. The main FPGA and one or more sub-FPGAs included in each group may be controlled by the user clock provided by the corresponding main FPGA.

The communication unit 270 may send and receive data (Rdata, Sdata) to and from the sub-FPGA (or the communication unit of the sub-FPGA). Rdata may refer to received data, and Sdata may refer to sent data. On the other hand, if the FPGA system includes a plurality of groups including the main FPGA and one or more sub-FPGAs, the communication unit 270 may be configured to communicate with the main FPGA of another group.

The communication unit 270 may send and receive data (e.g., packets) to and from another FPGA in the FPGA system using the so-called serial communication protocol which continuously sends data one bit at a time using a communication channel or bus. For example, the communication unit 270 may use any serial communication protocol Intellectual Property (IP) used in the chip field, for example, Chip-to-Chip (C2D), Die-to-Die (D2D), Ethernet, Peripheral Component Interconnect express (PCIe) IP, etc., but aspects are not limited thereto.

FIG. 3 illustrates an architecture of a sub-FPGA 300. The sub-FPGA 300 illustrated in FIG. 3 may include a clock gating unit 320, a user design logic 330, a processing unit 340, and the communication unit 370. The processing unit 340 may include a clock edge detection unit 350 and the packetization unit 360, and the communication unit 370 may include a receiver 380 and a transmitter 390. The clock gating unit 320, the user design logic 330, the processing unit 340, the clock edge detection unit 350, the packetization unit 360, the communication unit 370, the receiver 380, and the transmitter 390 of the sub-FPGA 300 may correspond to the clock gating unit 220, the user design logic 230, the processing unit 240, the clock edge detection unit 250, the packetization unit 260, the communication unit 270, the receiver 280, and the transmitter 290 of the main FPGA 200 of FIG. 2, and the same configuration as at least part of the configuration provided by each part of the main FPGA 200 of FIG. 2 may be provided. In the configurations provided by each part of the sub-FPGA 300 in FIG. 3, the description of the configurations that overlap with the configurations provided by each part of the main FPGA 200 is omitted.

The user design logic 330 may implement at least part of the overall logic that the user intends to implement, and may include a logic different from the user design logic 230 of the main FPGA 200. For example, the user design logic 330 may correspond to one or more of any blocks (BLK A, BLK B, BLK C, BLK DO, BLK D1, BLK D2, . . . , BLK D8) illustrated in FIG. 1, and at least some of the blocks not implemented in the main FPGA 200 of FIG. 2 may be implemented. That is, some of the logics implemented in the entire chip or chip design (e.g., the chip 100 in FIG. 1) may be implemented in the user design logic 330.

The receiver 380 of the communication unit 370 may receive one or more packets from the main FPGA.

The receiver 380 may receive a clock packet from the main FPGA. The sampling clock may be transferred to the packetization unit 360, and the packetization unit 360 may provide a clock packet detected using the sampling clock, or information on the detection of the clock packet to the clock gating unit 320. The clock gating unit 320 may generate a gated user clock and provide the generated clock to the user design logic 330. The data packet of the packets received from the main FPGA may be transferred to the user design logic 330 through the packetization unit 360 and processed using the gated user clock.

In another example, the receiver 380 may receive one or more data packets from the main FPGA. The packetization unit 360 may receive the gated user clock generated by the clock gating unit 320. The packetization unit 360 may provide one or more received data packets to the user design logic 330 using the gated user clock. In addition, the packetization unit 360 may receive a plurality of data items that are processed and output by the user design logic 330.

In addition, the packetization unit 360 may receive a sampling clock from the communication unit 370 and generate one or more data packets including a plurality of received data items. One or more data packets generated as described above may be provided to another FPGA (e.g., the main FPGA 200) through the transmitter 390 using the sampling clock.

FIG. 3 illustrates the sub-FPGA 300 which, unlike the main FPGA, does not include the clock control unit 210. That is, the sub-FPGA 300 may not generate a user clock using its internal clock, in which case the user clock gated by the clock gating unit 320 may be generated upon receiving the clock packet detected by the packetization unit 360 or the information on the detection of the clock packet. At this time, the cycle of the detected clock packets and the cycle of the gated user clock may be the same or similar to each other.

In addition, while FIG. 3 illustrates that the clock edge detection unit 350 is included in the processing unit 340, because the sub-FPGA 300 does not generate or control the user clock using the internal clock, the clock edge detection unit 350 may not perform any function. Under this configuration, the clock edge detection unit 350 may be processed as 0. Accordingly, instead of implementing an architecture different from that of the main FPGA 200, the sub-FPGA 300 may be implemented by omitting the clock control unit 210 of the main FPGA 200, for example, by omitting the PLL IP. At this time, the internal clock of the sub-FPGA 300 may be deactivated. Conversely, the clock edge detection unit 350 may not be included in the processing unit 340.

The internal configuration of the main FPGA 200 and the sub-FPGA 300 illustrated in FIGS. 2 and 3 are merely examples, and in some examples, configurations other than the internal configuration illustrated herein may be additionally included, and some configurations may be omitted. In addition, although the internal configurations of the main FPGA 200 and the sub-FPGA 300 are separated by function and described in FIGS. 2 and 3, this does not necessarily mean that they are physically separated. Although the internal configurations of the main FPGA 200 and the sub-FPGA 300 are separately described above, this is to aid understanding of the disclosure, and aspects are not limited thereto.

FIG. 4 illustrates an example in which the positive edge of a user clock 420 is detected using a sampling clock 410. The sampling clock 410 may include pulses that repeat 0 and 1 at regular intervals. At this time, the pulse included in the sampling clock 410 may be repeated at a faster cycle than the pulse included in the user clock 420. For example, unlike the example illustrated in FIG. 4, the pulses included in the sampling clock 410 may repeat at a much faster cycle (e.g., at a cycle about 10 to 100 times faster) than the pulses included in the user clock 420.

The sampling clock 410 may correspond to the sampling clock (Sampling Clk) provided by the main FPGA of FIG. 2. As described in FIG. 2, for example, a sampling clock (Sampling Clk) may be generated in the communication unit 270 and provided to the clock edge detection unit 250 and the packetization unit 260 of the processing unit 240.

The positive edge of the user clock 420 may be detected according to the product of the pulse of the sampling clock 410 and the pulse of the user clock 420. For example, as illustrated, the time point at which the product of the pulse of the sampling clock 410 and the pulse of the user clock 420 is 1 rather than 0 may be determined to be the time point at which the positive edge of the user clock 420 occurs. Accordingly, if the cycle of the sampling clock 410 is shortened, the difference between the time when the positive edge of the user clock 420 actually occurs and the time when it is detected may decrease. As described in FIG. 2, the positive edge of the user clock 420 may be detected by the clock edge detection unit 250 of the processing unit 240.

In response to detecting the positive edge, a flag 430 may be generated. That is, since the flag 430 includes information associated with a plurality of time points of detecting the positive edge, the cycle information of the user clock 420 may be acquired from the flag 430. The information associated with the flag 430 may be included in the clock packet and sent to the sub-FPGA.

In addition, as illustrated, the width of high level of one pulse included in the flag 430 may correspond to the width of one entire pulse (including the high level and low level) included in the sampling clock.

FIG. 5 illustrates an example of a clock packet 520 and a gated user clock 530 generated in response to detecting a user clock 510. A clock packet may be generated that includes information indicating a flag generated in response to detecting the user clock 510. As described in FIG. 2, this clock packet may be generated by the packetization unit 260 of the processing unit 240. The clock packet generated as described above may be provided from the main FPGA 200 to one of one or more sub-FPGAs (e.g., the sub-FPGA 300, etc.). In addition, through loopback, the clock packet may be provided to the packetization unit 260 of the main FPGA through the communication unit of the main FPGA.

If the clock packet is received or detected in the packetization unit of one or more sub-FPGAs and/or the packetization unit of the main FPGA, a detected clock packet 520 may be generated. As illustrated, the detected clock packet 520 may be configured such that a pulse is generated when the clock packet is received or detected.

The detected clock packet 520 may be provided to the clock gating unit of the corresponding FPGA, and the clock gating unit may generate a gated user clock (Gated User Clk) for the corresponding FPGA. The gated user clock generated as described above may be provided to the user design logic and the packetization unit of the corresponding FPGA, respectively.

Since the gated user clock is generated as soon as the detected clock packet 520 is provided to the clock gating unit, as illustrated in FIG. 5, the time point of start of the positive edge of the pulse of the detected clock packet 520 may be the same as the time point of start of the positive edge of the gated user clock. Conversely, the positive edge of the pulse of the gated user clock may be slightly later than the time point of start of the positive edge of the pulse of the detected clock packet 520. Under this configuration, the repetition cycle of the pulse of the gated user clock may be the same as the repetition cycle of the pulse of the detected clock packet 520.

The pulse width of the gated user clock may be the same as the pulse width of the clock packet 520. According to another example, the pulse width of the gated user clock may be different from the pulse width of the clock packet 520.

FIG. 6 illustrates an example of a packet 600. As described above in FIGS. 2 and 3, packets may be generated by each of the packetization unit 260 of the main FPGA 200 and the packetization unit 360 of the sub-FPGA (e.g., the sub-FPGA 300). The packetization unit 260 of the main FPGA 200 may generate the clock packets and the data packets, and the packetization unit 360 of the sub-FPGA may generate only the data packets.

The packet 600 illustrated in FIG. 6 may be an example of a format in which the clock packet and the data packet may be generated. For example, the packet 600 may be formed in a format that includes an operation mode bit (OP) area 610 indicating an operation mode, a counter bit (CNT) area 620 that is a set of bits indicating a packet number, and data bit (Data) area 630 that is a set of bits representing at least some of a plurality of data items. A specific example of generating a plurality of data packets including the clock packet and the data packet will be described in detail below with reference to FIG. 7.

FIG. 7 illustrates an example of a plurality of packets transferred to the FPGA. The plurality of packets may be provided from one FPGA to another FPGA. For example, a plurality of packets generated in the main FPGA may be provided to the sub-FPGA, and a plurality of packets generated in the sub-FPGA may be provided to the main FPGA.

The plurality of packets may include the data packets and the clock packets. For example, as illustrated in FIG. 7, the plurality of packets may include a plurality of data packets including [P0] to [Pn] (where, n is a natural number) and a clock packet including [Ck].

In FIG. 7, an OP area 710 of the packet may indicate the operation mode of the packet, a CNT area 720 of the packet may indicate the packet number, and a Data area 730 of the packet may represent at least some of a plurality of data items of the packet.

Each of the plurality of data packets ([P0] to [Pn]) may include one bit indicating a mode for sending data to the OP area 710. For example, the OP area 710 may include “0” as a data transmission mode. In addition, for the data packet, the CNT area 620 may include a set of bits indicating the data packet number, and the Data area 630 may include a set of bits indicating information on one or more data items. As illustrated in FIG. 7, the data packets ([P0] to [Pn]) may include information indicating the packet numbers in the CNT area in the order in which the data packets ([P0] to [Pn]) are arranged. In addition, each of the data packets ([P0] to [Pn]) may include at least some of a plurality of data items. For example, as shown in the data packet [P0], one data item may be completely included as in the case of DO and D1, or one item may be divided and stored as in the case of D2.1.

For the clock packet [Ck], the OP area 710 may include a flag or one bit indicating information on the detection of the clock edge. For example, in the clock packet, the OP area 710 may include “1” as a clock edge mode. In addition, in the clock packet [Ck], a set of bits included in each of the CNT area 720 and the Data area 730 may be empty or set to 0. For illustrative purposes, in FIG. 7, the CNT area 720 and the Data area 730 of the clock packet [Ck] are marked “X”, but the mark “X” may mean that the set of bits included in each of the CNT area 720 and the Data area 730 is empty or set to 0. That is, the CNT area 720 and the Data area 730 may be used only for packets including data items. Additionally, if there are multiple clocks, the CNT area 720 and/or the Data area 730 may be used to further indicate which of the multiple clocks represents an edge.

The FPGA (e.g., the packetization unit of the FPGA) receiving the packet may remap the data of the received packet based on the packet number indicated by the set of bits in the CNT area 720 if the bit in the OP area 710 included in the packet indicates “0”. Conversely, if the bit in the OP area 710 is 1, a gated user clock for the corresponding FPGA may be generated based on the sampling clock of the receiving FPGA.

In FIG. 7, the clock packet [Ck] is placed below the plurality of data packets, so it may be understood that a plurality of data packets are generated and then the clock packet is generated, but aspects are not limited thereto, and the clock packet [Ck] may be placed in the middle of or above the plurality of data packets. That is, the clock packet may be generated in the middle of or before a plurality of data packets.

FIG. 8 illustrates an example of a timing diagram in which a packet is transferred. As shown in FIG. 8, information on the cycle of the user clock of the main FPGA may be obtained from a flag generated using a sampling clock (Sampling Clk).

In the multiple FPGA system including a plurality of FPGAs, in order to provide the same functionality as that provided by one FPGA, all data packets generated from the main FPGA must be transferred to any one of one or more sub-FPGAs during one cycle of the user clock of the main FPGA. In addition, all data packets generated in the sub-FPGA must be transferred to the main FPGA during one cycle of the user clock of the main FPGA. To this end, the main FPGA generates a clock packet every time a flag is generated and transfers the generated clock to another FPGA (e.g., sub-FPGA) or receives a generated clock from another FPGA, so that all data packets that need to be processed or transferred in one cycle of the user clock may be transferred to or received from the sub-FPGA before the next clock packet is generated and sent.

Under this configuration, the frequency of the user clock may be predetermined based on a frequency of the sampling clock of the main FPGA (and/or sub-FPGA), a number of the one or more data packets, and data transfer time between any two FPGAs of the main FPGA and the one or more sub-FPGAs. At this time, the frequencies of the sampling clock of the main FPGA and sampling clock of the sub-FPGA may be the same as each other. The sampling clock of the main FPGA and the sampling clock of the sub-FPGA may be synchronized with each other. According to another example, the sampling clock of the main FPGA and the sampling clock of the sub-FPGA may be asynchronized with each other. In addition, as described above, since the main FPGA sends and receives the clock packets through the loopback function, the clock data transfer time may include the time for transferring data from the main FPGA to the main FPGA.

The cycle of the user clock may be equal to or greater than the resultant number obtained by adding the product of multiplying the cycle of the sampling clock of the main FPGA by the resultant number obtained by adding the number of the clock packets and the number of the one or more data packets, and the data transmission delay and reception delay between any two FPGAs of the main FPGA and the one or more sub-FPGAs. That is, the cycle of the user clock may have the magnitude relationship of Equation 2 below.

1

FREQ

user
clk

≥

1

FREQ

Sampling
clk

⁢

(

⌈

(

W

TOT
DATA

W
DATA

)

⌉

+
1

)

+

(

2
×

T
prop

)

+

T
margin

<

Equation
⁢

2

>

where,

1

FREQ

user
clk

may represent the cycle of the user clock of the main FPGA,

1

FREQ

Sampling
clk

may represent the cycle of the sampling clock 410 of the main FPGA or sub-FPGA, and

⌈

(

W

TOT
DATA

W
DATA

)

⌉

+
1

may represent the resultant number obtained by adding the number (that is, 1) of clock packets to the number of one or more data packets generated to include a plurality of data items received from the user design logic. As explained in Equation 1 above,

⌈

(

W

TOT
DATA

W
DATA

)

⌉

may represent the number of data packets calculated by dividing the total data size (W_TOT_DATA)) of a plurality of data items by the data size (W_DATA) included in one data packet and rounding up. The data size may be defined as bit width. In addition, (2×T_prop) may be the sum of the transmission time (e.g., transmission delay time) and reception time (e.g., reception delay time) between any two FPGAs, and T_marginmay be a margin time. For example, information on the processing speed of multiple data items in the user design logic (for example, information on the processing speed of multiple data items may be defined by sampling clock frequency information, total data size, data bit width, etc.), information on the frequency of the sampling clock, and information on the transmission and reception time (or, transmission and reception delay time) between FPGAs may be acquired in advance through the specifications of Serial IP. Accordingly, an amount to increase or adjust the frequency or speed of the internal clock of the main FPGA provided by the clock control unit (e.g., PLL IP) of the FPGA to satisfy Equation 2 above using the acquired information may be set in advance. That is, the output frequency may be determined in advance by adjusting the frequency of the internal clock of the main FPGA by the clock control unit of the main FPGA. This predetermined frequency may correspond to any one of various frequencies provided from PPL IP.

In addition, the bandwidth (BW) that may be sent at one time through the communication unit may be determined by Equation 3 below.

BW
=

FREQ
sampling_clk