540-1, a second SAS IOM 540-2, a first SAS expander 516-1 and a second SAS expander 516-2. The storage system comprises a storage sled 510, in which the interconnect module 518 comprising the identification and routing logic 519 is installed. The interconnect module 518 is connected to the first SAS IOM 540-1, the second SAS IOM 540-2, the first SAS expander 516-1 and the second SAS expander 516-2.

Identifying the plurality of hardware components installed in the chassis may comprise determining a hardware configuration of each of the plurality of hardware components. In illustrative embodiments, the determined hardware configuration for a given one of the hardware components may be maintained in a trusted platform module (TPM). TPMs use multiple physical security mechanisms to ensure that hardware components are tamper-resistant and that no malicious software is detected.

Step 204 comprises determining a failure of one or more of the first input-output module, the second input-output module, the first expander and the second expander, and step 206 comprises dynamically routing data through respective ones of the plurality of hardware components which remain operational in response to the determined failure. The interconnect module 518 is connected between the respective ones of the plurality of hardware components which remain operational and the process includes dynamically routing the data through the interconnect module 518.

As explained in more detail in connection with FIGS. 5 and 7, a first signal conditioner (e.g., SAS re-driver 517-1) is connected between the interconnect module (e.g., interconnect module 518) and the first expander (e.g., SAS expander 516-1), and a second signal conditioner (e.g., SAS re-driver 517-2) is connected between the interconnect module (e.g., interconnect module 518) and the second expander (e.g., SAS expander 516-2). The first expander (e.g., SAS expander 516-1) is also connected to the second expander (e.g., SAS expander 516-2) and the first IOM (e.g., SAS IOM 540-1) is connected to the second IOM (e.g., SAS IOM 540-2).

As explained in more detail in connection with FIGS. 5 and 7, interconnect module (e.g., interconnect module 518) is connected to the first expander (e.g., SAS expander 516-1) by a first plurality of SAS communication paths (e.g., X4 SAS) and the interconnect module (e.g., interconnect module 518) is connected to the second expander (e.g., SAS expander 516-2) by a second plurality of SAS communication paths (e.g., X4 SAS). The interconnect module (e.g., interconnect module 518) is connected to the first IOM (e.g., SAS IOM 540-1) by a first plurality of SAS communication paths (e.g., X4 SAS) and the interconnect module (e.g., interconnect module 518) is connected to the second IOM (e.g., SAS IOM 540-2) by a second plurality of SAS communication paths (e.g., X4 SAS). The interconnect module (e.g., interconnect module 518) is further connected to the first IOM (e.g., SAS IOM 540-1) and to the second IOM (e.g., SAS IOM 540-2) through a fabric connector (e.g., Fab-C connector 520), wherein the fabric connector is plugged into a power distribution board (e.g., power distribution board (PDB) 530).

It is realized herein that due to the hardware feasibility of accommodating a large number of hard disk drives (HDDs) or other storage devices, as well as the availability of centralized storage management functionality for multiple servers, various end-users utilize a “modular” server architecture and “blade” servers for applications which require a large amount of storage space. A modular server may include an enclosure or chassis, one or more blade servers, and one or more storage servers providing a storage pool that is utilized by the one or more blade servers. The chassis includes multiple slots in which the blade servers and storage servers may be installed. The chassis also includes management software (e.g., which may run as part of a chassis controller or chassis management console) providing various functionality for managing the blade servers and storage servers which are installed in the chassis. The chassis may also include one or more power supplies for powering the blade servers and storage servers installed in the chassis, cooling equipment (e.g., one or more fans) for cooling the blade servers and storage servers installed in the chassis, networking equipment (e.g., one or more network interface controllers, host adapters, etc.) which may be utilized by the blade servers and storage servers installed in the chassis, etc. In a modular server, the installed blade servers are physical servers configured to work independently, while the storage servers providing the storage pool may comprise a set of storage devices arranged in a Just a Bunch of Drives (JBOD) configuration.

By way of example only, FIG. 3 shows a storage architecture 300 of a modular server, which includes compute sleds 301-1 and 301-2 (collectively, compute sleds 301), a storage pool 303 including storage sleds 305-1 and 305-2 (collectively, storage sleds 305), a power distribution board (PDB) 307, serial attached Small Computer System Interface (SCSI) (SAS) controllers 309-1 and 309-2 (collectively, SAS controllers 309), and a JBOD controller 311. The compute sleds 301-1 and 301-2 are each connected to each of the SAS controllers 309-1 and 309-2, via the PDB 307. Similarly, the storage sleds 305-1 and 305-2 are each connected to each of the SAS controllers 309-1 and 309-2, via the PDB 307. The SAS controllers 309-1 and 309-2 are connected to one another, as well as the JBOD controller 311. The SAS controllers 309 enable users to assign HDDs or other storage devices (e.g., of storage servers installed in the storage sleds 305 providing the storage pool 303) to different blade servers (e.g., installed in the computed sleds 301). Storage devices will be accessible only to the respective blade servers to which they are assigned. The storage devices will be accessed only by the particular blade servers assigned thereto through an internal storage controller (e.g., a Dell PowerEdge Redundant Array of Independent Disks (RAID) Controller (PERC) which is part of a corresponding one of the compute sleds 301).

FIG. 4 shows an example of a modular server architecture 400, including a chassis 401 with a set of eight slots 403-1 through 403-8 (collectively, slots 403). A set of six blade servers 405-1 through 405-6 (collectively, blade servers 405) are installed in the slots 403-1 through 403-6 of the chassis 401, and two storage servers 407-1 and 407-2 (collectively, storage servers 407) are installed in the slots 403-7 and 403-8, respectively. The storage servers 407 may comprise Dell Insight storage pools (e.g., JBOD or other storage pools). In the FIG. 4 example, each of the storage servers 407 accommodates up to 16 HDDs or other storage devices, which are assigned to different ones of the blade servers 405 as illustrated (e.g., with six storage devices being assigned to each of the blade servers 405-1 through 405-4, and with four storage devices being assigned to the blade server 405-5 and the blade server 405-6). It should be appreciated, however, that the particular numbers of slots, blade servers, storage servers, storage devices, and the assignment of storage devices to blade servers shown in FIG. 4 is presented by way of non-limiting example only.

Referring to the information processing system 500 in FIG. 5, a chassis 501 includes a plurality of compute sleds 502-1, 502-2, . . . , 502-N (collectively “compute sleds 502”), and at least one storage sled 510. The compute sleds 502 respectively comprise storage drives 505-1, 505-2, . . . , 505-N (collectively “storage drives 505”), SAS host bus adaptors (HBAs) 504-1, 504-2, . . . , 504-N (collectively “SAS HBAs 504”) and PERCs 503-1, 503-2, . . . , 503-N (collectively “PERCs 503”). The storage sled 510 includes storage drives 515-1, . . . , 515-S (collectively “storage drives 515”), SAS expanders 516-1 and 516-2 (collectively “SAS expanders 516”), SAS re-drivers 517-1 and 517-2 (collectively “SAS re-drivers 517”), interconnect module 518 comprising the identification and routing logic 519 and a Fab-C connector 520.

The chassis 501 further includes the PDB 530, SAS IOMs 540-1 and 540-2 (“collectively “SAS IOMs 540”) and a chassis controller 550. The SAS IOMs 540-1 and 540-2 respectively comprise SAS expander 541-1 and SAS expander 541-2 (collectively “SAS expanders 541”) and fabric management processor (FMP) 543-1 and FMP 543-2 (collectively “FMPs 543”). The SAS IOM 540-1 further includes external connections (CONNs) 542-1, 542-2, 542-3 and 542-4, and the SAS IOM 540-2 further includes external connections (CONNs) 542-5, 542-6, 542-7 and 542-8. The external connections 542-1, 542-2, 542-3, 542-4, 542-5, 542-6, 542-7 and 542-8 are collectively referred to as “external connections 542.”

In illustrative embodiments, the SAS IOMs 540 and the storage sled 510 together create data paths to be used in connection with transmission and backing up of data. The SAS IOMs 540, which are examples of SAS controllers, function as managed SAS switches providing SAS attachments for end devices to associated compute servers (e.g., blade servers 110). SAS zoning is used to associate drive bays/slots within disk enclosures to the compute sleds 502. Communication from SAS IOM 540-1 to SAS IOM 540-2 and vice versa is implemented with a Gigabit Ethernet (GbE) network link, using, for example, inter-integrated circuit (I²C) protocol.

In illustrative embodiments, the storage sled 510 comprises 16 storage drives 515 such as, for example, HDDs and/or solid-state drives (SSDs). As described in more detail in connection with FIG. 7, the SAS expanders 516 collectively provide dual paths to each of the storage drives 515. In some embodiments, the SAS expanders 516 are hot-swappable, meaning that they can be removed or added to the storage sled 510 while the power remains on and without shutting down or rebooting a corresponding computer or server. The storage sled 510 provides, via the SAS expanders 516 and SAS re-drivers 517 dual X4 SAS links to a next-generation modular (NGM) SAS fabric (e.g., to Fab-C connector 520 via the interconnect module 518). The storage sled 510 is configured to provide 12 Gb SAS support.

In illustrative embodiments, the SAS IOMs 540 support SAS 3.0 connectors capable of 12 Gb/s data transmission speeds, and are backwards compatible to 6 Gb/s speeds. Each of the SAS IOMs 540 includes eight X4 internal SAS connections for connections to compute sleds (e.g., compute sleds 502) and/or storage sleds (e.g., storage sled 510). Each of the SAS IOMs 540 further includes four X4 external SAS connections (e.g., external connections 542) for connection to external SAS JBODs. In some embodiments, each of the SAS IOMs 540 includes two X4 external SAS connectors for chassis stacking. The FMPs 543 are management processors for SAS topology and JBOD management. The chassis controller 550 may include IOM common circuits for interfacing with chassis components including, but not necessarily limited to, flexible rugged external drives (FreDs), an enclosure controller (EC) and a megaRAID storage manager (MSM).

The SAS IOMs 540 are used to provide Fabric-C SAS connectivity between compute sleds (e.g., compute sleds 502) and storage sleds (e.g., storage sled 510). The two SAS IOMs 540 are installed as a redundant pair within the chassis 501. To operate as a redundant pair the SAS IOMs 540 communicate with each other. As noted herein above, the communication is implemented with a GbE link using I²C protocol. The communication between the two SAS IOMs 540 may further include a series of general-purpose input/output (GPIO) digital signal pins routed through the PDB 530.

The SAS expanders 516 provide disk expansion for the compute sleds 502. In an illustrative embodiment, the storage drives 515 are accessed by translating a storage drawer outward and accessing the storage drives 515 from the sides of the storage drawer. In an illustrative embodiment, electrically, the storage sled 510 comprises five different types of boards, plus a cable assembly. The five boards include a Fab-C SAS interface module (e.g., Fab-C SAS board 800 discussed in connection with FIG. 8), a power control module, an expander module, a backplane (4×) and a front panel large expensive disk (LED) board.

With conventional approaches, a first SAS IOM may be connected with a first SAS expander in a storage sled and vice versa, and a second SAS IOM may be connected with a second SAS expander in a storage sled and vice versa. With current configurations, if the first SAS IOM and the second SAS expander fail, or if the second SAS IOM and the first SAS expander fail, data paths can be lost even though there is redundancy with two SAS IOMs and two SAS expanders.

For example, users may need to access run-time data from drives present in a storage sled via an SAS IOM. When users are running critical applications, SAS IOMs and/or SAS expanders in a storage sled can fail due to, for example, SAS IOM firmware updates, storage sled firmware updates, shutdowns due to power fluctuations in a datacenter and/or high system workloads. Therefore, problems exist with current approaches in that even if redundant hardware components fail, data transmission paths may be lost.

Referring, for example, to the information processing system 500 in FIG. 5 and to the architecture 600 in FIG. 6, the illustrative embodiments provide a technical solution in which an interconnect module 518 is connected (e.g., cross-connected) to each of a first SAS IOM 540-1, a second SAS IOM 540-2, a first SAS expander 516-1 of a storage sled 510 and a second SAS expander 516-2 of the storage sled 510. As a result, even if the first SAS IOM 540-1 and the second SAS expander 516-2 fail, or if the second SAS IOM 540-2 and the first SAS expander 516-1 fail, data can be routed through the interconnect module 518 so that data transmission paths can be maintained. Unlike conventional approaches, the interconnect module 518 ensures data path redundancy with SAS IOMs 540 and the storage sled 510 even when components fail.

In illustrative embodiments, the chassis controller 550 communicates with both SAS IOMs 540 in the chassis 501. Both of the SAS IOMs 540 provide hardware configuration data and other identifying information (e.g., inventory data) to the chassis controller 550 (e.g., management logic 116 and management database 117) and/or interconnect module 518 (e.g., identification and routing logic 519). Active (e.g., operative) SAS IOMs 540 will provide an inventory of an SAS topology to the chassis controller 550 and/or interconnect module 518 and accept commands modifying a state of attached devices, while inactive (e.g., non-operational, failed) SAS IOMs 540 will not provide an inventory of an SAS topology to the chassis controller 550 and/or interconnect module 518 and will not accept commands modifying a state of attached devices.

Communication to and from the SAS IOMs 540 uses a GbE network. This management network is referred to as Fabric D (FAB-D). In illustrative embodiments, the chassis controller 550 provides SAS topology information to the FMPs 543 on the SAS IOMs 540 to allow the FMPs 543 to create and manage SAS zones. The chassis controller 550 also uses FAB-D to gather storage subsystem inventory and status information.

The storage sled 510 is used in the chassis 501 to provide disk expansion for compute sleds 502. The SAS expanders 516 in the storage sled 510 communicate through GPIO and LED pins, which allow the firmware to synchronize the activities and support failover. The diagram of the storage sled 510 in FIG. 7 shows the interconnection between SAS expanders 516-1 and 516-2. There is an X4 SAS link between the two SAS expanders 516-1 and 516-2.

Referring further to FIG. 7, the Fab-C SAS interface module via the Fab-C connector 520 connects into the PDB 530 to tie into the SAS Fabric. As shown in FIG. 7, the Fab-C interfaces support 2 groups of X4 SAS data paths extending between the Fab-C connector 520 and the interconnect module 518 and between the interconnect module 518 and the SAS re-drivers 517. Each group connects to an SAS re-driver 517-1 or 517-2 inside the storage sled 510. Eight SAS data paths (also referred to herein as “lanes”) are routed through flexible cables between the SAS re-drivers 517 and the two SAS expanders 516. Four SAS data paths are routed between the SAS re-driver 517-1 and the SAS expanders 516-1 and four SAS data paths are routed between the SAS re-driver 517-2 and the SAS expander 516-2. In addition, eight SAS data paths are routed between the interconnect module 518 and the SAS re-drivers 517. Four SAS data paths are routed between the SAS re-driver 517-1 and the interconnect module 518. Four SAS data paths are routed between the SAS re-driver 517-2 and the interconnect module 518. Two groups of four SAS data paths are routed between the interconnect module 518 and the Fab-C connector 520.

The SAS expanders 516-1 and 516-2 each connect to ports on all 16 storage devices 515-1, 515-2, 515-3, . . . , 515-S, where S is 16. As noted herein, the storage drives 515 can be, for example, HDDs or SSDs. SAS expander 516-1 connects to Port-A of a first group of eight storage drives 515 and to Port-B of a second group of eight storage drives 515. SAS expander 516-2 is configured opposite to SAS expander 516-1 in that SAS expander 516-2 connects to Port-B of the first group of eight storage drives 515 and to Port-A of the second group of eight storage drives 515. This forms redundancy from both of the SAS expanders 516-1 and 516-2. In an illustrative embodiment, half of the storage drives 515 are accessed through the SAS expander 516-1 and the other half are accessed through the SAS expander 516-2. As noted herein, there is also an X4 SAS link between the SAS expanders 516-1 and 516-2 to facilitate communication and synchronization between the SAS expanders 516.

As used herein, an X4 SAS communication channel uses SAS, which is a high-speed data transfer interface used for connecting storage devices such as HDDs, SSDs and other peripheral devices to a computer system. X4 SAS indicates that there are four data paths (lanes or pathways) used for data transmission. Each data path operates independently and contributes to an overall data transfer rate. The more data paths that are available, the higher the potential bandwidth for data transfer between devices. Each SAS lane may include multiple different pairs of electrical connections, which enable faster and more efficient data transfer.

Referring to the Fab-C SAS board 800 in FIG. 8, the Fab-C connector 520 functions as a re-driver board for two x4 SAS interfaces. These SAS signals are transmitted between the SAS IOMs 540 and the SAS expanders 516 in the storage sled 510. The Fab-C connector 520 plugs into the PDB 530. The Fab-C SAS board 800 depicts Fab-C SAS board functionality. In the Fab-C SAS board 800, the Fab-C connector 520 includes two X4 SAS cables used to transmit from the SAS IOM 540-2 (FAB-C IOM-C2 IOM) into a compute server of the Fab-C connector 520. Similarly, on the other side of Fab-C connector 520, data from SAS IOM 540-1 (FAB-C IOM C1 IOM) is transmitted into the compute server of the Fab-C connector 520. SAS re-drivers 517-1 and 517-2 are bi-directional, taking data to and from the SAS IOMs 540-1 and 540-2 and the compute servers, and sending the data to SAS expanders 516-1 and 516-2 via the interconnect module 518.

The redundant SAS re-drivers 517-1 and 517-2 are powered via the power board connector 521 using, for example, an auxiliary (AUX) cable. The power board connector 521 is also used to provide power for various functions, such as, but not necessarily limited to, “SHIFTY”, “I