In a second embodiment, the cloud-based controller may again include a fire and forget message broker that may be configured to receive messages from a plurality of different sources (e.g., application services running on the cloud-based controller) and forward those messages to a network device over a single, bidirectional, persistent HTTP connection. In this embodiment, however, the HTTP-based connection may be implemented as a WebSocket tunnel. The message broker may forward messages received from the cloud services to a WebSocket server, and the WebSocket server forwards the messages to an ingress controller of the cloud-based controller for transmission to a WebSocket client provided on the network device.

The WebSocket client receives the messages and forwards them to a message broker client library on the network device. The message broker client library passes the messages to a message handler which forwards the messages to the appropriate applications on the network device. The message broker client library may also generate acknowledgments in response to each message that are transmitted to the cloud-based controller over the WebSocket tunnel, where they are passed back to the cloud services. Messages may likewise be sent from the applications running on the network device to the cloud-based controller in the same fashion, with the message broker client library collecting the messages from the applications and passing them to the WebSocket client for transmission to the cloud-based controller over the WebSocket tunnel. The second embodiment may be appropriate for network devices that may not have sufficient resources to support a gRPC agent.

The techniques for exchanging messages between a cloud-based controller and the network devices of a WiFi network that are disclosed herein use a message broker to simplify having multiple cloud-based services communicate with the network devices while also providing benefits in terms of scalability and load balancing. The embodiments disclosed herein may use persistent bidirectional HTTP connections between the cloud-based controller and each network device, which allows the ingress controller of the cloud-based controller to perform load balancing. Moreover, since a single, persistent, bidirectional HTTP connection is used between the cloud-based controller and each network device, the communication technique may use less network resources than other approaches, and/or may require less resources on each network device. The use of the HTTP connection also makes it easier to operate through firewalls, and may avoid the loss in flexibility that some message brokers experience when operating, for example, through customer firewalls.

Embodiments of the present invention are described in further detail below with reference to the figures.

FIG. 1 is a block diagram illustrating a simplified WiFi network 100 in which the communications techniques according to embodiments of the present invention may be used. As shown in FIG. 1, the WiFi network 100 may include one or more access points 110, one or more client devices 120 (such as cellular telephones, computers, tablets, printers and a wide range of other WiFi-capable electronic devices), and one or more cloud-based controllers 140. Each access point 110 may include one or more radios 112 that operate, for example, in the 2.4 GHZ frequency band, the 5 GHz frequency band, and/or the 6 GHz frequency band. The client devices 120 may also include one or more client radios 122 that operate in the 2.4 GHZ, 5 GHZ and/or 6 GHz frequency bands. Only a single radio 112, 122 is shown in each access point 110 and client device 120 for simplicity, but it will be appreciated that many access points 110 and client devices 120 are capable of operating in multiple different frequency bands.

The access points 110 and the client devices 120 may communicate with each other via wireless communication that is compatible with an IEEE 802.11 standard. For example, a client device (e.g., client device 120-1) may associate with a particular access point (e.g., access point 110-1). The client device 120-1 may communicate with other client devices 120 and/or with external networks 150 via the access point 110-1. In order for a client device 120 to associate with an access point 110, the access point 110 and the client device 120 can transmit advertising frames on wireless channels, detect one another by scanning the wireless channels, exchange data/management frames (such as association requests and responses) to establish a connection and configure security options (e.g., Internet Protocol Security). Once the association is completed, the client device 120 can wirelessly communicate over the WiFi network 100 by transmitting and receiving frames or packets via the wireless connection to the access point 110. The wireless signals 114 that are transmitted from the access points 110 to the client devices 120 and the wireless signals 124 that are transmitted from the client devices 120 to the access points 110 are illustrated in FIG. 1 via jagged lines.

WiFi network 100 may further include a switch network 130 that includes one or more network switches and/or routers 132. As an example, a network switch 132-1 may include a number of communication interfaces or ports (not shown) that are connected to various of the network devices. During operation, a first of the communication interfaces may receive a packet or other data container from a first electronic device (e.g., a client device 120, an access point 110, another network switch 132, etc.). The packet may then be processed and forwarded to a second port associated with a second electronic device. The network switch and/or router 132 may be a layer-2 or layer-3 network switch or router. The switch network 130, and the network switches 132 thereof, may be coupled to access points 110 of the wireless network 102 via wired links 134.

The access points 110 and the switches/routers 132 may be connected to an external network 150. The network 150 may comprise, for example, the Internet, an intra-net and/or one or more dedicated communication links. The network 150 may be separated from the switch network 130 and the access points 110 by a firewall 160, which may monitor network traffic that is incoming to and outgoing from the switch network 130 and decide whether to permit or prohibit various traffic based on one or more security rules. The access points 110 may provide the client devices 120 access to the network 150. It will be appreciated that some access points 110 may only be connected to the network 150 through other access points 110 (e.g., in a mesh network implementation).

The WiFi network 100 further includes one or more cloud-based controllers 140 that are deployed at one or more locations that may be relatively remote from the access points 110 and/or the switch network 130. The cloud-based controllers 140 may communicate with the network switches 132 and the access points 110 via, for example, the network 150. The cloud-based controllers 140 may perform configuration operations and/or management operations that control functionality of the network devices of the WiFi network 100 such as the access points 110 and the switches and routers 132. For example, the cloud-based controllers 140 may define flow definitions comprising packet processing rules and corresponding actions and promulgate these rules to the network switches 132 of the switch network 130. As another example, the cloud-based controller 140 may manage the access points 110, for example by providing various configuration information, controlling settings, routing information, collecting statistics, providing authorization/authentication information and the like. The cloud-based controller 140 may communicate with the access point 110 and/or network switches 132 via one or more logical links 142, which in some embodiments may at least partially overlap the wired links 134.

As described in further detail below with reference to FIG. 7, the access points 110, client devices 120 and/or the cloud-based controllers 140 may include subsystems, such as a networking subsystem, a memory subsystem and a processor subsystem.

FIG. 2 is a block diagram illustrating an architecture for exchanging messages between a cloud-based controller 200 and a network device 250 according to certain embodiments of the present invention.

As shown in FIG. 2, a cloud-based controller 200 includes a plurality of application services 210, a message broker 220, a gRPC proxy 230 and an ingress controller 240. The services 210 may comprise a plurality of applications running on the cloud-based controller 200 that communicate with the network device 250. The network device 250 is a device that is part of a wireless communications network (e.g., WiFi network 100 of FIG. 1). The network device 250 may comprise, for example, an access point, a switch, a data plane, a router or the like. The network device 250 includes a gRPC agent 260, an embedded message broker 270, and a plurality of applications 280 that run on the network device 250 that communicate with the cloud-based controller 200. While only a single network device 250 is shown in FIG. 2 for simplicity, it will be appreciated that the cloud-based controller 200 may be in communication with a large number of different network devices 250.

As shown in FIG. 2, the services 210 running on the cloud-based controller 200 may need to send notifications, commands or the like to the network device 250. Messages comprising such notifications and/or commands are passed from the various services 210 to the message broker 220. Herein, references to “messages” refer to the information that is exchanged between the application services 210 of the cloud-based controller 200 and the applications 280 of the network device 250. These messages may comprise at least part of a “payload” of one or more larger transmissions that are used to pass the messages between the application services 210 of the cloud-based controller 200 and the applications 280 of the network device 250 through one or more intermediate elements. As is further shown in FIG. 2, the message broker 220 may include a plurality of queues 222. The messages forwarded by the services 210 may, for example, be directed to particular queues 222 based on the intended recipient(s) of the message(s). The message broker 220 may, for example, comprise software that translates a message from a messaging protocol of the sender (e.g., one of the services 210) to the messaging protocol of the intended recipient (e.g., one of the applications 280 on the network device 250). The message broker 220 takes the messages received from the plurality of different services 210 and outputs these messages to the gRPC proxy 230. The gRPC proxy 230 may comprise a software application that allows an application or device to communicate with an application running on an external device as if the external application were a local object.

In addition, the gRPC proxy 230 may operate as an acknowledgment receiver to ensure that messages sent from the cloud-based controller 200 are received by the network device 250. The message broker 220 may thus guarantee reliable delivery.

The gRPC proxy 230 may forward each message received from the message broker 220 to the ingress controller 240. The ingress controller 240 establishes a single, bidirectional, persistent HTTP-based connection 245 with the network device 250. The network device 250 may have default ports for HTTP and HTTPS traffic and hence by using an HTTP connection 245 between the cloud-based controller 200 and the network device 250 the need to configure a firewall that may be positioned between the cloud-based controller 200 and the network device 250 (e.g., a firewall of the organization deploying the WiFi network that the network device 250 is part of) may be eliminated. The need for opening additional ports for the connection may also be eliminated. Thus, by providing a gRPC-over-HTTP connection between the cloud-based controller 200 and the network device 250, the number of network resources consumed and the complexity of the connection can be reduced. This acts to improve scalability.

Still referring to FIG. 2, messages forwarded by the ingress controller 240 over the HTTP connection 245 are received at the gRPC agent 260 of the network device 250. The gRPC agent 260 is a counterpart to the gRPC proxy 230 that allows the network device 250 to appear as a local object to the cloud-based controller 200. The gRPC agent 260 also generates acknowledgments in response to each received message, and the acknowledgments are forwarded back to the ingress controller 240 of the cloud-based controller 200 over the HTTP connection 245. The ingress controller 240 forwards the acknowledgments to the gRPC proxy 230. If an acknowledgment is not received for a particular message within a specified time period, the gRPC proxy 230 may resend the message to guarantee that the messages received from the message broker 220 are all properly delivered to the network device 250. The gRPC agent 260 on the network device 250 forwards each received message to the embedded message broker 270, and each message is mapped to an appropriate queue 272 based on the particular application 280 the message is addressed to. The embedded message broker 270 then passes each received message from the queues 272 to the appropriate applications 280.

The applications 280 may also generate messages that are passed to the cloud-based controller 200, such as data, status messages, event notifications and the like. These messages are passed from the applications 280 to the embedded message broker 270, where they are routed to appropriate queues 272 within the message broker 270 based on the intended recipients (e.g., the different services 210 on the cloud-based controller 200) of the messages. The embedded message broker 270 forwards these messages to the gRPC agent 260. The gRPC agent 260 transmits the messages over the single, persistent, bidirectional HTTP connection 245 to the ingress controller 240 of the cloud-based controller 200. The ingress controller 240 may perform load balancing with respect to messages received from network device 250 (and from other devices communicating with the services 210), and thus may facilitate scalability for the cloud-based controller 200. The ingress controller 240 passes the received messages to the gRPC proxy 230, which forwards the messages to the message broker 220 where they are routed to appropriate queues 222. The gRPC proxy 230 also generates an acknowledgment for each received message that is passed to the ingress controller 240 for transmission to the gRPC agent 260 of the network device 250 over the HTTP connection 245. The message broker 220 then passes the messages from the network device 250 to the appropriate services 210. The messages may be routed to the appropriate services 210 based on uniform resource locators (“URLs”) for the different services 210 that are included in the messages.

As is further shown in FIG. 2, a so-called “callback URL” may be used as an alternative (or additional) technique for each service 210 to confirm that messages it sends to the network device 250 have in fact been received by the network device 250. The callback URL is an object that is embedded in a message sent by one of the services 210 that includes the URL for the particular service 210. When the application 280 to which the message is addressed reads the message, an application programming interface may generate, for example, both an acknowledgment and a result of execution of the command or other request that are routed to the ingress controller 240 via a separate path (e.g., through a path/ap/callback/traceroute/{app serial number}). The acknowledgment includes the URL for the service 210 that sent the original message, and the ingress controller 240 routes the acknowledgment to the service 210. The callback URL may be used, for example, with respect to messages having lower importance. For example, an access point might acknowledge a command from the cloud-based controller 200 instructing the access point to illuminate an LED thereof via a call-back URL acknowledgment. For more important commands (e.g., commands relating to troubleshooting), the acknowledgments may be sent through the above-described primary path for acknowledgments.

FIG. 3A is a flow chart diagram illustrating a method for a cloud-based controller to communicate with a network device of a wireless communications network. As shown in FIG. 3A, operations may begin with an application service of the cloud-based controller forwarding a message to a message broker of the cloud-based controller (operation 200). The message may thereafter be transmitted from the cloud-based controller to the network device over a persistent HTTP connection (operation 215). Thereafter, a gRPC proxy of the cloud-based controller may receive an acknowledgment that the message was received at the network device (operation 230).

FIG. 3B is a flow chart diagram illustrating further details of one possible implementation of the method of FIG. 3A. As shown in FIG. 3B, operations may again begin with an application service of the cloud-based controller forwarding a message to a message broker for the cloud-based controller (operation 200). The message may be stored in a queue within the message broker. The message broker may appropriately reformat the message and then forward the reformatted message to a gRPC proxy of the cloud-based controller (operation 205). The gRPC proxy may, in turn, forward the reformatted message to an ingress controller of the cloud-based controller (operation 210). Then, as discussed above with respect to FIG. 3A, the ingress controller may transmit the message to the network device over a persistent HTTP connection between the cloud-based controller and the network device (operation 215).

The gRPC agent of the network device may receive the message (operation 220). In response, the gRPC agent may generate an acknowledgment message that is forwarded over the persistent HTTP connection to the ingress controller of the cloud-based controller (operation 225). The ingress controller passes the acknowledgment to the gRPC proxy (operation 230). The gRPC proxy may then discard the original message (since it now knows that the message was received). As is further shown in FIG. 3B, in response to receiving the message, the gRPC agent (in addition to generating the acknowledgment) may pass the message to the embedded message broker, where the message is routed to an appropriate queue (operation 235). The embedded message broker may then forward the message to the appropriate application (operation 240).

FIG. 4A is a flow chart diagram illustrating a method for a network device of a wireless communications network to communicate with a cloud-based controller according to further embodiments of the present invention. As shown in FIG. 4A, a message generated by an application service of the cloud-based controller is received at the network device (operation 300). The message is passed to an application that is installed on the network device based on an address contained in the message (operation 310). The application then extracts a callback URL from the message (operation 315). The application may then transmit an acknowledgment addressed to a service running on the cloud-based controller in response to receiving the message (operation 330). The acknowledgment may be transmitted to the ingress controller of the cloud-based controller, and the ingress controller may forward the acknowledgment directly to the application service that sent the original message based on a URL contained within the acknowledgment. This acknowledgment is not received at the message broker.

FIG. 4B is a flow chart diagram illustrating more details of an example implementation of the method of FIG. 4A. As shown in FIG. 4B, a message generated by an application service of the cloud-based controller is received at the network device (operation 300). The message may be received at a gRPC agent of the network device and may be generated and forwarded to the gRPC agent in the manner discussed above with reference to operations 200, 205, 210, 215 and 220 of FIG. 3B. The message may then be pushed from the gRPC agent to an embedded message broker of the network device (operation 305). The message is passed to an application that is installed on the network device based on an address contained in the message (operation 310).

The application extracts a callback URL from the message (operation 315). The application may then transmit an acknowledgment addressed to a service running on the cloud-based controller to the ingress controller of the cloud-based controller (operation 320). The acknowledgment may be transmitted, for example, over a path/ap/callback/traceroute/{app serial number}. The ingress controller may then forward the acknowledgment directly to the application service that sent the original message based on a URL contained within the acknowledgment (operation 325).

Some network devices may only have limited memory and/or processing capabilities. These devices may not have sufficient resources to run gRPC and/or an embedded message broker. For example, some inexpensive access points may have limited memory. With these devices, there may not be sufficient resources to communicate with a cloud-based controller using the techniques discussed above with reference to FIGS. 2-4B.

FIG. 5 is a block diagram illustrating an architecture for exchanging messages between a cloud-based controller 400 and a network device 450 according to further embodiments of the present invention. The architecture shown in FIG. 5 may require less resources within the network device 450 as compared to the architecture shown in FIG. 2.

As shown in FIG. 5, the cloud-based controller 400 includes a plurality of services 410, a message broker 420, a WebSocket server 430 and an ingress controller 440. The services 410 may comprise a plurality of applications running on the cloud-based controller 400 that communicate with the network device 450. The network device 450 is a device that is part of the wireless communications (e.g., WiFi) network such as access point, a switch, a data plane, a router or the like. The network device 450 includes a WebSocket client 460 and a plurality of applications 470 that run on the network device 450 that communicate with the cloud-based controller 400.

As shown in FIG. 5, the application services 410 running on the cloud-based controller 400 may need to send notifications, commands and/or other information to the network device 450. Messages containing such notifications and/or commands are passed from the various services 410 to the message broker 420, where the messages are stored in queues 422 based on the intended recipients of the messages. The message broker 420 may, for example, be identical to the message broker 220 discussed above with reference to FIG. 2. The message broker 420 passes the received messages to the WebSocket server 430. The WebSocket server 430 may comprise a server that acts as a middleman to securely route communications through firewalls using secure socket layer (“SSL”) certificates. WebSocket is also HTTP-based and hence this approach again allows communications between the application services 410 of the cloud-based controller 400 and applications 470 running on a network device 450 to be routed over a single, persistent, bidirectional HTTP-based connection. The WebSocket server 430 forwards the messages to the ingress controller 440 for transmission over the WebSocket tunnel 445 that is formed between the cloud-based controller 400 and the network device 450.

The WebSocket client 460 on network device 450 is at the other end of the WebSocket tunnel 445 and receives the messages from the cloud-based controller 400. The applications 470 included on the client device 450 include a message broker client library 472. The messages are passed from the WebSocket client 460 to the message broker client library 472. The message broker client library 472 may be a counterpart to message broker 420. The payload in the transmission over the WebSocket tunnel is or includes the original message generated by the application service 410 running on the cloud-based controller 400. The WebSocket client 460 can extract the payload (including the message) and forward it to the message broker client library 472 as shown by the heading “packet forwarding” in FIG. 5. Upon receipt of a message, the message broker client library 472 generates an acknowledgment which is passed to the WebSocket client 460 and transmitted back to the message broker through the WebSocket tunnel 445, ingress controller 440 and WebSocket server 430. The message broker 410 may then forward the acknowledgment to the service 410 that generated the original message. If an acknowledgment is not received for a particular message within a specified time period, the service 410 that generated the original message may resend the message to guarantee that the message is properly delivered to the network device 250. Since the message broker client library 472 provides acknowledgments, the message broker 420 may be implemented as a simple “fire and forget” message broker. The message broker client library 472 may then pass the message to a message handler 474. It will be understood that both the message broker client library 472 and the message handler 474 are part of an application 470. The message handler 474 is a component of the application 470 that executes the message (e.g., responds to the message). For example, if the message is a “trace route” command, the message handler can execute that command and return the result to an application service 410 running on the cloud-based controller 400 via, for example, a callback URL.

The applications 470 running of the network device 450 may also generate messages that are passed to the cloud-based controller 400, such as data, status messages, event notifications and the like. These messages are passed from the applications 470 to the message handler 474, where they are routed to the message broker client library 472. The message broker client library 472 forwards these messages to the WebSocket client 460. The WebSocket client 460 transmits the messages over the WebSocket tunnel 445 to the ingress controller 440 of the cloud-based controller 400. The ingress controller 440 may perform load balancing with respect to messages received from network device 450 (and from other devices communicating with the services 410), and thus may facilitate scalability for the cloud-based controller 400. The ingress controller 440 passes the received messages to the WebSocket server 430, which forwards the messages to the message broker 420 where they are routed to appropriate queues 422. The message broker 420 then passes the messages from the network device 450 to the appropriate services 410. For return messages there may be no acknowledgments.

As is further shown in FIG. 5, an optional “callback URL” may be used as an alternative (or additional) technique for each service 410 to confirm that messages it sends to the network device 450 have in fact been received by the network device 450. The callback URL may be implemented in the same manner as discussed above with reference to FIG. 2, and hence further description thereof will be omitted here.

FIG. 6A is a flow chart diagram illustrating a method for a cloud-based controller to communicate with a network device of a wireless communications network according to embodiments of the present invention. As shown in FIG. 6A, operations may begin with an application service of the cloud-based controller forwarding a message to a message broker for the cloud-based controller (operation