200, according to exemplary embodiments of the present disclosure. Exemplary system 200 illustrated in FIGS. 2A and 2B may represent a system configured to select one or more paths for the transmission of data packets from a first device (e.g., a vehicle, etc.) to a second device (e.g., a computing resource and/or remote driving terminal, etc.), according to exemplary embodiments of the present disclosure.

As shown in FIGS. 2A and 2B, exemplary system 200 may include vehicle 210, which may transmit data packets 212 (e.g., data packets 212-1, 212-2 through 212-N), via paths 214 (e.g., paths 214-1, 241-2, through 214-N) of network 250, to computing resource 220. According to exemplary implementations of the present disclosure, data packets 212 may include data streams corresponding to images, video, and/or other information captured by imaging devices, cameras, sensors, and the like associated with vehicle 210 that are transmitted to computing resource 220 (e.g., a remote driving terminal, etc.), via network 250, to facilitate remote operation of vehicle 210. In an exemplary implementation, data packets 212 are sent from vehicle 210 to computing resource 220 to facilitate remote operation of vehicle 210 by a user operating computing resource 220.

In connection with the transmission of data packets 212, exemplary embodiments of the present disclosure may provide for the selection of one or more paths 214 of network 250, via which data packets 212 may be transmitted. For example, exemplary embodiments of the present disclosure may employ a token bucket technique in selecting one or more paths 214 on which to transmit data packets 212 by adaptively modifying a redundancy of data packets 212 being transmitted via the transmission paths of network 250 in view of current network parameters being experienced by network 250, so as to load balance data packets 212 between one or more paths 214 to avoid saturation and/or congestion of network 250, while also ensuring redundancy of data packets 212 and maintaining the data rate of the data stream. For example, the selection of one or more paths 214 may be based on current network information and/or parameters, such as a congestion indicator, the amount of data in flight (e.g., data that has been sent but not yet received), a data rate of the incoming data (e.g., a bitrate or byte-rate of the data streams) received from the various imaging devices, cameras, sensors, and the like, and a feedback period (e.g., the time between feedback received from computing resource 220 regarding receipt of data). The current network information and/or parameters may be used to determine refill rates of path token buckets associated with each path 214 (as well as a global token bucket) and may be periodically provided by computing system 2202 to vehicle 210 via feedback loop 252 to ensure that vehicle 210 has current network information and/or parameters. Further, the periodically received updated network information and/or parameters may be used to update the refill rates of the token buckets. Optionally, an alpha filter may be applied to the network information and/or parameters or the refill rates, so as to provide smoothed values and avoid large fluctuations in the refill rates.

In exemplary implementations of the present disclosure that employ a token bucket technique, a global token bucket for all paths 214 and a path token bucket for each path 214 may first be initialized. After initialization of the token buckets, network parameters, such as a data rate for the data stream, a congestion value (e.g., congestion window, etc.), a data in flight value, a feedback period, and the like, may be received from computing resource 220 and used to determine a refill rate for each path token bucket associated with each path 214. In exemplary implementations, the path token bucket refill rates determined using these network parameters may represent a redundancy capacity for each path 214. Further, the refill rates for paths 214 may be summed to determine a global redundancy refill rate for a global token bucket across all paths 214. The path token buckets for each path 214, as well as the global token bucket, may be refilled according to their respective refill rates, and one or more of paths 214 may be selected for transmitting data packets 212 based on the path token bucket values and the global token bucket. Accordingly, data packets 212 may be transmitted on each of the selected paths 214.

After transmission of data packets 212 on the selected paths 214, the path token bucket values of the selected paths 214 and the global token bucket may be updated in view of the transmission of data packets 212. After the path token bucket values of the selected paths 214 and the global token bucket are updated, the path token buckets for each path 214, as well as the global token bucket, may again be refilled according to their respective refill rates in preparation of the selection of one or more paths for the next transmission of packets. Accordingly, the refilling of the path token bucket values and the global token bucket, selection of one or more paths 214 for transmission of data packets 212 of the data stream, and updating of the path token bucket values of the selected paths 214 and the global token bucket may be continuously performed in connection with the transmission of data packets 212, so as to avoid saturation and/or alleviate congestion of network 250 while also maintaining the data rate of the data stream and redundancy of the packets being transmitted. Further, the refill rates may be periodically updated, as network conditions may change in view of the current network information that is periodically and continuously provided as feedback to vehicle 210 from computing resource 220.

According to certain aspects of the present disclosure, redundancy capping techniques may also be applied, so as to limit network costs, facilitate scaling, and the like, in connection with the selection of one or more paths 214 of network 250. Such a scenario may be experienced by vehicle 210 in areas where there may be strong network cover and/or limited/low network usage. Accordingly, in such implementations, the global token bucket may be capped to a predetermined threshold. Additionally, the token bucket values may not be refilled if the token bucket values for any of the paths is above a predetermined credit limit. Limiting the redundancy and/or refilling of token buckets above a credit limit can ensure a minimum redundancy and effective load balancing of the data stream across the paths of the data link.

Further, in scenarios where there is a degradation and/or congestion of network 250, data rate adaptation techniques may also be employed in connection with the selection of one or more paths of the network link in view of network conditions. Such a scenario may be experienced by vehicle 210 where network 250 may include poor network coverage and/or heavy network usage. For example, in situations where network conditions do not permit a desired redundancy in connection with the data stream, the data rate of the data stream may be adjusted (e.g., lowered, etc.) to compensate for the degraded network conditions. Additionally, exemplary implementations of the present disclosure also provide for adjusting the data rate (e.g., increasing, etc.) as network conditions improve. Further, the data rate may have maximum and minimum thresholds, so as to prevent the data rate from being below or above threshold values, and the adjustments to the data rate may be smoothed and/or filtered, so as to avoid large swings in the data rate.

FIG. 3 is a flow diagram of an exemplary adaptive path selection process 300, according to exemplary embodiments of the present disclosure. According to exemplary embodiments of the present disclosure, adaptive path selection process 300 may be performed in connection with the selection of one or more data paths for the transmission of data packets forming a data stream that includes images, video, and/or other information captured by imaging devices, cameras, sensors, etc. of a vehicle that is being remotely operated.

As shown in FIG. 3, exemplary adaptive path selection process 300 may begin at step 302, where network information may be determined and/or obtained. According to exemplary embodiments of the present disclosure, network information such as a data rate for the data stream, a congestion value (e.g., congestion window, etc.), a data in flight value, a feedback period, and the like, may be determined and/or obtained for each path. For example, the data rate may be determined based on the data stream being captured by the imaging devices, cameras, sensors, etc. of a vehicle that is being remotely operated, a congestion value (e.g., congestion window, etc.) may be determined based on timing associated with previously sent packets, a data in flight value may be determined based on feedback received regarding the receipt of previously sent packets, and the feedback period may be determined based on the timing associated with when feedback is received based on previously sent packets.

After the network information has been determined and/or obtained, in exemplary implementations of the present disclosure that employs a token bucket technique (e.g., a token bucket for each candidate path), refill rates may be determined for the path token buckets associated with each path, as well as for a global token bucket, as in step 304. For example, the refill rate for each path may represent a redundancy capacity and/or a margin that is available before the path becomes congested for each respective path and a refill rate for the global token bucket across all paths may be a sum of the refill rates for all paths (e.g., a global redundancy value). According to exemplary embodiments of the present disclosure, the refill rate for each path token bucket for each available path may be represented as:

$refill rate = \frac{Congestion - {Data}_{F l ight}}{Data Rate * Feedback Period}$

where Congestion may represent a congestion value, such as a congestion window, DataFlight may represent data in flight (e.g., expressed as bits, bytes, etc.), Data Rate may represent the data rate of the incoming data stream (e.g., expressed as a bitrate, a byte-rate, etc.), and Feedback Period may represent a time between feedback that is received.

In step 306, it may be determined if a data packet is received. Accordingly, if a data packet is not received, process 300 continues to loop until a data packet is received. Once a data packet is received, in step 308, the path token bucket values for each path and the global token bucket may be refilled according to their respective refill rates. For example, the corresponding refill rates may be added to the value of the path token buckets corresponding to each available path and the global token bucket.

After the token bucket values have been refilled, in step 310, one or more of the available paths are selected to determine the transmission paths based on the path token bucket values and the global token bucket. For example, the N number of paths having the greatest token bucket values may be selected, where N refers to a floor integer value of the global token bucket. Accordingly, in an exemplary implementation where the global redundancy value is 3.7, the three transmission paths having the highest path token bucket values may be selected. After selection of one or more paths, data packets may be transmitted on the selected paths.

In step 312, after transmission of the packets, the path token bucket values for the selected paths and the global token bucket may be updated. For example, the path token bucket values for each selected path may be subtracted by one (e.g., to represent the transmitted packet) and the global token bucket may be subtracted by the total number of transmitted packets across all selected paths. Continuing the above example where three transmission paths were selected and three packets were transmitted, the path token bucket values for each selected path on which a packet was transmitted may be subtracted by one and the global token bucket value may be subtracted by three.

In step 314, it may then be determined if the current feedback period has completed with the receipt of the next feedback information. In the event that the next feedback has not been received, exemplary adaptive path selection process 300 returns to step 308. Alternatively, adaptive selection process 300 returns to step 302, so as to determine and/or obtain updated current network information, which may be used to determine updated refill rates and an updated global redundancy value.

FIGS. 4A and 4B are exemplary tables 400 and 450 illustrating exemplary path token bucket values and global token bucket values, in accordance with exemplary embodiments of the present disclosure.

As shown in FIG. 4A, table 400 may present hypothetical values for path token buckets and a global token bucket used in connection with determining paths for transmitting packets of a data stream in an exemplary implementation of the present disclosure. In the illustrated implementation, four paths are available (e.g., Path 0, Path 1, Path 2, and Path 3) for transmitting packets of the data stream, and the network information and/or parameters utilized to determine the refill rates of the path token buckets and the global token bucket include a congestion indicator (e.g., a congestion window), a data in flight value, a data rate of the incoming data stream, and a feedback period. In the exemplary implementation illustrated in FIG. 4A, the current network information and/or parameters may indicate an input data stream rate of 8 Mb/s, a feedback period of 50 ms, and that each round (N) shown in table 400 is within a single feedback period. Further, the current network information and/or parameters may further indicate a difference between the congestion value and a data in flight value to be 25 KB for Path 0, 30 KB for Path 1, 75 KB for Path 2, and 15 KB for Path 3. Applying the network information and/or parameters to the following refill rate formula:

$refill rate = \frac{Congestion - {Data}_{F l ight}}{Data Rate * Feedback Period}$

a refill rate for Path 0 may be determined to be 0.5, a refill rate for Path 1 may be determined to be 0.6, a refill rate for Path 2 may be determined to be 1.0, and a refill rate for Path 3 may be determined to be 0.3. The refill rate for each path may represent a margin (e.g., a number of packets that may be sent) before the path is saturated. Further, summing the refill rates for all the paths may determine a global token bucket refill rate of 2.4, which may be added to the global redundancy value to obtain a refilled global redundancy value of 2.9 (e.g., 0.5+2.4).

As shown in FIG. 4A, in connection with Round N=0, the token bucket values for each path may be refilled by their corresponding refill rates. Accordingly, the refill rate of 0.5 for Path 0 may be added to the token bucket value for Path 0, the refill rate of 0.6 for Path 1 may be added to the token bucket value for Path 1, the refill rate of 1.0 for Path 2 may be added to the token bucket value for Path 2, and the refill rate of 0.3 for Path 3 may be added to the token bucket value for Path 3. In connection with Round N=0, prior to refilling the token bucket values for each path with the corresponding refill rates, each token bucket may have been initialized with an initial value of 0.5. For example, the initial value may be an average target redundancy value based on the number of paths. In the exemplary implementation, the initial value of 0.5 may correspond to a target redundancy value of 2.0 divided by the number of available paths (4), which yields the initial value of 0.5 for each path.

In connection with Round N=0, refilling each token bucket value for Path 0, Path 1, Path 2, and Path 3 may yield a token bucket value of 1.0 for Path 0, a token bucket value of 1.1 for Path 1, a token bucket value of 1.5 for Path 2, and a token bucket value of 0.8 for Path 3. After the refilled token bucket values have been determined for each path, the number of paths to be selected for transmitting packets may be determined by the global token bucket. As shown in FIG. 4A, the floor integer value of the global token bucket may be used to determine the number of paths to be selected. Accordingly, for Round N=0, two paths (e.g., |2.9|=2) may be selected from Path 0, Path 1, Path 2, and Path 3 for transmitting packets of the data stream in Round N=0. In selecting the paths for transmitting packets of the data stream, the paths with the highest associated token bucket values may be selected. Accordingly, as shown in FIG. 4A, the two paths with the highest token bucket values, namely Path 1 and Path 2, which have token bucket values of 1.1 and 1.5, respectively, are selected as the transmission paths for Round N=0. After transmission of the packets on Path 1 and Path 2, the token bucket values for the paths on which the packets were transmitted, as well as the global token bucket are updated. Accordingly, one is subtracted from the token bucket values associated with Path 1 and Path 2 to obtain updated token bucket values of 0.1 and 0.5, respectively, and two is subtracted from the global token bucket value, to obtain an updated global redundancy value of 0.9.

The updated token bucket values and updated global redundancy value are used to determine the paths in the next round. Further, prior to the selection of paths in the next round, it may be determined whether the feedback period has expired, and feedback information has been received with updated network information and/or parameters. In situations where feedback information has been received providing updated network information and/or parameters, updated refill rates may be determined based on the updated network information and/or parameters. Otherwise, the selection of paths may be performed with the previously determined refill rates based on the previous network information and/or parameters.

As illustrated in FIG. 4A, after the selection of Path 1 and Path 2 and updating of the token bucket values and global redundancy value in Round N=0, the updated token bucket values are refilled using the refill values for Round N=1. Accordingly, the corresponding refill rates are added to the updated token bucket values from Round N=0. As illustrated, the refill rate of 0.5 for Path 0 may be added to the token bucket value of 1.0 for Path 0 to obtain a refilled token bucket value of 1.5 for Path 0, the refill rate of 0.6 for Path 1 may be added to the token bucket value of 0.1 for Path 1 to obtain a refilled token bucket value of 0.7, the refill rate of 1.0 for Path 2 may be added to the token bucket value of 0.5 for Path 2 to obtain a refilled token bucket value of 1.5, and the refill rate of 0.3 for Path 3 may be added to the token bucket value of 0.8 for Path 3 to obtain a refilled token bucket value of 1.1 for Path 3. The global token bucket refill rate of 2.4 (e.g., the sum of the token bucket value refill rates −0.5+0.6+1.0+0.3=2.4) is added to the updated global token bucket value to obtain a refilled global redundancy value of 3.3.

After the path token bucket values have been refilled, the number of paths to be selected for transmitting packets may be determined by the global redundancy value. As shown in FIG. 4A, the floor integer value of the global token bucket value may be used to determine the number of paths to be selected. Accordingly, for Round N=1, three paths (e.g., |3.3|=3) may be selected from Path 0, Path 1, Path 2, and Path 3 for transmitting packets of the data stream in Round N=1. In selecting the paths for transmitting packets of the data stream, the paths with the highest associated token bucket values may be selected. Accordingly, as shown in FIG. 4A, the three paths with the highest token bucket values, namely Path 0, Path 2, and Path 3, which have token bucket values of 1.5, 1.5, and 1.1, respectively, are selected as the transmission paths for Round N=1. After transmission of the packets on Path 0, Path 2, and Path 3, the path token bucket values for the paths on which the packets were transmitted, as well as the global token bucket value are updated. Accordingly, one is subtracted from the token bucket values associated with Path 0, Path 2, and Path 3 to obtain updated token bucket values of 0.5, 0.5, and 0.1, respectively, and three is subtracted from the global token bucket value, to obtain an updated global redundancy value of 0.3.

The updated token bucket values and updated global token bucket value are then used to determine paths in the next round. Further, prior to the selection of paths in the next round, it may be determined whether the feedback period has expired, and feedback information has been received providing updated network information and/or parameters. In situations where feedback information providing updated network information and/or parameters has been received, updated refill rates may be determined based on the updated network information and/or parameters. Otherwise, the selection of paths may be performed with the previously determined refill rates based on the previous network information and/or parameters.

As illustrated in FIG. 4A, after the selection of Path 0, Path 2, and Path 3 and updating of the path token bucket values and global token bucket value in Round N=1, the updated path token bucket values are refilled using the refill values for Round N=2. Accordingly, the corresponding refill rates are added to the updated path token bucket values from Round N=1. As illustrated, the refill rate of 0.5 for Path 0 may be added to the token bucket value of 0.5 for Path 0 to obtain a refilled token bucket value of 1.0 for Path 0, the refill rate of 0.6 for Path 1 may be added to the token bucket value of 0.7 for Path 1 to obtain a refilled token bucket value of 1.3, the refill rate of 1.0 for Path 2 may be added to the token bucket value of 0.5 for Path 2 to obtain a refilled token bucket value of 1.5, and the refill rate of 0.3 for Path 3 may be added to the token bucket value of 0.1 for Path 3 to obtain a refilled token bucket value of 0.4 for Path 3. The global token bucket refill rate of 2.4 (e.g., the sum of the token bucket value refill rates −0.5+0.6+1.0+0.3=2.4) is added to the updated global token bucket value to obtain a refilled global redundancy value of 2.7.

After the path token bucket values have been refilled, the number of paths to be selected for transmitting packets may be determined by the global token bucket value. As shown in FIG. 4A, the floor integer value of the global token bucket value may be used to determine the number of paths to be selected. Accordingly, for Round N=2, two paths (e.g., |2.7|=2) may be selected from Path 0, Path 1, Path 2, and Path 3 for transmitting packets of the data stream in Round N=2. In selecting the paths for transmitting packets of the data stream, the paths with the highest associated token bucket values may be selected. Accordingly, as shown in FIG. 4A, the two paths with the highest token bucket values, namely Path 1 and Path 2, which have token bucket values of 1.3 and 1.5, respectively, are selected as the transmission paths for Round N=2. After transmission of the packets on Path 1 and Path 2, the token bucket values for the paths on which the packets were transmitted, as well as the global token bucket value are updated. Accordingly, one is subtracted from the token bucket values associated with Path 1 and Path 2 to obtain updated token bucket values of 0.3 and 0.5, respectively, and two is subtracted from the global redundancy value, to obtain an updated global redundancy value of 0.7.

As illustrated in FIG. 4A, paths may continue to be iteratively selected by refilling the path token bucket values for each path and the global token bucket value, selecting one or more paths based on the refilled path token bucket values and the refilled global token bucket value, and updating the path token bucket values associated with paths on which the packets were transmitted and the global token bucket value. Table 400 continues the above exemplary implementation of Rounds N=3 through N=9, with the assumption that no new feedback information is received between any of the rounds and the refill rates remain the same for each of Rounds N=3 through N=9.

FIG. 4B illustrates table 450, which may present hypothetical values for path token buckets and a global token bucket used in connection with determining paths for transmitting packets of a data stream in an exemplary implementation of the present disclosure. In the illustrated implementation, four paths are available (e.g., Path 0, Path 1, Path 2, and Path 3) for transmitting packets of the data stream, and the network information and/or parameters utilized to determine the refill rates of the path token buckets and the global token bucket include a congestion indicator (e.g., a congestion window), a data in flight value, a data rate of the incoming data stream, and a feedback period.

The exemplary implementation illustrated in FIG. 4B may be representative of a situation where there are poor network conditions (e.g., poor network coverage, the network is congested, etc.) are being experienced. Accordingly, the current network information and/or parameters may indicate a refill rate for Path 0 of 0.1, a refill rate for Path 1 of 0.2, a refill rate for Path 2 of 0.4, and a refill rate for Path 3 of 0.1, where the refill rates may be obtained by applying the network information and/or parameters to the following refill rate formula:

$refill rate = \frac{Congestion - {Data}_{F l ight}}{Data Rate * Feedback Period}$

The refill rate for each path may represent a margin (e.g., a number of packets that may be sent) before the path is saturated. Further, summing the refill rates for all the paths may determine a global token bucket refill rate of 0.8, which may be added to the global redundancy value to obtain a refilled global redundancy value of 1.3 (e.g., 0.5+0.8).

As shown in FIG. 4B, in connection with Round N=0, the token bucket values for each path may be refilled by their corresponding refill rates. Accordingly, the refill rate of 0.1 for Path 0 may be added to the token bucket value for Path 0, the refill rate of 0.2 for Path 1 may be added to the token bucket value for Path 1, the refill rate of 0.4 for Path 2 may be added to the token bucket value for Path 2, and the refill rate of 0.1 for Path 3 may be added to the token bucket value for Path 3. In connection with Round N=0, prior to refilling the token bucket values for each path with the corresponding refill rates, each token bucket may have been initialized with an initial value of 0.5. For example, the initial value may be an average target redundancy value based on the number of paths. In the exemplary implementation, the initial value of 0.5 may correspond to a target redundancy value of 2.0 divided by the number of available paths (4), which yields the initial value of 0.5 for each path.

In connection with Round N=0, refilling each token bucket value for Path 0, Path 1, Path 2, and Path 3 may yield a token bucket value of 0.6 for Path 0, a token bucket value of 0.7 for Path 1, a token bucket value of 0.9 for Path 2, and a token bucket value of 0.6 for Path 3. After the refilled token bucket values have been determined for each path, the number of paths to be selected for transmitting packets may be determined by the global token bucket. As shown in FIG. 4B, the floor integer value of the global token bucket may be used to determine the number of paths to be selected. Accordingly, for Round N=0, one path (e.g., |1.3|=2) may be selected from Path 0, Path 1, Path 2, and Path 3 for transmitting packets of the data stream in Round N=0. In implementations where the global token bucket is less than one, the global token bucket may be set to one to ensure selection of at least one path and transmission of the data path on the selected at least one path.

In selecting the paths for transmitting packets of the data stream, the path with the highest associated token bucket values may be selected. Accordingly, as shown in FIG. 4B, although none of the token buckets exceeds 1, the path with the highest token bucket value, namely Path 2, which has a token bucket value of 0.9, is selected as the transmission path for Round N=0. After transmission of the packet on Path 2, the token bucket value for the path on which the packets were transmitted, as well as the global token bucket are updated. Accordingly, one is subtracted from the token bucket values associated with Path 2 to obtain an updated token bucket value −0.1, and one is subtracted from the global token bucket value, to obtain an updated global redundancy value of 0.3.

As illustrated in FIG. 4B, after the selection of Path 2 and updating of the token bucket value and global redundancy value in Round N=0, the updated token bucket values are refilled using the refill values for Round N=1. Accordingly, the corresponding refill rates are added to the updated token bucket values from Round N=0. As illustrated, the refill rate of 0.1 for Path 0 may be added to the token bucket value of 0.6 for Path 0 to obtain a refilled token bucket value of 0.7 for Path 0, the refill rate of 0.2 for Path 1 may be added to the token bucket value of 0.7 for Path 1 to obtain a refilled token bucket value of 0.9, the refill rate of 0.4 for Path 2 may be added to the token bucket value of −0.1 for Path 2 to obtain a refilled token bucket value of 0.3, and the refill rate of 0.1 for Path 3 may be added to the token bucket value of 0.6 for Path 3 to obtain a refilled token bucket value of 0.7 for Path 3. The global token bucket refill rate of 0.8 (e.g., the sum of the token bucket value refill rates −0.1+0.2+0.4+0.1=0.8) is added to the updated global token bucket value to obtain a refilled global redundancy value of 1.1.

After the path token bucket values have been refilled, the number of paths to be selected for transmitting packets may be determined by the global redundancy value. As shown in FIG. 4B, the floor integer value of the global token bucket value may be used to determine the number of paths to be selected. Accordingly, for Round N=1, one path (e.g., |1.1|=1) may be selected from Path 0, Path 1, Path 2, and Path 3 for transmitting packets of the data stream in Round N=1. In selecting the paths for transmitting packets of the data stream, the path with the highest associated token bucket value may be selected. Accordingly, as shown in FIG. 4B, the path with the highest token bucket value, namely Path 1, which has a token bucket value of 0.9 is selected as the transmission path for Round N=1. As shown in FIG. 4B, although none of the token bucket values exceeds one, the path with the highest token bucket value, namely Path 1, which has a token bucket value of 0.9, is selected as the transmission path for Round N=0 to ensure selection of at least one path and transmission of the data path on the selected at least one path.

After transmission of the packets on Path 1, the path token bucket value for the path on which the packet was transmitted, as well as the global token bucket value are updated. Accordingly, one is subtracted from the token bucket value associated with Path 1 to obtain an updated token bucket value −0.1, and one is subtracted from the global token bucket value, to obtain an updated global redundancy value of 0.1.

As illustrated in FIG. 4B, after the selection of Path 1 and updating of the path token bucket values and global token bucket value in Round N=1, the updated path token bucket values are refilled using the refill values for Round N=2. Accordingly, the corresponding refill rates are added to the updated path token bucket values from Round N=1. As illustrated, the refill rate of 0.1 for Path 0 may be added to the token bucket value of 0.7 for Path 0 to obtain a refilled token bucket value of 0.8 for Path 0, the refill rate of 0.2 for Path 1 may be added to the token bucket value of −0.1 for Path 1 to obtain a refilled token bucket value of 0.1, the refill rate of 0.4 for Path 2 may be added to the token bucket value of 0.3 for Path 2 to obtain a refilled token bucket value of 0.7, and the refill rate of 0.1 for Path 3 may be added to the token bucket value of 0.7 for Path 3 to obtain a refilled token bucket value of 0.8 for Path 3. The global token bucket refill rate of 0.8 (e.g., the sum of the token bucket value refill rates −0.1+0.2+0.4+0.1=0.8) is added to the updated global token bucket value to obtain a refilled global redundancy value of 0.9. As shown, although the global redundancy value is less than one, the global token bucket may be set to one to ensure selection of at least one path and transmission of the data path on the selected at least one path.

After the path token bucket values have been refilled, the number of paths to be selected for transmitting packets may be determined by the global token bucket value. As shown in FIG. 4B, the floor integer value of the global token bucket value may be used to determine the number of paths to be selected. Accordingly, for Round N=2, one path (e.g., |1.0|=1) may be selected from Path 0, Path 1, Path 2, and Path 3 for transmitting packets of the data stream in Round N=2. In selecting the path for transmitting packets of the data stream, the path with the highest associated token bucket values may be selected. Accordingly, as shown in FIG. 4B, the path with the highest token bucket value, namely either Path 0 or Path 3, which both have token bucket values of 0.8, may be selected as the transmission path for Round N=2. After transmission of the packets on either Path 0 or Path 3, the token bucket value for the path on which the packets were transmitted, as well as the global token bucket value are updated. Accordingly, one is subtracted from the token bucket values associated with the selected path (e.g., Path 0 in the implementation illustrated in FIG. 4B) to obtain updated token bucket value of −0.2, and one is subtracted from the global redundancy value, to obtain an updated global redundancy value of 0.0.

As illustrated in FIG. 4B, paths may continue to be iteratively selected by refilling the path token bucket values for each path and the global token bucket value, selecting one or more paths based on the refilled path token bucket values and the refilled global token bucket value, and updating the path token bucket values associated with paths on which the packets were transmitted and the global token bucket value. As illustrated in FIG. 4B, exemplary embodiments of the present disclosure can continue to facilitate load balancing of the data packet across the “best,” least congested path, even in situations of poor network conditions.

FIG. 5 is a block diagram of an exemplary network link 500, according to exemplary embodiments of the present disclosure. The block diagram of FIG. 5 corresponds to the selection of paths and transmission of data packets on the selected paths in the exemplary embodiment detailed in FIG. 4A.

As shown in FIG. 5 (and FIG. 4A), during period N=0, Path 1 and Path 2 were selected, based on the refilled path token bucket values and the global token bucket value, and data packets 520-1 were transmitted on selected Path 1 and Path 2. Similarly, during period N=1, Path 0, Path 2, and Path 3 were selected, based on the refilled path token bucket values and the global token bucket value, and data packets 520-2 were transmitted on selected Path 0, Path 2, and Path 3. Additionally, during period N=2, Path 1 and Path 2 were selected, based on the refilled path token bucket values and the global token bucket value, and data packets 520-3 were transmitted on selected Path 1 and Path 2; during period N=3, Path 0, Path 1, and Path 2 were selected, based on the refilled path token bucket values and the global token bucket value, and data packets 520-4 were transmitted on selected Path 0, Path 1, and Path 2; during period N=4, Path 0 and Path 2 were selected, based on the refilled path token bucket values and the global token bucket value, and data packets 520-5 were transmitted on selected Path 0 and Path 2; during period N=5, Path 2 and Path 3 were selected, based on the refilled path token bucket values and the global token bucket value, and data packets 520-6 were transmitted on selected Path 2 and Path 3; during period N=6, Path 0, Path 1, and Path 2 were selected, based on the refilled path token bucket values and the global token bucket value, and data packets 520-7 were transmitted on selected Path 0, Path 1, and Path 2; during period N=7, Path 1 and Path 2 were selected, based on the refilled path token bucket values and the global token bucket value, and data packets 520-8 were transmitted on selected Path 1 and Path 2; during period N=8, Path 0, Path 1, and Path 2 were selected, based on the refilled path token bucket values and the global token bucket value, and data packets 520-9 were transmitted on selected Path 0, Path 1, and Path 2; and during period N=9, Path 2 and Path 3 were selected, based on the refilled path token bucket values and the global token bucket value, and data packets 520-10 were transmitted on selected Path 2 and Path 3.

FIG. 6 is a flow diagram of an exemplary adaptive path selection process 600, according to exemplary embodiments of the present disclosure. According to exemplary embodiments of the present disclosure, adaptive path selection process 600 may be performed in connection with the selection of one or more data paths for the transmission of data packets forming a data stream that includes images, video, and/or other information captured by imaging devices, cameras, sensors, etc. of a vehicle that is being remotely operated in circumstances where it may be desirable to limit network costs, facilitate scaling, and the like.

As shown in FIG. 6, exemplary adaptive path selection process 600 may begin at step 602, where network information may be determined and/or obtained. According to exemplary embodiments of the present disclosure, network information such as a data rate for the data stream, a congestion value (e.g., congestion window, etc.), a data in flight value, a feedback period, and the like, may be determined and/or obtained for each candidate path. For example, the data rate may be determined based on the data stream being captured by the imaging devices, cameras, sensors, etc. of a vehicle that is being remotely operated, a congestion value (e.g., congestion window, etc.) may be determined based on timing associated with previously sent packets, a data in flight value may be determined based on feedback received regarding the receipt of previously sent packets, and the feedback period may be determined on the timing associated with when feedback is received based on previously sent packets.

After the network information has been determined and/or obtained, in exemplary implementations of the present disclosure that employs a token bucket technique (e.g., a token bucket for each candidate path), a refill rate may be determined for the path token buckets associated with each path, as well as for a global token bucket, as in step 604. The refill rate for the token bucket associated with each path may represent for example, a redundancy capacity and/or a margin that is available for the path before the path becomes congested. In exemplary implementations, the refill rate for each path may represent a redundancy capacity for each path and the global redundancy refill rate may be a sum of the refill rates for all available paths. According to exemplary embodiments of the present disclosure, the refill rate for each candidate path may be represented as:

In step 606, the global token bucket value may be refilled, according to the respective refill rates. For example, the global token bucket may be refilled by adding the global redundancy refill rate to the global token bucket.

After the global token bucket value has been refilled, in step 608, it may be determined whether the global token bucket value exceeds a threshold value. If the global token bucket value exceeds the threshold value, the global token bucket value may be capped at a redundancy limit value, as in step 610.

In step 612, it may be determined if any of the path token bucket values exceeds a token bucket limit cap. In the event that one or more of the path token bucket values exceeds the token bucket limit cap, only the token bucket values not exceeding the cap are refilled, as in step 614. If none of the path token bucket values exceeds the token bucket limit cap, the token bucket values are refilled for each path, as in step 616.

In step 618, one or more of the candidate paths are selected to determine the transmission paths based on the path token bucket values and the global token bucket value. For example, the N number of paths having the greatest path token bucket value may be selected, where N refers to a floor integer value of the global token bucket value. Accordingly, in an exemplary implementation where the global token bucket value is 3.7, the three transmission paths having the highest path token bucket values may be selected. After selection of one or more paths, data packets may be transmitted on the selected paths.

In step 620, after transmission of the packets, the path token bucket values for the selected paths and the global token bucket value may be updated. For example, the path token bucket values for each selected path may be subtracted by one (e.g., to represent the transmitted packet) and the global token bucket value may be reduced by the total number of transmitted packets. Continuing the above example where three transmission paths were selected and three packets were transmitted, the path token bucket values for each selected path on which a packet was transmitted may be subtracted by one and the global token bucket value may be subtracted by three.

In step 622, it may then be determined if the current feedback period has completed with the receipt of the next feedback information. In the event that the next feedback has not been received, exemplary adaptive path selection process 600 returns to step 606. Alternatively, adaptive selection process 600 returns to step 602, so as to determine and/or obtain updated current network information, which may be used to determine updated refill rates and an updated global redundancy value.

FIG. 7 is a flow diagram of an exemplary adaptive path selection and data rate adaptation process 700, according to exemplary embodiments of the present disclosure. According to exemplary embodiments of the present disclosure, adaptive path selection process 700 may be performed to modify and/or adjust the incoming data rate of an incoming data stream in connection with the selection of one or more data paths for the transmission of data packets forming the data stream that includes images, video, and/or other information captured by imaging devices, cameras, sensors, etc. of a vehicle that is being remotely operated in situations where network conditions do not permit a desired redundancy in connection with the data stream.

As shown in FIG. 7, exemplary adaptive path selection process 700 may begin at step 702, where network information may be determined and/or obtained. According to exemplary embodiments of the present disclosure, network information such as a data rate for the data stream, a congestion value (e.g., congestion window, etc.), a data in flight value, a feedback period, and the like, may be determined and/or obtained for each candidate path. For example, the data rate may be determined based on the data stream being captured by the imaging devices, cameras, sensors, etc. of a vehicle that is being remotely operated, a congestion value (e.g., congestion window, etc.) may be determined based on timing associated with previously sent packets, a data in flight value may be determined based on feedback received regarding the receipt of previously sent packets, and the feedback period may be determined on the timing associated with when feedback is received based on previously sent packets.

After the network information has been determined and/or obtained, in exemplary implementations of the present disclosure that employs a token bucket technique (e.g., a token bucket for each candidate path), a refill rate may be determined for the token buckets associated with each path, as well as for a global token bucket, as in step 704. The refill rate for each path may represent for example, a redundancy capacity for each path and/or a margin, in terms of tokens, that is available before the path becomes congested. For example, the refill rate for each path may represent a redundancy capacity for each path and the global token bucket refill rate may be a sum of the refill rates for all paths. According to exemplary embodiments of the present disclosure, the refill rate for each available path may be represented as:

In step 706, the current redundancy of data packets being achieved may be compared to a target and/or desired redundancy. According to exemplary embodiments of the present disclosure, the current redundancy may be represented by the global token bucket refill rate and the target and/or desired redundancy may be the amount of redundancy desired in connection with the data packets via the one or more selected paths, a redundancy limit, as described in connection with FIG. 6, and the like. In exemplary implementations of the present disclosure, the comparison of the current redundancy to the target redundancy may be represented as a ratio, percentage, and the like.

In step 708, the comparison of the current redundancy to the target/desired redundancy (e.g., in the form of a ratio, percentage, etc.) may be compared against a limit or threshold value. In the event that the comparison of the current redundancy to the target/desired redundancy is not less than the limit/threshold value, the incoming bitrate is not modified, and exemplary method 700 proceeds to step 716, where the path token bucket values for each path and the global token bucket may be refilled according to their respective refill rates. For example, the corresponding refill rates may be added to the value of the path token buckets corresponding to each candidate path and the global token bucket value.

After the token bucket values have been refilled, in step 718, one or more of the candidate paths are selected to determine the transmission paths based on the path token bucket values and the global redundancy value. For example, the N number of paths having the greatest path token bucket values may be selected, where N refers to a floor integer value of the global redundancy value. Accordingly, in an exemplary implementation where the global redundancy value is 3.7, the three transmission paths having the highest token bucket values may be selected. After selection of one or more paths, data packets may be transmitted on the selected paths.

In step 720, after transmission of the packets, the path token bucket values for the selected paths and the global token bucket value may be updated. For example, the path token bucket values for each selected path may be subtracted by one (e.g., to represent the transmitted packet) and the global token bucket value may be subtracted by the total number of transmitted packets across all selected paths. Continuing the above example where three transmission paths were selected and three packets were transmitted, the token bucket values for each selected path on which a packet was transmitted may be subtracted by one and the global token bucket value may be subtracted by three.

In step 722, it may then be determined if the current feedback period has completed with the receipt of the next feedback information. In the event that the next feedback has not been received, exemplary adaptive path selection process 700 returns to step 716. Alternatively, adaptive selection process 700 returns to step 702, so as to determine and/or obtain updated current network information, which may be used to determine updated refill rates and an updated global redundancy value.

In exemplary embodiments where the current redundancy to the target/desired redundancy is less than the limit/threshold value, the incoming data rate may be modified, as shown in FIG. 7. As illustrated, it may then be determined whether the data rate is within a defined range modified, as in step