>2 is a block diagram illustrating components of the compute service manager 108, in accordance with some embodiments of the present disclosure. As shown in FIG. 2, the compute service manager 108 includes an access manager 202 and a key manager 204 coupled to a data store 206 that stores access information. Access manager 202 handles authentication and authorization tasks for the systems described herein. Key manager 204 manages storage and authentication of keys used during authentication and authorization tasks. For example, access manager 202 and key manager 204 manage the keys used to access data stored in remote storage devices (e.g., data storage devices in data storage 104).

A request processing service 208 manages received data storage requests and data retrieval requests (e.g., jobs to be performed on database data). For example, the request processing service 208 may determine the data necessary to process a received query (e.g., a data storage request or data retrieval request). The data may be stored in a cache within the execution platform 110 or in a data storage device in data storage 104.

A management console service 210 supports access to various systems and processes by administrators and other system managers. Additionally, the management console service 210 may receive a request to execute a job and monitor the workload on the system.

The compute service manager 108 also includes a job compiler 212, a job optimizer 214, and a job executor 216. The job compiler 212 parses a job into multiple discrete tasks and generates the execution code for each of the multiple discrete tasks. The job optimizer 214 determines the best method to execute the multiple discrete tasks based on the data that needs to be processed. The job optimizer 214 also handles various data pruning operations and other data optimization techniques to improve the speed and efficiency of executing the job. The job executor 216 executes the execution code for jobs received from a queue or determined by the compute service manager 108.

A job scheduler and coordinator 218 sends received jobs to the appropriate services or systems for compilation, optimization, and dispatch to the execution platform 110. For example, jobs may be prioritized and processed in that prioritized order. In some examples, the job scheduler and coordinator 218 identifies or assigns particular nodes in the execution platform 110 to process particular tasks.

A virtual warehouse manager 220 manages the operation of multiple virtual warehouses implemented in the execution platform 110. As discussed below, each virtual warehouse includes multiple execution nodes that each include a cache and a processor.

Additionally, the compute service manager 108 includes a configuration and metadata manager 222, which manages the information related to the data stored in the remote data storage devices and in the local caches (e.g., the caches in execution platform 110). The configuration and metadata manager 222 uses the metadata to determine which storage units need to be accessed to retrieve data for processing a particular task or job. A monitor and workload analyzer 224 oversees processes performed by the compute service manager 108 and manages the distribution of tasks (e.g., workload) across the virtual warehouses and execution nodes in the execution platform 110. The monitor and workload analyzer 224 also redistributes tasks, as needed, based on changing workloads throughout the cloud data platform 102 and may further redistribute tasks based on a user (e.g., “external”) query workload that may also be processed by the execution platform 110. The configuration and metadata manager 222 and the monitor and workload analyzer 224 are coupled to a data store 226. Data store 226 in FIG. 2 represents any data repository or device within the cloud data platform 102. For example, data store 226 may represent caches in execution platform 110, storage devices in data storage 104, the metadata data store 113, or any other storage device or system.

In addition, as mentioned above, the compute service manager 108 includes a free pool sizing manager 109 that is responsible for maintaining the free pool from which different virtual warehouses can obtain VMs (also referred to as compute resources or servers). Further details regarding the functionality of the free pool sizing manager 109 109 are discussed below.

FIG. 3 is a simplified block diagram showing a framework for free pool provisioning, in accordance with some embodiments of the present disclosure. A plurality of virtual warehouses 302A-302Z are provided by the cloud data platform, as described above. Each virtual warehouse includes one or more execution nodes 312A-312AN (also referred to as virtual machines (VMs)), as described above. The virtual warehouses 302A-302Z may be provided for a plurality of accounts of the cloud data platform as described above.

The virtual warehouses 302A-302Z may be started, stopped, and resumed based on workloads. The execution nodes 312A-312AZ may be added to the virtual warehouses 302A-302Z from a free pool 304 or released from the virtual warehouses 302A-302Z back to the free pool 304. For example, when stopped or paused, the virtual warehouse may release its VMs back to the free pool. When the virtual warehouses are resumed, the virtual warehouse may add VMs from the free pool.

The free pool 304 may include preprovisioned VMs 306A-306N that have been provisioned from a cloud provider service 308 (e.g., AWS, Azure, GCP). The VMS 306A-306N are preprovisioned by having the software of the cloud data platform loaded and set up in advance so that they are made ready in advance before they are actually needed by a workload request. The free pool 304 serves as a buffer of preprovisioned VMs that are ready to be assigned to a virtual warehouse. The free pool 304 may add VMs from the cloud provider service 308 as demand increases, and the free pool 304 may release VMs to the cloud provider service 308 as demand decreases.

Next, different techniques for sizing the free pool are described. In cloud resource management, the fundamental challenge is to ensure that the free pool contains a sufficient number of servers to meet predicted future demand at any given time. The process may begin with a demand prediction that estimates the number of servers required for upcoming workloads based on historical and real-time data. Once this demand is forecasted, the system may determine not only how many VMs to provision, but also when to initiate provisioning to guarantee availability while minimizing idle costs.

In some aspects, the system can be configured to predict provisioning latency to optimize the size of the VM free pool. For example, the system may replace fixed or static time windows with real-time and dynamic provisioning durations. The system may also determine an overprovisioning factor where a greater number of VMs are provisioned as needed to ensure that the needed number of VMs are provisioned on time.

Determining dynamic provisioning times and an optimal overprovisioning factor provides substantial technical benefits for cloud resource management. By continuously analyzing real-time and historical data, the system can accurately predict when to initiate provisioning actions and how many extra VMs to launch beyond the immediate demand. These optimizations allow the platform to minimize the risk of delayed resource availability caused by unpredictable or long-tail provisioning latencies, while also reducing unnecessary idle time and associated costs. The dynamic approach ensures that resources are provisioned just early enough to meet service-level objectives, but not so early that they sit unused. The calculated overprovisioning factor increases the probability that the required number of VMs will be ready on time, even under adverse or variable cloud provider conditions. These techniques enable the system to deliver highly reliable, cost-efficient, and responsive cloud services, adapting in real time to changing workloads and infrastructure performance.

As described in further detail below, the predicting provisioning latency component can leverage advanced statistical and machine learning techniques to accurately estimate how long it will take to provision a specified number of VMS under current and anticipated conditions. By maintaining a historical database of provisioning events and continuously modeling real-time latency trends, the system can use a k-nearest neighbor (k-NN) approach to identify historical situations similar to the present context, such as time of day, workload, and recent provisioning activity. For each candidate provisioning lead time, the system can calculate a conservative probability that a VM will be ready in time and apply a binomial survival function to determine the optimal number of VMs to launch in order to meet a target confidence level (e.g., 99%). This approach enables the system to dynamically adjust both the timing and quantity of provisioning actions, accounting for long-tail latency events and cloud provider variability.

FIG. 4 is a flowchart of a method 400 for predicting provisioning latency, in accordance with some embodiments of the present disclosure. In some examples, the method 400 may be executed by a cloud data platform 102, in particular a free pool sizing manager 109, as described above. The objective or goal of method 400 is to estimate how many VMs to initiate provisioning (M) from the cloud provider service and when to start provisioning them (T_prov), so that the immediate demand (N) of VMs are provisioned by the target time (T) with a specified level of confidence (e.g., 99% confidence).

At operation 402, the system (e.g., cloud data platform 102, free pool sizing manager 109) determines a set of plausible or candidate pre-provisioning times based on N and T. Next, an iterative process for each candidate pre-provisioning times is executed to determine a value M for each candidate pre-provisioning time.

At operation 404, for the respective candidate pre-provisioning time, the system finds similar historical periods based on historical distribution of provisioning latencies. In some examples, a k-Nearest Neighbor (k-NN) approach may be employed to find historical periods (e.g., minutes) similar to the current context (e.g., time of day, day of week, number of VMs being provisioned, recent launches).

At operation 406, for each historical period, the system determines a probability that a single VM would complete within the pre-provisioning time based on current and historical data. A set of probabilities are generated based on the historical periods.

At operation 408, the system selects a specified probability. For example, the system may select a conservative probability (e.g., 10^thpercentile) to account for variance and risk.

At operation 410, the system determines a minimum number of VMS (M) to provision at the given pre-provisioning time so that at least N VMs are ready by the target time with the desired confidence C (e.g., 99% confidence) based on the selected probability. For example, the system applies a binomial survival function to determine a minimum number of VMs (M) to provision at the given pre-provisioning time so that at least N VMs are ready by the target time with the desired confidence C (e.g., 99% confidence), where the selected probability is a variable in the binomial survival function.

In some examples, the system may use other techniques, such as bootstrapping. For example, the system may identify a set of historical periods (using features such as time of day, workload, and recent activity) that closely match the present context. The system may then repeatedly sample from these historical periods, with replacement, to simulate multiple possible outcomes for provisioning events. By aggregating the results of these simulated samples, the system constructs an empirical distribution of provisioning latencies, which can be used to estimate the likelihood that a required number of resources will be ready within a given time window.

At operation 412, the system calculates the cost of the pair of M and pre-provisioning time. The calculated cost may include a cost metric associated with provisioning the servers at the specified pre-provisioning time. Operations 404-412 may be iteratively performed for each candidate pre-provisioning time in the set of candidate pre-provisioning times. At operation 414, the system selects the pair of M and pre-provisioning time that yields the lowest cost.

At operation 416, the system returns the selected M number of VMs to provision with optimal pre-provisioning time such that at least N number of VMs are provisioned by the target time with the specified confidence C.

FIG. 5 shows a graph showing an example relationship of pre-provisioning time, overprovisioning, and cost, in accordance with some embodiments of the present disclosure. The x-axis shows the various pre-provisioning times (T_prov) in seconds. Relative to the left axis, plot 502 shows the overprovisioning coefficient (how many times more instances to provision than the desired 100 instances). Relative to the right axis, plot 504 shows the relative cost (in instance-minutes).

With respect to the example in the graph of FIG. 5, to achieve 100 VMs with 99% confidence at the lowest cost proxy, the optimal (minimum cost proxy) solution is pre-provisioning time (T_prov) of 103 seconds, 1.58 overprovisioning coefficient (M/N) of 1.580 (i.e., M=158 VMs to provision) with a minimum cost proxy of 16274. In comparison, a naive solution (i.e., no overprovisioning so that M=N) would yield starting 100 VMs with a provisioning time of 325 seconds with 99% confidence and a cost proxy of about 32500. Thus, the naive strategy cost would be 99.7% more expensive than the optimal (minimum cost proxy) solution with over-provisioning.

FIG. 6 is a flowchart of a method 600 for predicting provisioning latency, in accordance with some embodiments of the present disclosure. In some examples, the method 600 may be executed by a cloud data platform 102, in particular a free pool sizing manager 109, as described above.

At operation 602, the system sets an objective for finding a cost-optimal solution, which includes a M number of servers (i.e., VMs) and start time (t_start) in order to obtain N servers by a target time T with confidence C, where N, T, and C are known and set.

At operation 604, the system defines a search space for t_prov, where a set of plausible pre-provisioning times, S_t_prov={t_prov₁, t_prov₂, . . . , t_prov_j}, is selected. This range can be heuristically determined from historical latency percentiles (e.g., P₅₀to P₁₀₀×factor).

At operation 606, an iterative process for each candidate pre-provisioning times is executed to determine a value M for each candidate pre-provisioning time (t_provE Stprov).

At operation 608, for the respective candidate pre-provisioning time, the system determines a start time for the pre-provisioning time, t_start=T-t_prov.

At operation 610, the system queries a k-NN model to find K historical minutes, Neighbors (t_start)={minute₁, . . . , minute_K}, that are most similar to the conditions at t_start. Similarity may be based on features like minute of hour, hour of day, day of week, recent launches, and M (the number of servers being provisioned).

At operation 612, the system calculates historical probabilities. For each neighboring historical minute (minute_k∈Neighbors(t_start)), the system determines the empirical probability p_hist_k(t_prov) that a single server launched in minute_kwould complete within t_prov. This may be done based on the actual latencies D_kof servers launched in minute_k:

$p_{{hist}_{k}} (t_{prov}) = \frac{count of l \in D_{k} such that l \leq t_{prov}}{total number of servers in D_{k}}$

At operation 614, the system selects a conservative probability (p_conservative). That is, from the set of probabilities {p_hist₁(t_prov), . . . , p_hist_K(t_prov)}, the system selects a conservative value. For example, the system may select the 10^thpercentile: p_conservative(t_prov)=P₁₀({p_hist_k(t_prov)}). The choice of percentile can be a function of Cost_VM.

At operations 616-622, the system determines an optimal M. For example, the system determines the smallest integer M≥N such that a binomial survival function meets the target confidence C: M*(t_prov)=min{M∈Z⁺|M≥N∧BSF(N, M, p_conservative(t_prov))≥C}. The target confidence C itself can also be a function of Cost_VM.

At operation 624, the system calculates the cost of calculated pair of M and pre-provisioning time, (t_prov, M*(t_prov)): Cost(t_prov, M*(t_prov))=M*(t_prov)×t_prov.

At operation 626, the system checks if there are more candidate pre-provisioning times. If yes, then operations 608-624 are performed for the additional candidate pre-provisioning times.

If there are no remaining candidate pre-provisioning times, the system, at operation 628, selects the overall optimal solution (e.g., optimal pair of M and pre-provisioning time that yields the minimal cost). For example, among the evaluated t_provvalues, the system may choose the pair

$(t_{prov}^{opt}, M^{opt})$

that yields the minimum cost:

$(t_{p r o v}^{opt}, M^{opt}) = \arg \min_{(t_{prov}, M^{*} (t_{p r o v}))} {Cost (t_{p r o v}, M^{*} (t_{p r o v}))} .$

At operation 630, the system returns the requested values based on the selected pair. For example, the system returns the optimal pre-provisioning time,

$t_{p r o v}^{opt};$

the optimal number of servers to launch, M^opt; and the optimal provisioning start time:

$t_{start}^{opt} = T - t_{p r o v}^{opt} .$

The system may also return the predicted latency distribution for the M servers launched a

$t_{start}^{opt},$

such that the system can determine when each of the N servers before T will arrive, as well as when the remaining M−N overprovisioned servers are likely to arrive. This latency distribution can be the cumulative distribution function (CDF) of the neighbor minute from which p_conservativewas chosen.

In another aspect of the present disclosure, the system may use rapid predictive preprovisioning techniques for managing the size of a free pool to optimize cost and service reliability. These techniques can continuously project future free pool states by analyzing predicted demand, anticipated server returns, and real-time provisioning latency. For each future time window, the system determines whether a deficit in available VMs is likely to occur. If a deficit is projected, a predictive provisioning latency model, such as those described above with reference to FIGS. 4-6, may be used to calculate the optimal time to start provisioning and the number of VMs to launch. Overprovisioning techniques can be used to account for long-tail latency events and to meet a specified confidence level. Based on the predictive provisioning latency model, if no immediate provisioning is required, the system may evaluate whether excess VMs can be safely released from the free pool without risking future shortages. The system can use a cooldown mechanism to prevent unnecessary churn. The rapid predictive preprovisioning techniques may also incorporate net demand and peak intra-window demand to ensure the free pool is robust against transient spikes as well as sustained load.

FIG. 7 is a flowchart of a method 700 for rapid predictive provisioning, in accordance with some embodiments of the present disclosure. In some examples, the method 700 may be executed by a cloud data platform 102, in particular a free pool sizing manager 109, as described above. The objective of method 700 is to determine, at each decision point, whether to provision or release VMs, and in what quantity, to maintain the free pool at an optimal size for upcoming demand.

At operation 702, the system projects the free pool size for each future time window. The system may have a plurality of future time windows for a defined future time period, such as 30 second future time windows for the next 4 minutes. The projected free pool size for each future time window may be based on a predicted net demand and peak demand. Net demand refers to the total change in the number of VMs required over a specific time window, calculated as the sum of all VMs needed minus any VMs returned to the free pool during that period. Net demand represents the overall increase or decrease in resource requirements from the start to the end of the window, providing a long-term view of how the free pool size should evolve.

Peak demand (or peak intra-window drawdown), on the other hand, captures the maximum number of VMs needed at any single point within the time window, regardless of any subsequent returns or reductions. It reflects the highest instantaneous resource requirement that occurs due to transient spikes in workload. Hence, net demand is used to project the overall trend in resource usage across time windows, while the use of peak demand ensures the system can absorb temporary spikes without running out of available VMs. The system may also factor in known VM returns (VMs expected to become available from virtual warehouses and be released back into the free pool).

At operation 704, for each future time window, the system determines whether a deficit of VMs in the free pool is projected. If no deficit across all future time windows is projected, the system may proceed to check if any VMs can be released (e.g., operation 712), as described in further detail below.

Consider an example scenario in FIG. 8, which shows an example scenario of projected free pool sizes and demands during future time windows. In this example scenario, there is a consistent demand of 100 servers (VMs) in each time window T. The system currently has 200 VMs in the free pool. However, based on the projections, if the system does nothing, the system will run out of VMs during window T₂(the free pool will be short 100 servers by the start of T₃). Hence, the system may need to provision VMs in advance such that they will be ready by time T₂.

Returning to FIG. 7, if a deficit is projected in respective future time windows, the system, at operation 706, determines the number of VMs to provision and their respective start time. In some examples, the system may use the predicting provisioning latency techniques described above (e.g., FIGS. 4-6). In some examples, the system may use different provisioning techniques, such as naive strategies with no overprovisioning or dynamic provisioning times.

At operation 708, the system may determine whether provisioning is needed immediately for any of the future time windows. For example, the system may check whether the start time for provisioning additional VMs for any of the future time windows is at or before the current time (e.g., before the next decision point), so that immediate provisioning is needed.

If immediate provisioning is needed, the system, at operation 710, may start provisioning of additional VMs based on the determined additional VMs needed. If, however, none of the future time windows require immediate provisioning, the system may proceed to check if any VMs can be released. The release process is described next.

At operation 712, the system may check whether a cooldown period from a previous free pool size adjustment is still underway. For example, the system may check whether the cooldown period expires before the next decision point. If the cooldown period is still underway, the system may maintain the current size of the free pool and update information related to the free pool accordingly (operation 718).

In some examples, the cooldown period check may be incorporated into future release decisions. For example, the system may include a cooldown period in its assessment of when a release activity might trigger the next provisioning action to occur. If the next provisioning action is after the cooldown period, the system may proceed with the release activity.

If the cooldown period has expired, the system, at operation 714, may sort release candidates set of VMs in descending order, from greatest to smallest number of VMs to release. At operation 716, the system may determine the maximum safe quantity of VMs to release, if any, based on the sorted list. For example, the system may check if future deficits can still be met without violating cooldown restraints if VMs are released. In some examples, the system may perform a binary search over the search space of the release candidates set to determine the quantity of VMs to release, if any. At operation 718, the system may update its free pool size by maintaining current size or releasing one or more VMs, if needed.

Method 700 may be continuously performed at each decision point. Notably, VMs are provisioned only when the system determines that, for a future deficit, the predicted start provisioning time is at or before the current time (or before the next decision point). If the predicted start provisioning time for the deficit is not at or before the current time (i.e., at or after the next decision point), the system does not start provisioning any VMs for that deficit. Instead, the system continuously runs the method 700 at each decision point because conditions, such as the predicted provisioning times and demand may change. Thus, the rapid predictive provisioning techniques can enable the cloud data platforms to maintain high availability of VMs in the free pool while significantly reducing idle resource costs.

FIG. 9 is a flowchart of a method 900 for a provisioning section of rapid predictive provisioning, in accordance with some embodiments of the present disclosure. In some examples, the method 900 may be executed by a cloud data platform 102, in particular a free pool sizing manager 109, as described above. The objective of method 900 is to calculate the number of servers P₀, if any, that should start provisioning at the current time, T₀, to meet future demands that cannot wait. Provisioning that does not need to start at T₀are not relevant because the system will not provision them now.

At operation 902, the system initiates the provisioning section of the rapid predictive provisioning algorithm at the current time T₀. The system initializes the provisioning quantity, setting P₀=0. This variable will accumulate the maximum number of servers needing provisioning initiated at T₀.

At operation 904, an iterative process for each future time window is executed to determine the provisioning quantity for the respective future time windows (k∈{0,1, . . . , K−1}).

At operation 906, the system calculates cumulative net demand, which is the total servers consumed (or returned) from T₀up to the beginning of window T_k: using advanced statistical methods:

$C_{D} (k) = \sum_{j = 0}^{k - 1} D_{j}$

C_D(k) represents the total change in the free pool size due to demands (D) from T₀until T_kbegins. Note: For k=0, C_D(0)=0 as the sum is empty.

At operation 908, the system calculates the cumulative net servers anticipated to complete provisioning C_s(k) between T₀and the start of T_k:

$C_{s} (k) = \sum_{j = 0}^{k - 1} S_{j}$

C_s(k) represents the total number of already provisioning servers that the system requested for in prior windows that will arrive by the start of T_k.

At operation 910, the system projects baseline free pool size if no action is taken by the system. The system calculates the free pool size at the start of T_kassuming no new provisioning (P₀=0) or releases (R₀=0) occur at T₀:

FP_k=FP₀−C_D(k)+C_s(k)

FP_krepresents the “do nothing now” projection of the free pool at the start of window k. The system takes the current free pool size FP₀minus the demand C_D(k), plus the already in-provisioning servers C_s(k) that would arrive by this time.

At operation 912, the system determines a required target level at the start of T_k:

target=B_min+Peak_K,

where Peak_Kis the predicted peak drawdown between T_kand T_k+1.

At operation 914, the system checks for a deficit by determining if the projected baseline free pool drops below the target level (FP_k<target). If there is no deficit, P₀remains at zero, and the system proceeds to processing the next future time window if any (operation 924).

If there is a deficit, the system, at operation 916, calculates the number of servers needed:

N_needed,k=target−FP_k

At operation 918, the system determines the provisioning start time for the needed servers. For example, the system may employ the latency predictor L, as described above, to determine when provisioning for N_needed,kmust begin to be ready by T_k:

T_start,k=L(T_k,N_needed,k)

The function L provides the cost-optimal time T_start,kto start provisioning of N_needed,kservers, such that they are available by the start of T_k.

At operation 920, the system determines if provisioning action is needed by comparing the provisioning start time with the current time (T_start,k≤T₀).

At operation 922, the provisioning quantity is determined:

P₀=max(P₀,N_needed,k)

If multiple future deficits (T_k1, T_k2, . . . ) require provisioning to start by T₀, P₀must be large enough to cover the single largest deficit amount them (N_needed,k_i). This assumes a single batch of P₀servers provisioned at T₀would cover these needs if P₀is greater than or equal to each such N_needed,k_i.

At operation 924, the system checks if there are more future time windows. If yes, then operations 906-922 are performed for the additional future time windows.

If there are no remaining future time windows, the system, at operation 926, determines if the provisioning quantity P₀that has been determined based on the set of future time windows is greater than 0 (P₀>0).

If P₀is greater than 0, the system, at operation 928, starts provisioning P₀servers, starting at T₀. Consequently, the system will not be releasing any servers at T₀(R₀=0).

If P₀is equal to 0, no immediate provisioning is required based on the current state. The system may consider releasing VMs, as described below.

FIG. 10 is a flowchart of a method 1000 for a releasing section of rapid predictive provisioning, in accordance with some embodiments of the present disclosure. In some examples, the method 1000 may be executed by a cloud data platform 102, in particular a free pool sizing manager 109, as described above. The system checks if any VMs can be safely released after the system determines that no VMs need to be provisioned at the current time (P₀=0). The objective of method 1000 is to determine the maximum R₀(servers to release at T₀) such that no future deficit is created that would require provisioning to start before T_{next_run}+Δ_cooldown. Hence, the release quantity is initialized, by setting R₀=0. This is the default (conservative) decision if the system is unable to find a safer larger release amount.

At operation 1002, the system defines the cooldown threshold based on the next run time (decision point) and the cooldown period:

T_{threshold_start_time}=T_{next_run}+Δ_cooldown

The cooldown threshold is the earliest time a provisioning task triggered by this release can safely start.

At operation 1004, the system generates set of candidate release amounts Candidates_R. These candidates are amounts that, if released, would cause the free pool at some future time T_Kto be exactly the target, considering only demands and arriving servers up to T_K.

Candidates_R={0}

For k∈{0,1, . . . ,K−1}:

- Let R_cand,k=FP₀−C_D(k)+C_s(k)−B_min.
  
  If R_cand,k>0 add R_cand,kto Candidates_R.
  
  Candidates_Ris a set, so it stores only unique values. Each R_cand,kis the number of servers that could be released if T_Kwere the only future constraint, aiming for FP_k=target. Releasing nothing (R=0) is always a candidate. We only consider positive release amounts. FP_o−target is effectively included via R_cand,0if C_d(0)=0. The

At operation 1006, the system sorts the Candidates_Rin descending order. This is an optimization to ensure the system tests larger potential releases first.

At operation 1008, the system tests each candidate release amounts R_testin the sorted candidate list Candidates_Rin an iterative outer loop of the process.

At operation 1010, the system checks the impact of the respective R_teston the future windows in an iterative inner loop of the process.

At operation 1012, the system projects free pool size with hypothetical release. For example, system calculates pool size at T_kif R_testservers are released:

FP_k=(FP₀−R_test)−C_D(k)+C_s(k)

At operation 1014, the system checks if the projected pool size would result in a deficit based on the target free pool size (FP_k<target). If no deficit is projected, then the system checks the next future window size, if any, in the inner loop (operation 1024).

If a deficit is projected, the system, at operation 1016, determines the servers needed by T_k:

N_needed,k=target−FP_k

- At operation 1018, the system determines the required provisioning start time for the projected deficit:
  
  T_start,k=L(T_k,N_needed,k)

At operation 1020, the system checks against cooldown threshold. For example, the system checks if the provisioning for this new deficit is before cooldown threshold (T_start,k≤T_{threshold_start}).

If the projected start time is less than or equal to the cooldown threshold, the system may break the inner loop, at operation 1022, and check the next candidate release amounts R_testin the sorted candidate list Candidates_R(operation 1008). The reason is that releasing R_testservers here creates a deficit at T_kthat requires provisioning sooner than the acceptable T_{threshold_start}(which incorporates T_{next_run}and Δ_cooldown). This violates the anti-churn/cooldown objective, and thus, R_testis considered an unsafe release amount

If the projected start time does not exceed the cooldown threshold, R_testis considered safe for that future time window. R_testis evaluated for the remaining future time windows in the inner loop (operation 1024). If after evaluating all k windows, R_testis considered a safe release amount, then R₀=R_testis set, at operation 1026, as the largest safe release quantity and is released. The system breaks from the outer loop (over R_test) because the system sorted candidates from largest first, this R_testis the maximum safe amount to release. Thus, the system determines this R_testwill not cause any unmanageable short-term deficits in any future window.

In view of the disclosure above, various examples are set forth below. It should be noted that one or more features of an example, taken in isolation or combination, should be considered within the disclosure of this application.

Example 1. A method comprising: determining, by a cloud data platform, a set of candidate pre-provisioning times for compute resources to be provisioned from a cloud provider service to a free pool maintained by the cloud platform, the set of candidate pre-provisioning times being based on a demand of compute resources (N) and a target time; for each candidate pre-provisioning time: finding similar historical periods based on historical distribution of provisioning latencies; for each historical period of the similar historical periods, determining a minimum number of compute resources (M) to provision at the candidate pre-provisioning time so that at least N compute resources are ready by the target time within a specified confidence; and calculating a cost metric for pair of M and candidate pre-provisioning time; and selecting a pair of M and candidate pre-provisioning time based on the calculated cost metrics; and causing the provisioning of the selected M number of compute resources at the selected pre-provisioning time from the cloud provider service to add to the free pool

Example 2. The method of example 1, wherein M is greater than or equal to N.

Example 3. The method of any of examples 1-2, wherein provisioning comprises loading software related to the cloud data platform on the provisioned compute resources.

Example 4. The method of any of examples 1-3, wherein finding the similar historical periods is performed using a k-nearest neighbor algorithm based on context features.

Example 5. The method of any of examples 1-4, further comprising: determining a probability that provisioning a single compute resource is completed within the candidate pre-provisioning time; and selecting a specified probability, wherein determining the minimum number of M compute resources is based on the specified probability.

Example 6. The method of any of examples 1-5, wherein determining the minimum number of M compute resources is performed using a binomial survival function where the specified probability is a variable in the binomial survival function or a bootstrapping technique.

Example 7. The method of any of examples 1-6, further comprising: projecting free pool size for each of a set of future time windows; for at least one future time window of the set of future time windows, determining a deficit of compute resources in the free pool is projected; determining whether to provision new compute resources at a current time based on a future number of compute resources and start time to meet the deficit by comparing the current time to the start time; initiating provisioning of the future number of compute resources if provisioning is needed at the current time; and not initiating provisioning if no provisioning is not needed at the current time.

Example 8. The method of any of examples 1-7, wherein the future number of compute resources and start time is determined based on an overprovisioning algorithm.

Example 9. The method of any of examples 1-8, further comprising: determining that no compute resources are to be provisioned at a current time; sorting release candidates in a release candidate set in descending order; determining a maximum safe quantity of compute resources to release based on the sorted release candidates; and releasing the maximum safe quantity of compute resources from the free pool to the cloud provider service.

Example 10. A system comprising: one or more processors of a machine; and a memory storing instructions that, when executed by the one or more processors, cause the machine to perform operations implementing any one of example methods 1 to 9.

Example 11. A machine-storage medium embodying instructions that, when executed by a machine, cause the machine to perform operations implementing any one of example methods 1 to 9.

FIG. 11 illustrates a diagrammatic representation of a machine 1100 in the form of a computer system within which a set of instructions may be executed for causing the machine 1100 to perform any one or more of the methodologies discussed herein, according to an example embodiment. Specifically, FIG. 11 shows a diagrammatic representation of the machine 1100 in the example form of a computer system, within which instructions 1116 (e.g., a software, a program, an application, an applet, an app, or other executable code) for causing the machine 1100 to perform any one or more of the methodologies discussed herein may be executed. For example, the instructions 1116 may cause the machine 1100 to execute any one or more operations of the free pool sizing methods described herein. As another example, the instructions 1116 may cause the machine 1100 to implement any one or more portions of the functionality illustrated in any one of FIGS. 1, 2, and 3. In this way, the instructions 1116 transform a general, non-programmed machine into a particular machine that is specially configured to carry out any one of the described and illustrated functions of the cloud data platform 102 such as the compute service manager 108 (or a component thereof such as the free pool sizing manager 109) or an execution node of the execution platform 110.

In some embodiments, the machine 1100 operates as a standalone device or may be coupled (e.g., networked) to other machines. In a networked deployment, the machine 1100 may operate in the capacity of a server machine or a client machine in a server-client network environment, or as a peer machine in a peer-to-peer (or distributed) network environment. The machine 1100 may comprise, but not be limited to, a server computer, a client computer, a personal computer (PC), a tablet computer, a laptop computer, a netbook, a smart phone, a mobile device, a network router, a network switch, a network bridge, or any machine capable of executing the instructions 1116, sequentially or otherwise, that specify actions to be taken by the machine 1100. Further, while only a single machine 1100 is illustrated, the term “machine” shall also be taken to include a collection of machines 1100 that individually or jointly execute the instructions 1116 to perform any one or more of the methodologies discussed herein.

The machine 1100 includes processors 1110, memory 1130, and I/O components 1150 configured to communicate with each other such as via a bus 1102. In an example embodiment, the processors 1110 (e.g., a central processing unit (CPU), a reduced instruction set computing (RISC) processor, a complex instruction set computing (CISC) processor, a graphics processing unit (GPU), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a radio-frequency integrated circuit (RFIC), another processor, or any suitable combination thereof) may include, for example, a processor 1114 and a processor 1112 that may execute the instructions 1116. The term “processor” is intended to include multi-core processors 1110 that may comprise two or more independent processors (sometimes referred to as “cores”) that may execute instructions 1116 contemporaneously. Although FIG. 11 shows multiple processors 1110, the machine 1100 may include a single processor with a single core, a single processor with multiple cores (e.g., a multi-core processor), multiple processors with a single core, multiple processors with multiple cores, or any combination thereof.

The memory 1130 may include a main memory 1132, a static memory 1134, and a storage unit 1136, all accessible to the processors 1110 such as via the bus 1102. The main memory 1132, the static memory 1134, and the storage unit 1136 store the instructions 1116 embodying any one or more of the methodologies or functions described herein. The instructions 1116 may also reside, completely or partially, within the main memory 1132, within the static memory 1134, within the storage unit 1136, within at least one of the processors 1110 (e.g., within the processor's cache memory), or any suitable combination thereof, during execution thereof by the machine 1100.

The I/O components 1150 include components to receive input, provide output, produce output, transmit information, exchange information, capture measurements, and so on. The specific I/O components 1150 that are included in a particular machine 1100 will depend on the type of machine. For example, portable machines such as mobile phones will likely include a touch input device or other such input mechanisms, while a headless server machine will likely not include such a touch input device. It will be appreciated that the I/O components 1150 may include many other components that are not shown in FIG. 11. The I/O components 1150 are grouped according to functionality merely for simplifying the following discussion and the grouping is in no way limiting. In various example embodiments, the I/O components 1150 may include output components 1152 and input components 1154. The output components 1152 may include visual components (e.g., a display such as a plasma display panel (PDP), a light emitting diode (LED) display, a liquid crystal display (LCD), a projector, or a cathode ray tube (CRT)), acoustic components (e.g., speakers), other signal generators, and so forth. The input components 1154 may include alphanumeric input components (e.g., a keyboard, a touch screen configured to receive alphanumeric input, a photo-optical keyboard, or other alphanumeric input components), point-based input components (e.g., a mouse, a touchpad, a trackball, a joystick, a motion sensor, or another pointing instrument), tactile input components (e.g., a physical button, a touch screen that provides location and/or force of touches or touch gestures, or other tactile input components), audio input components (e.g., a microphone), and the like.

Communication may be implemented using a wide variety of technologies. The I/O components 1150 may include communication components 1164 operable to couple the machine 1100 to a network 1180 or devices 1170 via a coupling 1182 and a coupling 1172, respectively. For example, the communication components 1164 may include a network interface component or another suitable device to interface with the network 1180. In further examples, the communication components 1164 may include wired communication components, wireless communication components, cellular communication components, and other communication components to provide communication via other modalities. The devices 1170 may be another machine or any of a wide variety of peripheral devices (e.g., a peripheral device coupled via a universal serial bus (USB)). For example, as noted above, the machine