(e.g., priority subsystem 118) may determine, based on the first predicted path, a priority of the first candidate. For example, a priority may be a relative importance compared to other candidates within the network. This prioritization may be particularly important in scenarios where network resources are constrained or specific data types need expedited handling. If a packet is marked with high priority, it may be processed and transmitted more swiftly by routers and switches, ensuring reduced latency and improved performance for time-sensitive data (e.g., voice or video streams). Conversely, packets with lower priority may experience delays or be queued during periods of high network congestion. Priority levels may be set according to the type of service field in a data packet's header, which routers and switches may then interpret to make routing decisions. For example, if a network were to support Quality of Service (QoS) protocols, these may actively manage packet priorities, ensuring that high-priority traffic, such as real-time communications, maintains its quality by receiving preferential treatment over less critical data, such as standard file transfers. Therefore, the priority assigned to a data packet significantly influences its pathways through the network, balancing the need for efficiency and the requirements of various types of network traffic.

In some embodiments, priority subsystem 118 may use a Markov decision process (MDP) to determine priority. The MDP may use probabilistic determinations to determine a priority of a candidate. For example, priority subsystem 118 may use the MDP to model the network's state and decision-making process as a series of probabilities and rewards. Each state in this context may represent a specific set of network conditions, such as traffic load, types of data packets waiting for transmission, available bandwidth, the status of network nodes, or other conditions. The actions may be the various routing decisions, including prioritization of certain packets. In such a model, transition probabilities may be crucial. These probabilities may represent the likelihood of moving from one network state to another after taking a specific action, for example, prioritizing a video packet over a standard data packet under high network congestion. As the network conditions change, these probabilities may dynamically adjust, reflecting the evolving nature of the network. In some embodiments, the MDP may include a reward function. This function may quantify the benefit or cost of taking certain actions in specific states. For example, prioritizing time-sensitive packets might yield a higher reward in states where network congestion is high, as it would help maintain QoS. By continuously observing the network's state and making decisions based on the expected rewards, an MDP-based system may aim to optimize packet routing. This optimization may involve assigning priorities to packets in a way that balances the need for efficient network utilization with the QoS requirements of different types of data. For example, a goal may be to ensure that high-priority packets, such as those for real-time applications, are routed expediently, especially under challenging network conditions, while still maintaining overall network performance.

In some embodiments, the MDP may use agent input. The agent may be an entity that interacts with the network to make routing decisions. For example, the agent may continuously observe the current state of the network. This may include monitoring network conditions such as traffic load, bandwidth availability, types of data packets in the queue, network latency, and the status of routers and switches. Based on these observations, the agent may then evaluate possible actions, such as assigning different priorities to packets. In scenarios where the agent employs reinforcement learning, the agent may learn from the outcomes of its actions. For example, if prioritizing certain types of packets led to improved network efficiency or maintained service quality, the agent may receive a positive reinforcement (e.g., reward), encouraging similar decisions in the future. Furthermore, the agent's decision-making may be dynamic, adjusting to fluctuating network conditions. If, for example, network congestion were to increase suddenly, the agent may prioritize packets differently compared to times of lower congestion.

In some embodiments, the MDP may rely on a model of a candidate's behavior (e.g., the first predicted path) to determine the priority of the candidate. For example, the first predicted path may be a personalized prediction of a candidate's behavior or pathway in the short term (e.g., 1-30 days). In some embodiments, the MDP may thus rely on first predicted path 306, as shown in FIG. 3, to determine priority. First predicted path 306 may illustrate future actions of the candidate, future value of the candidate, or other information about the candidate. The MDP may rely on first predicted path 306 when assessing which transitions are likely given a particular action in a particular state. For example, the MDP may assign transition probabilities based on the first predicted path. This may allow the MDP to assign a priority to a candidate that reflects the behavior or predicted path of the candidate. In some embodiments, the MDP may calculate priorities associated with counter-factual scenarios, thereby providing a more accurate prediction of priority. For example, the MDP may calculate the priority of the candidate based on one or more different pathways. The MDP may calculate the priority of the candidate based on different actions or behaviors (e.g., past or future) of the candidate. The MDP may thereby assess the priority of the candidate according to various counter-factual scenarios (e.g., various paths).

FIG. 4 illustrates a Markov decision process (MDP) 400, in accordance with one or more embodiments. MDP 400 may include state 402, state 404, state 406, state 408, state 410, and state 412. The states may represent traffic load, types of data packets waiting for transmission, available bandwidth, the status of network nodes, or other network conditions. In some embodiments, each status may be associated with a reward (e.g., R=0, R=1, R=−1, etc.). The arrows between states may be actions. For example, the arrows may represent routing decisions. As an illustrative example, state 402 may represent high network congestion and may be associated with a reward of R=−1. An action that prioritizes every data packet in a network as high priority may cause the system to remain in state 402 (e.g., high congestion), thus earning an agent a negative reward (e.g., R=−1). An action prioritizing the most important 50% of data packets may result in the state transitioning from state 402 to state 404. State 404 may represent medium network congestion and may be associated with a reward of R=1. The transition to state 404 may thus earn the agent a positive reward (e.g., R=1). The agent may take other actions to transition to state 406, state 408, or state 410. For example, an action prioritizing data packets according to a range of different levels of importance may result in the state transitioning from state 404 to state 406. State 406 may represent low network congestion and may be associated with a reward of R=2. The transition to state 406 may thus earn the agent a larger positive reward (e.g., R=2). In some embodiments, state 410 and state 412 may represent terminal states. For example, state 410 may represent a state in which one or more data packets have failed to reach their destination. For example, one or more data packets may have been lost or stuck somewhere in the network. State 412 may represent a state in which all data packets have been successfully routed from a source or a destination. In a terminal state, prioritizing packets for a given session is completed, and no further actions may be required in the context of the MDP.

As an illustrative example, priority subsystem 118 may use an MDP to determine value of people (e.g., customers) within an organization. This may involve modeling a person's interactions and behavior as a series of states and decisions. Each state in the MDP model may represent a different level of customer engagement or satisfaction, ranging from highly satisfied to at risk of leaving the organization. The actions may include various customer retention strategies, such as offering discounts, communication efforts, or other strategies. Transition probabilities in the MDP may reflect the likelihood of a person moving from one state of engagement to another, depending on the action taken. For example, if an organization were to offer a special discount to a person showing signs of reduced engagement (e.g., state 406), the probability of this customer moving to a more engaged state (e.g., state 404) may be considered in the model. Similarly, lack of action or an ineffective strategy may increase the probability of the person transitioning to a state closer to leaving the organization (e.g., from state 406 to state 408). The reward function in this MDP may quantify the benefit of retaining a customer at each engagement level. Actions that successfully enhance customer satisfaction and loyalty may yield higher rewards, reflecting an increased lifetime value and potential revenue from the customer. Actions that fail to engage or, worse, that alienate the customer, may result in lower rewards. By continually assessing the state of each customer and making decisions based on the expected rewards, an MDP may aim to optimize customer retention strategies based on how people in the organization are prioritized. The MDP may aim to set each person's priority such that the system can implement actions that maximize the likelihood of retaining valuable people, especially those at risk of leaving, while efficiently allocating resources to maintain overall satisfaction and loyalty. In some embodiments, state 410 and state 412 may represent terminal states. For example, state 410 may represent a person charging off, while state 412 may represent a cure for a person's situation. In some embodiments, the states may represent other scenarios in other contexts, and the MDP may be used to predict these and other transitions.

Returning to FIG. 1, priority subsystem 118 may determine a first confidence score for the first predicted path (e.g., first predicted path 306) based on the priority of the first candidate and a first distance associated with the first predicted path. For example, a first confidence score may indicate a likelihood that the data packet will follow the first predicted path. In some embodiments, the first confidence score may indicate a desirability of the data packet following the first predicted path. For example, based on the priority of the first candidate and the first distance of the first predicted path, priority subsystem 118 may assign a low confidence score based on determining that the first distance is too long given the priority of the first candidate. In some embodiments, priority subsystem 118 may determine the first confidence score further based on the first predicted destination. For example, priority subsystem 118 may assign a low confidence score based on determining that the first predicted destination is not appropriate given the priority of the first candidate. As an illustrative example, the first candidate may be a high-priority data packet and the first predicted destination may be a public server. Priority subsystem 118 may assign a low confidence score to the first predicted path based on determining that the public server is not an appropriate destination for the first candidate. For example, a secure server may be a more appropriate destination given the high priority.

Priority subsystem 118 may determine whether the first confidence score associated with selecting the first predicted path for the first message exceeds a threshold. For example, a threshold may be a minimum confidence in the first predicted path that is acceptable. In some embodiments, the threshold may differ based on priority. For example, higher-priority candidates may require higher confidence thresholds than lower-priority candidates. In some embodiments, the threshold may be predetermined, based on other factors, or based on a combination of factors.

Based on determining that the first confidence score for the first predicted path does not exceed the threshold, routing subsystem 116 may select a different path for the first candidate. In some embodiments, routing subsystem 116 may select the different path such that a second confidence score for the different path exceeds the threshold. For example, routing subsystem 116 may select the different path based on the priority of the first candidate and a second distance associated with the different path. For example, the priority of the first candidate may be low. Accordingly, routing subsystem 116 may select a different path that has a longer distance than the first predicted path. The longer distance may represent a lower cost associated with the different path. Thus, the longer path may be associated with the low-priority candidate. In some embodiments, routing subsystem 116 may select the different path further based on the different path connecting the source and the first predicted destination via a different route than the first predicted path. For example, routing subsystem 116 may select a different route that is shorter than the first predicted path, that travels along different nodes, or that otherwise travels a different route. In some embodiments, routing subsystem 116 may select the different path based on the different path connecting the source with a different destination than the first predicted destination. For example, the first predicted destination for a high-priority candidate may be a public server. Routing subsystem 116 may select a different path connecting the source with a secure server, as a secure destination may be more appropriate given the high priority of the candidate.

FIG. 5 illustrates a second predicted path 506 between a source 504 and a second destination 508, in accordance with one or more embodiments. In some embodiments, FIG. 5 may include a plurality of paths 500. In some embodiments, plurality of paths 500 may be the same as plurality of paths 300, as shown in FIG. 3. Plurality of paths 500 may include path 502 that a candidate has already followed to reach source 504. Source 504 may be a current location or a new starting point of the candidate. In some embodiments, multiple paths may connect source 504 with second destination 508, including second predicted path 506. In some embodiments, second destination 508 may represent a new outcome for the candidate. For example, second destination 508 may be a desired outcome for the candidate. As an illustrative example, source 504 may represent a current status of a person within an organization. For example, the current status may include two accounts, one of which is overdrawn. In some embodiments, second predicted path 506 may represent a series of future actions for the person. For example, nodes along second predicted path 506 may include on-time payments, payment extensions, customer service offers, or other actions. In some embodiments, second destination 508 may include an outcome expected for the person. For example, second destination 508 may include the person paying off all balances owed to the organization.

In some embodiments, routing subsystem 116 may re-direct a candidate along the different path. Returning to FIG. 1, communication subsystem 112 may receive a plurality of rules for re-directing candidates to selected paths. In some embodiments, the rules may include external rules, such as local, state, and federal regulations. As an example, external rules may include prohibitions against contacting certain candidates using certain methods of contact. In some embodiments, the rules may include internal organization rules. For example, internal rules may include a prohibition against multiple contacts to a candidate within one day.

In some embodiments, routing subsystem 116 may determine a strategy for re-directing a first candidate from a first predicted path to a different path. As an illustrative example, the strategy may involve methods for contacting the first candidate. Methods for contacting may involve a medium of contact (e.g., telephone, in-person, text, email, or other methods of reaching the candidate). The methods of contact may include content of the contact (e.g., a particular message that is delivered, how that message is delivered, what information is or is not included with the message, or other content). The strategy may further involve a schedule for contacting the first candidate (e.g., a time, a date, a frequency, or another schedule). For example, the schedule may include an interval for contacting the candidate and which methods of contact will be used for each instance of contact. In some embodiments, the methods of contact may include circumstances surrounding the contact (e.g., contacting the person at a workplace, on a payday, on rainy days, or other circumstances). In some embodiments, the methods of contact may account for scenarios related to the reason for contact (e.g., a day prior to an escalation in interest rate, a day prior to a debt being transferred to a third-party collection agency, etc.). An example schedule may include emailing the candidate one week before a due date, texting the candidate two days before the due date during work hours, and calling the candidate on the day of the due date before work hours. In some embodiments, the methods of contact may include user-initiated contact strategies, provider-initiated contact strategies, trigger-based contact strategies, or other types of strategies. In some embodiments, various strategies may be associated with different costs. For example, different methods of contact may have different costs for an organization. In some embodiments, more contacts may be costlier to the organization than fewer contacts. In some embodiments, the strategy may include offering various rewards, extensions, or other incentives to a person in order to re-direct them. These incentives may each have an associated cost. In some embodiments, routing subsystem 116 may determine the strategy for re-routing the candidate based on the costs associated with different strategies. In some embodiments, routing subsystem 116 may compare the costs of a strategy to an outstanding debt or a probability of a result using the contact strategy before determining that a particular contact strategy is available for the person.

In some embodiments, routing subsystem 116 may compare the strategy with the received rules to determine whether the strategy violates any rules. If the strategy does not violate the rules, routing subsystem 116 may approve the strategy. In some embodiments, a candidate may be on a do-not-call list. In this case, a contact strategy involving calling the candidate may violate the rules. Routing subsystem 116 may determine a different strategy and may compare the different strategy to the rules to determine that it does not violate the rules. Once the different strategy does not violate the rules, routing subsystem 116 may approve the different strategy.

In some embodiments, the strategy may be adjusted or canceled based on actions or updates from the candidate. For example, priority subsystem 118 may monitor the first candidate for updates to the priority of the first candidate. As an example, a first candidate's priority may change dynamically based on actions taken by the first candidate. For example, a candidate may open a new account, close an account, overdraw an account, make a payment, call customer service, request an extension, or take another action. Each action or a combination of actions may affect the candidate's priority. In some embodiments, the first candidate's priority may change based on actions of other candidates. For example, the first candidate's priority may be relative to other candidates' priorities and thus a change to other candidates' priorities may affect the first candidate's priority. Priority subsystem 118 may detect an updated priority of the first candidate. Based on detecting the updated priority, routing subsystem 116 may select an updated path for the first candidate. Routing subsystem 116 may select the updated path based on the updated priority of the first candidate and a third distance associated with the updated path. For example, the third distance may represent a different cost associated with routing the first candidate along the updated path. In some embodiments, routing subsystem 116 may select the updated path using any of the methods previously discussed herein. In some embodiments, routing subsystem 116 may select the updated path such that a third confidence score for the updated path exceeds a threshold.

As an illustrative example, message routing system 102 may determine strategies for directing people within an organization from a current state to a desired outcome. Routing subsystem 116 may use a first machine learning model to determine, for a first person, a first predicted path. The first predicted path may represent a series of future actions that the person is predicted to take. For example, the first predicted path may involve the person missing payments, overdrawing accounts, and eventually charging off. Priority subsystem 118 may determine, based on the first predicted path, a priority of the first person using a Markov decision process. Message routing system 102 may determine whether the first predicted path is appropriate for the first person based on the priority of the first person. For example, the person may be valuable to the organization but the first predicted path may lead the person to leave the organization. Priority subsystem 118 may thus determine that the first predicted path is not appropriate for the person based on the person's high priority. Based on determining that the first predicted path is not appropriate, routing subsystem 116 may select a different path for the person. The different path may connect the source with a different destination (e.g., the person remaining in the organization and continuing to contribute value). For example, the different path may involve the person making on-time payments, contacting customer service, and ultimately maintaining their accounts with the organization. Message routing system 102 may select the different path based on the priority of the person, the costs of pursuing the first predicted path versus the different path, and the different destination. Thus, routing subsystem 116 may re-direct the person down a path (e.g., the different path) that is appropriate given the priority of the person. In some embodiments, routing subsystem 116 may additionally account for rules (e.g., local, state, and federal regulations, internal organization rules, or other rules) and may re-direct the person using methods that are in compliance with those rules. For example, if a strategy for re-directing the person involves contacting the person with payment reminders but the person is on a do-not-call list, communication subsystem 112 may select alternative methods of contact (e.g., email) to communicate with the person. Communication subsystem 112 may then initiate permissible methods of re-directing the person to the different path.

FIG. 6 shows an example computing system 600 that may be used in accordance with some embodiments of this disclosure. In some instances, computing system 600 is referred to as a computer system 600. A person skilled in the art would understand that those terms may be used interchangeably. The components of FIG. 6 may be used to perform some or all operations discussed in relation to FIGS. 1-5. Furthermore, various portions of the systems and methods described herein may include or be executed on one or more computer systems similar to computing system 600. Further, processes and modules described herein may be executed by one or more processing systems similar to that of computing system 600.

Computing system 600 may include one or more processors (e.g., processors 610a-610n) coupled to system memory 620, an input/output (I/O) device interface 630, and a network interface 640 via an I/O interface 650. A processor may include a single processor, or a plurality of processors (e.g., distributed processors). A processor may be any suitable processor capable of executing or otherwise performing instructions. A processor may include a central processing unit (CPU) that carries out program instructions to perform the arithmetical, logical, and I/O operations of computing system 600. A processor may include a graphics processing unit (GPU) that supports a CPU. A processor may execute code (e.g., processor firmware, a protocol stack, a database management system, an operating system, or a combination thereof) that creates an execution environment for program instructions. A processor may include a programmable processor. A processor may include general or special purpose microprocessors. A processor may receive instructions and data from a memory (e.g., system memory 620). Computing system 600 may be a uni-processor system including one processor (e.g., processor 610a), or a multi-processor system including any number of suitable processors (e.g., 610a-610n). Multiple processors may be employed to provide for parallel or sequential execution of one or more portions of the techniques described herein. Processes, such as logic flows, described herein may be performed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and generating corresponding output. Processes described herein may be performed by, and apparatus can also be implemented as, special purpose logic circuitry, for example, an FPGA (field-programmable gate array) or an ASIC (application-specific integrated circuit). Computing system 600 may include a plurality of computing devices (e.g., distributed computer systems) to implement various processing functions.

I/O device interface 630 may provide an interface for connection of one or more I/O devices 660 to computing system 600. I/O devices may include devices that receive input (e.g., from a user) or output information (e.g., to a user). I/O devices 660 may include, for example, a graphical user interface presented on displays (e.g., a cathode ray tube (CRT) or liquid crystal display (LCD) monitor), pointing devices (e.g., a computer mouse or trackball), keyboards, keypads, touchpads, scanning devices, voice recognition devices, gesture recognition devices, printers, audio speakers, microphones, cameras, or the like. I/O devices 660 may be connected to computing system 600 through a wired or wireless connection. I/O devices 660 may be connected to computing system 600 from a remote location. I/O devices 660 located on remote computer systems, for example, may be connected to computing system 600 via a network and network interface 640.

Network interface 640 may include a network adapter that provides for connection of computing system 600 to a network. Network interface 640 may facilitate data exchange between computing system 600 and other devices connected to the network. Network interface 640 may support wired or wireless communication. The network may include an electronic communication network, such as the Internet, a local area network (LAN), a wide area network (WAN), a cellular communications network, or the like.

System memory 620 may be configured to store program instructions 670 or data 680. Program instructions 670 may be executable by a processor (e.g., one or more of processors 610a-610n) to implement one or more embodiments of the present techniques. Program instructions 670 may include modules of computer program instructions for implementing one or more techniques described herein with regard to various processing modules. Program instructions may include a computer program (which in certain forms is known as a program, software, software application, script, or code). A computer program may be written in a programming language, including compiled or interpreted languages, or declarative or procedural languages. A computer program may include a unit suitable for use in a computing environment, including as a stand-alone program, a module, a component, or a subroutine. A computer program may or may not correspond to a file in a file system. A program may be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files that store one or more modules, subprograms, or portions of code). A computer program may be deployed to be executed on one or more computer processors located locally at one site or distributed across multiple remote sites and interconnected by a communication network.

System memory 620 may include a tangible program carrier having program instructions stored thereon. A tangible program carrier may include a non-transitory computer-readable storage medium. A non-transitory computer-readable storage medium may include a machine-readable storage device, a machine-readable storage substrate, a memory device, or any combination thereof. A non-transitory computer-readable storage medium may include non-volatile memory (e.g., flash memory, ROM, PROM, EPROM, EEPROM), volatile memory (e.g., random access memory (RAM), static random access memory (SRAM), synchronous dynamic RAM (SDRAM)), bulk storage memory (e.g., CD-ROM and/or DVD-ROM, hard drives), or the like. System memory 620 may include a non-transitory computer-readable storage medium that may have program instructions stored thereon that are executable by a computer processor (e.g., one or more of processors 610a-610n) to cause the subject matter and the functional operations described herein. A memory (e.g., system memory 620) may include a single memory device and/or a plurality of memory devices (e.g., distributed memory devices).

I/O interface 650 may be configured to coordinate I/O traffic between processors 610a-610n, system memory 620, network interface 640, I/O devices 660, and/or other peripheral devices. I/O interface 650 may perform protocol, timing, or other data transformations to convert data signals from one component (e.g., system memory 620) into a format suitable for use by another component (e.g., processors