102 (e.g., images of the pathway on which the ego agent 102 is traveling). The one or more cameras of the environment sensing system 112 may be disposed at external front portions of the ego agent 102, including, but not limited to different portions of a front panel, a dashboard, a bumper, front lighting units, fenders, and a windshield. In one embodiment, the one or more cameras may be configured as RGB cameras that may capture RGB bands that are configured to capture rich information about global constraints that pertain to roadway lane markings, roadway/pathway markers, and/or roadway/pathway infrastructure (e.g., guardrails, walls), and the like that encompass the surrounding environment of the ego agent 102.

In other embodiments, the one or more cameras may be configured as stereoscopic cameras that are configured to capture environmental information in the form of three-dimensional images. In one or more configurations, the one or more cameras may be configured to capture one or more first person viewpoint RGB images/videos of the surrounding environment of the ego agent 102 from the perspective of the ego agent 102. In one embodiment, the environment sensing system 112 may be configured to convert one or more RGB images/videos (e.g., sequences of images) into image data that is communicated to the MPC application 106 to be analyzed.

In some embodiments, the environment sensing system 112 of the ego agent 102 may additionally or alternatively include a plurality of LiDAR sensors (not shown). The plurality of LiDAR sensors may be configured as one or more planar sweep lasers that include respective three-dimensional LiDAR sensors that may be configured to oscillate and emit one or more laser beams of ultraviolet, visible, or near infrared light toward the scene of the surrounding environment of the ego agent 102. The plurality of LiDAR sensors may be configured to receive one or more reflected laser waves (e.g., signals) that are reflected off one or more objects such as surrounding vehicles located within the surrounding environment of the ego agent 102. In other words, upon transmitting the one or more laser beams to the surrounding environment of the ego agent 102, the one or more laser beams may be reflected as laser waves by one or more obstacles that include static objects and/or dynamic objects that may be located within the surrounding environment of the ego agent 102 at one or more points in time.

In one embodiment, the environment sensing system 112 may be configured to analyze the reflected laser waves and output respective LiDAR data. The LiDAR data may include LiDAR coordinates that may be associated with the locations, positions, depths, and/or dimensions (e.g., measurements) of one or more traffic agents that may be located within the surrounding environment of the ego agent 102 that is communicated to the MPC application 106 to be analyzed.

In one embodiment, the neural network 108 may be hosted upon an external server 114 that may be owned, operated, and/or managed by an OEM, a third-party administrator, and/or a dataset manager that manages data that is associated with the operation of the MPC application 106. The external server 114 may be operably controlled by a processor 124 that may be configured to execute the MPC application 106. In particular, the processor 124 may be configured to execute one or more applications, operating systems, database, and the like. The processor 124 may also include internal processing memory, an interface circuit, and bus lines for transferring data, sending commands, and communicating with the plurality of components of the external server 114.

In one embodiment, the processor 124 may be operably connected to a memory 126 of the external server 114. Generally, the processor 124 may communicate with the memory 126 to execute the one or more applications, operating systems, and the like that are stored within the memory 126. In one embodiment, the memory 126 may store one or more executable application files that are associated with the MPC application 106.

In an exemplary embodiment, the memory 126 of the external server 114 may be configured to store the neural network 108. The neural network 108 may be configured as a long short-term memory recurrent neural network (LSTM neural network). As an LSTM neural network, the neural network 108 may execute machine learning/deep learning techniques to process and analyze sequences of data points such as image data and/or LiDAR data provided by the environment sensing system 112 of the ego agent 102 and dynamic data provided by the dynamic sensors 122 of the ego agent 102.

In an exemplary embodiment, the neural network 108 may include the variational autoencoders 110a and decoders 110b. In one embodiment, the variational autoencoders 110a may be configured as multi-pronged ensemble of variational autoencoders. Environment data and dynamic data may be input to the variational autoencoders 110a. Each of the autoencoders of the variational autoencoders 110a may model a single optimization process of the MPC, meaning the computational complexity increases linearly with the number of constraints. The variational autoencoders 110a may be configured to reduce an environment state dimension to a latent space.

The decoders 110b may be configured as multiple decoder prongs that are configured to decode data that is output by the variational autoencoders 110a. By conditioning on the latent environment state as well as specific optimizations through designed cost functions, the decoders 110b may be able to produce probabilistic action estimates which may be combined to form an approximate MPC solution.

II. The Probabilistic Model Predictive Control Application and Related Methods

Components of the MPC application 106 will now be described according to an exemplary embodiment and with continued reference to FIG. 1. In an exemplary embodiment, the MPC application 106 may be stored on the storage unit 116 and executed by the ECU 104 of the ego agent 102. In another embodiment, the MPC application 106 may be stored on the memory 126 of the external server 114 and may be accessed by a telematics control unit of the ego agent 102 to be executed by the ECU 104 of the ego agent 102.

FIG. 2 is a schematic view of an overall structure of the MPC application 106 according to an exemplary embodiment of the present disclosure. A general functionality of the MPC application 106 will now be discussed with respect to FIG. 2. In an exemplary embodiment, the MPC application 106 may include a plurality of modules 202-208 that may be configured to utilizing model predictive control for optimal interactions. The plurality of modules 202-208 may include a data reception module 202, an encoding decoding module 204, a control trajectory determinant module 206, and an agent control module 208. However, it is appreciated that the MPC application 106 may include one or more additional modules and/or sub-modules that are included in lieu of the modules 202-208.

FIG. 3 is a process flow diagram of a method 300 for outputting an estimated optimal control trajectory of the ego agent 102 according to an exemplary embodiment of the present disclosure. FIG. 3 will be described with reference to the components of FIG. 1 and FIG. 2 though it is to be appreciated that the method 300 of FIG. 3 may be used with other systems/components. The method 300 may begin at block 302, wherein the method 300 may include receiving environment data associated with the driving scene of the ego agent 102.

In an exemplary embodiment, at one or more past time steps and/or at a current time step, the data reception module 202 of the MPC application 106 may be configured to communicate with the environment sensing system 112 to collect image data associated with untrimmed images/video of the surrounding environment of the ego agent 102. In some configurations, the image data may pertain to one or more first person viewpoint RGB images/videos of the driving scene of the ego agent 102 captured at particular time steps. The image data may be configured to include rich information about object appearance that pertain to roadway lane markings, roadway/pathway markers, roadway/pathway infrastructure within the driving scene of the ego agent 102 at one or more time steps. In some embodiments, the data reception module 202 may package and store the image data on the storage unit 116 to be evaluated at one or more points in time. In one embodiment, the data reception module 202 may be configured to communicate the image data as environment data to the encoder decoder module 204 of the MPC application 106.

The method 300 may proceed to block 304, wherein the method 300 may include receiving dynamic data associated with the surrounding environment of the ego agent 102. In an exemplary embodiment, the data reception module 202 of the MPC application 106 may be configured to communicate with the dynamic sensors 122 of the ego agent 102. As discussed above, the dynamic sensors 122 may be associated with a real time dynamic operation of the ego agent 102 as it is traveling within the surrounding environment of the ego agent 102. The dynamic data may be provided in the form of one or more values (e.g., numeric levels) that are associated with the real-time dynamic performance of the ego agent 102 as one or more maneuvers are conducted and/or as the ego agent 102 is controlled to be autonomously driven. In some embodiments, the data reception module 202 may package and store the dynamic data on the storage unit 116 to be evaluated at one or more points in time. In one embodiment, the data reception module 202 may be configured to communicate the dynamic data to the encoder decoder module 204 of the MPC application 106.

The method 300 may proceed to block 306, wherein the method 300 may include inputting the environment data and the dynamic data to the variational autoencoders 110a. In an exemplary embodiment, the encoder decoder module 204 of the MPC application 106 may be configured to communicate the environment data that is associated with the global constraints of surrounding environment of the ego agent 102 to the neural network 108 to be inputted to the variational autoencoders 110a. Additionally, the encoder decoder module 204 may be configured to communicate the dynamic data associated with the dynamic operation of the ego agent 102 to the neural network 108 to be inputted to the variational autoencoders 110a.

In one embodiment, each of the autoencoders of the variational autoencoders 110a may model a single optimization process of the MPC, meaning the computational complexity increases linearly with the number of constraints. The variational autoencoders 110a may be configured to analyze the environment data and the dynamic data and reduce an environment state dimension to a latent space. The variational autoencoders 110a may be configured as a multi-pronged variational auto encoders that are conditioned on both the cost of moving from one state to another as well as the costs related to whether each state action pair produces optimal future interactions with respect to the operation of the ego agent 102.

Accordingly, the variational autoencoders 110 may be configured to reduce an environment state as well as specific optimizations through designed cost functions and input respective encoded data to the decoders 110b of the neural network 108. The decoders 110b of the neural network 108 may decode the encoded data to output probabilistic action estimates which may be combined to form an approximate MPC solution. In an exemplary embodiment, data pertaining to the probabilistic action estimates may be communicated to the encoding decoding module 204 of the MPC application 106. The encoding decoding module 204 may be configured to communicate respective data to the control trajectory determinant module 206 of the MPC application 106.

The method 300 may proceed to block 308, wherein the method 300 may include recombining outputs to form an estimated optimal control trajectory. In an exemplary embodiment, the control trajectory determinant module 206 may be configured to utilize the neural network 108 to place Gaussian assumptions on parameterized probabilistic outputs of the decoders 110b. Additional prior distributions with higher order statistical moments may be used as well. In particular, the data pertaining to the probabilistic action estimates output by the decoders 110b may be recombined with Gaussian assumptions to form an estimated optimal control trajectory that is associated with the ego agent 102 at one or more future time steps (t+1, t+2, t+n). The control loop is closed in normal MPC fashion by actuating the current control action while iteratively updating based on the sensed environment. The neural network 108 accordingly operates as an approximate MPC controller which may operate on highly nonlinear systems while retaining lower computational complexity and robust closed-loop stability. The neural network 108 may output data pertaining to the estimated optimal control trajectory that is associated with the ego agent 102 at one or more future time steps (t+1, t+2, t+n) to the control trajectory determinant module 206.

More specifically, the neural network 108 utilizes MPC to exploit suboptimal interactions to search a space of surrounding environment of the ego agent 102 for optimality, to learn system dynamics and constraints based on the dynamic data and environment data provided by the ego agent 102, and to learn the relative importance of optimality to one or more interacting agents. The modeling of the MPC by the neural network 108 allows the MPC application 106 to learn the optimal policy:

π*:S→A,π(s,a)=p(u_t=a|[x_t,r_t,e_t]=s)

which infers an optimal control action A, from the current state S which includes environment variables e_t, ego agent motion x_t, and reference optimizations r_t.

The policy is therefore learned from iterative samples (x,u) of demonstrated, likely stochastic, states and corresponding actions, typically recorded in demonstrations of the form: