A standard regularizer in statistical learning literature is the [Image Omitted] penalty Ω(w)=λ∥w∥₁, which originates from LASSO. As a variant of LASSO, Adaptive LASSO also incorporates the prior knowledge of w into the LASSO. More specifically, given a prior w₀of w and w₀'s elementwise reciprocal r, the regularizer is given by Ω(w)=λ∥rew∥₁. The idea of Adaptive LASSO is also used for the learning of vector autoregressive matrix A and partial correlation ρ. Both LASSO and Adaptive LASSO are convex relaxations of the [Image Omitted] regularizer Ω_ρ(ρ), which is equivalent to the best subset selection problem in regression. Best selection problem or its variants can be targeted via integer programming for large-size problem, an hybrid regularizer in the form of Ω(w)=λ₀∥w∥₀+λ₂∥w∥₂², can be considered.

Methodology

Problem Setup

Time Series Model

The system, for example, may consider a multivariate time series model with nonstandard multivariate Gaussian noise, given by

y
t

=

f
⁡
(

y

t
-
1

,
…

,

y

t
-
p

)

+

ε
t

,

ε
t

∼

𝒩
⁡
(

0
,
 

C

-
1

)

(
1
)

where each y_t∈ [Image Omitted] ^Ndenotes the target variables of N different companies at time t, C∈^N×Nin (1) denotes the inverse covariance matrix of the Gaussian error, and f:⊗_k=1^p^N→^Ndenotes the prediction function given the past p realizations of the target variables. The goal is to learn the function f and the inverse covariance matrix C from the data set {y_t}. Here, the system can further impose sparsity assumption on the inverse covariance matrix C, meaning that there are only a few strong conditional correlation between different companies.

Specification and Assumption

For financial time series applications, the target variable y_tcan be stock returns, stock volatilities or bond yields of N different companies; it can be option, future or derivative prices of N different assets; it can also be currency exchange rate of different countries/regions, etc. f can either be a simple linear model (e.g., vector autoregression (VAR)) or a sophisticated neural network architecture (e.g., long short term memories (LSTMs), Graph Convolutional Networks). For illustration purpose, two classes of functions are described. Other classes or functions can be considered. One class is the high-dimensional linear models—the first-order Vector Autoregression (VAR), i.e., f(y_t−1)=Ay_t−1. The other class can be a single-layered graph convolutional network (GCN) based on knowledge graph. This is a more structural linear function class compared to VAR. Other types of neural network and/or GCN can be implemented.

In financial markets, there are common systematic risks associated with either the whole market or a certain market sector. Therefore, simply assuming the sparsity structure of C is not enough. For example, financial econometrics and statistics can use factor models to remove the market and sector trend and impose the structured assumption on the residual part of the factor model instead. After this detrending process, the residual can be considered as the individual risk of each company. It can be assumed the sparse inverse covariance structure of the residual. In an aspect, the system can use the residual part after removing the market/sector factors as a target variable y_tand assume the inverse covariance matrix C is sparse.

A challenge of learning this graphical time series model (1) is jointly learning both time series prediction function f and the inverse covariance structure C. With a large pool of companies in the market, there are a large amount of parameters in f and C, but, on the contrary, the data samples for these time series learning are limited. The system in an aspect may provide solutions with the help of knowledge graph information.

Learning Error Correlations: A Pseudolikelihood Framework

The sparsity of the inverse covariance matrix C is closely related to Gaussian graphical model, which assumes the partial correlation structure forms a graph. Specifically, the partial correlation ρ_ijbetween ε_itand ε_jthas following expression

ρ
ij

=

corr
(

ε
it

,
 

ε
jt

⁢

❘
"[LeftBracketingBar]"

{

ε
kt

:
k

≠
i

,
j

}

)

=

-

c
ij

c
ii

⁢

c
jj

.

(
2
)

For example, there can be two approaches to estimate the sparse inverse covariance matrix. One method is through the penalized likelihood method, particularly Graphical Lasso. However, the Graphical Lasso is a positive semi-definite (PSD) optimization problem. Since there is a need to jointly learn f and the PSD matrix C, the problem cannot scale well when the number of companies is large.

Another approach exploits the conditional distribution of multivariate Gaussian variables, and avoids positive semidefinite program by pseudo-likelihood method. Given {ε_kt: k≠i}, the conditional distribution of ε_itis

ε
it

|

{

ε
kt

:
k

≠
i

}

∼

𝒩
⁡
(

∑

k
≠
i

ρ

i
⁢
k

⁢

c

k
⁢
k

c
ii

⁢

ε
kt

,

1

c
ii

)

.

(
3
)

The pseudo-likelihood method considers maximizing an approximation of the likelihood function, PL(θ;{ε_t})=Σ_t,ilog p_θ(ε_it|ε_−it), and the estimator is shown to be consistent under certain regularity condition. For Gaussian graphical model, according to (3), the pseudo-likelihood becomes least-squares between ε_itand {ε_kt: k≠i}. More specifically, ignoring the constants, the pseudo-likelihood function is

PL
⁡
(

ρ
,
c

)

=

∑

t
,
i

{

log
⁢

c

i
⁢
i

-

c

i
⁢
i

(

ε
it

-

∑

k
≠
i

ρ

i
⁢
k

⁢

c

k
⁢
k

c
ii

⁢

ε
kt

)

2

}

To further learn the graph connectivity pattern, the system may impose sparsity inducing penalty on ρ's.

Furthermore, the system may consider a more general loss

ℓ
⁡
(

ρ
,
f
,

c
;

y

t
-
p

t

)

=

∑

i
=
1

N

w
i

(

y
it

-

f
i

(

y

t
-
1

,
…

,

y

t
-
p

)

+

∑

h
=
1

,

h
≠
i

N

ρ

i
⁢
h

⁢

c

h
⁢
h

c
ii

⁢

(

f
h

(

y

i
-
1

,
…

,
 

y

t
-
p

)

-

y
ht

)

)

2

,

(
4
)

where w_i's are nonnegative weights.

It is worth noting that when c is fixed, (4) can be decomposed into N subproblems, which reduces to the column-by-column estimator. Furthermore, when the weights w_iare taken to be equal to c_ii, this loss (4) is equivalent to the pseudo-likelihood function PL(ρ, c). Therefore, a statistical interpretation is provided for (4) as an extension of pseudo-likelihood loss. However, since the system may learn c and f and ρ at the same time, the optimal solution of c to this formulation does not necessarily minimize the original pseudo-likelihood loss, because of the missing logarithmic terms in (4). Therefore, the system may correct c in each step to ensure c_ii⁻¹is the conditional variance of ε_itgiven ε_−it. (details are shown in Algorithm 1).

This loss (4), along with some regularization on f and ρ, leads to the final optimization problem that is looked into, given by

L
⁡
(

ρ
,

f
;
c

)

=

1
T

⁢

∑

t
=
1

T

ℓ
⁡
(

ρ
,

f
;
c

,

y

t
-
p

t

)

+

Ω
f

(
f
)

+

Ω
ρ

(
ρ
)

(
5
)

for some prespecified function class and regularizers Ω_f, f_ρ. Here, Ω_fis a regularizer that controls the complexity and provides certain interpretability of the function class. For a 1st-order VAR model, f(y_t−1)=Ay_t−1, and a regularizer will be Ω_f(f)=λ_A∥B⊙A∥₁, where B is the elementwise reciprocal of the pre-estimator Â and ⊙ is the elementwise multiplication operation. As for a graph convolution network (GCN) model, a regularizer is specified, for example, below.

For Ω_ρ, it induces the sparsity structure of ρ (i.e. C). Additionally, in this context, knowledge graph can serve as guideposts for efficient variable selection in inverse covariance estimation in the form of additional priors on the partial correlation parameters, described below.

Masked Regularizers Ω_ρ

In an aspect, since the system is jointly learning the time-series model f and the partial correlation structure ρ with (5), the problem may suffer from the issue of limited sample size, as there are at least O(N²) parameters but only O(T) observations. Also, the nonconvex objective with large number of parameters may lead to slow learning process. Therefore, based on the sparsity inducing regularizers, the system may take advantage of knowledge graphs to provide additional information on the sparsity pattern of ρ. This can help reduce the number of parameters and improve the optimization efficiency.

The knowledge graph information describes some connectivity/similarity between companies, and the system can compute a weight matrix representing the degree of this connectivity, so the system can use these weights to inform the sparsity pattern of the inverse covariance matrix C and thus restrict the number of parameters in the partial correlation ρ.

In an embodiment, specifically, given a knowledge graph, the system creates a symmetric weight matrix G so that each entry G_ijmeasures how strong the connectivity between i and j is in the knowledge graph. Given a prespecified sparsity level K, for any company i, the system can define its neighbor N(i) as those companies j's that have top K weights G_ijamong j≠i. For some knowledge graph, entries of the weight matrix G can only take value among a few numbers. In this case, taking top K weights for each i may incur a lot of ties, and an alternative way to create the neighbors is by thresholding on the whole weight matrix G.

Given the neighbors N(i) for each i, the system may define hard masking matrix M^hardas follows

M
ij
hard

=

{

1
,

if
⁢

i

∈

N
⁡
(
j
)

⁢

or
⁢

j

∈

N
⁡
(
i
)

∞
,

otherwise

.

(
6
)

Let G_maxbe the maximum entry of G, the system may define soft masking matrix M^softas follows

M

i
⁢
j

soft

=

{

G

m
⁢
ax

G
ij

 
,

if
⁢

i

∈

N
⁡
(
j
)

⁢

or
⁢

j

∈

N
⁡
(
i
)

∞
,

otherwise

.

(
7
)

In the cases where the weight matrix is binary or integer, additional handling of soft masking matrix can be performed.

These masking matrices—M∈{M^hard, M_soft}—are used to impose modified regularization penalty on ρ. As the masking entry becomes larger, the penalty imposed on the corresponding entry of ρ increases. ∞ masking entry essentially zeros out the corresponding element in the ρ parameter matrix, reducing the model complexity. With these additional masking weights, the system may re-define our new masked regularizers as follows (In the table, r is the elementwise reciprocal of the prior knowledge/pre-estimator of ρ).

TABLE 1

Summary of Masked Regularizers

Ω_ρ(ρ)

[Image Omitted]

λ₀ Σ_{i < h} M_ih1[ρ_ih≠ 0] + λ₂∥ρ∥₂²

Lasso
λ₁∥Meρ∥₁

Adaptive Lasso
λ₁∥Mereρ∥₁

In an aspect, reduction in model complexity allows for a significantly faster algorithm for the optimization problem, because these masking matrices ensure sparser gradient computations in the optimization scheme. This improvement in efficiency can be significant, for example, in cases where there are a large pool of companies in the market to consider (e.g., there are about 3,000 stocks traded on New York Stock Exchange (NYSE)). In an aspect, the model complexity for the partial correlation component only grows as [Image Omitted] (KN) as opposed to (N²) without any knowledge graph.

Graph Convolutional Networks

The model complexity of the forecasting component, f in general high-dimensional time-series model, presented above, grows quickly as higher time lags are considered in f with high-dimensional vector autoregressive models. The model requires estimation of pN²parameters for the autoregressive matrices in addition to N(N−1)/2 parameters for partial correlation. This high-dimensionality, coupled with limited sample sizes, may lead to overfitting in high-dimensional VARs. The availability of knowledge graphs can help correct for large model complexity in VARs in a transparent way with the use of graph-based learning approaches. The graph information can restrict the really high-dimensional linear class of functions explored by higher-order VARs. In an embodiment, according to this modelling approach for time-series prediction, the past time-samples of multivariate time-series are processed with graph operations, e.g., graph convolution, before being fed into a linear model of significantly reduced model complexity. This model compression aspect of graph-based linear models over high-dimensional VARs can provide for improved predictive performance, for example, in financial network applications with limited data.

In an aspect, a single-layered Graph Convolution Network (GCN) falls under the linear model class. In an aspect, the following can be defined as the time-series forecasting function f(y_t−1, . . . , y_t−p, G) for GCN:

y
t

=

G
~

⁢

X
t

⁢
W

(
8
)

where X_t=[y_t−1, . . . , y_t−p] is the input data,

G
~

=

D
~

1
2

(

G
+
I

)

⁢

D
~

1
2

is the normalized graph matrix apriori defined based on the knowledge graph, {tilde over (D)}_ii=Σ_j{tilde over (G)}_ij, and W∈ [Image Omitted] is the weight vector to be estimated. The system may impose regularization on W, i.e. Ω_f(f)=λ_w∥W∥₂². In an aspect, this is an adaptation of the original Graph Convolution Networks to cater to additional (time) sample dimension at each node for learning with multiple time instances. In an aspect, a single-layered graph convolution network is a highly structured instance of a high-dimensional VAR model. This can be understood by expanding out the VAR and GCN functions as given below:

GCN
:

f
it

=

∑

k
=
1

p

∑

j
=
1

N

G
˜

ij

⁢

y

jt
-
k

⁢

w
k

⁢

VAR
:

f
it

=

∑

k
=
1

p

∑

j
=
1

N

A
ijk

⁢

y

jt
-
k

(
9
)

Comparing the two linear models, it can be seen that the product {tilde over (G)}_ijw_kin GCN are more specialized instances of the general A_ijkentries in the VAR case. The model complexity for the forecasting component reduces from N²p for VAR to p in GCN because of parameter sharing and availability of additional graph connectivity. This compression shifts the overall model complexity from the forecasting component (O(pN²)) in VARs with partial correlation to the correlation component (O(N²)) in GCN with partial correlation. The sample scenarios benefit from this model compression considering the knowledge graph has informative connectivity structure. Improved predictive performance can be shown on financial markets under a general time-series modelling approach.

Algorithm

An algorithm for (5) is developed, for example (but not limited to), for the time-series model such as VAR or single-layered GCN. Since both models are linear in their parameters, it can be assumed f is parametrized by θ and the corresponding Ω_f(f) becomes Ω_θ(θ). The algorithm is alternately updating w=(θ,ρ) by proximal gradient descent and updating c via the conditional variance formula (3). The algorithm is detailed below in Algorithm 1 in an embodiment.

Algorithm 1 - Algorithm for (5)

Input: Learning rate γ, Initialization θ⁽⁰⁾, ρ⁽⁰⁾, c⁽⁰⁾

1:
for k = 0, 1, 2, . . . do

2:
Update w^(k+1) = (θ^(k+1), ρ^(k+1)) by solving

min

θ
,
ρ

γ
2

⁢



w
-

w
~



2

+

Ω
θ

(
θ
)

+

Ω
ρ

(
ρ
)

,

(
10
)

where {tilde over (w)} = w^(k)− γ∇_wL(θ^(k), ρ^(k), c^(k))

3:
Compute ε_it^(k+1) = y_it− f_it(y_t−p^t−1; θ)

4:
Compute u_it^(k+1) via

u
it

(

k
+
1

)

=

ε
it

(

k
+
1

)

-

∑

j
≠
i

ρ
ij

(

k
+
1

)

⁢

c
jj

(
k
)

c
ii

(
k
)

⁢

ε
jt

(

k
+
1

)

5:
Compute c_ii^(k+1) = 1/Var(u_it^(k+1))

6:
end for

For solving the proximal problem (10), from the separability of the objective, for each component of θ, the proximal problem becomes the following problem

min
a

1
2

⁢

(

a
-

a
˜

)

2

+

λ
⁢

❘
"[LeftBracketingBar]"

a

❘
"[RightBracketingBar]"

,

which admits a closed form solution a*=sign(ã)(|ã|−λ)₊. This also applies to ρ in the cases where Ω_ρ is lasso or adaptive lasso penalty. In the [Image Omitted] case, the system can compute each ρ_ijby solving the following problem

min
a

1
2

⁢

(

a
-

a
˜

)

2

+

λ
0

⁢

1
[

a
≠
0

]

+

λ
2

⁢

a
2

,

which is given by

a
*

=

{

a
~

1
+

2
⁢

λ
2

if
⁢

a
~

2

>

2
⁢

λ
0

(

1
+

2
⁢

λ
2

)

0

if
⁢

a
~

2

<

2
⁢

λ
0

(

1
+

2
⁢

λ
2

)

{

0
,

a
~

1
+

2
⁢

λ
2

}

if
⁢

a
~

2

=

2
⁢

λ
0

(

1
+

2
⁢

λ
2

)

The following definition and theorem provide some insight on the convergence of the algorithm.

Definition 1 For a lower semi-continuous (lsc) function φ, its Fréchet subdifferential is defined as

∂
F

φ
⁡
(

w
0

)

=

{

ξ
:

liminf

h
→
0

⁢

φ
⁡
(

x
0

+
h

)

-

φ
⁡
(

x
0

)

-

〈

ξ
,
h

〉


h


≥
0

}

Let f(x)=g(x)+φ(x), where g is continuously differentiable and φ is lsc. The set of stationarity point of f is defined as

X
*

=

{

x
:

0

∈

∇

g
⁡
(
x
)

+

∂

φ
⁡
(
x
)

}

.

Theorem 1 Assume that ∇_wL(w; c) is σ-Lipschitz for any c. For any fixed c, if the system updates w^(k)iteratively via (10) with

γ
∈

(

0
,

1
σ

)

,

and let w be a limiting point of {w^(k)}, then w is a stationary point of L(⋅; c).

This theorem shows that if the system fixes c and keep updating w=(θ,ρ), the algorithm may converge to a stationary point. Since the system is updating c via the conditional variance formula, c will move towards the optimal c for pseudo-likelihood loss. At the same time, the update (10) will drive w towards the stationary point of L given c. Therefore, this provide heuristics on how Algorithm 1 may work.

In an aspect, an implementation of the algorithm exploits the sparsity structure of the masking matrix and performs sparse gradient computation and proximal update. Therefore, there can be optimization boost using the sparse graph masking matrix.

FIG. 1 is a diagram illustrating a method in an embodiment. The method can be implemented or run on one or more computer processors, for example, including hardware processors. Data 102 such as asset universe, historical financial indicators, other data such as news, knowledge graphs and/or parameter constraints can be received or obtained. For instance, there can be application programming interfaces (APIs) and/or data connectors 104, providing access to such data or information.

Selections or configurations 106 for modeling can also be received or obtained. For instance, a user may provide via a user interface configuration setting for asset selection, forecasting target selection (e.g., target prediction such as but not limited to, stock returns, stock volatilities or bond yields of N different companies, option, future or derivative prices of N different assets, currency exchange rate of different countries/regions, and/or others) and/or forecasting horizon (time line or period).

At 108, a multivariate time series model with Gaussian noise can be generated, defined or structured. An example is shown in Eq. (1) above.

At 110, partial correlation between two elements on the inverse covariance matrix of the Gaussian error can be obtained, generated, defined or structured. An example is shown in Eq. (2) above.

At 112, pseudo-likelihood method is constructed to estimate the elements in covariance matrix. An example is shown and described above as

PL
⁡
(

ρ
,
c

)

=

∑

t
,
i

{

log
⁢

c
ii

-

c
ii

(

ε
it

-

∑

k
≠
i

ρ
ik

⁢

c
kk

c
ii

⁢

ε
kt

)

2

}

⁢

where

ε
it

|

{

ε
kt

:

k

≠
i

}

∼

𝒩
⁡
(

∑

k
≠
i

ρ
ik

⁢

c
kk

c
ii

⁢

ε
kt

,

1

c
ii

)

as in Eq. (3).

At 114, given the neighbors N(i) for each i, hard or soft masking matrix is applied to partial correlation matrix ρ. At 116, the pseudo-likelihood for time series models can be maximized or optimized, where f can be VAR, NN, etc. times series model. At 114 and 116, joint optimization with respect to both the time series and inverse covariance structure can be performed. An example is shown and described in Eq. (4) and Eq. (5) above. In Eq. (5), for example, f can be any time series model such as VAR, LSTM, GCN and/or others. In Eq. (5), masked regularizer (Ω_ρ(ρ)) based on knowledge graphs is used to learn the partial correlation structure. Masked regularizers are shown and described in Eq. (6) and Eq. (7) above.

At 118, based on the trained model, financial indicator forecasting. At 120, based on the trained model, estimation of covariance structure can be performed. In an aspect, the financial indicator forecasting and estimation of covariance structure are performed jointly.

At 122, portfolio optimization can be performed. For example, running the trained model can predict asset allocation such as portfolio composition.

Experimental Results

Data

Stock Volatilities Time-Series

Experimental evaluation is performed on the combination of time-series prediction with Knowledge Graph in the context of market volatilities. The experiment considers two different financial markets S&P500 and S&P1500 and defines the daily volatility using the daily high and low stock price:

σ
~

it
2

=

0.361

(

log
⁢

p
it
high

-

log
⁢

p
it
low

)

2

(
11
)

where p_it^highand p_it^lowdenote the maximum and minimum price of stock i on day t. The experiment conditioned on the corresponding market index: S&P500 or S&P1500 and 9 sector indices. This reduces the number of companies to 383 for S&P500 and 1,009 for S&P1500 markets. In the experiment, the target variable y is the idiosyncratic volatility residuals computed via least squares optimization.

Knowledge Graph (KG) Using Cosearch

For the experiment, a KG is generated between firms by collecting cosearch of peer firms. Analysts collectively search for financial data on economically-related or similar firms. The cosearch of peer firms by users can explain a degree of similarity between firms. The experiment used cosearch to extract meaningful information about graph connectivity and construct a knowledge graph. For example, for each pair of firms (i,j), the number of unique users searching for both firms i and j is used to define a daily co-search proportion, which can be aggregated annually. These search peers may reflect the collective view of similarity between firms across all users. The experiment evaluates the effectiveness of KG in categorizing long-term contemporaneous correlation behavior through masked regularizers on partial correlation and identifying adjacency for forecasting in graph convolutional networks.

Other Knowledge Graphs

Other knowledge graphs can also be employed. Such knowledge graph may include information about entities and relationships (e.g., supply chain relationships), similarities among entities, and other metadata.

Setup

An example experimental setup considers two financial markets S&P500 and S&P1500, using daily stock volatilities residuals from 2013-2016. The experiment set 2013-2014 as the training period, 2015 as hyperparameter validation period and 2016 for final model evaluation. There are 504 days in the training period (as compared to the 383 companies for SP500 and 1,009 companies for SP1500) to evaluate proposed methodology on small sample size regime. In an aspect, the modelling approach is evaluated in terms of R²performance, sparsity of partial correlation structure, optimization runtimes' improvement and model compression (demonstrated by #params). The experiment follows R²definition where the computation of R²uses zero-mean prediction in the denominator (as the market and sector factors have been accounted for).

The experiment evaluated the utility of knowledge-graph based regularizers (constructed from different knowledge graphs) by comparison with various regularizers that do not use additional knowledge graph connectivity information. The experiment performed the optimization using the algorithm (Algorithm 1). The baseline scenario refers to non-masked regularizered models optimized via the algorithm; the shooting method for optimizing high-dimensional vector autoregressive models with partial correlation (VAR-PC) is not state-of-the-art for solving lasso-type problems and not easily generalizable to general time-series models beyond VAR forecasting components. The experiment also evaluated the effectiveness of aforementioned knowledge graphs as adjacency matrices in graph convolutional networks with partial correlation (GCN-PC).

Results Discussion

Table 2 presents results for prediction of daily volatilities residuals defined as data as described above. Table 2 confirms that the KG-masked regularizers provide faster optimization because of reduction in the number of partial correlation parameters. The optimization gains depend on the number of neighbors in the masking matrices. In this experiment, empirical investigations suggest that 75 peers are sufficient to ensure that there is no degradation in performance in terms of R²performance and provide significant computational gains (up to 3 and 10 times for S&P500 and S&P1500 markets respectively). There may be tradeoff between model complexity with computational scalability.

TABLE 2

Evaluating different KG-masked regularizers

for VAR-PC in predicting daily

volatilities for (S&P500).

R-type of

regularizer

Type

specified in

of

Table 1
nnz(ρ)
Speed-

Mask
Ω_ρ(ρ)
(%)
(%)
up

None

34.44
16.77
Baseline

Hard
Lasso
34.07
17.62
3.8 ×

Soft

34.51
13.42
2.9 ×

None

33.32
10.49
Baseline

Hard
Adaptive
33.59
9.88
2.6 ×

Soft
Lasso
33.56
9.25
2.6 ×

None

32.64
34.35
Baseline

Hard
[Image Omitted]

33.79
12.61
3.1 ×

Soft

33.65
8.35
3.0 ×

Additionally, the KG-input also results in more compact correlation structure as observed by the percentage of nonzeros of the recovered partial correlation ρ∈ [Image Omitted] ^N×Ndenoted by nnz(ρ) (%) ρ in the table. The masked-regularizers can give up to 5 times sparser partial correlation estimates. The partial correlation sparsity patterns under different regularizers can be visualized. The correlation heatmaps, which can be plotted show that although the values of ρ are different, different regularizers can give similar sparsity patterns in ρ, demonstrated by the correspondences between the spikes in ρ and the spikes in G. In an aspect, a pseudolikelihood framework disclosed herein uses knowledge graph as a guide to arrive at the true correlation structure.

Table 3 reports a graph convolutional network with partial correlation results. In an example experiment, 17% improvement in R², 9 times reduction in model complexity and 22% improvement in sparsity of partial correlation over VAR-PC are observed.

TABLE 3

Evaluating VAR-PC and GCN-PC in predicting daily

volatilities for (S&P500).

Model
R²
nnz(ρ)
#

Lasso(ρ)
(%)
(%)
params

VAR-PC
34.44
16.77
220k

GCN-PC (herein)
39.51
21.01
73k

GCN-PC-Hard (herein)
0.28
13.08
25k

GCN-PC-Soft (herein)
40.06
19.92
25k

In general, the experiment results show improvement in prediction performance, more computational efficiency, more sparsity on the covariance structure for structured explanation. Sparse granger causality interpretation via linear time model or structured GCNs can be performed.

A general framework is disclosed for jointly learning forecasting models (e.g., high-dimensional time-series models, GCNs), correlation structures with models based on knowledge graphs under a pseudolikelihood framework. Based on empirical results, the approach leads to improved prediction, reduced model complexity and computational efficiency. Hence, for example, computing and/or memory efficiency of a computer system can be improved.

FIG. 3 is another diagram illustrating a method in an embodiment. The method can be implemented or run on one or more computer processors, for example, including hardware processors. At 302, time series data is received. For example, data can be received via an API or another data connector, for example, remotely or locally. The data can be stored on a storage device.

At 304, one or more knowledge graph structures can be received. A knowledge graph structure can include nodes and edges. The nodes represent entities associated with the time series data and the edges representing relationships between the nodes connected by the edges. Nodes can also contain data about entities, e.g., metadata. An example knowledge graph structure is shown in FIG. 2.

At 306, a machine learning model is structure or generated to forecast a prediction using the time series data. The machine learning model can be structured to integrate the knowledge graph structure as an error term in the machine learning model.

At 308, the machine learning model is trained to forecast the prediction based on the time series data and the knowledge graph structure. The error term can be regularized for sparsity in training.

In an aspect, the machine learning model includes a neural network. In another aspect, the machine learning model can include a graph neural network such as, but not limited to, a graph convolutional network.

In an aspect, the knowledge graph structure is transformed to an inverse covariance matrix of a Gaussian error to be expressed as the error term in the machine learning model. The training in an aspect jointly learns a prediction function of the machine learning model and the knowledge graph structure integrated as the error term represented by the inverse covariance matrix of the Gaussian error.

In an aspect, regularizing includes using a soft mask including a real value. In another aspect, regularizing includes using a hard mask including a binary value. In an aspect, the machine learning model can be run to predict financial portfolio composition, for example, to optimize an investment portfolio.

In an aspect, a user interface can be provided for allowing a user to configure a type of the machine learning model, the time series data and the knowledge graph structure.

FIG. 4 is a diagram illustrating a tool for building a machine learning model in a cloud-based computing environment in an embodiment. Users such as analysts or portfolio managers 402, 404 may access the functionalities of the tool via a user interface and network such as an Internet and/or intranet 406, 408. Data and model repositories 410, 412 can store input and output data, such as the time series data, knowledge graph structure data, and also a plurality of machine learning models. API services 414, 416, 418 provide interfaces to a model service 420 building and/or running a model. A deep learning and/or machine learning computing service engine 422 works with the service 420 and a model builder 424 to provide a requested service, for example, building a machine learning model as described herein, and/or running a built machine learning model. The cloud-based system can be different types of cloud, e.g., as shown at 426. To end users, for example, a knowledge-based portfolio optimization system can be provided for forecasting asset selection, forecasting results and optimized portfolio.

FIG. 5 is a diagram showing a user interface of a tool for performing forecasting asset selection in an embodiment. The user interface, for example, can be run on a user machine (e.g., shown at 406, 418 in FIG. 5), which can include communication or network capabilities, for instance, for connecting and communicating with one or more API services on a remote system. In another aspect, the user interface can interact with a standalone system, for example, not limited to a cloud-based system. The display screen of the user interface shows an interface for performing forecasting asset selection. For example, asset class, market and sector can be selected for the time series model. Types of knowledge graphs can also be selected or configured. In an embodiment, more than one knowledge graph can be selected. For example, the user can select or directly upload list of assets to forecast its future and construct the portfolio. Running the built model can produce forecast prediction, for example, which can be visualized graphically. For example, a plot of prediction values can be provided along a time line, e.g., a 2-dimensional graph having x-y axis, one axis representing time and another axis representing the prediction value. Learned correlation structure (e.g., knowledge graph structure) among entities associated with the time series data can also be visualized, for example, with nodes and edges connecting the nodes having correlations.

FIG. 6 is a diagram showing components of a system in one embodiment that can perform knowledge graph-based learning, for example, for time series prediction, in an embodiment. The system, for example, can perform multi-dimensional financial prediction. One or more hardware processors 602 such as a central processing unit (CPU), a graphic process unit (GPU), and/or a Field Programmable Gate Array (FPGA), an application specific integrated circuit (ASIC), and/or another processor, may be coupled with a memory device 604, and generate a prediction model and forecast predictions. A memory device 604 may include random access memory (RAM), read-only memory (ROM) or another memory device, and may store data and/or processor instructions for implementing various functionalities associated with the methods and/or systems described herein. One or more processors 602 may execute computer instructions stored in memory 604 or received from another computer device or medium. A memory device 604 may, for example, store instructions and/or data for functioning of one or more hardware processors 602, and may include an operating system and other program of instructions and/or data. One or more hardware processors 602 may receive input, for example, including time series data and one or more knowledge graphs or knowledge graph structures. For instance, at least one hardware processor 602 may structure a machine learning model to forecast a prediction using the time series data, the machine learning model structured to integrate the knowledge graph structure as an error term in the machine learning model. At least one hardware processor 602 may train the machine learning model to forecast the prediction based on the time series data and the knowledge graph structure, at least one hardware processor 602 configured to regularize the error term for sparsity during training. In one aspect, time series data and/or knowledge graph structure data may be stored in a storage device 606 or received via a network interface 608 from a remote device, and may be temporarily loaded into a memory device 604 for building or generating the prediction model. The learned prediction model may be stored on a memory device 604, for example, for running by one or more hardware processors 602. One or more hardware processors 602 may be coupled with interface devices such as a network interface 608 for communicating with remote systems, for example, via a network, and an input/output interface 610 for communicating with input and/or output devices such as a keyboard, mouse, display, and/or others.

FIG. 7 illustrates a schematic of an example computer or processing system that may implement a system in one embodiment. The computer system is only one example of a suitable processing system and is not intended to suggest any limitation as to the scope of use or functionality of embodiments of the methodology described herein. The processing system shown may be operational with numerous other general purpose or special purpose computing system environments or configurations. Examples of well-known computing systems, environments, and/or configurations that may be suitable for use with the processing system shown in FIG. 7 may include, but are not limited to, personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, multiprocessor systems, microprocessor-based systems, set top boxes, programmable consumer electronics, network PCs, minicomputer systems, mainframe computer systems, and distributed cloud computing environments that include any of the above systems or devices, and the like.

The computer system may be described in the general context of computer system executable instructions, such as program modules, being run by a computer system. Generally, program modules may include routines, programs, objects, components, logic, data structures, and so on that perform particular tasks or implement particular abstract data types. The computer system may be practiced in distributed cloud computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed cloud computing environment, program modules may be located in both local and remote computer system storage media including memory storage devices.

The components of computer system may include, but are not limited to, one or more processors or processing units 12, a system memory 16, and a bus 14 that couples various system components including system memory 16 to processor 12. The processor 12 may include a module 30 that performs the methods described herein. The module 30 may be programmed into the integrated circuits of the processor 12, or loaded from memory 16, storage device 18, or network 24 or combinations thereof.

Bus 14 may represent one or more of any of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, and a processor or local bus using any of a variety of bus architectures. By way of example, and not limitation, such architectures include Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MCA) bus, Enhanced ISA (EISA) bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnects (PCI) bus.

Computer system may include a variety of computer system readable media. Such media may be any available media that is accessible by computer system, and it may include both volatile and non-volatile media, removable and non-removable media.

System memory 16 can include computer system readable media in the form of volatile memory, such as random access memory (RAM) and/or cache memory or others. Computer system may further include other removable/non-removable, volatile/non-volatile computer system storage media. By way of example only, storage system 18 can be provided for reading from and writing to a non-removable, non-volatile magnetic media (e.g., a “hard drive”). Although not shown, a magnetic disk drive for reading from and writing to a removable, non-volatile magnetic disk (e.g., a “floppy disk”), and an optical disk drive for reading from or writing to a removable, non-volatile optical disk such as a CD-ROM, DVD-ROM or other optical media can be provided. In such instances, each can be connected to bus 14 by one or more data media interfaces.

Computer system may also communicate with one or more external devices 26 such as a keyboard, a pointing device, a display 28, etc.; one or more devices that enable a user to interact with computer system; and/or any devices (e.g., network card, modem, etc.) that enable computer system to communicate with one or more other computing devices. Such communication can occur via Input/Output (I/O) interfaces 20.

Still yet, computer system can communicate with one or more networks 24 such as a local area network (LAN), a general wide area network (WAN), and/or a public network (e.g., the Internet) via network adapter 22. As depicted, network adapter 22 communicates with the other components of computer system via bus 14. It should be understood that although not shown, other hardware and/or software components could be used in conjunction with computer system. Examples include, but are not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archival storage systems, etc.

It is understood in advance that although this disclosure may include a description on cloud computing, implementation of the teachings recited herein are not limited to a cloud computing environment. Rather, embodiments of the present invention are capable of being implemented in conjunction with any other type of computing environment now known or later developed. Cloud computing is a model of service delivery for enabling convenient, on-demand network access to a shared pool of configurable computing resources (e.g. networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services) that can be rapidly provisioned and released with minimal management effort or interaction with a provider of the service. This cloud model may include at least five characteristics, at least three service models, and at least four deployment models.

Characteristics are as follows:

On-demand self-service: a cloud consumer can unilaterally provision computing capabilities, such as server time and network storage, as needed automatically without requiring human interaction with the service's provider.

Broad network access: capabilities are available over a network and accessed through standard mechanisms that promote use by heterogeneous thin or thick client platforms (e.g., mobile phones, laptops, and PDAs).

Resource pooling: the provider's computing resources are pooled to serve multiple consumers using a multi-tenant model, with different physical and virtual resources dynamically assigned and reassigned according to demand. There is a sense of location independence in that the consumer generally has no control or knowledge over the exact location of the provided resources but may be able to specify location at a higher level of abstraction (e.g., country, state, or datacenter).

Rapid elasticity: capabilities can be rapidly and elastically provisioned, in some cases automatically, to quickly scale out and rapidly released to quickly scale in. To the consumer, the capabilities available for provisioning often appear to be unlimited and can be purchased in any quantity at any time.

Measured service: cloud systems automatically control and optimize resource use by leveraging a metering capability at some level of abstraction appropriate to the type of service (e.g., storage, processing, bandwidth, and active user accounts). Resource usage can be monitored, controlled, and reported providing transparency for both the provider and consumer of the utilized service.