In some examples, the relations matrix may also be used to provide understanding of the event that may be occurring. For example, if there is a 0.75 probability of attack states which have the word escalation in the database, the present approach may provide an analyst with the estimation that they may be now in a privilege escalation stage. This may provide the analyst with context about the activity which may be occurring and enable insight into an adversary's actions. If multiple words appear with probability above a threshold, they may all be provided.

In some examples, the analyst then may choose one of the keywords based on their estimation of the activities by the adversary. This may be used to collapse the potential states in the graph to those states which include the keyword or similar words. Similar words may be defined based on lexicographic stemming, manual annotation or co-occurrence in attack states or attack history. Other methods may also be used. For example, in some embodiments, the system may use the output of the algorithm with analyst input to take action. Such action may include slowing down particular types of activities, preventing certain types of activities, raising new alerts, or any other type of action.

FIG. 3 depicts a flow chart of a linguistic analysis procedure 300, according to example embodiments of the present disclosure.

At step 302, linguistic analysis module 110 of server system 104 may retrieve a threat intelligence text article automatically (e.g., through threat intel feeds, such as really simple syndication (RSS) feeds, spiders/web crawlers, or other methods of retrieving data). Alternatively, a user may manually retrieve the threat intel feeds. In one example, a search on the entity name may yield basic attack state as an initial search result. Name entity recognition may be performed by methods apparent to one of ordinary skill in the art. For example, at step 304, linguistic analysis module 110 may perform rule based parsing process that identifies entities name pairs, or their synonym defines a section. In one example, each combination of identified pairs of names may be truncated, leaving margins of X words before and after the names or based on specific stop text terms. The result may be a set of sections, each representing a different pair combination. Other methods of detecting sections may be used. At step 306, linguistic analysis module 110 may automatically mark up the identified entity names (i.e., attack step) which may be used where each marked-up section may be processed by the NLP ML process. Specifically, at step 308, linguistic analysis module 110 may define a test section for analysis per pair, while at step 310 linguistic analysis module 110 section may perform processing via ML. At step 312, linguistic analysis module 110 may then output pair relationships. In other words, the process output by linguistic analysis module 110 of server system 104 may be the identified relation type, (e.g., entity A followed by entity B, or entity B followed by entity A, where entity A may be an example of entity B, entity A may be in the same list of entity B). Other relation types may be of course possible.

In some embodiments, this matrix may represent the probabilities of cause-and-effect relationships between attack states (entities). These cause-and-effect relationships may be defined as “transitions” which may therefore be referred to as a “transition matrix”. The transition matrix may be generated/updated based on a process that analyzes the outputs of the linguistic model predictions and security expert inputs. This will be further discussed in more detail below.

Attack steps may be represented through different threat description conventions. However, one of the more effective and popular conventions may be based on the MITRE ATT&CK terminology. This terminology defines attacks, tactics, techniques, sub-techniques and software. In general, a tactic may represent a more general attack intent, while techniques and sub techniques may represent more specific attack steps that serves the intent. Software might be used to describe the tool that enables the execution of a technique or a sub-technique, and thus might provide a more accurate context to the attack stage, (e.g., spear-phishing attachment sub-technique with a key logger software that provides better understanding of the attacker's intent than specifying the attack state only by a sub-technique which doesn't indicate what the attacker's intent may be and the possible next steps). Such a representation may teach that the intent may be to infect a user machine with a malicious software that may be used for personal data theft. Although the solution may not be limited to a specific attack state naming convention, this disclosure uses the MITRE ATT&CK terminology as an example in order to represent an attack state (an entity).

FIG. 4 depicts a transition matrix 400, according to example embodiments of the present disclosure. Transition matrix 400 includes first column 402 and first row 404. The column represents the “FROM” entity, the row represents the “TO” entity, and the value in each cell represents the transition conditional probability between the entities. The ID of each row and column may uniquely define the entity as follows: 1^sttwo digits represent the tactic, digits 3 and 4 represent the attack technique, digits 5 and 6 represents the attack sub-technique. For simplicity reasons, software type representation was excluded from the entity representation. It should be also noted that the transition matrix may allow for representation of bi-directional transitions, (i.e., from A to B and B to A). The transition matrix may also define an entity that includes only tactic (without technique and sub technique), or tactic and technique (without sub technique). For example: entity ID 050000 may represent only a tactic, and entity ID 051400 may represent an attack state by its tactic and technique. It also should be noted that that entity might be defined with more characterization parameters such as attack state role (victim or performer). The role may teach that if the defined entity refers to the victim or to the performer of the malicious attack step. This information might influence the transition probability. For example, and in the form of MITRE terminology, a “brute force” attack technique may be typically followed with a high probability by a “privilege escalation” attack technique if it refers to the same victim entity. The transition probability in this example may be lower if the transition represents a victim (to the 1^stattack state) that turns into a performer of the 2^ndattack state.

FIG. 5 depicts a process 500 for transition matrix generation, according to example embodiments of the present disclosure. In step 502, linguistic analysis module 110 outputs the linguistic analysis in the form of from-to attack entity transition per each textual section analysis. The linguistic analysis output may be either derived from the threat intelligence reports (using linguistic analysis) or may be defined manually by the security expert based on their domain expertise. At step 504, transition matrix module 114 may perform an aggregation process that accumulates all from-to entity transitions pairs and creates transition relation statistics which represents the number of recognized unique entity transitions, (e.g., count each from entity ID and to entity ID, while the numbers of entities may be from 1 to N, where N represents the number of attack states. The output format is shown in FIG. 2A. With reference to FIG. 2A, the transition probability calculation, and the aggregations may be performed on type 1 (e.g. transition type).

At steps 506 and 508, transition matrix module 114 may perform normalization and probability calculations. For example, at step 506, transition matrix module 114 may compute upper stats propagation, while at step 508, transition matrix module 114 may compute regulated conditional probabilities to compute the transition matrix. In one example, a pre-normalized matrix T(k,w) may be created, a 2-d array of size N×N. This matrix describes the number of from-to entity transitions per each from-to entity (k,w). So ‘w’ represents the from entity ID, and ‘k’ the to entity ID, and where k≠w (as entities repetitions may not be predicted) so by default when k=0, T(k,w)=0. This may be because typically the method may not learn entity repetition probabilities from the threat intelligence reports. Transition counting may also be performed automatically according to entities pair profile, (e.g., if entity 1 is an example of entity 3, then pair counters may be incremented accordingly in the transition count, i.e., entity 3 count will be incremented each time entity 1 was recognized). Transition types 2 and 3 may be used for this pair profiling. Upper entity propagation may involve propagating transition counters of sub-techniques to its parent technique, and techniques transition counters to their parent tactic.

As the relation matrix may include entities IDs that define tactic-only, and technique-only (e.g., 010000 defines tactic only, and 000100 defines technique only) these tactics and techniques transition counters may be adapted in the example as follows: adapt tactics-only counters based on type 1 transitions counters of their child techniques, and adapt techniques-only counters based on type 1 transition counters of their child sub-techniques.

This process may be an upper propagation of counters that creates an upper propagation matrix (UT). FIG. 6A depicts UT 600, where 602 is the first column and 604 is the first row, according to example embodiments of the present disclosure. With reference to FIG. 6A, row 2 represents technique-only “A”, (e.g., phishing technique), whereas rows 5, 6, and 7 represent sub-techniques of technique “A”, (e.g., spear-phishing attachment, spear-phishing link, spear-phishing via service). Adapting the transition counters of row 2 (technique A) in column 1, may be defined as summing all transition counters of sub techniques of “A” in column 1, and setting the technique counter of “A” as a maximum (sum of sub techniques counter, current counter). In the case above, the technique counter within the 1^stcolumn may be set to 4. To complete the adaptation process, the same process may be repeated for each column in the relation matrix where adapting all techniques-only counters may be performed, and the whole process may be repeated again for tactics-only counters and their associated techniques. The results may be set for the final upper matrix of UT. A more formal definition of the upper propagation procedure may be given in Equation (1) below:

If ID_m is a parent node of ID_i1,ID_i2, . . . ,ID_iM, then

UT(m,k)=max(T(m,k),T(j_i1)+T(j_i2)+ . . . +T(j_iM) Equation (1)

Probability calculation and “regulated” probability approaches may include setting the layer of probabilities at the last stage in the creation of the transition matrix, which forms the result in Equation (2) below:

TrM={p(m,j)|1≤m,j≤N} Equation (2)

Each p(m,j) for m≠j may be a probability of transition from entity ID_j to entity with ID_m. The columns of the matrix may be relevant to “from” entities, while the rows may be relevant to “to” entities. The probability of transition may be defined in Equation (3) below:

p(m,j)=0, where m=j Equation (3)

The matrix elements may be considered as a conditional probabilities p(ID_m|ID_j) to get entity D_m after ID_j. An example of a straightforward approach may be to set p(m,j) in Equation (4) below:

p
⁡
(

m
,
j

)

=

T
⁡
(

m
,
j

)

∑

m
=
1

N

⁢

T
⁡
(

m
,
j

)

Equation
⁢

(
4
)

The above probability calculation may be a more naïve conditional probability function. The approach may be relevant if a set of entities is exclusive and may not be possible after entity ID_j to go to both ID_m1 and ID_m1. Setting the final conditional probability of the transitions may be performed based on the final adapted matrix UT. Normalizers may be used where each entity belongs to one of the hierarchy layers (L), where L=1 represents tactics, L=2 represents techniques, and L=3 represents sub-techniques. For each column k of matrix UT(m,k) (connected with information from ID_k) and each level L of hierarchy define norm(L,k) which is a sum of elements UT(i,k) for i such that entity ID_i is a node of level L in the hierarchy. So, norm(L,k)=sum({UT(i,k) Ilevel(Id_i)=L}). According to the definition, norm(L1,k)>norm(L2,k) if L1<L2 and norm(L1,k)>UT(i,k) if level(Id_i)=L1. So, norm(1,k) may be the largest from all normalizers.

In regulated conditional probability calculations, there may be some possible definitions of conditional probabilities. An example definition of conditional probabilities may be shown in Equation (5) below:

p(Id_k|Id_m)=UT(m,k)/norm(1,k) Equation (5)

By using this method, low probabilities of transitions in certain cases may be observed. In order to provide a more effective method, a “regulated” probability calculation may be used as described below.

Configurable conditional probabilities depend on an additional parameter alpha, where 0<alpha<½. This may be defined in the system configuration and formula for conditional probability which may be different for different levels of target entity (i.e., to entity) such as in Equation 6 below:

For ID_m of level 1:

p(Id_k|Id_m)=UT(m,k)/norm(1,k)

For ID_m of level 2:

p(Id_k|Id_m)=UT(m,k)/(1−alpha)*norm(2,k)+alpha*norm(1,k))

For ID_m of level 3:

p(Id_k|Id_m)=UT(m,k)/(1−alpha−alpha{circumflex over ( )}2)*norm(3,k)+alpha{circumflex over ( )}2*norm(1,k)+alpha*norm(2,k))

Generic formula for level L:

p(Id_k|Id_m)=UT(m,k)/norm(L,k,alpha)

Where norm(L,k,alpha)=(1−norm(L,k)*sum({alpha{circumflex over ( )}i|i=1, . . . ,L−1}+sum(norm(i,k){alpha{circumflex over ( )}i|i=1, . . . ,L−1}. Equation (6)

A reason for using the alpha and polynom functions in this example may be to increase the influence of higher-level entities such as sub-techniques elements. If these functions are not used, the probabilities of some of these elements may be too low.

In some embodiments, the probability model(s) may be generated and updated based on the transition matrix details. There may be a few prediction models that may be used separately, or combined, to generate predictions of attack states transitions. Each prediction model has its advantages and disadvantages. Therefore, it may be beneficial to create an ensemble multiple models that may allow increased prediction precision. For example, models may be combined using one or more techniques, such as boosting.

FIG. 6B depicts a quasi-linear prediction model 620, according to example embodiments of the present disclosure. As shown, Equation 622 may be in matrix/vector form. Transition matrix 624 may be a matrix of the previously described entity transitions in FIG. 5, where each element in the transition matrix 624 may describe the probability of transition from a specific entity (j) to entity (m), such that each T(m,j) element may represent the transition probability. It is noted that J and m may be in the range of 1 . . . N, while N may be the number of all possible entities. Evidence input vector 626 ‘E’ may include the set of the entities evidence (attack states). Each element ‘e’ may be assigned with a value between 0 to 1. A value of 1 may indicate that the entity exists, while a value of 0 may indicate that the entity doesn't exist. Any value between 0 to 1 may represent the level of probability (certainty) that the entity exists or does not exist. The sum of all probabilities may reach above 1 because there may be simultaneous attack states (e.g., both brute-force and spear phishing attachment threats techniques were discovered with high probability). This may be addressed by the “par” operator described below.

The quasi-linear operator 630 may multiply the input vector E(j) with each row in the matrix and sum the results in the form of the output prediction vector P(m) 628. The quasi operator 630 may normalize and sum the probabilities, so each output element may be in the range of 0 to 1.

The operation may be generally defined in Equation (7) below:

P
⁡
(
m
)

=

par
⁢

∑

j
=
1

N

⁢

T
⁡
(

m
,
j

)

×

E
⁡
(
j
)

Equation
⁢

(
7
)

The par operator may be defined as a probability addition rule in Equation (8) below:

P(A or B)=P(A)+P(B)−P(A and B) Equation (8)

FIG. 7 depicts a probability addition rule 700, according to example embodiments of the present disclosure. The par has associative and commutative properties as illustrated by 702 and 704, and shown in Equation (9) below:

A par B=B par A

(A par B)par C=A par(B par C) Equation (9)

It is also should be noted that the par operator characteristics may be consistent with matrix decomposition and summation so that the following Equation (10) may be true:

TrM_1@a par TrM_2@a=(TrM_1@+TrM_2)@a Equation (10)

This property may be useful for cases where it may be more convenient to use subsets of the entire matrix and make the prediction on each one separately, (e.g., cases of multiple iterations in order to identify specific influence order etc.). Given the evidence inputs (Ej), the prediction vector P(m) (m=1 . . . N) describes the transition probability to each entity. In this example, the 1^stentry (p1) represents entity “brute force” technique where if the result probability value is 1, while all other entries may be set 0 (in p2 . . . pN), then the prediction of the following attack state entity may be a “brute force” attack technique, and no other prediction exist. The prediction vector in this case may be shown in Equation (11) below:

(

1

0

0

0

…

…

0

)

(

Equation
⁢

11

)

The prediction input vector may include “fuzzy” evidence representation in the form of values between 0 and 1 which may be used to represent the level of certainty (probability) that each entity evidence really occurred. Use cases for using this fuzzy evidence is further described in the classification methods description. Advantages to the quasi-linear prediction model may be that the model is simple to construct and may include cyclic relationships, (i.e., the model may represent a situation that entity A may transit to entity B, and entity B may transit to entity A. In order to overcome some limitations, the quasi-linear model may be used repeatedly, by feeding the model iteratively with a different set of evidence so the prediction impact of each input may be analyzed separately.

Another effective option for prediction model may be a Bayesian Belief Network (BBN) prediction model based on BBN influence diagram that describes the conditional probabilities for transition between nodes in the Bayesian network where each node represents an entity (an attack state).

FIG. 8 depicts a BBN influence diagram 800, according to example embodiments of the present disclosure. Each node represents an entity, which in the above example may be represented by the MITRE tactic, technique, sub-technique and software class (SW). The edges may define the cause-and-effect direction (transition direction) between the entities. Each node may be assigned with a conditional probability table that describes the transition probability into it from all direct connected nodes. T may represent the probability of the occurrence of the entity, while F may represent the complementary probability (sum of T and F is 1). Once evidence is observed (e.g., a system detected the occurrence of an attack state), the BBN model may propagate the probabilities through the entire network. The conditional probability tables, tactics, techniques and sub-techniques are shown as 802, 804, 806 and 808.

For example, FIG. 9 depicts a probabilities propagation 900, according to example embodiments of the present disclosure. Node 2 may be the input evidence 906 with 100% certainty (True state=1; False state=0). Probability propagation may take place according to all conditional probability tables and may set the prediction probabilities for all other nodes accordingly. This may include nodes that may not be directly connected with the evidence node, (i.e., for a distance that is larger than 1 from the evidence node). The conditional probability tables, tactics, techniques and sub-techniques are shown as 902, 904, 908 and 910.

There are advantages to the quasi-linear prediction model. For example, FIG. 10 depicts a BBN model illustration 1000, according to example embodiments of the present disclosure. For example, the BBN model provides wide context of possible transitions including: past entities predictions (e.g. nodes 1014 and 1016); future entities predictions (nodes 1004, 1006, 1008, 1010, 1012); clear indication of the relationship between the entities (in case the input includes multiple evidence input); clear indication of the order of the predicted entities, (i.e., which attack state follows the other); providing “distance” (defines as ‘d’ in FIG. 10) analysis capabilities, (i.e., prediction per the distance from the evidence (parent) entity); and providing attack states “gap” predictions (node 1018). These may be prediction of attack state entities in between two existing evidence inputs (nodes 1002).

The overall process of generating the final prediction models is illustrated in FIG. 11 which depicts a prediction models generation process 1100, according to example embodiments of the present disclosure.

In order to create an influence diagram for BBN based model, and given that BBN model cannot include cyclic transitions, there may be a need to identify cyclic transitions cases and to decompose in step 1104 the transition matrix 1102 into a matrix that may include only non-cyclic transitions (for generating the BBN influence diagram), and another matrix that includes the remaining transitions. These steps may include computing a decomposition quality score in step 1106, selecting the best decomposition matrix in step 1108 and generating the BBN influence diagram in step 1110. In some embodiments, the values of the known or labeled nodes may be fixed and may not change during the calculation.

It is noted that in some embodiments, other methods for handling cyclic transitions, such as, but not limited to, loopy belief, propagation, Markov random fields, factor graphs or independent component analysis, can be used.

For example, in the decomposition method, 2 matrix types may be defined, which may be subset of the transition matrix (TrM), meaning that the sum of the matrices may be equal to the TrM. In this example, main matrix (Main_TrM) may include only non-cyclic transitions, while supplement (Sup_TrM) may include the cyclic transitions. In some embodiments low probability elements may be removed from the TrM so it may be less complex to create an effective influence diagram for the BBN prediction model. All excluded elements may join the supplement matrix

Specifically, FIG. 12 depicts transitions cases 1200 with two transition types (cyclic and non-cyclic), according to example embodiments of the present disclosure. In the non-cyclic case 1202 on the left of FIG. 12, there may be no transition of entities that influence themselves, while in the cyclic case 1204 (on the right in FIG. 12), entity A may have influence on entity B and C, which influences on D and that influence back on A. Therefore, in this example, path A-B-C-D and path A-C-D are cyclic.

There may be a few decomposition methods that may be applied, and the overall goal may be to reach an optimized case so the size of the supplement matrix may be minimized while still containing all possible cyclic cases. One example of a decomposition method could be ordering which may be a method that organizes in sequence all entities so cyclic cases may be excluded (and kept in the supplement matrix). If entities may be organized in ordered sequence then the transition matrix for N entities may be represented as a 2D array TrM={T(i,j); 1<=i,j,<=N}, so TrM=TrM_Increasing+TrM_Decreasing, where TrM_Increasing={T(i,j); 1<=i>j,<=N} (“From_ID” less than “To_ID”) is the low triangular submatrix of TrM matrix, and where TrM_Decreasing={T(i,j); 1<=i<j,<=N}. (“To_ID” less then “From_ID”) is the upper triangular submatrix of TrM matrix.

The system may then choose the upper or lower diagonal (e.g. select diagonal with maximum non-zero elements) and set the selection as the main matrix. In that way it may be possible to exclude all possible cases that entity will eventually influence itself. FIG. 13 depicts a simple ordering decomposition 1300, according to example embodiments of the present disclosure. FIG. 13 shows an illustration of upper sub matrix 1304 and lower triangular sub matrix 1306 of transition matrix 1302.

Another method may be to define different orders of the elements by for example: selecting outgoing-only elements, (i.e., entities that only influence others, where outgoing-only indicates that that the element's row may be equal to zero); selecting the incoming-only elements that may be only influenced by others, where incoming-only indicates that the element's column may be equal to zero; setting the outgoing 1^stand then the incoming elements in the matrix, and then for all the rest defining another order.

For example, the method may include the following steps: assign each pair with the following four parameters that may take a value of 0 or 1 where the role order=0 if victim before performer, and role order=1 if the opposite is true; tactics order=0 indicates a “natural” order of tactics based on the MITRE order, (e.g., initial access is execution), and tactics order=1 if the opposite is true; hierarchy order=0 if the influence direction is from parent to child (e.g., tactic to technique . . . ), and hierarchy order=1 if the opposite is true; child order=0 if ID FROM is higher than ID TO such that all pairs may be equal according to previous rules. In this example, there may be 2{circumflex over ( )}4=16 variants from 0 up to 15, which may be indexed numbers in binary format. For example, 0110 indicates role order=0, tactics order=1, hierarchy order=1 and child order=0. The order of these pairs may be placed according to the binary value, and the lower or upper triangle matrix part may be again set based on the one that has higher number of non-zero elements.