During the model training process, the weights of each connection and/or node may be adjusted in a learning process as the model adapts to generate more accurate predictions on a training set. The weights assigned to each connection and/or node may be referred to as the model parameters. The model may be initialized with a random or white noise set of initial model parameters. The model parameters may then be iteratively adjusted using, for example, stochastic gradient descent algorithms that seek to minimize errors in the model.

FIG. 3 depicts a system for authenticating a user device 301. The user device 301 is shown as connected, via the network 103, to an authentication server 302, a transactions database 303, a user database 304, an e-mail database 305, and an organizations database 306. The network 103 may be the same or similar as the network 103 of FIG. 1. Each of the user device 301, the authentication server 302, the transactions database 303, the user database 304, the e-mail database 305, and/or the organizations database 306 may be one or more computing devices, such as a computing device comprising one or more processors and memory storing instructions that, when executed by the one or more processors, perform one or more steps as described further herein. For example, any of those devices might be the same or similar as the computing devices 101, 105, 107, and 109 of FIG. 1.

As part of an authentication process, the user device 301 might communicate, via the network 103, to access the authentication server 302 to request access (e.g., to a user account). For example, the user device 301 might access a website or user interface associated with the authentication server 302. The authentication server 302 may then prompt (e.g., via a user interface) the user device 301 to answer one or more authentication questions. These questions might relate to a username, a password, a user PIN, or the like. In the case of questions relating to a username, a password, or the like, the questions might be generated based on data stored by the user database 304. Such authentication questions might additionally and/or alternatively relate to one or more transactions conducted by a user. Such questions (e.g., those relating to one or more transactions conducted by a user) might be generated based on data stored by the transactions database 303. Such questions might comprise one or more false answers, which might be based on one or more organizations indicated in the data of the organizations database 306. For example, the organizations database 306 might store a long list of random merchants, and the false answers might comprise indications of one or more of those random merchants. Based on the answers provided by a user of the user device 301, the user might be provided access (to, e.g., a user account).

The user database 304 might store information about one or more user accounts, such as a username, password, or the like. For example, as part of creating an account, a user might provide a username, a password, and/or one or more answers to predetermined authentication questions (e.g., “What is the name of your childhood dog?”), and this information might be stored by the user database 304. The authentication server 302 might use this data to generate authentication questions. The user database 304 might additionally and/or alternatively store preferences for one or more user accounts relating to authentication. For example, the user database 304 might store, for a particular user account, an e-mail for a user, such as an e-mail account managed by the e-mail database 305. As another example, the user database 304 might store preferences indicating that certain users are required to use two-factor authentication as part of authentication.

The transactions database 303 might comprise data relating to one or more transactions conducted by one or more financial accounts. For example, the transactions database 303 might maintain all or portions of a general ledger for various financial accounts associated with one or more users. The data stored by the transactions database 303 may indicate one or more merchants (e.g., where funds were spent), an amount spent (e.g., in one or more currencies), a date and/or time (e.g., when funds were spent), or the like. The data stored by the transactions database 303 might be generated based on one or more transactions conducted by one or more users. For example, a new transaction entry might be stored in the transactions database 303 based on a user purchasing an item at a store online and/or in a physical store. As another example, a new transaction entry might be stored in the transactions database 303 based on a recurring charge (e.g., a subscription fee) being charged to a financial account.

The account data stored by the user database 304 and the transactions database 303 may, but need not be related. For example, the account data stored by the user database 304 might correspond to a user account for a bank website, whereas the financial account data stored by the transactions database 303 might be for a variety of financial accounts (e.g., credit cards, checking accounts, savings accounts) managed by the bank. As such, a single user account might provide access to one or more different financial accounts, and the accounts need not be the same. For example, a user account might be identified by a username and/or password combination, whereas a financial account might be identified using a unique number or series of characters.

The e-mail database 305 might comprise data associated with one or more e-mails corresponding to one or more e-mail accounts. For example, the e-mail database 305 might store data associated with one or more e-mails associated with a user's e-mail account. These e-mails may comprise information such as a sender, a recipient, a subject line, body text, one or more attachments, or the like. The user database 304 might indicate one or more e-mail accounts associated with a particular user. For example, the user database 304 might store information indicating, for a particular user, one or more corresponding e-mail accounts managed by the e-mail database 305. Thus, for example, the same user might have two e-mail accounts (with corresponding data stored in the e-mail database 305), a single user account (with corresponding data stored in the user database 305), and the single user account might provide the user online access to three financial accounts (with corresponding data stored in the transactions database 303).

The organizations database 306 might store data relating to one or more organizations, including indications (e.g., names) of organizations, aliases of the organizations, and the like. That data might be used to generate authentication questions that comprise both correct answers (e.g., based on data from the transactions database 303 indicating one or more organizations where a user has conducted a transaction) and real-looking but false answers (e.g., based on data from the organizations database 306, which might be randomly-selected organizations where a user has not conducted a transaction). For example, a computing device might generate an authentication question comprising both correct and correct options by querying the organizations database 306 for a list of organizations (e.g., popular organizations where average customers in a particular area commonly shop). The computing device might then generate a question comprising one correct option (e.g., an organization indicated by the transactions database 303; that is, where a user has in fact shopped) and three incorrect options (e.g., three organizations from the organizations database 306; that is, where a user might appear to have shopped but did not in fact shop).

FIG. 4A depicts an illustrative user interface 400a comprising a username field 401a, a password field 401b, and a submit button 401c. The user interface 400a might be provided by, e.g., the authentication server 302 as part of determining, using data stored by the user database 304, whether to provide the user device 301 access to an account. For example, a user might be prompted to type in a candidate username into the username field 401a and a candidate password into the password field 401b, and the user might be provided access to an account based on whether the candidate username and the candidate password match data stored by the user database 304. As such, the user interface 400a depicts a circumstance where a user is provided two different authentication questions: specifically, as shown in FIG. 4A, they are prompted to provide both a valid username and a valid password.

FIG. 4B depicts a user interface 400b comprising an authentication question 402a which allows a user to select one of a plurality of options 403a-403c and submit the selection using a submit button 402b. As previously discussed, an authentication question, such as the authentication question 402a, might relate to a past transaction of a user, such as (as shown in FIG. 4B) a question relating to where a user purchased an item (coffee) at a particular time (last week). The plurality of options 403a-403c might comprise one or more correct answers and one or more incorrect answers. For example, in the example provided in FIG. 4B, the user might have in fact purchased coffee at Bob's Coffee (option 403b) and not Joe's Coffee (option 403a) and/or Tim's Coffee (option 403c).

FIG. 4B illustrates that authentication questions, such as the authentication question 402a, might be generated, in whole or in part, based on false information. For example, while Joe's Coffee (option 403a) and/or Tim's Coffee (option 403c) might be real merchants and real coffee shops, they might have been selected based on determining that, according to data stored by the transactions database 303, a user has not shopped at these locations. That said, Joe's Coffee (option 403a) and/or Tim's Coffee (option 403c) need not be real coffee shops at all: the names might have been randomly generated (e.g., by a machine learning algorithm trained using input comprising a variety of legitimate store names) to appear genuine.

FIG. 4B also illustrates a potential source of confusion for authentication questions such as the authentication question 402a. In the circumstance depicted in FIG. 4B, the computing device may assume that only option 403b is correct. That said, the user might have actually conducted a transaction at a “Tim's Coffee” in the last week, albeit paying in cash or using a credit card not tracked by the transactions database 303, such that data about that transaction might not be stored by the transactions database 303. Additionally and/or alternatively, the user might have in fact actually conducted a transaction at a “Timothy's Coffee” in the last week, such that they might be confused as to whether “Tim's Coffee” and “Timothy's Coffee” are the same.

Having discussed several examples of computing devices which may be used to implement some aspects as discussed further below, discussion will now turn to a method for generating authentication questions in a manner that avoids confusion regarding organizations where a user may have conducted a transaction.

FIG. 5 illustrates an example method 500 for generating authentication questions in accordance with one or more aspects described herein. The method 500 may be implemented by a suitable computing system, as described further herein. For example, the method 500 may be implemented by any suitable computing environment by a computing device and/or combination of computing devices, such as one or more of the computing devices 101, 105, 107, and 109 of FIG. 1, and/or any computing device comprising one or more processors and memory storing instructions that, when executed by the one or more processors, cause the performance of one or more of the steps of FIG. 5. The method 500 may be implemented in suitable program instructions, such as in machine learning software 127, and may operate on a suitable training set, such as training set data 129. The steps shown in the method 500 are illustrative, and may be re-arranged or otherwise modified as desired.

In step 501, a computing device might receive a request for access. The request for access might be for an account associated with a user. The request for access might be associated with a user accessing, via a user device, a website, a log-in screen, or the like. The request or access might comprise a request for access to an account, such as a financial services account, a banking account, or the like. The request for access might comprise, for instance, a user clicking a “Log In” button on a website. With that said, the request for access need not originate in any particular format or manner.

In step 502, the computing device may identify a plurality of organizations by querying an organizations database, such as the organizations database 306. The organizations in this list might comprise one or more merchants where any customer (not necessarily just the user associated with the request for access in step 501) might have conducted a transaction. For instance, the plurality of organizations may comprise one or more stores, restaurants, and vendors where a customer might spend money. In some instances, the organizations database might store a list of the most popular merchants in the United States, among customers of a particular bank, or the like. In this manner, the plurality of organizations might be identified for use in later generating an authentication question. This plurality of organizations need not comprise organizations where a user (e.g., the user associated with the request for access in step 501) has conducted a transaction: in fact, as will be discussed below, this plurality will be processed to remove organizations where the user is known to have conducted transactions, such that a plurality of organizations where a user has not conducted a transaction may be determined.

Querying the organizations database might comprise querying the organizations database based on information about the user associated with the request for access received in step 501. For instance, because organizations where a user might conduct a transaction might vary based on user demographics (e.g., the age of the user, where the user is located), the organizations database might be queried based on demographics of the user associated with the request for access received in step 501. For example, geographically-identifying information for a user (e.g., their ZIP code) might be identified, and the organizations database might be queried using the geographically-identifying data associated with the user to retrieve, from the organizations database, a plurality of organizations associated with a particular region associated with the user. As another example, the organizations database might be queried using an age of the user to retrieve, from the organizations database, a plurality of organizations where customers around that age customarily shop.

In step 503, the computing device may receive e-mails associated with the account. The e-mails may be received from an e-mail server, such as the e-mail database 305. The e-mails may be received by determining one or more e-mail accounts corresponding to the user associated with the request for access in step 501 and querying, based on the one or more e-mail accounts, the e-mail server. For example, the computing device might use the user database 304 to determine one or more e-mail accounts corresponding to the user associated with the request for access in step 501 and then, using an identifier of those e-mail accounts (e.g., an e-mail address), query the e-mail database 305.

The computing device might require permission, by a user, to receive the e-mails in the e-mail database 305. For example, the e-mail database 305 might store personal e-mail data for a user, such that the user might have to provide the computing device permission to access the e-mails. As such, step 503 might comprise requesting and receiving permission to analyze the e-mails of the user. Such permission might additionally and/or alternatively be received well in advance of the authentication process. For example, as part of registering for an account with a banking website, a user might provide the banking website access to their e-mail account.

The e-mail database 305 might be configured to periodically receive and store e-mails from external e-mail accounts. For example, the e-mail database might be an e-mail server managed by a financial institution that periodically retrieves e-mail data from a plurality of different external e-mail accounts via different e-mail servers. In this manner, the e-mail database 305 might be a cache e-mails from one or more e-mail accounts of a user. This might be advantageous because the cache could be managed to protect user security: for example, the cache may be periodically cleared and/or pruned to remove e-mails that do not relate to transactions.

Receiving the one or more e-mails associated with the account may comprise querying an e-mail server (e.g., the e-mail database 305) for e-mails associated with a time period. Retrieving excessively old e-mails might be undesirable because the transactions indicated by those e-mails might be rather old. For example, a user might not remember whether they shopped at a particular grocery store three years ago. As such, the query might be limited to a particular period of time, such as within the last year, within the last month, or the like. That said, certain transactions (e.g., for very significant purchases, such as for a car or a very expensive item), though old, might be nonetheless memorable to a user. As such, the e-mail server might be queried for all e-mails for a particular period (e.g., the last month), but might additionally and/or alternatively be queried for only particular e-mails (e.g., those containing dollar amounts over ten thousand dollars) for a broader period (e.g., the last year).

In step 504, the computing device may process the received e-mails. Processing the e-mails may comprise identifying one or more organizations in the received e-mails. The one or more organizations identified during processing may correspond to one or more transactions conducted by the user. Advantageously, such e-mails might be for transactions where the user used cash and/or another financial account to pay, such that the e-mails might more fully represent spending of the user as compared to the data in the transactions database 303. As such, the processing in step 504 might comprise using a computing device to analyze all or portions of e-mails to identify one or more organizations where a user has conducted a transaction, whether or not that transaction is reflected in the data stored by the transactions database 303. As will also be detailed further below, such processing might be performed using a natural language processing algorithm, a trained machine learning model, or the like.

Processing the e-mails might comprise use of a machine learning model, such as might be implemented using the deep neural network 200. A computing device may train, using input data comprising a plurality of e-mails with tagged organizations, a machine learning model to identify indicators of organizations in e-mail data. For example, the input data might comprise large quantities of different e-mails, with various aspects of those e-mails (e.g., organizations, goods/services involved in a transaction, dollar amounts) tagged. This input data might train the machine learning model to identify, e.g., an organization in an e-mail. Then, the computing device may provide, as input to the machine learning model, the one or more e-mails received in step 503. The computing device may receive, as output from the machine learning model, an indication of one or more organizations. In this manner, the machine learning model may learn to tag organizations in e-mails.

Processing the e-mails may entail use of e-mail templates. For example, a particular organization might use a certain format of e-mail (e.g., a certain layout that uses certain stylesheets, a certain sender e-mail address, a certain subject line format, etc.) to confirm a purchase made by a user. Such templates might be determined by analyzing one or more previous e-mails by an organization. Using these templates, e-mails having that format might be identified and associated with the organization. In this manner, even though such purchase confirmation e-mails might vary (e.g., in terms of the number/nature of goods/services purchased), the e-mails might nonetheless be more readily identified as originating from an organization. In this way, an e-mail template corresponding to a first organization might be determined and, based on comparing the e-mail template to at least one of the one or more e-mails, an organization might be identified.

The e-mails processed need not always be received directly from an organization, but might instead be received by another organization. For example, an e-mail from a first organization (e.g., a payment processing organization, such as PayPal Holdings, Inc. of San Jose, California) might correspond to a transaction conducted at another organization (e.g., an online store, and/or a physical store that uses a payment processing network associated with the first organization). In such a circumstance, processing the e-mails may entail use of an e-mail template for a first organization to identify a second organization. For example, using an e-mail template associated with PayPal (a first organization), an e-mail might be processed to identify an online store where the user recently shopped (a second organization). As another example, using an e-mail template associated with an insurance company (a first organization), an e-mail from that insurance company might be processed to determine where a customer might have recently purchased a vehicle (e.g., a specific dealership indicated in the insurance policy). Accordingly, templates might be determined not only for merchants themselves, but also other organizations through which e-mails reflecting purchasing activity might be received.