As demonstrated in the Examples below, the disclosed MPARRM method provides an approach to faithfully remove stimulation artifacts without corrupting the electrophysiologic signal components. MPARRM can remove stimulation artifacts without spectral leakage or temporal spread problems. It works for variable stimulation parameters and can recover the early response of SPES in different frequency bands. Specifically, during the early response window (5 to 10 ms following the SPES onset), the broadband gamma power (70-170 Hz) of the clean signal was highly correlated with the raw signal (R=0.98, Pearson), and the broadband gamma of the clean signal could faithfully reveal the auditory modulation in the raw signal with a 94% sensitivity and 99% specificity. The disclosed MPARRM method facilitates the understanding of neural response mechanisms including, but not limited to, responses to single-pulse electrical signals (SPES).

In various aspects, computing systems and computer-implemented methods to remove brain stimulation artifacts from neural signals are disclosed herein. The disclosed systems and methods provide for the precise extraction of brain stimulation artifacts and removal of these artifacts from neural signals, yielding a practical measurement of neural responses to brain stimulation. The disclosed method is based on the observation that matching pursuit algorithm (see generally Mallat et al., IEEE Trans. Signal Process. 41, 3397-3415, 1993; Chandran K S et al., J. Neurosci. 36, 3399-3408, 2016) was capable of accurately extracting stimulation artifacts. The matching pursuit algorithm, as used herein, refers to a method of iteratively decomposing a signal into a linear expansion of waveforms (atoms) selected from a large over-complete dictionary, in which the waveforms are selected to best match the local signal structures.

In various aspects, the disclosed MPARRM method makes use of a matching pursuit (MP) algorithm. The MP algorithm is an iterative decomposition technique that approximates a time-domain signal to a linear combination of waveforms called atoms. MP-based algorithms have been previously used to separate the spikes from oscillatory activities or broadband activity, estimate the gamma duration, detect epileptic activity, and estimate the single-trial evoked brain responses. In various aspects, the disclosed MPARRM method sequentially extracts the line noise, evoked potentials, and stimulation artifacts using the MP algorithm. In various aspects, the disclosed MPARRM method accurately removes stimulation artifacts without affecting the physiological components of the electrophysiological signal.

By way of non-limiting example, the disclosed MPARRM method is used to remove stimulation artifacts from data obtained during stimulation using single pulse electrical signals (SPES). Without being limited to any particular theory, early electrophysiological responses to SPES could potentially provide vital information. However, the understanding of the spectral responses during the early period is largely unknown, mainly because the stimulation artifact dramatically contaminates the signals nearby due to a filter issue. These spectral responses carry a lot of important information; the power spectral changes of the broadband gamma signals could represent the average firing rate of neurons located directly underneath the recording electrodes. As demonstrated in the Examples herein, the disclosed MPARRM method effectively removes the stimulation artifacts, and the early response of broadband gamma (within 5 ms to 10 ms following stimulation onset) is highly correlated (R=0.98, Pearson correlation) with the ground truth (FIG. 9B).

In various aspects, the disclosed MPARRM method removes the artifact from each trial by only involving information of the single-trial signal itself. Consequently, no information from other trials and channels is required. The artifact shapes can vary among different electrodes, times, and even sample rates (FIG. 11B). Because of the greedy fashion of the MP algorithm used by the disclosed MPARRM method, artifacts with different pulse shapes may be removed (FIGS. 11D and 13). In some aspects, the stimulation artifact is assumed to be within a 10 ms window (i.e., 5 ms preceding and 5 ms following stimulation onset). In other aspects, this window length is extended or reduced to any suitable length without limitation to ensure that all stimulation artifacts have been removed.

In various aspects, the number of iterations used to implement the disclosed MPARRM method may be set to any suitable number without limitation. Suitable numbers of iterations may range from about 10 iterations to about 50 or more iterations. Without being limited to any particular theory, reducing the number of iterations results in faster processing of data to remove artifacts using the disclosed MPARRM method.

FIG. 5A is a schematic diagram illustrating an exemplary process of brain stimulation accompanied by the recording of neural signals. The traces shown in the right panel of FIG. 5A are stereo-electroencephalographic signals in the human frontal brain area with 30 Hz vagus nerve stimulation; the dashed lines indicate the onset of individual stimulation pulses. As illustrated in FIG. 1A, brain stimulation techniques typically introduce strong stimulation artifacts around the stimulation period into neural signal traces.

The disclosed method overcomes at least a portion of the limitations of existing artifact extraction methods. FIG. 5B schematically illustrates three existing signal processing techniques to remove stimulation artifacts. However, all these methods are implemented subject to assumptions that are invalid under conditions that typically occur for many brain stimulation methods.

As illustrated in the top row of FIG. 5B, removing stimulation artifacts by filtering via a low-pass filter assumes that the stimulation artifacts occur at a frequency that is higher than naturally occurring neural oscillations. However, in general, stimulation waveforms may be characterized by a variety of frequencies including, but not limited to, frequencies that fall within a frequency range of naturally occurring neural oscillations. FIG. 5C further illustrates the shortcomings of removing stimulation artifacts from neural signal traces by applying one or more filters to the neural trace. As illustrated in FIG. 5C, the stimulation artifact (top graph) is a burst signal with a sharp shape, which will cause spectral leakage around the stimulation period after filtering using a notch filter (middle graph) or bandpass filter (bottom graph). The filters introduce additional artifacts that may be interpreted as fake neural responses, rather than merely removing the stimulation artifacts as intended.

As illustrated in the middle row of FIG. 5B, removing stimulation artifacts by substitution of a template waveform assumes that the stimulation artifact waveforms are regular and constant over repeated stimulation cycles. However, in general, stimulation waveforms may result in artifact waveforms that vary between individual waveforms.

As illustrated in the bottom row of FIG. 5B, removing stimulation artifacts by interpolation of the neural signal trace based on data immediately preceding and following the stimulation artifact essentially discards the neural signal trace obtained during the stimulation cycle and assumes that the neural trace in the immediate vicinity of the stimulation cycle does not contain any useable information.

In various aspects, the disclosed method for removing stimulation artifacts from neural signal traces makes use of the matching pursuit algorithm to decompose the signal trace into a plurality of waveforms selected from a library of atoms/functions, identifying those waveforms within the decomposition that correspond to stimulation artifacts, removing the artifact-associated waveforms, and regenerating a modified neural signal trace using the remaining waveforms, in which the modified neural signal trace includes the raw neural signal trace with the stimulation artifacts removed.

The matching pursuit algorithm (MP) is an iterative procedure to decompose a signal as a linear combination of members of a specified family of functions, referred to herein as “atoms”. The matching pursuit algorithm is described in Mallat et al., IEEE Trans. Signal Process. 41, 3397-3415, 1993, the content of which is incorporated by reference herein in its entirety. Briefly, the MP algorithm decomposes any signal into a linear expansion of waveforms that belong to a redundant dictionary, in which each waveform is selected to best match the signal structure. As applied to neural signal traces, the MP algorithm implements an adaptive time-frequency decomposition, in which the neural signal trace is decomposed into a plurality of waveforms selected from a dictionary of time-frequency atoms. The term “atoms”, as used herein, refers to dilations, translations, and modulations of single window functions. The MP algorithm is a greedy algorithm that chooses a waveform at each iteration that is best adapted to approximate at least a part of the signal trace. The linear expansion of waveforms is produced over a plurality of iterations, and conservation of energy is invoked to ensure convergence.

In various aspects, the MPARRM method includes subjecting the electrophysiological data to three rounds of matching pursuit to identify line noise, evoked potentials, and stimulation artifacts, respectively. Each round of matching pursuit limits the characteristics of the atoms selected according to separate criteria that represent the expected characteristics of line noise, evoked potential, and stimulation artifact, respectively. Without being limited to any particular theory, the summation of all atoms identified by each round of matching pursuit is thought to correspond to the line noise, evoked potential or stimulation artifact portion of the raw signal, and is subtracted accordingly after each matching pursuit decomposition to produce the data used for the next matching pursuit decomposition.

In various aspects, the atoms associated with line noise are selected if the atom frequency is greater than 55 Hz, and extend along the entire duration of the raw data. In various other aspects, the atoms associated with evoked potential artifacts are selected if the atom frequency is less than 70 Hz. In various additional aspects, the disclosed method makes use of the matching pursuit algorithm to accurately retrieve the stimulation artifact by choosing atoms centered on the stimulation period with specific characteristics. In various aspects, the atoms associated with the stimulation artifact are identified according to at least three selection criteria. In one aspect, an atom is associated with a stimulation artifact if the atom is centered around the stimulation onset time. In another aspect, an atom is associated with a stimulation artifact if the atom frequency is larger than about 70 Hz, or the atom is a Dirac atom. In a third aspect, an atom is associated with a stimulation artifact if the atom is a short atom (i.e. the atom does not extend along the whole time axis).

In various aspects, the MP algorithm selects waveforms from a large over-complete dictionary that includes a plurality of atom types including, but not limited to, Gabor atoms, Dirac atoms, and Fourier atoms. Gabor atoms are sine-modulated Gaussian waveforms that represent a compromise between the frequency and time resolution of a waveform. Dirac atoms are discrete waveforms equal to 1 at one data point and equal to 0 elsewhere and are typically used to extract sharp or abrupt signal waveforms including, but not limited to, stimulation artifacts. Fourier atoms are pure sinusoid signals that are typically used to extract periodic signals including, but not limited to, line noise.

By way of non-limiting example, FIGS. 6A, 6B, 6C, 6D, and 6E illustrate the method for removing stimulation artifacts from a neural signal trace as disclosed herein. The method includes decomposing a raw signal trace, shown in FIG. 6B into a plurality of atoms. FIG. 6A contains graphs of 50 atoms selected using the MP algorithm for the linear decomposition of the raw signal trace containing a signal artifact (FIG. 6B). FIG. 6C compares the raw signal trace (black) to a trace reconstructed using all 50 atoms shown in FIG. 6A (orange line); the reconstructed trace (orange line) is highly similar to the raw trace (black line). The method further includes identifying and selecting a subset of the atoms of the MP-generated decomposition according to the three criteria described above (centered around stimulation onsite time, <70 Hz/Dirac, and short duration). The atoms of the decomposition identified using these criteria are denoted as red traces in FIG. 6A. FIG. 6D shows a signal trace reconstructed by adding the atoms associated with the stimulation artifact (i.e. the red traces of FIG. 6A. The red signal trace shown in FIG. 6D represents the stimulation artifact identified by the disclosed method. The disclosed method further includes determining a clean trace (or residual trace) by subtracting the stimulation artifact shown in FIG. 6D from the raw signal trace shown in FIG. 6B. FIG. 6E shows a comparison of the clean trace (green) and the raw signal trace (black).

By way of another non-limiting example, FIGS. 7A and 7B illustrate the disclosed method of removing brain stimulation artifacts from neural recordings in another aspect. FIG. 7A is a schematic overview of the disclosed method in this aspect. The method includes receiving a raw neural signal trace (top graph) that may include a superposition of signals indicative of a variety of phenomena including, but not limited to, a stimulation evoked potential, a stimulation artifact, and line noise. As illustrated in FIG. 7A, to remove the stimulation artifact using the disclosed method in this aspect, the line noise is removed, followed by the removal of the evoked potential, and finally the removal of the stimulation artifact.

In this example, the steps of the method illustrated in FIG. 7A are implemented by decomposing the raw signal into a plurality of atoms using the MP algorithm as described herein and then selecting and removing subsets of the plurality of atoms associated with each of the three signal subsets (line noise, stimulation evoked potential, and stimulation artifact) as identified by specific criteria defining the characteristics of the atoms associated with each signal subset. Without being limited to any particular theory, the line noise within the raw signal trace is removed using the MP-based method as described herein to avoid spectral leakage associated with the use of other existing methods of noise removal such as notch filtering (see FIG. 5C, middle graph). Further, because the frequency components of the stimulation evoked potential (EP) are complex and may overlap with the corresponding frequency components of the stimulation artifact trace, the EP is removed from the signal trace prior to extracting the stimulation artifact trace from the residual signal remaining after removing the EP (i.e., without EP). To maintain an intact EP signal, the stimulation artifact is subtracted from the signal trace with only the line noise removed to produce a clean signal trace that contains just the EP with the line noise and stimulation artifact removed.

FIG. 7B is a flow chart illustrating the specific steps of the method illustrated in FIG. 7A. As illustrated in FIG. 3B, the method includes receiving a raw signal trace at (1) and performing MP to decompose the raw signal trace into a plurality of atoms. At (2) a subset of the plurality of atoms from (1) is selected according to a set of line noise criteria. The subset of the plurality of atoms is summed to produce a line noise trace (red line), shown superimposed over the raw signal trace (dashed) at (2). In some aspects, the line noise criteria for selecting atoms associated with line noise include atoms with frequencies larger than about 55 Hz, and long atoms that extend along the entire time axis.

Referring again to FIG. 7B, the red signal trace associated with line noise is subtracted from the raw signal trace at (3) to produce a second signal trace that includes the EP and the stimulation artifact, but not the signal noise. The second signal trace without line noise is determined at (3) by subtracting the line noise trace (2) from the raw signal trace (1), i.e. (3)=(1)-(2). At (4) the stimulation time window in the second trace is replaced with a straight line (red portion) and then MP is performed on the resulting third signal trace at (5) to decompose the third signal trace into a second plurality of atoms. A subset of the second plurality of atoms associated with the EP is selected according to a set of EP criteria that includes, but is not limited to, those atoms with frequencies of less than about 70 Hz. The subset of the second plurality of atoms is summed to produce an EP signal trace (red line), shown compared to the third signal trace from (4) (dashed line). The EP signal trace of (5) is subtracted from the second signal trace of (3) to produce a fourth signal trace at (6). The fourth signal trace is decomposed into a third plurality of atoms at (7). A subset of the third plurality of atoms associated with the stimulation artifact is selected according to the stimulation artifact criteria, including, but not limited to, atoms centered around the stimulation onset time, atoms with frequencies greater than about 70 Hz or a Dirac atom, and a short atom that does not extend along the entire time axis. (short atom). The subset of the third plurality of atoms is summed to produce the stimulation artifact trace (red) shown compared to the fourth signal trace at (7).

Referring again to FIG. 7B, the stimulation artifact trace is subtracted from the second signal trace without line noise at (8) to produce an artifact-free trace (green). After these procedures, only the line noise from (2) and the stimulation artifact from (7) have been removed from the raw trace, while all other brain signals remain intact within the artifact-free trace.

The recorded signal during electrical stimulation typically has three major components (FIGS. 7A and 15), which are electrical line noise, stimulation artifact, and the electrophysiologic activity. It has proven difficult to remove stimulation artifacts and line noise from recorded signals without losing information from the electrophysiologic signal components. We therefore developed a matching pursuit algorithm (MP) to accomplish this task without the same limitations of previously published approaches.

By way of another non-limiting example, the MP algorithm is an iterative procedure that decomposes a signal including, but not limited to, electrophysiological signals, to a linear combination of basis functions, referred to herein as “atoms”. Without being limited to any particular theory, Gabor, sharp Gaussian, Dirac (1 at t=0; otherwise 0), and Fourier (pure sinusoids) atoms are useful as basis functions in the signal decomposition of electrophysiological signals obtained during SPEP stimulation. Without being limited to any particular theory, Gabor atoms provide a good compromise between frequency and time resolution, sharp Gaussian and Dirac atoms are useful to represent sharp and transient signals, and Fourier atoms are useful to represent periodic signals such as line noise.

In this non-limiting example, the denoising procedure by which MPARRM extracts stimulation artifacts can be broken into seven steps, whose signals are represented by Roman numeral I-VII in FIG. 7D. The line noise (e.g., 60 Hz and harmonics) can be represented by long atoms that are spread along the time axis and concentrated at their characteristic frequencies which are eliminated from the signal in steps 1 (matching pursuit #1) and 2 (subtract line noise). Without being limited to any particular theory, this procedure is more powerful than traditional notch filtering, which can create strong artifactual oscillations around sharp stimulation artifacts. Such filtering can obscure strong potentials induced shortly after stimulation, such as early negative potentials (N1) occurring at 10-50 ms. Evoked potentials are temporally removed from the signal at steps 3 (replace stimulation time window with a line), 4 (matching pursuit #2, and 5 (subtract evoked potential) to ensure they are not falsely identified as a stimulation artifact. Finally, the stimulation artifact is extracted after the line noise and evoked potential have been removed in steps 6 (matching pursuit #3) and 7 ((artifact-free trace). In various aspects, the stimulation artifact is represented by atoms including, but not limited to, short Gabor, short Gaussian, and Dirac atoms at the time of stimulation onset. Table 2 summarized the steps of the disclosed MPARRM method in some aspects; the steps in Table 2 refer to portions of FIG. 7B. The criteria used for the MP phases of the method of TABLE 1 are summarized in TABLE23.

TABLE 1

MPARRM Method Steps

Step
Signal Processing

1
Decomposing the raw trace (I) using the MP

algorithm and reconstructing the line noise

(II) based on criterion #1

2
Removing the line noise from the raw trace

(i.e., III = I-II)

3
Interpolating the stimulation time window

with a straight line from 2.5 ms preceding to

2.5 ms following stimulation onset (IV)

4
Decomposing the interpolated trace with MP

algorithm and reconstructing the evoked

potential (V) based on criterion #2

5
Removing the evoked potential from trace-III

(i.e., VI = III-V)

6
Decomposing the residual signal (VI) with MP

algorithm and reconstructing the stimulation

artifact (VII) based on criterion #3. Specifically,

the center time of the atom is within the

artifact window (i.e., 5 ms preceding and

5 ms following stimulation onset);

7
Removing the stimulation artifact from trace-III

(i.e., VIII = III-VII). The green trace (VIII)

represents the final clean signal. The line noise

and the stimulation artifacts are removed,

while the brain signals (including the evoked

potentials) are reserved

TABLE 2

Criterion For Matching Pursuit Atoms

MP to
Limitations for

Criterion
Remove
Atoms Selected by MP

1
Line Noise
Atom frequency >55 Hz

Atom is spread along the whole

time axis, i.e., long atom

2
Evoked
Atom frequency <70 Hz

Potential

3
Stimulation
Atom duration is centered around

Artifact
stimulation onset

Atom frequency >70 Hz or is Dirac

atom;

Atom is not spread along the whole

time axis, i.e., short atom.

FIGS. 8A, 8B, 8C, 8D, and 8E contain a series of graphs comparing clean traces (green) resulting from the application of the disclosed method for the purpose of removing stimulation artifacts. All the black lines indicate the raw trace, and the green lines indicate the artifact-free signal after applying our novel method. FIG. 8A shows one trace of an electroencephalographic signal (black line) near the human parietal-frontal areas with 1 Hz of DBS in the thalamic nucleus. FIG. 8B shows one stereo-electroencephalographic signal (black line) near the human hippocampus with 1 Hz of single-pulse electrical stimulation in the amygdala. FIG. 8C shows one stereo-electroencephalographic signal (black line) near the human temporal lobe with 1 Hz of single-pulse electrical stimulation in the amygdala. FIG. 8D shows one stereo-25 electroencephalographic signal (black line) near the human frontal lobe with 50-Hz high-frequency electrical stimulation in the amygdala. FIG. 8E shows one stereo-electroencephalographic signal (black line) in the human frontal brain area with 30 Hz vagus nerve stimulation.

In various aspects, the disclosed method is configured to extract artifacts from neural signals obtained in association with multiple kinds of brain stimulation, including, but not limited to, rTMS, single-pulse TMS, tDCS, tRNS, tACS, tFUS, VNS, SEPS, high-frequency stimulation, and micro-stimulation.

In various aspects, the computer-implemented methods disclosed herein are implemented using various computing devices and systems that include computing devices as described herein.

FIG. 1 depicts a simplified block diagram of the system for implementing the computer-aided methods of removing stimulation artifacts from neural signals as described herein. As illustrated in FIG. 1, the computing device 300 may be configured to implement at least a portion of the tasks associated with implementing the methods described herein. The computer system 300 may include a computing device 302. In one aspect, the computing device 302 is part of a server system 304, which also includes a database server 306. The computing device 302 is in communication with a database 308 through the database server 306. The computing device 302 is communicably coupled to a user computing device 330 through a network 350. The network 350 may be any network that allows local area or wide area communication between the devices. For example, the network 350 may allow communicative coupling to the Internet through at least one of many interfaces including, but not limited to, at least one of a network, such as the Internet, a local area network (LAN), a wide area network (WAN), an integrated services digital network (ISDN), a dial-up-connection, a digital subscriber line (DSL), a cellular phone connection, and a cable modem. The user computing device 330 may be any device capable of accessing the Internet including, but not limited to, a desktop computer, a laptop computer, a personal digital assistant (PDA), a cellular phone, a smartphone, a tablet, a phablet, wearable electronics, smartwatch, or other web-based connectable equipment or mobile devices.

In other aspects, the computing device 302 is configured to perform a plurality of tasks associated with the disclosed methods. FIG. 2 depicts a component configuration 400 of computing device 402, which includes database 410 along with other related computing components. In some aspects, computing device 402 is similar to computing device 302 (shown in FIG. 1). A user 404 may access components of computing device 402. In some aspects, database 410 is similar to database 308 (shown in FIG. 1).

In one aspect, database 410 includes neural signal data 412 and matching pursuit (MP) data 418. Non-limiting examples of suitable neural signal data 412 include any values of parameters defining the raw neural signal traces, clean signal traces, and/or the intermediate signal traces generated during implementation of the disclosed method as described herein. In one aspect, the MP data 412 includes any values defining the atoms used to decompose the signal traces, the selection criteria used to select subsets of the pluralities of atoms defining a decomposed signal trace, and any other parameters associated with the implementation of the MP algorithm to practice the method as described herein.

Computing device 402 also includes a number of components that perform specific tasks. In an exemplary aspect, computing device 402 includes a data storage device 430, matching pursuit (MP) component 440, and communication component 460. Data storage device 430 is configured to store data received or generated by computing device 402, such as any of the data stored in database 410 or any outputs of processes implemented by any component of computing device 402.

In various aspects, the matching pursuit (MP) component 440 is configured to implement the matching pursuit algorithm and to implement the method of removing stimulation artifacts from neural signals using the matching pursuit algorithm, selection of atom subsets, and reconstruction/removal of signal subsets as described herein.

Communication component 460 is configured to enable communications between computing device 402 and other devices (e.g. user computing device 330 shown in FIG. 1) over a network, such as a network 350 (shown in FIG. 1), or a plurality of network connections using predefined network protocols such as TCP/IP (Transmission Control Protocol/Internet Protocol).

FIG. 3 depicts a configuration of a remote or user computing device 502, such as user computing device 330 (shown in FIG. 1). Computing device 502 may include a processor 505 for executing instructions. In some aspects, executable instructions may be stored in a memory area 510. Processor 505 may include one or more processing units (e.g., in a multi-core configuration). Memory area 510 may be any device allowing information such as executable instructions and/or other data to be stored and retrieved. Memory area 510 may include one or more computer-readable media.

Computing device 502 may also include at least one media output component 515 for presenting information to a user 501. Media output component 515 may be any component capable of conveying information to user 501. In some aspects, media output component 515 may include an output adapter, such as a video adapter and/or an audio adapter. An output adapter may be operatively coupled to processor 505 and operatively coupleable to an output device such as a display device (e.g., a liquid crystal display (LCD), organic light-emitting diode (OLED) display, cathode ray tube (CRT), or “electronic ink” display) or an audio output device (e.g., a speaker or headphones). In some aspects, media output component 515 may be configured to present an interactive user interface (e.g., a web browser or client application) to user 501.

In some aspects, computing device 502 may include an input device 520 for receiving input from user 501. Input device 520 may include, for example, a keyboard, a pointing device, a mouse, a stylus, a touch-sensitive panel (e.g., a touchpad or a touch screen), a camera, a gyroscope, an accelerometer, a position detector, and/or an audio input device. A single component such as a touch screen may function as both an output device of media output component 515 and input device 520.

Computing device 502 may also include a communication interface 525, which may be communicatively coupleable to a remote device. Communication interface 525 may include, for example, a wired or wireless network adapter or a wireless data transceiver for use with a mobile phone network (e.g., Global System for Mobile communications (GSM), 3G, 4G or Bluetooth) or other mobile data network (e.g., Worldwide Interoperability for Microwave Access (WIMAX)).

Stored in memory area 510 are, for example, computer-readable instructions for providing a user interface to user 501 via media output component 515 and, optionally, receiving and processing input from input device 520. A user interface may include, among other possibilities, a web browser and client application. Web browsers enable users 501 to display and interact with media and other information typically embedded on a web page or a website from a web server. A client application allows users 501 to interact with a server application associated with, for example, a vendor or business.

FIG. 4 illustrates an example configuration of a server system 602. Server system 602 may include, but is not limited to, database server 306 and computing device 302 (both shown in FIG. 1). In some aspects, server system 602 is similar to server system 304 (shown in FIG. 1). Server system 602 may include a processor 605 for executing instructions. Instructions may be stored in a memory area 625, for example. Processor 605 may include one or more processing units (e.g., in a multi-core configuration).

Processor 605 may be operatively coupled to a communication interface 615 such that server system 602 may be capable of communicating with a remote device such as user computing device 330 (shown in FIG. 1) or another server system 602. For example, communication interface 615 may receive requests from user computing device 330 via a network 350 (shown in FIG. 1).

Processor 605 may also be operatively coupled to a storage device 625. Storage device 625 may be any computer-operated hardware suitable for storing and/or retrieving data. In some aspects, storage device 625 may be integrated into server system 602. For example, server system 602 may include one or more hard disk drives as storage device 625. In other aspects, storage device 625 may be external to server system 602 and may be accessed by a plurality of server systems 602. For example, storage device 625 may include multiple storage units such as hard disks or solid-state disks in a redundant array of inexpensive disks (RAID) configuration. Storage device 625 may include a storage area network (SAN) and/or a network attached storage (NAS) system.

In some aspects, processor 605 may be operatively coupled to storage device 625 via a storage interface 620. Storage interface 620 may be any component capable of providing processor 605 with access to storage device 625. Storage interface 620 may include, for example, an Advanced Technology Attachment (ATA) adapter, a Serial ATA (SATA) adapter, a Small Computer System Interface (SCSI) adapter, a RAID controller, a SAN adapter, a network adapter, and/or any component providing processor 605 with access to storage device 625.

Memory areas 510 (shown in FIG. 3) and 610 may include, but are not limited to, random access memory (RAM) such as dynamic RAM (DRAM) or static RAM (SRAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), and non-volatile RAM (NVRAM). The above memory types are examples only, and are thus not limiting as to the types of memory usable for the storage of a computer program.

The computer systems and computer-aided methods discussed herein may include additional, less, or alternate actions and/or functionalities, including those discussed elsewhere herein. The computer systems may include or be implemented via computer-executable instructions stored on non-transitory computer-readable media. The methods may be implemented via one or more local or remote processors, transceivers, servers, and/or sensors (such as processors, transceivers, servers, and/or sensors mounted on vehicle or mobile devices, or associated with smart infrastructure or remote servers), and/or via computer-executable instructions stored on non-transitory computer-readable media or medium.

In some aspects, a computing device is configured to implement machine learning, such that the computing device “learns” to analyze, organize, and/or process data without being explicitly programmed. Machine learning may be implemented through machine learning (ML) methods and algorithms. In one aspect, a machine learning (ML) module is configured to implement ML methods and algorithms. In some aspects, ML methods and algorithms are applied to data inputs and generate machine learning (ML) outputs. Data inputs may include but are not limited to images or frames of a video, object characteristics, and object categorizations. Data inputs may further include sensor data, image data, video data, telematics data, authentication data, authorization data, security data, mobile device data, geolocation information, transaction data, personal identification data, financial data, usage data, weather pattern data, “big data” sets, and/or user preference data. ML outputs may include but are not limited to: a tracked shape output, categorization of an object, categorization of a region within a medical image (segmentation), categorization of a type of motion, a diagnosis based on the motion of an object, motion analysis of an object, and trained model parameters ML outputs may further include: speech recognition, image or video recognition, medical diagnoses, statistical or financial models, autonomous vehicle decision-making models, robotics behavior modeling, fraud detection analysis, user recommendations and personalization, game AI, skill acquisition, targeted marketing, big data visualization, weather forecasting, and/or information extracted about a computer device, a user, a home, a vehicle, or a party of a transaction. In some aspects, data inputs may include certain ML outputs.

In some aspects, at least one of a plurality of ML methods and algorithms may be applied, which may include but are not limited to: genetic algorithms, linear or logistic regressions, instance-based algorithms, regularization algorithms, decision trees, Bayesian networks, cluster analysis, association rule learning, artificial neural networks, deep learning, dimensionality reduction, and support vector machines. In various aspects, the implemented ML methods and algorithms are directed toward at least one of a plurality of categorizations of machine learning, such as supervised learning, unsupervised learning, adversarial learning, and reinforcement learning.

The methods and algorithms of the invention may be enclosed in a controller or processor. Furthermore, methods and algorithms of the present invention, can be embodied as a computer-implemented method or methods for performing such computer-implemented method or methods, and can also be embodied in the form of a tangible or non-transitory computer-readable storage medium containing a computer program or other machine-readable instructions (herein “computer program”), wherein when the computer program is loaded into a computer or other processor (herein “computer”) and/or is executed by the computer, the computer becomes an apparatus for practicing the method or methods. Storage media for containing such computer programs include, for example, floppy disks and diskettes, compact disk (CD)-ROMs (whether or not writeable), DVD digital disks, RAM and ROM memories, computer hard drives and backup drives, external hard drives, “thumb” drives, and any other storage medium readable by a computer. The method or methods can also be embodied in the form of a computer program, for example, whether stored in a storage medium or transmitted over a transmission medium such as electrical conductors, fiber optics or other light conductors, or by electromagnetic radiation, wherein when the computer program is loaded into a computer and/or is executed by the computer, the computer becomes an apparatus for practicing the method or methods. The method or methods may be implemented on a general-purpose microprocessor or on a digital processor specifically configured to practice the process or processes. When a general-purpose microprocessor is employed, the computer program code configures the circuitry of the microprocessor to create specific logic circuit arrangements. Storage medium readable by a computer includes medium being readable by a computer per se or by another machine that reads the computer instructions for providing those instructions to a computer for controlling its operation. Such machines may include, for example, machines for reading the storage media mentioned above.

Definitions and methods described herein are provided to better define the present disclosure and to guide those of ordinary skill in the art in the practice of the present disclosure. Unless otherwise noted, terms are to be understood according to conventional usage by those of ordinary skill in the relevant art.

In some embodiments, numbers expressing quantities of ingredients, properties such as molecular weight, reaction conditions, and so forth, used to describe and claim certain embodiments of the present disclosure are to be understood as being modified in some instances by the term “about.” In some embodiments, the term “about” is used to indicate that a value includes the standard deviation of the mean for the device or method being employed to determine the value. In some embodiments, the numerical parameters set forth in the written description and attached claims are approximations that can vary depending upon the desired properties sought to be obtained by a particular embodiment. In some embodiments, the numerical parameters should be construed in light of the number of reported significant digits and by applying ordinary rounding techniques. Notwithstanding that the numerical ranges and parameters setting forth the broad scope of some embodiments of the present disclosure are approximations, the numerical values set forth in the specific examples are reported as precisely as practicable. The numerical values presented in some embodiments of the present disclosure may contain certain errors necessarily resulting from the standard deviation found in their respective testing measurements. The recitation of ranges of values herein is merely intended to serve as a shorthand method of referring individually to each separate value falling within the range. Unless otherwise indicated herein, each individual value is incorporated into the specification as if it were individually recited herein. The recitation of discrete values is understood to include ranges between each value.

In some embodiments, the terms “a” and “an” and “the” and similar references used in the context of describing a particular embodiment (especially in the context of certain of the following claims) can be construed to cover both the singular and the plural, unless specifically noted otherwise. In some embodiments, the term “or” as used herein, including the claims, is used to mean “and/or” unless explicitly indicated to refer to alternatives only or the alternatives are mutually exclusive.

The terms “comprise,” “have” and “include” are open-ended linking verbs. Any forms or tenses of one or more of these verbs, such as “comprises,” “comprising,” “has,” “having,” “includes” and “including,” are also open-ended. For example, any method that “comprises,” “has” or “includes” one or more steps is not limited to possessing only those one or more steps and can also cover other unlisted steps. Similarly, any composition or device that “comprises,” “has” or “includes” one or more features is not limited to possessing only those one or more features and can cover other unlisted features.

All methods described herein can be performed in any suitable order unless otherwise indicated herein or otherwise clearly contradicted by context. The use of any and all examples, or exemplary language (e.g., “such as”) provided with respect to certain embodiments herein is intended merely to better illuminate the present disclosure and does not pose a limitation on the scope of the present disclosure otherwise claimed. No language in the specification should be construed as indicating any non-claimed element essential to the practice of the present disclosure.

Groupings of alternative elements or embodiments of the present disclosure disclosed herein are not to be construed as limitations. Each group member can be referred to and claimed individually or in any combination with other members of the group or other elements found herein. One or more members of a group can be included in, or deleted from, a group for reasons of convenience or patentability. When any such inclusion or deletion occurs, the specification is herein deemed to contain the group as modified thus fulfilling the written description of all Markush groups used in the appended claims.

All publications, patents, patent applications, and other references cited in this application are incorporated herein by reference in their entirety for all purposes to the same extent as if each individual publication, patent, patent application, or other reference was specifically and individually indicated to be incorporated by reference in its entirety for all purposes. Citation of a reference herein shall not be construed as an admission that such is prior art to the present disclosure.

Having described the present disclosure in detail, it will be apparent that modifications, variations, and equivalent embodiments are possible without departing from the scope of the present disclosure defined in the appended claims. Furthermore, it should be appreciated that all examples in the present disclosure are provided as non-limiting examples.

As used herein, the term “multiple types of brain stimulation” refers to different types of both non-invasive and invasive brain stimulation techniques. Non-limiting examples of non-invasive brain stimulation include repetitive transcranial magnetic stimulation (rTMS), single-pulse TMS, transcranial direct current stimulation (tDCS), transcranial random noise stimulation (tRNS), transcranial alternating current stimulation (tACS), transcranial focused ultrasound stimulation (tFUS), vagus nerve stimulation (VNS); invasive brain stimulation including single-pulse electrical stimulation (SEPS), high-frequency stimulation, and micro-stimulation.

As used herein, the term “neural signal” refers broadly to any signal reflecting the electromagnetic (EM) activity of the brain, without limitation. Non-limiting examples of neural signals include signals indicative of electroencephalographic activity (EEG), electrocorticographic (ECoG) activity, stereo-electroencephalographic (SEEG) activity, magnetoencephalographic (MEG) activity, and local field potential (LFP) activity.

As used herein, the term “neural targets” refers broadly to any brain site that is chosen as the target of brain stimulation.

As used herein, the term “neural response” refers broadly to the neural signal that is related (or response) to the brain stimulation.

As used herein, the term “stimulation artifacts” refers broadly to the non-neural signal that is not originally from neural activity, but instead, from the stimulation.

In various aspects, the disclosed method can be used to remove stimulation artifacts from different types of stimulation techniques: including repetitive transcranial magnetic stimulation (rTMS), single-pulse TMS, transcranial direct current stimulation (tDCS), transcranial random noise stimulation (tRNS), transcranial alternating current stimulation (tACS), transcranial focused ultrasound stimulation (tFUS), vagus nerve stimulation (VNS); single-pulse electrical stimulation (SEPS), high-frequency stimulation, and micro-stimulation

In various aspects, the disclosed method can be used to remove stimulation artifacts of different neural signals, including any signal reflecting the electromagnetic (EM) activity of the brain without limitation including, but not limited to, electroencephalographic activity (EEG), electrocorticographic (ECoG) activity, stereo-electroencephalographic (SEEG) activity, magnetoencephalographic (MEG) activity, and local field potential (LFP) activity.

The function dictionaries involved in the matching pursuit algorithm include any suitable functions/atoms for extracting the stimulation artifact without limitation. Non-limiting examples of suitable functions/atoms include Gabor functions, Dirac functions, Fourier functions, and Rectangle functions.

In various aspects, the frequency restriction to extract the line noise can be any frequency without limitation including, but not limited to, 55 Hz.

The frequency restriction to extract the evoked potential can be any frequency without limitation including, but not limited to, 70 Hz.

The frequency restriction to extract the stimulation artifact can be any frequency without limitation including, but not limited to, 70 Hz.

Although the present disclosure describes various criteria for selecting subsets of atoms that include particular ranges of frequencies, center times, and time spread lengths herein, any suitable selection criteria may be used without limitation. In various aspects, additional selection criteria related to a variety of atom characteristics may be used to better extract the line-noise, evoked potential, and stimulation artifact including, but not limited to, the center time of atoms, frequency of atoms, type of atoms, time spread length of atoms, and any combination thereof.

EXAMPLES

The following non-limiting examples are provided to further illustrate the present disclosure. It should be appreciated by those of skill in the art that the techniques disclosed in the examples that follow represent approaches the inventors have found function well in the practice of the present disclosure, and thus can be considered to constitute examples of modes for its practice. However, those of skill in the art should, in light of the present disclosure, appreciate that many changes can be made in the specific embodiments that are disclosed and still obtain a like or similar result without departing from the spirit and scope of the present disclosure.

Example 1: Validation of MPARRM Artifact Removal Method

To validate the disclosed MPARRM artifact removal method across a wide range of potential stimulation artifact types the following experiments were conducted. Simulated artifacts were generated using recordings from electrodes submerged in a saline solution to generate 110 types of SPES artifacts, added these artifacts to the stereo-electroencephalographic (SEEG) signals recorded from 9 human subjects during a receptive speech task, and then removed the simulated artifacts from the added signals using MPARRM. The effectiveness of MPARRM was quantified by comparing the raw SEEG signals (prior to adding artifacts) with the clean signal (SEEG signals with added simulated artifacts after MPARRM denoising).

Experimental Methods

Subjects. Human SEEG Recordings and Electrode Localization SEEG signals were recorded in 9 human subjects (see FIG. 12A). Subjects underwent placement of SEEG electrodes (PMT) for Stage-2 diagnostic electrocorticography to help localize epileptogenic focus for clinical applications. Implantation was performed according to the standard of care at our institution.

SEEG signals were recorded using signal acquisition hardware and general-purpose brain-computer interface software BC12000. SEEG signals were amplified and digitized at 2000 Hz in parallel to the clinical monitoring system to ensure that clinical data collection was uninterrupted. We utilized preoperative MRI imaging to produce three-dimensional brain models with Freesurfer (http://surfer.nmr.mgh.harvard.edu) for anatomically accurate visualization. We localized implanted electrodes through co-registration of post-operative computer tomography (CT) scans using SPM (http://www.fil.ion.ucl.ac. uk/spm).

Auditory Stimulus. We used a receptive speech protocol to evoke a cortical response in all 9 subjects. The stimuli consisted of 32 unique words presented to patients via over-ear headphones (12 Hz-23.5 kHz audio bandwidth, 20 dB isolation from environmental noise). The duration of each stimulus was 700 ms, and the inter-stimulus interval was 1000 ms. The selected words were simple, meaningful verbs spoken by a female native English speaker. Stimuli were presented in two identical stimulus blocks composed of all 32 stimuli. Blocks were separated by a 20-second silent period during which the subject was instructed to relax. We defined individual trials as a 1700 ms-long interval (1000 ms preceding the stimulus onset and 1700 ms after the stimulus onset).

Single-Pulse Electrical Stimulation. One human subject underwent single-pulse electrical stimulation. During SEEG recording, 200 consecutive electrical pulses (biphasic, pulse width μs, current amplitude *mA) were generated between pairs of adjacent SEEG contacts, using an inter-stimulus interval of 500 ms. This was performed across five different pairs of adjacent electrode contacts (FIGS. 10A and 10B).

Recording Stimulation Artifact in Saline. We recorded isolated SPES artifacts by suspending two standard clinical stereo-electroencephalographic (SEEG) probes (FIG. 11A) in a 0.9% saline solution. The SEEG probes (PMT, Chanhassen, Minnesota) consisted of platinum/iridium contacts 0.8 mm in diameter, spaced 3.5 mm apart (contact length 2 mm, insulation length 1.5 mm). Similar to the clinical recordings, one SEEG probe was used for stimulation and a second for recording. Both electrodes were placed in a saline bath which simulated the clinical environment to record stimulation artifacts (FIG. 11A). Stimulation was performed using a 16-channel headstage (M4016) controlled by the Intan Stim/Recording System (Intan Tech., USA). SPES was delivered through a middle contact of the first SEEG probe using a monopolar configuration. The stimulation artifact was simultaneously recorded from a middle contact of a second SEEG probe connected to a separate headstage. Stimulation was administered at 0.5 Hz with an inter-stimulus interval of 2s. 110 different biphasic pulse shapes were generated (FIGS. 11B and 11E). Stimulus parameters were based on commonly used configuration settings, and each pulse was analyzed across 60 trials. The resulting signal, which included the recurring stimulation artifacts was amplified, digitized at 30,000 Hz, and saved in the RHS file format for subsequent analysis.

To generate a template of the saline SPES artifact, we extracted the recording around each stimulus pulse (2000 ms preceding and 2000 ms following onset) and averaged the signal across all trials to reduce the background noise. We assumed the majority of the artifact was within the artifact window (1 ms preceding and 4 ms following pulse onset) and thus set the value as zeros outside of the artifact window for better comparison. We further reduced the sampling of the saline SPES artifact to 2000 Hz (MATLAB decimate( )) to match the human SEEG recordings.

Validation of MPARRM: The performance of the MPARRM method was evaluated using the combined signals from the human SEEG data and the simulated artifacts/saline data. Specifically, the human SEEG signals during the auditory stimulus trial were used as a “ground truth” signal, which ideally would be recovered by MPARRM denoising. SPES artifact from the saline recordings was added to the SEEG signal to generate a mixed pseudo-signal which simulates a SEEG study with SPEP and stimulation artifacts. The MPARRM denoising method was then applied to the mixed signal to remove the stimulation artifact. The resulting “clean” signal was then compared to the original SEEG signal, which is expected to have a strong correlation for effective denoising methods. We also applied an existing interpolate denoising method (i.e., linear merging with signals 5 ms preceding and 5 ms following stimulation onset) as a control/comparison (Crowther et al., 2019).

We processed the human SEEG recordings by visually inspecting the signals recorded from electrodes and rejected those that exhibited potential artifacts. The signal was then high-pass filtered at 0.5 Hz to remove slow drifts. For signals obtained during the auditory stimulus trial, we re-referenced signals using a common average reference spatial filter, extracted individual trials (1000 ms preceding and 700 ms following auditory stimulus onset), and randomly selected 50 electrodes from each subject (450 electrodes in total, FIG. 12B) to reduce the computation time. For the signals obtained during the SPES trial, we re-referenced the signals using the averaged signals of electrodes that did not show strong CCEPs, and extracted the recording from each trial 500 ms preceding and 500 ms following stimulation onset.

We added the saline SPES artifact to the human SEEG recordings around 300 ms following the auditory stimulus onset, where it showed strong auditory-induced response (FIG. 9B). We validated the results within 0 to 50 ms following saline stimulation onset among various frequency bands, including theta (θ, 4-7 Hz), alpha (α, 8-12 Hz), low beta (lβ, 13-20 Hz), high beta (hβ, 20-30 Hz), low gamma (lγ, 30-50 Hz), and broadband gamma (bγ, 70-170 Hz). In brief, for each frequency band, we filtered the raw and clean signals (after denoising) from each electrode and took the absolute value of the Hilbert transform of the resulting signal. We then baseline-corrected and normalized the individual trials for each electrode by subtracting the average power during each trial's baseline period (500 ms to 100 ms preceding the stimulus onset), and dividing it through the standard deviation across all trial's baseline periods. We used forward-backward filters for all the filtering processes, in particular using pop eeg f iltnew( ) in EEGLAB.