The abnormalities module 110 may measure distances between a number of salient features of one vertebra with salient features of another vertebra, for identifying disk pinches or bulges. Based on the identified disk pinches or bulges, herniated disks may be identified in the patients. It should be obvious to those skilled in the art, that given a wide variety of salient features and geometric rules, many spinal abnormalities could be identified. If the spinal abnormalities are identified, the PSNB module 108 may graphically identify areas having the spinal abnormalities and may send such information to a user device 118.

In one embodiment, information related to spinal surgeries may be displayed through a Graphical User Interface (GUI) of the user device 118, as illustrated using FIG. 1B. A smart phone is shown as the user device 118 in FIG. 1B, as an example. Further, the user device 118 may be any other device including a GUI, for example, a laptop, desktop, tablet, phablet, or other such devices known in the art.

The XZ screw angle module 112 may determine an XZ angle of a spinal screw or other implant to be used during the surgery. Further, the XY screw angle module 114 may determine an XY angle of the implant. The XZ screw angle module 112 and the XY screw angle module 114 may determine a position entry point for at least one spinal screw. The XZ screw angle module 112 and the XY screw angle module 114 may graphically represent the identified data and may send such information to the user device 118.

The screw size module 116 may be used to determine a screw diameter (e.g., a maximum screw diameter) and a length of the screw based on the salient features identified from the images of the patients.

In some implementations, the XZ screw angle module 112, the XY screw angle module 114, and the screw size module 116 can identify implant configurations for other types of implants in addition to, or other than screws (e.g., pedicle screws, facet screws, etc.) such as cages, plates, rods, disks, fusions devices, spacers, rods, expandable devices, etc. In addition, these modules may suggest implant configurations in relation to references other than an X, Y, Z, coordinate system. For example, in a spinal surgery, the suggestions can be in reference to the sagittal plane, mid-sagittal plane, coronal plane, frontal plane, or transverse plane. As another example, in an ACL replacement surgery, the suggestions can be an angle for a tibial tunnel in reference to the frontal plane of the femur. In various implementations, the XZ screw angle module 112, the XY screw angle module 114, or screw size module 116 can identify implant configurations using machine learning modules, algorithms, or combinations thereof, as described below in relation to FIGS. 6-9.

In one embodiment, referring to FIG. 2, a block diagram showing different components of the system 102 is explained. The system 102 includes a processor 202, interface(s) 204, and a memory 206. The processor 202 may execute an algorithm stored in the memory 206 for augmenting an implant surgery, e.g. by providing assistance to a surgeon during a spinal or other implant surgery, by providing controls to a robotic apparatus (e.g., robotic surgery systems, navigation system, etc.) for an implant surgery or by generating suggestions for implant configurations to be used in an implant surgery. The processor 202 may also be configured to decode and execute any instructions received from one or more other electronic devices or server(s). The processor 202 may include one or more general purpose processors (e.g., INTEL® or Advanced Micro Devices® (AMD) microprocessors) and/or one or more special purpose processors (e.g., digital signal processors or Xilinx® System On Chip (SOC) Field Programmable Gate Array (FPGA) processor). The processor 202 may be configured to execute one or more computer-readable program instructions, such as program instructions to carry out any of the functions described in this description.

The interface(s) 204 may help a user to interact with the system 102. The user may be any of an operator, a technician, a doctor, a doctor's assistant, or another automated system controlled by the system 102. The interface(s) 204 of the system 102 may either accept an input from the user or provide an output to the user, or may perform both the actions. The interface(s) 204 may either be a Command Line Interface (CLI), Graphical User Interface (GUI), or a voice interface.

The memory 206 may include, but is not limited to, fixed (hard) drives, magnetic tape, floppy diskettes, optical disks, Compact Disc Read-Only Memories (CD-ROMs), and magneto-optical disks, semiconductor memories, such as ROMs, Random Access Memories (RAMs), Programmable Read-Only Memories (PROMs), Erasable PROMs (EPROMs), Electrically Erasable PROMs (EEPROMs), flash memory, magnetic or optical cards, or other type of media/machine-readable medium suitable for storing electronic instructions.

The memory 206 may include modules, implemented as programmed instructions executed by the processor 202. In one case, the memory 206 may include a design module 208 for receiving information from the abnormalities module 110. The design module 208 may poll the surgeon for an information request. The design module 208 may allow the surgeon to design the spinal screw and change the generated implant configurations, such as the entry point (e.g., entry point into the patient, entry points into a vertebra, entry points to the implantation site, etc.), and screw or other implant angles in any of various planes. If the surgeon changes the entry point or angles, the system can automatically update other features of the implant configuration to account for the changes, such as the implant dimensions (e.g. screw diameter, thread pitch, or length). The design module 208 may include patient data 210. The patient data 210 may include images of patients and may allow the surgeon to identify the patients. A patient may refer to a person on whom and operations is to be performed. The patient data 210 may include images of patients, received from the user device 118.

In one embodiment, areas of interest may be defined in diagnostic data of a patient. In one case, the system 102 may determine the areas of interest based on pre-defined rules or using machine learning models, as described below in relation to FIGS. 6-9. In another case, the areas of interest may be defined based on a surgeon's input. In one case, the diagnostic data may include images of the patient. The images may be any of camera images, Magnetic Resonance Imaging (MRI) images, ultrasound images, Computerized Aided Tomography (CAT) scan images, Positron Emission Tomography (PET) images, and X-Ray images. In one case, the images of the patients may be stored in the patient surgeon database 210.

Post defining the areas of interest, a screw bone type may be defined based on various models and/or the surgeon's input. Successively, salient features of the areas of interest may be identified in the images of the patients, e.g. by applying the procedures described below. FIG. 3A shows salient points present in a top view of a vertebra of the patient, according to an embodiment. The salient points are shown as bubbles i.e. ‘e₁,’ ‘e₂,’ and ‘f₂.’ Further, FIG. 3B shows salient points present in a side view of the vertebra of the patient, according to an embodiment. The salient points are shown as bubbles i.e. ‘k_i,’ ‘k_u,’ ‘h_u,’ ‘i_m,’ and ‘z’.’

Successively, based on the salient points of the areas of interest, the system 102 may determine implant configurations (e.g. angles and entry point, implant orientation, implant movement, etc.) using the analysis procedures. FIG. 4A illustrates computations for determining the XZ angle (ϕ) 402 using the salient points, according to an embodiment. It should be noted that positions of X and Y co-ordinates of the regions of interest may be determined based on a location of at least one salient feature present in the image.

FIG. 4B illustrates computations for determining the XY angle (θ) 406 using the salient points, according to an embodiment. It should be noted that positions of X and Y co-ordinates of the regions of interest may be determined based on a location of at least one salient feature present in the image. Further, FIG. 4A illustrates a position entry point 404 for the spinal screw, and FIG. 4B illustrates a position entry point 408 for the spinal screw. Upon determining, MRI data including the XY angle, the XZ angle, and the position entry point for the spinal screw, may be stored in the abnormalities module 110.

Post identification of the angels and the entry point for an implant, the system 102 may determine additional implant configuration features. For example, the system 102 can determine a maximum implant (e.g. spinal screw) diameter, a minimum implant diameter, and a length of the implant to be used during a spinal surgery. For example, upon determining the maximum spinal screw diameter and the length of the spinal screw, the procedure MRI data may be updated in the abnormalities module 110.

In the spinal surgery example, the spinal screw having the determined maximum screw diameter and the length may be identified. The spinal screw may be suggested, to the surgeon, for usage during the spinal surgery. In one case, a spinal screw HA and dimensions of the spinal screw HA may be illustrated for the surgeon's selection, as shown in FIG. 5. As illustrated in FIG. 5, a schematic showing different parameters of the spinal screw HA, dimensions of the spinal screw HA, and a schematic of threads of the spinal screw HA are shown, according to an embodiment. Further, different such details related to spinal screws HB, spinal screws HD, and other known spinal screws may be presented to the surgeon for usage during the spinal surgery, thereby assisting the surgeon during the spinal surgery.

As another example, for an ACL replacement, upon determining the entry point and angle for a tibial tunnel for attaching a replacement graft, the system 102 can identify a depth for the tibial tunnel such that it will end above the center of the knee joint without piercing surrounding tissue. In addition, dimensions for the ACL graft itself and/or for screws or other fastening components can be suggested.

The flowchart 600 of FIG. 6 shows the architecture, functionality, and operation for providing assistance to a surgeon during a spinal surgery, according to an embodiment. One skilled in the art will appreciate that, for this and other processes and methods disclosed herein, the functions performed in the processes and methods may be implemented in differing order. Furthermore, the outlined steps and operations are only provided as examples, and some of the steps and operations may be optional, combined into fewer steps and operations, or expanded into additional steps and operations without detracting from the essence of the disclosed embodiments. For example, two blocks shown in succession in FIG. 6 may in fact be executed substantially concurrently or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. Any process descriptions or blocks in flowcharts should be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps in the process, and alternate implementations are included within the scope of the example embodiments in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved. In addition, the process descriptions or blocks in flow charts should be understood as representing decisions made by a hardware structure such as a state machine. The flowchart 600 starts at step 602 and concludes at step 610.

At step 602, areas of interest may be defined in diagnostic data of a patient and a screw bone type may be defined, during a spinal surgery. The diagnostic data may include images of the patient. The images may be any of camera images, Magnetic Resonance Imaging (MRI) images, ultrasound images, Computerized Aided Tomography (CAT) scan images, Positron Emission Tomography (PET) images, and X-Ray images.

At step 604, salient features of areas of interest may be identified from the diagnostic data. In one case, the images may be analyzed to identify abnormalities and the salient features, for performing spinal surgeries on the patients.

At step 606, an XZ angle, an XY angle, and a position entry point for an implant (e.g. a spinal screw) are determined. In one case, the XZ angle, the XY angle, and the position entry point may be determined based on the salient features.

At step 608, a maximum screw diameter and a length of the screw to be used during the spinal surgery may be determined based on the XY angle, the XZ angle, and the position entry point of the screw. Upon determining the maximum screw diameter and the length of the screw, the procedure MRI data may be updated in an abnormalities module 110.

At step 610, the screw implant to be used during a surgery may be identified and suggested to a surgeon. The screw implant may be identified based on the maximum screw diameter and the length of the screw.

FIG. 7 illustrates a flowchart showing a method 700 for generating implant configurations. At block 702, method 700 can obtain implant surgery information such as images, patent history, circumstance, test results, biographic data, surgeon recommendations, implant specifics, etc. Implant surgery images can be of parts of a patient, such as camera images, Magnetic Resonance Imaging (MRI) images, ultrasound images, Computerized Aided Tomography (CAT) scan images, Positron Emission Tomography (PET) images, X-Ray images, 2D or 3D virtual models, CAD models, etc. Additional implant surgery information can include, e.g. sex, age, height, weight, type of pathology, occupation, activity level, implant types and dimensions, availability of available implants, or aspects of a surgeon's preoperative plan (e.g. surgeon's initial implant configuration, detection and measurement of the patient's anatomy on images, etc.)

The implant surgery information can be obtained in various manners such as through direct user input (e.g. through a terminal or by interacting with a web service), through automatic interfacing with networked databases (e.g. connecting to patient records stored by a hospital, laboratory, medical data repositories, etc.), by scanning documents, or through connected scanning, imaging, or other equipment. The patient data can be gathered with appropriate consent and safeguards to remain HIPPA compliant.

At block 704, method 700 can convert the implant surgery information obtained at block 702 to be compatible with analysis procedures. The conversion can depend on the analysis procedure that will be used. As discussed below in relation to block 706, analysis procedures can include directly applying a machine learning model, applying an algorithm with multiple stages where any of the stages can provide machine learning model predictions (see FIG. 8A), or applying a virtual modeling system (see FIG. 8B).

In some implementations, the conversion of the implant surgery information can include formatting the implant surgery information for entry to a machine learning model. For example, information such as patient sex, height, weight, etc. can be entered in a feature vector, such as a sparse vector with values corresponding to available patient characteristics. In some implementations, the conversions can include transforming images from the implant surgery information into a format suitable for input to a machine learning model, e.g. an array of integers representing pixels of the image, histograms, etc. In some implementations, the conversion can include identifying surgery target features (detection and measurement of the patient's anatomy on images), characterizing surgery targets, or modeling (i.e. creating a virtual model of) the implant surgery target. For example, in a spinal surgery, this can include measuring vertebrae features on an image, converting 2D images of vertebrae into a 3D model, or identifying which vertebrae from a series of images are to be the target of the implant operation. As another example, in an ACL replacement surgery, this can include identifying and measuring features in an image such as location, size, and spacing of anatomy such as of the femur, patella, remaining portion of meniscus, other ligaments, etc., converting 2D images of the knee into a 3D model, or identifying other areas of damage (e.g. fractures, torn cartilage, other ligament tears, etc.).

In various implementations, the conversion process can be automatic, human supervised, or performed by a human technician, e.g. using tools such as a digital ruler and a digital angle finder. Further in the spinal surgery example, the conversion can include identifying a target set of vertebrae, initially localizing and marking the target set of vertebrae, performing segmentation for each of the target set of vertebrae, and marking cortical boundaries. In some implementations, input for the spinal implant surgery can specify a target set of vertebrae, however the surgical assistance system 164 can automatically perform calculations for additional vertebrae that weren't specified in the inputs. This can give the surgeon an option to expand the set of vertebrae to be fused, either prior to the operation or even during the procedure. In the ACL replacement surgery example, the conversion can include identifying a graft type (e.g. patella tendon, hamstring, cadaver ACL, etc.), initially localizing or marking the target drilling sites, performing segmentation for the target features (e.g. end of the femur), and marking boundaries (e.g. bone edges, meniscus edges, synovial membrane, etc.).

At block 706, method 700 can apply analysis procedures, using the converted implant surgery information from block 704, to identify implant configuration(s). In various implementations, the analysis procedures can include directly applying a machine learning model, applying a sequence of steps that can include one or more machine learning models, and/or generating a virtual model of the surgery target area and applying heuristics for implant configuration selection.

To apply a machine learning model directly, method 700 can provide the converted implant information to a machine learning model trained to specify implant configurations. A machine learning model can be trained to take input such as representations of a series of images and a feature vector for the patient and other aspects of the surgery (e.g. implant availability, surgeon specialties or ability indicators, equipment available, etc.) and can produce results that implant configurations. For example, for a spinal surgery, the machine learning model can suggest pedicle screw configurations, e.g. characteristics such as screw diameter, length, threading and application parameters such as screw insertion point, angle, rotation speed, etc. As another example, for an ACL replacement surgery, the machine learning model can suggest graft type, attachment type (e.g. screw material, length, or configuration features), graft attachment locations, drill depths, etc.

In some implementations, the converted implant information can be used in a multi-stage process for selecting aspects of an implant configuration. For example, for a spinal surgery, the multi-stage process can include method 800 or method 850, discussed below. In various steps of this these processes, either an algorithm can be used to generate results for that step or a machine learning model, trained for that step, can be applied.

In some implementations, the procedure for identifying implant configurations for a spinal surgery can include processing implant surgery information to locate targeted vertebrae and their pedicles in images, on available axes; identifying and tagging vertebrae characteristics; determining a preferred screw insertion point based on a mapping between tags and insertion point criteria (e.g. where the mapping can be a representation of a medical definition of a pedicle screw insertion point-described below); performing measurements, on the images, of the pedicle isthmus width and height and length of the pedicle and vertebral body, starting at the preferred insertion point; measuring the angle between the line used to determine length and the sagittal plane, in the axial view; and measuring the angle between that length line and the axial plane.

In some implementations, machine learning models can be trained to perform some of these tasks for identifying implant configurations. For example, machine learning models can be trained to identify particular vertebral pedicles in various images, which can then be atomically measured and aggregated across images, e.g. storing minimal, maximal, median, or average values, as appropriate given the target being measured. As another example, a machine learning model can receive the set of images and determine an order or can select a subset of the images, for automatic or manual processing. In some implementations, a machine learning model can be used to localize and classify the target within an image, such as by identifying a target vertebra or localizing the end of the femur or meniscus edges. In some implementations, a machine learning model can be used to segment target vertebrae, femur, tibia, or other anatomical features in the target area, to determine their boundaries. In some implementations, a machine learning model can be used to localize insertion points. In some implementations, a machine learning model can be used to segment images to determine boundaries of anatomical structures (e.g., boundaries of bones, organs, vessels, etc.), density of tissue, characteristics of tissue, or the like.

In various implementations, the results from the above stages can be used in inference formulae to compute the implant configurations. For example, a maximal screw diameter can be determined using the smallest pedicle isthmus dimension found across all the images of the target vertebrae (which can be adjusted to include a safety buffer). As another example, a maximal screw length can be determined using the smallest measurement of pedicle and vertebral body length, across all the target vertebra in question (which can be adjusted to include a safety buffer).

Machine learning models, as used herein, can be of various types, such as Convolutional Neural Networks (CNNs), other types of neural networks (e.g. fully connected), decision trees, forests of classification trees, Support Vector Machines, etc. Machine learning models can be trained to produce particular types of results, as discussed below in relation to FIG. 9. For example, a training procedure can include obtaining suitable training items with input associated with a result, applying each training item to the model, and updating model parameters based on comparison of model result to training item result.

In some implementations, automated selection of implant configurations can be limited to only cases likely to produce good results, e.g. only for certain pathologies, types of patients, surgery targets (e.g. the part of the spine that needs to be fused), or where confidence scores associated with machine learning model outputs are above a threshold. For example, in the spinal surgery example, automation can be limited to common types of spinal fusions, such as L3/L4, L4/L5, or L5/S1, certain pathologies such as spondylolisthesis or trauma, or patients with certain characteristics, such as being in a certain age group. As another example, for an ACL replacement, automation can be limited to cases without other ligament tears.

At block 708, method 700 can provide results specifying one or more features of an implant configuration. For example, the results for a spinal surgery can include selection of pedicle screw type and dimensions for each vertebra and guidance on an insertion point and angle for each screw. As another example, results for an ACL replacement surgery can include selection of implant graft type, connection type, joint dimensions, and guidance on connection points such as drill locations and depths. In some implementations, the results can be specified in a natural language, e.g. using templates that can be filled in with recommendations. In some cases, the results from the analysis of block 706 can be mapped to particular reasons for the implant configuration recommendations, and these reasons can be supplied along with the recommendations.

In some implementations, the results can be based on medical definitions of preferred implant configurations, where the preferred implant configurations can be mapped to a particular surgical target area based on the results from block 706. For example, results for spinal surgery pedicle screws can include a preferred insertion point, e.g. defined, for lumbar vertebrae, at the intersection of the superior articular facet, transverse process, and pars interarticularis; and for thoracic spine or cervical spine, at the intersection of the superior articular facet plane and transverse process. As another example, a preferred screw angle can be, in axial view, the angle between the sagittal plane and the line defined by the insertion point and midpoint of the pedicle isthmus. In sagittal view the preferred screw angle can be parallel to the superior vertebral endplate. In addition, a maximal screw length can be defined as the distance between the insertion point and the far cortical boundary of the vertebra, at a particular screw angle. A maximal screw diameter can be the minimal width of the pedicle isthmus, on any axis. Each of these can be modified to include a certain safety buffer, which can vary depending on the size of the vertebra. The results from block 706 can identify features of an individual patient, which can be used in conjunction with the foregoing implant configuration definitions to specify patient specific implant configurations, e.g. in natural language, as image annotations, in a 3D model, as discussed below.

The implant configuration results can be specified in various formats. For example, results can be in a natural language, coordinates one of various coordinate systems, as instructions for a robotic system, as annotations to one or more images, or as a virtual model of the surgery target area. The results can be used to augment the implant surgery in multiple ways. For example, results can be added to a preoperative plan. Results can trigger acquisition of implant materials, such as by having selected implants ordered automatically or having designs for patient-specific screws provided for 3D-printing. As another example, results can be used to provide recommendations during a surgical procedure, e.g. with text or visual annotations provided as overlies on a flat panel display, through auditory or haptic feedback alerts, or using an AR or VR system, e.g. to display an overlay of the implant on the patient anatomy or to display guidance on the suggested insertion point and angle. In some implementations, the results can be used to control robotic systems, e.g. causing a robotic arm to align itself according to the recommended insertion point and angle, which may be first confirmed by a surgeon.

The method 700 can be used in a wide range of procedures, e.g. open procedures, minimally invasive procedures, orthopedic procedures, neurological procedures, reconstructive implants, maxillofacial procedures (e.g., maxillary implants), or other procedure. In some surgical procedures, the implant information at block 702 can include implant dimensions, material information (e.g., composition of implant), and images of the patient. At block 706, the implant configuration can be implant dimensions (e.g., when in a delivery state or implanted state), implant functionality, or the like.

FIG. 8A illustrates a flowchart showing a method 800 for applying analysis procedures that can utilize machine learning models, according to an embodiment. In some implementations, method 800 is performed as a sub-process of block 706. At block 802, method 800 can receive implant surgery information. This can be some of the converted implant surgery information from block 704. In some implementations, the implant surgery information can include one or more images of the surgery target area, e.g. MRI scans of a spine, X-rays of a wrist, ultrasound images of an abdomen, etc.

At block 804, method 800 can localize and classify a target in one or more of the images of the surgery target area. In various implementations, this can be accomplished by applying a machine learning model trained for the particular target area to identify surgical targets or by finding a centroid point of each displayed vertebra, performing vertebral classification using image recognition algorithms, and determining whether the classified vertebrae match a list of vertebrae identified for the surgery. In some implementations, if the image does not contain at least one target of interest, the image can be disregarded from further processing.

At block 806, method 800 can segment the identified target(s) from block 804 to determine their boundaries. At block 808, method 800 can localize implant insertion points. In some implementations, blocks 806 and 808 can be performed using machine learning models or algorithms, e.g. that identify particular patterns, changes in color, shapes, etc.

At block 810, method 800 can localize and segment individual target features. For example, in a spinal surgery where targets are vertebrae, at block 810 method 800 can identify vertebrae pedicles and their isthmus, and measure these features. In some implementations, this can be accomplished using a machine learning model trained to detect each type of feature. In some implementations, detecting the pedicle and the isthmus of vertebra from annotated images can include measuring the isthmus width and tracking the minimal value across images and planes, defining the angle between the line that passes through at least two midpoints in the pedicle, and the reference plane, measuring the maximal length through that line, and tracking the minimal value across measurements. In some implementations, isthmus determination and measurement can be accomplished by starting at a point inside a pedicle, computing the distance to pedicle borders in multiple directions, taking the minimum length. In other implementations, the isthmus determination and measurement can be accomplished by scanning, for example using horizontal lines that intersect with pedicle borders in an axial view, and finding the minimum-length line.

In some implementations, the steps performed at any of blocks 804-810 can be repeated for each of multiple target area images, aggregating results from the multiple images. For example, in a step for identifying and measuring vertebrae pedicles, an aggregated measurement for a particular pedicle can be the minimum measured width of the pedicle from all of the images showing that particular pedicle.

At block 812, method 800 can use results from any of blocks 804-810 to compute an implant configuration (e.g. characteristics and application parameters). For example, the minimum width of a pedicle found across the images showing that pedicle, with some buffer added, can be the selected width characteristic of a pedicle screw implant. As another example, a screw angle could be determined using an identified insertion point and a center of the pedicle isthmus, with respect to center axis, depending on the image plane. The angles in axial and sagittal planes can be either the median or average angles across the multiple images. As a further example, a maximal screw length can be determined as the length of the line defined by the insertion point, the insertion angle, and the point where the line hits the cortical wall of the vertebra, minus some safety buffer. This length can be computed from multiple images and the minimum across all the images can be used for of this screw.

FIG. 8B illustrates a flowchart showing a method 850 for applying analysis procedures that can utilize virtual models, according to an embodiment. In some implementations, method 850 is performed as a sub-process of block 706. At block 852, method 850 can receive implant surgery information. This can be some of the converted implant surgery information from block 704. In some implementations, the implant surgery information can include one or more images of the surgery target area.

At block 854, method 850 can build one or more virtual models of the target surgery area based on the images and/or other measurement data in the implant surgery information. A virtual model, as used herein, is a computerized representation of physical objects, such as the target area of a surgery (e.g. portions of a patient's spine) and/or implants (e.g. screws, rods, etc.). In some implementations, virtual models can be operated according to known physical properties, e.g. reactions to forces can be predicted according to known causal relationships. In various implementations, the virtual models generated by method 850 can be two-dimensional models or three-dimensional models. For example, a two-dimensional model can be generated by identifying portions of an image as corresponding to parts of a patient's anatomy, such that a computing system can determine how implant characteristics would fit in relation to the determined anatomy parts. As another example, a three-dimensional model can be generated by identifying shapes and features in individual images, from a set of successive images, and mapping the identified shapes and features into a virtual three-dimensional space, using relationships between images. Finite element analysis techniques can be used to predict stresses, strains, pressures, facture, and other information and be used to design implants, surgical tools, surgical techniques, etc. For example, the implant configuration can be determined based on predetermined stresses (e.g., maximum allowable stresses in the tissue and/or implant, yield strength of anatomical structures and/or implant components, etc.), fracture mechanics, or other criteria defined by the physician or automatically determined based on, for example, tissue characteristics, implant design, or the like. In some embodiments, fatigue life can be predicted using stress or strain based techniques.

A virtual model can also analyze mechanical interaction between a patient's vertebrae, loading of implants, and other devices (e.g., rods, ties, brackets, plates, etc.) coupled to those implants. The output of these analyses can be used to select pedicle screw configurations, insertion trajectories, and placement location to optimize screw pull-out strength, maximum allowable loading (e.g., axial loads, shear loads, moments, etc.) to manage stresses between adjacent vertebrae, or maximum allowable stress in regions of the bone at risk for fracture.

In some embodiments, a user could identify areas of weakened bone or areas on images of the patient where there is risk of a fracture due to the presence of a screw or other implant. This information can be provided to the virtual model. The virtual model can be used to evaluate whether the configuration or location of the implant would create an unacceptable risk of fracture in the identified region. If so, the system could alert the user to that risk or modify the implant configuration or the procedure to reduce the risk to an acceptable level. In other embodiments, the system could identify these areas of high fracture risk automatically. In yet another embodiment, the system could provide data to the user such as the maximum torque to apply to a given pedicle screw during the surgical procedure such that tissue trauma, risk of fracture, or adverse advents is minimized.

At block 856, method 850 can localize and classify areas of interest within the virtual model(s) from block 854. This can be accomplished using object recognition that matches shapes of known objects to shapes within the virtual models. For example, in a virtual model for a spinal surgery, the images can be MRI images of vertebrae. The virtual vertebrae can be labeled (e.g. c1-s5) and virtual model vertebrae corresponding to the vertebrae for which the spinal procedure is planned can be selected as the areas of interest. In some implementations, additional areas around the selected areas can be added to the areas of interest, allowing the surgeon to select alternative options before or during the procedure. For example, the one or two vertebrae adjacent, on one or both sides, to the planned vertebrae can be additionally selected.

At block 858, method 850 can segment the areas of interest, identified at block 856, to determine various boundaries and other features, such as the pedicle boundaries and the pedicle isthmus. In some implementations, the segmentation or boundary determinations can be performed using a machine learning model. The machine learning model can be trained, for the type of implant surgery to be performed, to receive a portion of a virtual model and identify target portion segmentations or boundaries.

At block 860, method 850 can localize an insertion point for the implant in the target area. In some implementations, this can be accomplished by applying a machine learning model trained to identify insertion points. In some implementations, localizing insertion points can be accomplish using an algorithm, e.g. that identify particular patterns, changes in color, shapes, etc. identified as corresponding to preferred implant insertion points.

At block 862, method 850 can compute an implant configuration based on the virtual model(s) and/or determinations made in blocks 856-860. In some implementations, the implant can be associated with requirements for their application and properties to maximize or minimize. In these cases, the implant configuration can be specified as the configuration that fits with the virtual model, achieving all the requirements, and optimizing the maximizable or minimizable properties. For example, when method 850 is performed to determine pedicle screw configurations for a spinal surgery, virtual pedicle screws can be placed in a virtual model generated at block