As shown in FIG. 2, an example embodiment of an aircraft 100 is illustrated in which embodiments of systems and methods for obtaining electronic or digital information (e.g., 3D digital models, schematics, drawings, etc.) related to areas of interest of a physical object, such as the aircraft 100, can be implemented. As shown in FIG. 2, the aircraft 100 includes a fuselage 102 having a left side 104, a right side 106, a nose end 108 and a tail end 110. A first wing 112 is coupled to the left side 104 of the fuselage 102. A second wing 114 is coupled to the right side 106 of the fuselage 102. In the illustrated example, the aircraft 100 includes a door 116 disposed on the left side 104 of the fuselage 102. Passengers and/or crew may enter (e.g., board) and/or exit (e.g., disembark) the aircraft 100 via the door 116. The aircraft 100 of FIG. 2 is merely an example and, thus, the embodiments disclosed herein may be used with other aircrafts or vehicles without departing from the scope of this disclosure.

The locations or positions of the structures and systems of the aircraft 100 may be specified with respect to local coordinates of the aircraft 100. The coordinate system for the aircraft 100 is indicated in FIG. 2, with the x-axis indicating the fore-aft direction (e.g., front to rear of aircraft 100), the y-axis indicating the port-starboard direction (e.g., left and right of the center of the aircraft 100), and the z-axis indicating the up-down direction (e.g., bottom to top of aircraft 100). The x, y, and z coordinates with respect to the aircraft 100 may also be called the station (or fuselage station), the butt line, and the water line, respectively.

FIG. 3 illustrates of a system 200 for obtaining electronic or digital information about an area of interest 202 of a physical object 204 in a real-world environment 206, according to an exemplary embodiment. As shown in FIG. 3, the system 200 includes a portable electronic device 208, a communication network 210, a database 212, and communication devices 214 and 216. As illustrated, the portable electronic device 208 is in communication with the communication network 210. The communication network 210 of the system 200 may be used to provide communications links between various devices and computers connected together within the system 200. The communication network 210 may include connections, such as wires, wireless communication links, or fiber optic cables.

The portable electronic device 208 of the system 200 may be configured to display a live view of the physical object 204 in the real-world environment 206 (or a portion thereof) such that a user may view a representation of the physical object 204 in real time. The portable electronic device 208 may also be configured to determine a position or location of the area of interest 202 of the physical object 204 in the real-world environment 206, such as a location of an anomaly (e.g., a defect, a fault, a condition, damage, etc.) of the aircraft 100 of FIG. 2. Further, the portable electronic device 208 may be configured to obtain electronic or digital information (e.g., 3D digital models, virtual representations, technical documentation, schematics, maintenance reports, system data, photographic records, silhouette imagery, etc.) related to the area of interest 202 of the physical object 204.

The physical object 204 of the system 200 may include a vehicle, a building, a structure, a system, a subsystem, a power plant, a ship, a spacecraft, a surface or skin of a vehicle, a component, a part, and/or any other suitable physical object or article in the real-world environment 206. The real-world environment 206 may be any type of environment in the physical world, such as a workspace. In the illustrated example, the real-world environment 206 may be within the physical object 204, such as a fuselage of the aircraft 100 of FIG. 2. In other examples, the real-world environment 206 may be, without limitation, a maintenance environment, a manufacturing environment, a production environment, a design environment, an installation environment, and/or any other suitable environment.

As show in FIG. 3, the communication devices 214 and 216 of the system 200 may be connected to the communication network 210. The communication devices 214 and 216 may be, for example, wireless or computing devices operated by various personnel, such as maintenance personnel, mechanics, technicians, analysts, engineers, etc. The communication devices 214 and 216 may be located within the physical object 204 (e.g., inside an aircraft) or located at facilities remote from the physical object 204 (e.g. maintenance facilities).

The database 212 of the system 200 may be connected to the communication network 210. The database 212 may store information relating to the physical object 204. For example, the database 212 may include electronic or digital information (e.g., 3D digital models, virtual representations, schematics, specifications, designs, photographic records, silhouette imagery, etc.) about the architecture and structure of the physical object 204. The electronic or digital information may also include maintenance information (e.g., maintenance actions and messages, component installations and removals, etc.) about the physical object 204. Further, the electronic or digital information may include information about systems, subsystems, components, parts, etc. of the physical object 204. The electronic or digital information stored in the database 212 may be accessed and/or retrieved based on a coordinate system of the object and/or the relevancy to a position or location (e.g., coordinates) of the physical object 204. The electronic or digital information stored in the database 212 may also contain information in a graph database format, which stores data as nodes (entities) and relationships between them. These relationships are represented as edges, which can have various properties.

The portable electronic device 208 of the system 200 may be configured to capture physical or sensor data about the physical object 204 and determine geo-centric locational data (e.g., GPS location information) as well as the object-centric relative locational data (e.g., aircraft-centric) of the portable electronic device 208 within the real-world environment 206. Based on the sensor data and/or the locational data, the portable electronic device 208 may be configured to determine a location of the portable electronic device 208 in the real-world environment and relative to the physical object 204. The portable electronic device 208 may also be configured to measure a distance or range from the portable electronic device 208 to the area of interest 202 (e.g., an anomaly, a defect, a fault, a component, a part, damage, a condition, etc.) of the physical object 204. In some examples, the portable electronic device 208 may use a measurement device to project a laser or light beam 218 onto the physical object to illuminate the area of interest 202 of the physical object 204.

Once the portable electronic device 208 determines a distance to the area of interest 202 of the physical object 204, the portable electronic device 208 may determine the location or position of the area of interest 202 of the physical object 204 in a physical coordinate system. For example, the portable electronic device 208 may determine coordinates of the area of interest 202 in a physical coordinate system of the real-world environment or the portable electronic device 208. The portable electronic device 208 may convert the location of the area of interest 202 in the physical coordinate system into a position in a coordinate system of the physical object (e.g., aircraft 100 of FIG. 2) as further described below. For example, the portable electronic device 208 may translate or transform the location of the area of interest 202 of the physical object 204 in the physical coordinate system to a corresponding position in a coordinate system of the physical object (e.g., a virtual coordinate system of a digital model of the physical object 204).

After the location of the area of interest 202 of the physical object 204 is determined in the coordinate system of the object, the portable electronic device 208 may identify and obtain electronic or digital information (e.g., digital models, virtual representations, schematics, maintenance reports, specifications, designs, installation diagrams, digital twins, etc.) associated with or related to the area of interest 202 of the physical object 204. The portable electronic device 208 may retrieve the electronic or digital information from the database 212 of the system 200 or from the memory of the portable electronic device 208. To obtain the electronic or digital information, the portable electronic device 208 may generate a query based on the location of the area of interest 202 of the physical object 204 in the coordinate system of the object. The portable electronic device 208 may use the query to retrieve the electronic or digital information from the database 212 and/or the memory of the portable electronic device 208. For example, the portable electronic device 208 may retrieve electronic or digital information that matches or relates to the area of interest 202 of the physical object 204. In some examples, the portable electronic device 208 may retrieve a 3-D digital model of the physical object (e.g., a digital model of an aircraft) or a 3-D model that corresponds to the area of interest of physical object (e.g., a digital model of a structure, system or component of an aircraft).

The portable electronic device 208 may display the electronic or digital information (e.g., virtual representations, schematics, specifications, etc.) obtained from memory or the database 212 and/or send the electronic or digital information to other communication devices, such as the communication devices 214 and 216. For example, the portable electronic device 208 may display a virtual representation of the area of interest 202 of the physical object 204 and send the virtual representation to the communication devices 214 and 216. The communication devices 214 and 216 may be configured to display the electronic or digital information received from the portable electronic device 208. In some examples, the portable electronic device 208 may send the location of the area of interest 202 of the physical object 204 in the coordinate system of the object to the communication devices 214 and 216. The communication devices 214 and 216 may use the location of the area of interest 202 to access the electronic or digital information from the database 212 or local memory. For example, on-site and/or off-site analysts may use the communication devices 214 and 216 to access and retrieve electronic or digital information from the database 212 based on the position or location of the area of interest 202 of the physical object 204 in the coordinate system of the object.

As showing in FIGS. 4A and 4B, the portable electronic device 208 of the system 200 may be a compact device that may be handled or carried by a user or maintenance personnel. The portable electronic device 208 may include a smartphone, tablet computer, laptop computer, or any other suitable device. As shown in FIG. 4A, the portable electronic device 208 may display an image of the area of interest of the physical object along with graphic or virtual representations of the area of interest. The portable electronic device 208 may correlate and match-up a low fidelity digital image (based on a model) with the high fidelity digital image (based on a photographic database). For example, once the portable electronic device 208 determines a location of an area of interest or object (in both digital and physical worlds), the portable electronic device 208 may match-up a low fidelity digital image (based on a model) to a high fidelity digital image (based on a photographic database). As an example, a user or mechanic, performing an evaluation to verify that wiring is placed in the correct location, may view reality or the physical world (e.g., aircraft interior) as well as a digital photo from a database (e.g., digital world) displaying where the wires should be. In some examples, the portable electronic device 208 may include a wearable device (e.g., a pair of augmented reality glasses). For example, the wearable device may provide a live view of the real-world environment by superimposing a representation of the real-world environment on a transparent or translucent display that functions similar to eyeglass lenses, such that a user is able to view the real-world environment through the display.

Referring now to FIG. 5, a schematic illustration of the components of the portable electronic device 208 is shown. The portable electronic device 208 may include a communication unit 520, a sensor system 522, a storage device 524 (e.g., a memory or a database), a processing unit 526, a user interface 528, and a display device 530. In other examples, the portable electronic device 208 may include additional components, hardware, or functionality. A bus 532 may couple the communication unit 520, the sensor system 522, the storage device 524, the processing unit 526, the user interface 528, and the display device 530 together to enable communication there-between. Although only one bus is depicted, the portable electronic device 208 may include multiple buses or other types of communication pathways between any of its elements or components.

The communications unit 520 of the portable electronic device 208 may be configured to be connected to a communication network (e.g., the communication network 210 of FIG. 3). The communication units 520 may receive data/communications from and send data/communications to other devices, such as remote communication and/or computing devices (e.g., the communication devices 214 and 216 of FIG. 3) within the communication network. The communications unit 520 may enable the portable electronic device 208 to communicate, via a wireless channel or a wired communication link, with other devices. For example, the communications unit 520 may enable the portable electronic device 208 to wirelessly transmit a position or location of an area of interest of a physical object (e.g., the physical object 204 of FIG. 3) to other devices. The communications unit 520 may also enable the portable electronic device 208 to wirelessly transmit electronic or digital information (e.g., virtual representations, schematics, maintenance reports, drawings, component information, etc.) retrieved from the storage device 524 to other devices.

The communications unit 520 may include wireless connections, wired connections, cable connections, fiber optics connections, etc. and may communicate via a wide area network (WAN), a local area network (LAN), a cellular network, a peer-to-peer communication network, or any other suitable network. The communications unit 520 may also be operative to interface with a communications network using any type of communication protocol, such as, for example, Wi-Fi (e.g., 802.xx protocols), a radio frequency (RF) protocol (e.g., 900 MHz, 1.4 GHz, and 5.6 GHZ), Bluetooth®, a cellular communication protocol (e.g., 2G, 3G, 4G, 5G, etc.), or any other communication protocol.

The sensor system 522 of the portable electronic device 208 may be configured to capture and collect physical or sensor data (e.g., image data, ranges, distances, position information, etc.) about one or more physical objects in a real-world environment. The sensor system 522 can include various types of sensors, such as GPS sensors, inertial measurement units (IMU) or sensors, measurement sensors, image sensors (e.g., image capture devices for capturing images of physical objects in the real-world environment), or any other suitable sensor. The sensor system 522 may send the sensor data to the processing unit 526. The sensor data may be processed by the processing unit 526 to determine the position and orientation of the portable electronic device 208 in the real-world environment and/or relative to the physical objects in the real-world environment as further described below.

The GPS sensor of the sensor system 522 may be configured to provide information regarding the position or location (e.g., location coordinates) of the portable electronic device 208 in the real-world environment. The IMU of the sensor system 522 may sense position and orientation changes of the portable electronic device 208 based on inertial acceleration. For example, the IMU may detect a pitch and yaw of the portable electronic device 208 while the portable electronic device 208 is stationary or in motion. The IMU may include one or more accelerometers that generate acceleration sensor data. The one or more accelerometers may be used to measure static acceleration, such as the tilt of the portable electronic device 208 relative to gravity, as well as dynamic acceleration resulting from motion of the portable electronic device 208. The IMU may also include one or more gyroscopes configured to generate sensor data indicating a current location or orientation of the portable electronic device 208.

The image capture device or sensor of the sensor system 522 may be configured to capture image data of the real-world environment within its field of view. The image capture device may add geographical location data into metadata fields of the captured image data. The captured image data may be used to determine the location or position of the portable electronic device 208 relative to the physical objects in the real-world environment. The captured image data may also be displayed on the display device 530 of the portable electronic device 208. In some examples, the image capture device may be a camera including three-dimensional capabilities. As shown in FIG. 4B, the imaging capture device may be located at the rear or back of the portable electronic device 208.

The measurement sensor of the sensor system 522 may be configured to measure ranges and/or distances from the portable electronic device 208 to areas of interest of the physical object in the real-world environment. For example, the portable electronic device 208 may measure the distance from the portable electronic device 208 to an anomaly (e.g., damage, a defect, etc.) of a physical object (e.g., aircraft) in the real-world environment. In some examples, the measurement device may be configured to project a laser or light beam onto the physical object to illuminate an area of interest of the physical object. The measurement sensor may include a light emitting device, a laser device, an optical device, or any other suitable measurement sensor.

Referring still to FIG. 5, the storage device 524 of the portable electronic device 208 may store the physical or sensor data captured by the sensors of the sensor system 522. The storage device 524 may also store information relating to one or more physical objects. For example, the storage device 524 may include electronic or digital information (e.g., digital models, virtual representations, schematics, specifications, designs, installation diagrams, system information, etc.) about the physical objects. Further, the storage device 524 may store digital models and/or virtual representations of the physical objects. The digital models may include representations of structures, assemblies, systems, and subsystems of the physical object. In some examples, the storage device 524 may store a digital model representing a fuselage of an aircraft. The storage device 524 may also store mapping or positional data that represents a spatial or physical coordinate-based map of the physical objects in the real-world environment.

The storage device 524 of the portable electronic device 208 may also store program instructions that are executed or carried out by the processing unit 526 of the portable electronic device 208. The storage device 524 may include physical, non-transitory, computer-readable memory that stores data on a temporary or permanent basis for use by the processing unit 526. The memory may include one or more volatile and/or non-volatile memory devices, such as random access memory (RAM), static random access memory (SRAM), dynamic RAM (DRAM), read only memory (ROM), flash memory, or any other suitable medium or memory which may be used to store desired information (e.g., system information, virtual models, mapping information, etc.).

The processing unit 526 of the portable electronic device 208 may be in communication with the various components of the portable electronic device 208. The processing unit 526 may include one or more processors. For example, the processing unit 526 may include one or more central processing units (CPU), one or more graphical processing units (GPU), one or more digital signal processors (DSP), one or more peripheral interface controllers (PIC), or another type of microprocessors.

The processing unit 526 may be configured to identify and select a digital model of a physical object in the real-world environment. The processing unit 526 may retrieve the digital model of the physical object from the storage device 524 or a remote database, such as the database 212 of FIG. 3. In some examples, the processing unit 526 may select and/or retrieve a digital model representing a fuselage of an aircraft. In other examples, the processing unit 526 may retrieve, based on the physical object, a digital model representing a building, a vehicle, an industrial facility, a power plant, a ship, a spacecraft, a submarine, or any other suitable object.

The digital model may represent a 3-D model or representation of the physical object including the systems and structures of the physical object. In some examples, the digital model may be a computer-aided design (CAD) model and may use a coordinate system to identify positions of spatial or virtual content of the digital model of the physical object. The digital model may be based on the design, testing, manufacturing, installation, and/or operational phases of the physical object.

Once the processing unit 526 selects and retrieves the digital model of the physical object, a user may use the portable electronic device 208 to identify positions or locations of areas of interest of physical objects in the real-world environment. When the user is using the portable electronic device 208, the processing unit 526 may receive and collect physical or sensor data associated with the physical objects in the real-world environment 206 from the sensor system 522. For example, the processing unit 526 may capture image data about the physical objects in the real-world environment. Based on the sensor data, the processing unit 526 may be configured to determine the location of the portable electronic device 208 in the real-world environment and the locations or positions of the physical objects in the real-world environment. For example, the processing unit 526 may use GPS data to determine the location of the portable electronic device 208 in the real-world environment (e.g., geo-centric or aircraft centric (relative to the aircraft)) and may use image data to determine the position and orientation of the portable electronic device 208 in the real-world environment (e.g., geo-centric or aircraft centric (relative to the aircraft)). Further, the processing unit 526 may be configured to determine the distance of the portable electronic device 208 to the physical objects and/or items or markers on the physical object in the real-world environment. Additionally, the processing unit 526 may be configured to determine the location of the portable electronic device 208 relative to the physical objects and items or markers of the physical object.

The processing unit 526 may also be configured to map the sensor data of the real-world environment in a physical coordinate system or reference frame. For example, the processing unit 526 may generate mapping or positional data that represents a spatial or physical coordinate-based map of the real-world environment. The processing unit 526 may map the real-world environment to establish a relationship between a position or location of the portable electronic device 208 and the positions of the physical objects within the real-world environment 206 such that, upon mapping the real-world environment, the physical objects are assigned specific positional coordinates within the physical coordinate system. The physical coordinate system may be based on the position of the portable electronic device 208 within the real-world environment 206. In some examples, the physical coordinate system may be three-dimensional and include three mutually-perpendicular axes.

The processing unit 526 may also be configured to track the position and orientation of the portable electronic device 208 in the real-world environment. For example, the processing unit 526 may process the sensor or physical data received from the sensor system 522 to determine the position and orientation of the portable electronic device 208 relative to the physical objects in the real-world environment. As the portable electronic device 208 moves within the real-world environment (e.g., a fuselage of an aircraft), the processing unit 526 may be configured to track the physical objects in the real-world environment 206 for determining the position and orientation of the portable electronic device 208 relative to the physical objects and/or items or markers of the physical objects in the real-world environment. For example, the processing unit 526 may track changes in the proximity and angle of the portable electronic device 208 relative to the physical objects or items of the physical object in the real-world environment. In some examples, the items may include fiducial markers. Based on the perceived changes in the real-world environment surrounding the portable electronic device 208, the processing unit 526 may calculate movement (e.g., translation and/or rotation) of the portable electronic device 208 and determine a current position and orientation of the portable electronic device 208 relative to the physical objects in the real-world environment 206.

The processing unit 526 may be configured to align or spatially-register the physical object in the real-world environment 206 to a digital model or representation of the physical object. The processing unit 526 may use a transformation or transfer function to align the physical object defined in a physical coordinate system to digital or virtual content of a digital model of the physical object defined in a local or object coordinate system. Once the physical object in the real-world environment is aligned with the digital model of the object, the portable electronic device 208 may be configured to determine positions of areas of interest of a physical object in the object coordinate system.

As shown in FIG. 6, the portable electronic device 208 may use triangulation techniques to determine a position or location of an area of interest 602 of a physical object 604 in a real-world environment 606. In other embodiments, the portable electronic device 208 may use trilateration techniques, multilateration techniques, or any other suitable technique to determine the location of the area of interest 602 of the physical object 604 in the real-world environment. As shown in FIG. 6, the portable electronic device 208 can determine a first position (P1) of the portable electronic device at a first time. From the first position (P1), the portable electronic device 208 may determine a distance or length from the first position (P1) to the area of interest 602 of the physical object 604. At a second time, the portable electronic device 208 can determine a second position (P2) of the portable electronic device 208 and a distance or length from the second position (P2) to the area of interest 602 of the physical object 604.

After the portable electronic device 208 determines the location of the first and second positions, the portable electronic device 208 may calculate the distance (D) between the first position (P1) and the second position (P2). Based on these calculations, the portable electronic device 208 can calculate the location of the area of interest 602 of the physical object 604 based on the following expressions:

tan
⁡
(

angle
⁢

1

)

=

d
/
x

(
1
)

tan
⁡
(

angle
⁢

2

)

=

d
/

(

D
-
x

)

(
2
)

d
=

x
⋆

tan
⁡
(

angle
⁢

1

)

(
3
)

d
=

(

D
-
x

)

⋆

tan
(

Angle
⁢

2

)

(
4
)

x
⋆

tan
⁡
(

angle
⁢

1

)

=

(

D
-
x

)

⋆

tan
⁡
(

angle
⁢

2

)

(
5
)

x
⋆

tan
⁡
(

angle
⁢

1

)

=

D
⋆

tan
⁡
(

angle
⁢

2

)

-

x
⋆

tan
(

angle
⁢

2

)

(
6
)

x
⋆

tan
⁡
(

angle
⁢

1

)

+

x
⋆

tan
⁡
(

angle
⁢

2

)

-

D
⋆

tan
(

angle
⁢

2

)

(
7
)

x
⋆

[

tan
⁡
(

angle
⁢

1

)

+

tan
⁡
(

angle
⁢

2

)

]

-

D
⋆

tan
(

angle
⁢

2

)

(
8
)

x
=

D
⋆

tan
⁡
(

angle
⁢

2

)

⁢

/
[

tan
⁡
(

angle
⁢

1

)

+

tan
(

angle
⁢

2

)

]

(
9
)

The portable electronic device 208 may use other techniques to determine a position or location of an area of interest 602 of a physical object 604 in a real-world environment 606. As shown in FIGS. 7A-7E, the portable electronic device may determine the position or location of the electronic device using trilateration of the portable electronic device relative to three (3) known locations of the object. Initially, the portable electronic device may be geo-located at three (3) positions relative to three (3) known and/or fixed features or items (P1, P2 and P3) of the object as shown in FIGS. 7A-7C. For example, for an aircraft, the portable electronic device may determine the location of the portable electronic device within the aircraft based on three known and fixed items, such as an RFID chip, a QR code at a known reference point, and a door hinge, within the aircraft. The portable electronic device may determine a first location or position (PA) of the portable electronic device relative to the three known items (P1, P2, and P3) at a first time as shown in FIG. 7A. The portable electronic device may also determine a second location (PB) of the portable electronic device relative to the three known items (P1, P2, and P3) at a second time as shown in FIG. 7B. Further, the portable electronic device may determine a third location (PC) of the portable electronic device relative to the three known items (P1, P2, and P3) at a third time as shown in FIG. 7C.

After the portable electronic device determines the three locations of the portable electronic device relative to the three known items, the portable electronic device may determine a position or location of an area of interest of a physical object in a real-world based on the first position (PA), the second position (PB), and the third position (PC) of the portable electronic device as shown in FIG. 7D. The portable electronic device may calculate a distance from the first position (PA) area to the area of interest (PX), calculate a distance from the second position (PB) to the area of interest (PX), and calculate the distance from the third position (PC) to the area of interest (PX). Based on these calculations, the portable electronic device 208 can calculate the location of the area of interest (PX) (e.g., an area of interest of the object) in three dimensions as shown in FIG. 7D-7E based at least on the following expressions:

(

D
z

-

A
z

)

2

+

(

D
y

-

A
y

)

2

-

❘
"[LeftBracketingBar]"

AD

❘
"[RightBracketingBar]"

2

=
0

(
10
)

(

D
z

-

B
z

)

2

+

(

D
y

-

B
y

)

2

-

❘
"[LeftBracketingBar]"

BD

❘
"[RightBracketingBar]"

2

=
0

(
11
)

(

D
z

-

C
z

)

2

+

(

D
y

-

C
y

)

2

-

❘
"[LeftBracketingBar]"

CD

❘
"[RightBracketingBar]"

2

=
0

(
12
)

D
z

=

(

(

(

-

A
x

)

2

+

(

B
x

)

2

-

(

A
y

)

2

+

(

B
y

)

2

-

❘
"[LeftBracketingBar]"

BD

❘
"[RightBracketingBar]"

2

+

❘
"[LeftBracketingBar]"

AD

❘
"[RightBracketingBar]"

2

)

/
2

)

-

(

D
y

(

B
y

-

A
y

)

)

/

(

B
x

-

A
x

)

(
13
)

where A(A_z,A_y) is the location of PA, where B(B_z,B_y) is the location of PB, where C(C_z,C_y) is the location of PC, where D(D_z,D_y) is the location of PX, and where the distances |AD|, |BD|, |CD| are known.

Referring again to FIG. 6, the portable electronic device 208 may utilize any suitable coordinate system for determining the position of the portable electronic device 208 within the real-world environment and/or the position of the area of interest 602 of the physical object 604. For example, the portable electronic device 208 may use a Geodetic Coordinate System in which a location on the Earth is specified by longitude (e.g., in degrees East or West of the Prime Meridian) and latitude (e.g., degrees North or South from the Equator), and altitude is specified by height above Mean Sea Level (MSL). This coordinate system provides spherical coordinates (e.g., approximating the shape of the Earth).

The portable electronic device 208 may also use a local coordinate system of East, North, and Up (ENU). In this coordinate system, the location is specified by units East and North of the coordinate system origin (e.g., located on the Earth by a geodetic pair), and altitude is specified by height above Mean Sea Level (MSL). Further, the portable electronic device 208 may use a local coordinate system of North, East, and Down (NED). This coordinate system is similar to the ENU system wherein the x component is the same as the ENU East component, the y component is the same as the ENU Up component, and the z component is the negative of the ENU North component. The NED coordinate system is similar to the earth-centered-earth fixed (ECEF) coordinate system. The relationship between the NED coordinate system and the ECEF coordinate system is given by the following expression:

P
NED

=

R
T

(

P
ECEF

-

P
Ref

)

where P_NEDis a 3D position in a NED system, P_ECEFis the corresponding ECEF position, P_Refis the reference ECEF position (where the local tangent plane originates), and where R is a rotation matrix with columns in the north, east, and down axes and may be defined from the latitude phi and longitude lambda corresponding to P_Refas follows:

R
=

[

−
⁢

sin
⁡

(
ϕ
)

⁢

cos
⁡

(
λ
)

−
⁢

sin
⁡

(
ϕ
)

⁢

sin
⁡

(
λ
)

cos
⁡

(
ϕ
)

−
⁢

sin
⁡

(
λ
)

cos
⁡

(
λ
)

0

−
⁢

cos
⁡

(
ϕ
)

⁢

cos
⁡

(
λ
)

−
⁢

cos
⁡

(
ϕ
)

⁢

sin
⁡

(
λ
)

−
⁢

sin
⁡

(
ϕ
)

]

In some implementations, the portable electronic device 208 may use GPS location information to determine the location of the area of interest 602 of the physical object 604 (e.g., aircraft) in the real-world environment (e.g., geo-centric or object/aircraft centric (relative to the object/aircraft)). For example, the portable electronic device 208 may determine a GPS location of the area of interest 602 and GPS locations of known fixed items or markers of the physical object 604 (e.g., aircraft). The portable electronic device 208 may then calculate an object-centric location (e.g., aircraft-centric location) of the area of interest 602 relative to the known fixed items or markers of the object (e.g., aircraft). The portable electronic device 208 may also use GPS location information to determine the location of the portable electronic device 208 in the real-world environment (e.g., geo-centric or object centric (relative to the object)).

Referring again to FIG. 5, once the processing unit 526 of the portable electronic device 208 determines a location of an area of interest of a physical object in the real-world environment defined in a physical coordinate system, the processing unit 526 may be configured to determine a location of the area of interest of the physical object in coordinates of the coordinate system of the physical object. The processing unit 526 may use a transformation or transfer function to transform or convert the coordinates of the location of the area of interest of the physical object in the real-world environment into coordinates of the coordinate system of the physical object. In some examples, the coordinates of the coordinate system of the physical object may be three-dimensional coordinates defined along three mutually perpendicular axes within the coordinate system of the object.

Once the coordinates of the area of interest of the physical object are determined in the coordinate system of the object, the processing unit 526 may retrieve electronic or digital information about the area of interest of the physical object from the storage device 524 or a database. The processing unit 526 may be configured to search electronic or digital records and reports (e.g., damage report records, service requests, etc.) in one or more databases for information relating to the areas of interest of the physical object. For example, the processing unit 526 may search electronic reports and records for problems related to the location of the damage for a particular type of vehicle or aircraft. The processing unit 526 may identify and cause the relevant electronic or digital records to be displayed to support or maintenance personnel. For example, the processing unit 526 may be configured to cause the electronic or digital information (e.g., graphic or virtual representations of the area of interest) to be displayed on the display device 530 of the portable electronic device 208. Further, the processing unit 526 may send the electronic or digital information to other communication or computing devices to enable remote personnel to analyze the area of interest of the physical object. Additionally, the processing unit 526 may also store or log any current damage reports of the physical object in a database as a new record or report. For example, the processing unit 526 may create and store a service request or problem report, including the description and type of damage, object or vehicle type, photos and sketches of the damage, etc., in a database (e.g., airline database) based on the physical location of the damage.

Referring still to FIG. 5, the user interface 528 of the portable electronic device 208 may allow a user or maintenance personnel to interact with the portable electronic device 208. The user interface 528 may include an interactive touchscreen. In other examples, the user interface 528 may include a keyboard, a mouse, microphones, or any other suitable input/output device. The user interface 528 of the portable electronic device 208 may be configured to receive inputs and/or user selections to enable the portable electronic device 208 to select one or more digital models representative of a physical object. For example, a user may input into the user interface 528 information relating to a physical object in the real-world environment (e.g., an aircraft), a system of the physical object (e.g., an electrical wiring system of an aircraft), a component of a system and/or other information related to the physical object. After the information is received, the portable electronic device 208 may display graphical or virtual content on the user interface 528 or the display device 530 related to the area of interest of the physical object as further described below.

The display device 530 of the portable electronic device 208 may be configured to present visual, audio, and/or tactile information to the user or maintenance personnel. The display device 530 may include a screen or any another suitable type of display. In other examples, the display device 530 may be integrated into a transparent or translucent visor of an optical see-through augmented reality (AR) imaging device and viewable by a user or technician wearing the AR imaging device.

The display device 530 of the portable electronic device 208 may be configured to display a live view of the real-world environment such that a user is able to view a representation of physical objects in the real-world environment in real time. In some examples, the display device 530 may display a representation of a fuselage of an aircraft (or a portion thereof). The display device 530 may also display augmented-reality (AR) content, such as graphical or virtual content. The display device 530 of the portable electronic device 208 may also be configured to display graphical or virtual representations of the area of interest of the physical object and/or images of the areas of interest as shown in FIG. 4A.

FIG. 8 illustrates a flow diagram of a method 800 of obtaining electronic or digital information (e.g., 3D digital models, virtual representations, schematics, drawings, etc.) related to an area of interest of a physical object (e.g., an aircraft) in a real-world environment, according to an exemplary embodiment. The method may be performed or implemented entirely, or in part, by a portable electronic device or AR device, such as the portable electronic device