7A and 7B are diagrams illustrating examples of a three-dimensional model. FIG. 7A is a diagram illustrating a point cloud 500 (an example of a point cloud). While size is not calculated by SfM, it is possible to perform association with the actual scale by, for example, performing comparison with a member having known dimensions or performing imaging with a scaler having a known dimension installed on the photographic subject.

The generation processing unit 106 applies, for example, a triangulated irregular network (TIN) model to the data of the point cloud obtained in the way described above to approximate the surface of the building with triangles, and can obtain another three-dimensional model (a polygon model, a mesh model, a surface model, a solid model, or the like) on the basis of the result. In the three-dimensional solid model among these models, the three-dimensional shape of a building is constructed as a combination of solid three-dimensional members such as blocks. To obtain the solid model, the user may designate “which range of the point cloud belongs to the same surface” via the operation unit 300, and the generation processing unit 106 may use the result. Alternatively, the generation processing unit 106 may use an algorithm such as random sample consensus (RANSAC) to estimate points belonging to the same surface and automatically generate the solid model without the user's operation. To generate the solid model, the generation processing unit 106 may use information such as the three-dimensional positions, colors (R, G, and B), and luminance of the point cloud and a change in the information.

If a three-dimensional model has already been generated or acquired by previous inspection or the like, the generation processing unit 106 may read the model (see FIG. 18 and the description related to FIG. 18 described below).

Specification of Members

The generation processing unit 106 (processor) preferably specifies members constituting the building in the three-dimensional model (specification process or specification step). That is, the generation processing unit 106 preferably specifies “to which member of the building each region of the three-dimensional model corresponds”. The generation processing unit 106 may specify the members in accordance with the user's operation (for example, designating a range constituting one member with the mouse 320) or may specify the members without the user specifying the members. When specifying a member, the generation processing unit 106 may use information on the shape and dimensions of the member. For example, information such as information indicating that “a member extending two-dimensionally in a horizontal plane and having an area equal to or greater than a threshold is a floor slab” or information indicating that “a member attached to a floor slab and extending one-dimensionally is a main girder” can be used. Further, the generation processing unit 106 may specify the members using a learning machine such as a DNN configured by machine learning in which members constituting a three-dimensional model are given as correct labels. FIG. 7B is a diagram illustrating an example of a three-dimensional model in which members are specified. In the illustrated example, a three-dimensional model 510 of a bridge is constituted by members, namely, a floor slab 512, a wall 514, and a leg 516.

Reconstruction of Three-Dimensional Model

Depending on the image-capturing conditions or the state of the photographic subject, a sufficient number of feature points are not acquired from captured images, and as a result, an accurate model cannot be generated (for example, one surface is incorrectly divided into a plurality of portions, or portions that would be on different surfaces are recognized as being included in the same surface). Accordingly, the generation processing unit 106 (processor) can reconstruct the three-dimensional model in response to an operation (for example, an operation of designating regions that are included in the same surface) performed by the user (reconstruction process or reconstruction step). Such reconstruction enables the user to obtain an appropriate three-dimensional model.

Development of Three-Dimensional Model into Two-Dimensional Image

The development processing unit 108 (processor) designates a condition (development condition) for developing the generated three-dimensional model into a two-dimensional image (step S130: designation process or designation step). For example, the development processing unit 108 can designate a range of development and an edge for development (the boundary of surfaces constituting the three-dimensional model) in accordance with an operation performed by the user. The development processing unit 108 may designate the development condition without the user's operation. FIGS. 8A, 8B, and 8C are diagrams illustrating how a development condition is designated for a three-dimensional model. In FIG. 8A, the three-dimensional model 510 (part of the bridge) is constituted by members such as a floor slab 520 and a leg 530. FIG. 8B illustrates designation of the leg 530 (portion indicated by thick lines) as the development range in accordance with a selection operation with the mouse 320, for example. FIG. 8C illustrates designation of the development range and an edge (portions both indicated by thick lines). The development range may be part or the whole of the three-dimensional model. The designation described above enables the user to develop a desired range with a desired edge. In addition, no need exists to develop an unnecessary portion. This can reduce a load on the system due to the development process.

The development processing unit 108 develops the three-dimensional model into a two-dimensional image under the designated condition (step S140: development process or development step). FIG. 9 is a diagram illustrating that the designated range (the leg 530) of the three-dimensional model 510 illustrated in FIG. 8C is developed along the designated edge. In FIG. 9, the designated range is developed into a left side surface 532, a wall surface 534, and a right side surface 536.

Development with Normal Directions Aligned

FIG. 10 is a diagram illustrating the leg 530 viewed from the side corresponding to the left side surface 532. The wall surface 534 is constituted by regions 534A, 534B, and 534C whose normal directions are directions indicated by vectors 537A, 537B, and 537C, respectively, the normal directions being different from each other. In this case, the development processing unit 108 can align the normal directions of regions constituting a two-dimensional image. For example, as illustrated in FIG. 11, the development processing unit 108 can develop the wall surface 534 to produce a wall surface 535 constituted by regions 535A, 535B, and 535C with the direction perpendicular to the plane of the figure (the side closer to the viewer) being represented by a normal vector. The development described above enables the user to observe individual regions of a subject from the same direction.

Division-Based Development of Curved Surface

The development processing unit 108 can develop a three-dimensional model such that a portion of the three-dimensional model including a curved surface is divided into planes having a designated size. FIGS. 12A, 12B, and 12C are diagrams illustrating a designation of the size of a plane (the width of a mesh). FIG. 12A is a diagram illustrating that the width of a plane (the width of a portion indicated by a dotted-line box) is designated in a side surface 542 of a cylindrical three-dimensional model 540 by a rectangular region 543. FIG. 12B illustrates that a portion of the side surface 542 is divided into regions 542A having the designated width. The development processing unit 108 can designate the size of a plane for division, such as the rectangular region 543, in response to selection by the user with the mouse 320, for example. FIG. 12C illustrates that the side surface 542 is divided into a plurality of planes 544 having the designated size (width) and two-dimensionally developed.

FIGS. 13A, 13B, and 13C are diagrams illustrating another designation of the size of a plane (the width of a mesh). FIG. 13A is a diagram illustrating that the width of a plane is designated for the three-dimensional model 540 by a rectangular region 545. The rectangular region 545 has a narrower width than the rectangular region 543 illustrated in FIG. 12A. Likewise, FIG. 13B illustrates that a portion of the side surface 542 is divided into regions 545A having a designated width, and FIG. 13C illustrates that the side surface 542 is divided into a plurality of planes 546 having the designated size (width) and two-dimensionally developed.

The development processing unit 108 can designate the size of a plane (the width of a mesh) in consideration of the accuracy of development, the calculation load, and the like. The size of a plane (the width of a mesh) may be changed for each region of a three-dimensional model. For example, it is possible to increase the width of meshes (coarse meshes) for regions having a flat shape and a nearly flat shape and decrease the width of meshes (fine meshes) for a region having a high curvature. The designation of the width of meshes allows appropriate plane development in consideration of the shape of the building, development accuracy, and the like.

Designation of Edge in Point Cloud Data

While FIGS. 8A to 8C illustrate an example in which a development range and an edge are designated for a three-dimensional solid model, a development range and an edge may be designated for three-dimensional point cloud data. FIG. 14 is a diagram illustrating designation of an edge 550 for the point cloud 500. The development processing unit 108 can perform this designation in response to selection by the user with the mouse 320, for example.

Generation and Mapping of Composite Image

If the three-dimensional model generated in step S120 is a model of only a shape, the mapping processing unit 112 (processor) can combine a plurality of images (captured images) (step S142: image combining process or image combining step) in a way described below and map the resulting image (composite image) to a two-dimensional image (or the original three-dimensional model) (step S144: mapping process or mapping step).

The mapping processing unit 112 can perform movement, rotation, deformation, or the like such that corresponding feature points between a plurality of images (captured images) can match to combine the images. This method is particularly effective for a planar object, and a high-quality image is obtained. The mapping processing unit 112 can also apply orthogonal projection to project a plurality of images onto the same plane or perform mosaic processing to combine the images. The orthogonal projection and the mosaic processing are effective for an object with irregularities. Accordingly, the mapping processing unit 112 may adopt an image combining method in accordance with the shape of the target (a photographic subject or a building).

Alternatively, the mapping processing unit 112 can perform mapping by using, for example, surface mapping, UV mapping (mapping using the uv coordinate system in addition to the xyz coordinate system), plane mapping, plane projection, or the like. The mapping processing unit 112 may use a plurality of mapping methods in combination, or may selectively use a plurality of methods in accordance with the shape of the three-dimensional model. FIGS. 15A and 15B are diagrams illustrating combining of images. In FIG. 15A, dotted-line boxes indicate the ranges of captured images, and a solid-line box indicates the range of a composite image. FIG. 15B illustrates an example of the composite image. FIG. 16 is a diagram illustrating a mapping image 538 obtained by mapping the composite image to a wall surface 535 of the leg 530. Regions 538A, 538B, and 538C are regions corresponding to the regions 534A, 534B, and 534C in FIG. 9 and the regions 535A, 535B, and 535C in FIG. 11, respectively.

The mapping processing unit 112 may generate a composite image from an image group constituting some of the captured images, or may generate a composite image from an image group selected by the user's operation or the like. For example, the mapping processing unit 112 can generate a composite image from high-quality images (for example, images with appropriate luminance, less noisy images with less blurring, or images captured from the front).

Mapping of Defect Information

The mapping processing unit 112 (processor) can map the defect information extracted in step S110 to a two-dimensional image into which the three-dimensional model is developed or a two-dimensional image to which the composite image is mapped (step S150: mapping process or mapping step). The output processing unit 118 (processor) outputs the two-dimensional image obtained by mapping by displaying the two-dimensional image on the display device 20, storing the two-dimensional image in the storage device 200 (the two-dimensional developed image 210), or using any other method (step S160: output process or output step).

Mapping to Three-Dimensional Image

The mapping processing unit 112 can also map the composite image to the three-dimensional model.