>Each FIG. 7 and FIG. 8 illustrates a display switch example by the linked display system 10.

In a display state A (the all-content display state) of FIG. 7 and FIG. 8, when connection between the information device 2 and the HMD 3 is established and then the information device 2 transmits the text data to the HMD 3, transition of the display state A1 to a display state B1 (see FIG. 7) or a display state B2 (see FIG. 8) is made. When the information device 2 or the HMD 3 receives an instruction for terminating the linkage, transition from the display state B1 or the display state B2 to the display state A is made.

In the display state A, that is, in the display state where the content is displayed only by the information device, the content containing the text data and an image A is displayed on the information device 2. At the right end of the screen 2g of the touch screen display 204, a scroll bar 2g1 for moving the display screen in the front and rear directions is provided.

In the display state B1, only the image A is displayed on the information device 2 while only the text data is displayed on the HMD 3. The display size of the image A in the display state B1 is larger than the display size of the image A in the display state A. In this connection, although it is often the case to refer an object displayed on the HMD 3 as an “AR”, a virtual object (corresponding to the text data in FIG. 7) displayed on the HMD 3 is not necessarily associated with a real object or a position.

In the display state B1, the text data is gathered and displayed on the left side of the display area (virtual screen) of the HMD 3 so as to ensure the visibility of the center of the visual field of the wearer (user 1) of the HMD 3.

As illustrated in the display state B2, the text data may be gathered and displayed near the top of the virtual screen of the HMD 3 so as to ensure the visibility of the center of the visual field of the wearer of the HMD 3. In particular, in the case where the wearer of the HMD 3 is working while looking down at his or her hand or a real object, such as for cooking, displaying a virtual object only on the upper part of the virtual screen of the HMD 3 can ensure the visibility during cooking. Depending on working states, the position on which the virtual object is to be displayed may be set to either the lower part or the left or right part. The display position may be arbitrarily determined by the wearer, or may be automatically set by the HMD 3. In the case where the HMD 3 automatically sets the position, the out-camera detects a moving range of the wearer's hand so that the virtual object can be displayed on a position of the display area of the HMD 3 where the virtual object does not overlap the detected moving range of the wearer's hand.

In each of the display states B1 and B2, in the case where the number of characters in the text data is so large that they cannot fit within the virtual screen, the display control section 501 displays a virtual object M having an arrow shape for scrolling the virtual screen together with the text data on the virtual screen of the HMD 3. An arrow selection area M1 is formed by a closed area including the virtual object M. The arrow selection area M1 is an area in which the display control section 501 determines that the line of sight has selected the arrow.

The line-of-sight detection section 504 is configured to cause the in-camera to capture an image of the face of the wearer of the HMD 3 so as to detect a line-of-sight direction based on the captured face image. Then, upon detecting that the line of sight of the wearer has reached the arrow selection area M1, the display control section 501 makes a displayable area of the HMD 3 move in the direction of the arrow. The HMD 3 notifies the information device 2 that the screen has been scrolled. When an image associated with newly displayed text data as a result of scroll is an image B which is different from the image A, the information device 2 switches the image A to the image B.

Each FIG. 9 and FIG. 10 explains a flow of processing of the linked display system. In step S100, an application is started. The following processing is started from a state in which a main power of the information device 2 and that of the HMD 3 are turned on, the HMD linking application is started in each of the information device 2 and the HMD 3, and pairing therebetween is completed.

FIG. 9 illustrates a flowchart of processing of analyzing the content on the side of the information device 2.

In a state where the text data and the image A are displayed (display state A) on the information device 2, a user operates the information device 2 to switch to a mode for browsing the content on the HMD 3 (step S101).

The communication control section 403 of the information device 2 confirms whether connection with the HMD has been established (step S102).

The content analysis section 402 of the information device 2 extracts only text from the displayed content to generate text data (step S103).

The content analysis section 402 outputs the generated text data to the communication control section 403. The communication control section 403 transmits the text data via the wireless communication unit 206 to the HMD 3 (step S104).

The communication control section 503 of the HMD 3 receives the text data via the wireless communication unit 354, and transmits a reception completion signal to the information device 2 (step S105).

The display control section 501 of the HMD 3 adjusts the received text data, for example, fonts, size, and colors thereof to those for HMD display (step S106), and causes the projection optical system 32 and the display optical system 33 to display the text data on the HMD 3 (step S107).

The display control section 501 causes the wireless communication unit 354 to transmit the text which has been displayed to the information device 2 (step S108).

The content analysis section 402 determines an image corresponding to the text which has been displayed on the HMD 3, and the display control section 401 causes the touch screen display 204 of the information device 2 to enlarge and display the image (step S109).

In the case where the entire text is not displayed (step S110/NO), the display control section 501 continues to display the text on the HMD 3 (step S107).

In the case where the entire text is displayed (step S110/YES) but display of the text on the HMD 3 is requested to continue (step S111/NO), the display control section 501 continues to display the text on the HMD 3 (step S107).

In the case where display of the text on the HMD 3 is requested to be terminated (step S111/YES), the HMD 3 transmits a display termination signal to the information device 2 (step S112).

Upon receiving the display termination signal, the display control section 401 of the information device 2 restores the original display state of the information device 2 (step S113).

FIG. 10 illustrates a flowchart of the processing of analyzing the content on the side of the HMD 3. The same steps as those in FIG. 9 are provided with the same reference signs, and a repetitive explanation therefor is omitted.

After confirming that connection with the HMD 3 is established (step S102), the information device 2 transmits the content data (text data and image data) to the HMD 3 (step S201).

The HMD 3 receives the content data, and causes the communication control section 503 to transmit the reception completion signal to the information device 2 (step S202).

The content analysis section 502 of the HMD 3 extracts the text part from the received content data to generate the text data (step S203), performs the processing for displaying the text data on the HMD 3 (step S106), and displays the text (step S107). Thereafter, in the same manner as the flow of FIG. 9, the information device 2 enlarges and displays the image.

In the steps described above, there may be various methods in which the content analysis section 502 of the HMD 3 and the content analysis section 402 of the information device 2 discriminate between text and images. For example, in the case of a website or e-book data in EPUB format, the content analysis section 502 and the content analysis section 402 may analyze the sources of HTML, XML, and XHTML to separate the text to be displayed from the image files, and extract the text being displayed (excluding a part of the text that is not being displayed on the information device 2, such as comments) to generate the text data.

On the other hand, since the image has the directory information of storage location in its tag, the content analysis section 502 and the content analysis section 402 may be configured to directly acquire the corresponding address therefrom so that the image can be displayed on the information device 2.

In the case of PDF data, the content analysis section 502 and the content analysis section 402 may extract a portion that can be written out by such as the OCR as the text data.

In addition, as a method of displaying only an image on the information device 2, for example, in the case where a user is using the information device 2 to browse a site, since the URL (storage location of the image file on the server) has been already read at the time of displaying the image, it may be only needed to display its address.

On the other hand, in the case where the content data does not include any image, the displayed content may be displayed as it is on the information device 2. This case is an exception of the above-described processing using the content including text and an image.

According to the first embodiment, at the time of displaying the content including text and an image by linking the information device 2 with the HMD 3, it is possible to display only the text data on the HMD 3 while enlarging and displaying the image on the information device 2. With this configuration, since the image is not displayed on the HMD 3, in the state where the user wears the HMD 3 and visually recognizes the text information displayed thereon, the visibility of the text information displayed on the HMD 3 can be improved. Furthermore, since the image is enlarged and displayed on the information device 2, the user who wants to watch the image is only required to look at the information device 2 to visually recognize the enlarged image as a real image, whereby the visibility of the image can be improved.

Second Embodiment

The difference between the second embodiment and the first embodiment can be found in content. In particular, the linked display system according to the second embodiment is configured to link and display content in which a part of text and an image are associated with other for each operation process (procedure), which is different from the linked display system according to the first embodiment.

FIG. 11 illustrates an example of content used in the second embodiment. FIG. 12 illustrates a display example on the HMD according to the second embodiment. FIG. 13 illustrates an example of content used in the first embodiment. FIG. 14 illustrates a display example on the HMD 3 according to the second embodiment.

The content 1 of FIG. 11 is content in which an image such as a figure or a picture is associated with a part of text in one-to-one relation. The content 1 includes content involving activities, for example, cooking recipes, guides for sewing and gardening, training manuals of stretching and workout, and methods for dressing kimonos and tying obi belts for kimonos. The image associated with the part of the text may contain a still image or video.

The image and the part of the text in the content are placed such that, in the case of the cooking recipes, an Image 1 showing a preliminary cooking procedure is displayed on the left side of the screen while the part of the text corresponding to the Image 1 describing the procedure is displayed on the right side of the Image 1. An Image 2 showing the next procedure is arranged on the lower side of the Image 1 while another part of the text corresponding to the Image 2 is arranged on the lower side of the text corresponding to the Image 1.

As a criterion in which the content analysis section 502 of the HMD 3 and the content analysis section 402 of the information device 2 recognize that the parts of the content are displayed along the processes, at least one of the following criteria or any combination thereof may be used, for example.

- (Criteria 1) Each part of text and each image are placed on a frame or a table per paragraph.
- (Criteria 2) A number or alphabet indicating the order is provided at the top of each text.
- (Criteria 3) A number or alphabet indicating the order is provided on a caption of each image (in the case of a web site, an <ALT> tag may be used).
- (Criteria 4) Each title sentence contains a word such as “procedure”, “recipe”, “how to”, or “method”, which means the activity procedure.