18a. Furthermore, the transmission/reception unit 18b transmits and receives various data to and from the external power supply 70 via the second communication unit 13 based on the control of the communication control unit 18a.

The display control unit 18c corresponds to a drive unit of the display unit 11 and causes the display unit 11 to display a screen for the sub display 10 transmitted from the smartphone 100. Hereinafter, the screen for the sub display 10 is appropriately referred to as a “sub screen”.

3. Configuration of Smartphone

Next, a configuration example of the smartphone 100 is described. FIG. 4 is a block diagram illustrating a configuration example of the smartphone 100 according to the embodiment of the present disclosure.

As illustrated in FIG. 4, the smartphone 100 includes a camera 101, a communication unit 102, a display unit 103, an operation unit 104, a storage unit 105, and a control unit 106.

The camera 101 is a camera mounted on the main body of the smartphone 100 and includes the above-described out-camera 101a. The communication unit 102 is a connection interface component with the sub display 10 and performs USB connection with the sub display 10 in the embodiment of the present disclosure.

The display unit 103 is a display device that is mounted on the main body of the smartphone 100 and presents visual information, and presents visual information such as a moving image and text and visual information related to an application under the control of the control unit 106 described below. Examples of the display device include the LCD and the OLED described above. The operation unit 104 is an operation component mounted on the main body of the smartphone 100. Note that the operation unit 104 may be configured integrally with the display unit 103 by a touch panel. Therefore, the operation unit 104 may be a software component, and in the embodiment of the present disclosure, the operation unit 104 may be, for example, a graphical user interface (GUI) operably displayed on the display unit 103 by the camera application.

The storage unit 105 is implemented by, for example, a semiconductor memory element such as a random access memory (RAM), a read only memory (ROM), or a flash memory. In the example illustrated in FIG. 4, the storage unit 105 stores application information 105a.

The application information 105a is information including a program of the camera application, various parameters used during start of the camera application, and the like.

The control unit 106 is a controller and is implemented by, for example, a CPU, an MPU, or the like executing various programs stored in the storage unit 105 using the RAM as a work area. Furthermore, the control unit 106 can be implemented, for example, by an integrated circuit such as an ASIC or an FPGA.

The control unit 106 includes an application execution unit 106a, a communication control unit 106b, and a transmission/reception unit 106c and implements or executes a function and an action of information processing described below.

The application execution unit 106a reads the application information 105a stored in the storage unit 105 and executes the camera application. In a state where the sub display 10 is not connected, the application execution unit 106a performs display control, on the display unit 103 of the smartphone 100, of a screen related to the execution of the camera application.

In addition, in a state where the sub display 10 is connected and a communication connection with the sub display 10 is established, the application execution unit 106a performs display control of the sub screen on the sub display 10. For easy understanding of this point, a basic data flow between the smartphone 100 and the sub display 10 is described with reference to FIGS. 5 and 6.

FIG. 5 is a basic data flow (part 1) between the smartphone 100 and the sub display 10. Also, FIG. 6 is a basic data flow (part 2) between the smartphone 100 and the sub display 10.

As shown in FIG. 5, in the embodiment of the present disclosure, when the smartphone 100 and the sub display 10 are connected by USB, and a communication connection, here, USB connection is established, the smartphone 100 transmits an identification code “show” to the sub display 10 via such a connection.

The identification code is code information for identifying a valid communication partner as viewed from the sub display 10 and “show” is also a command for displaying image information from the valid communication partner.

The sub display 10 opens the own display port when being confirmed that it is a valid communication partner by the identification code “show”. Note that the smartphone 100 continues to send the identification code “show” at a predetermined cycle (for example, every 400 ms), and the sub display 10 opens the port while receiving the “show”.

Meanwhile, as illustrated in FIG. 6, the smartphone 100 transmits an identification code “stop” to the sub display 10 when the application is ended. The sub display 10 receives the “stop” and closes the own display port.

Note that FIGS. 5 and 6 merely illustrate a basic data flow illustrating features of the embodiment of the present disclosure. Processing sequences of various situations are described below with reference to FIG. 13 and subsequent drawings.

The description returns to FIG. 4. The communication control unit 106b controls communication with the sub display 10 via the communication unit 102 according to the execution content of the camera application of the application execution unit 106a. The transmission/reception unit 106c transmits and receives various data to and from the sub display 10 via the communication unit 102 based on the control of the communication control unit 106b.

4. Display Example

<4-1. Display Example of Sub Screen>

Next, a display example of various types of information in the external display system 1 is described. First, display contents of the sub screen are described.

FIG. 7 is an explanatory diagram of display contents of the sub screen. Also, FIG. 8 is a diagram illustrating a display example of the sub screen during still image shooting. Also, FIG. 9 is a diagram illustrating a display example of the sub screen during moving image shooting. Note that “F” illustrated in FIGS. 8 and 9 schematically represents a subject, and the same applies to FIGS. 11 and 12 described below.

The sub display 10 displays a sub screen including various types of information illustrated in FIG. 7 based on display control of the smartphone 100. For example, FIG. 8 illustrates an example in which, at the time of still image shooting, the sub display 10 displays, on the display unit 11, a sub screen including a camera-through image including a subject, a focus frame indicated by a thick rectangle in the drawing, a remaining amount of the battery of the smartphone 100, and a high temperature detection mark indicated by a triangle in the drawing.

Furthermore, for example, FIG. 9 illustrates an example in which the sub display 10 displays a sub screen including a Rec status and an Audio level on the display unit 11 in addition to the display content of FIG. 8 at the time of moving image shooting.

<4-2. Display Example on Smartphone Side>

Meanwhile, while the sub display 10 is connected to the smartphone 100, and the sub screen is displayed on the sub display 10, a screen with the following display contents is displayed on the display unit 103 of the smartphone 100. FIG. 10 is a diagram illustrating a display example on the smartphone 100 side when the sub display 10 is connected.

As illustrated in FIG. 10, while the sub screen is displayed on the sub display 10, the smartphone 100 sets a preview area in which the camera-through image is displayed when the sub display 10 is not connected to, for example, black, and then displays a message such as “sub display being connected” on the display unit 103.

Note that, as illustrated in FIG. 10, for an application operation area and the application information area, it is possible to display operation components of a camera application that can be operated on the smartphone 100 side even when the sub display 10 is connected.

<4-3. Display Example of Sub Screen During Change of Aspect Ratio>

For example, FIG. 10 illustrates an example in which an aspect ratio change button asp is displayed. A display example in a case where the aspect ratio change button asp is operated is described.

FIG. 11 is a diagram illustrating a display example of the sub screen when the aspect ratio is changed. In a case where the aspect ratio change button asp is operated, as illustrated in FIG. 11, the sub display 10 can display the sub screen on the display unit 11 while switching the aspect ratio at each ratio such as “16:9”, “3:2”, “4:3”, and “1:1” based on the display control of the smartphone 100.

<4-4. Display Example of Sub Screen During Operation of Flip Key>

Next, a display example of the sub screen at the time of operating the flip key 16 is described. FIG. 12 is a diagram illustrating a display example of the sub screen at the time of operating the flip key 16.

In the arrangement state of each device illustrated in FIG. 1, it is described above that the external display system 1 provides a self-portrait photograph function via a camera application, and the sub display 10 functions as a camera through monitor for such a self-portrait photograph function.

Normally, when a self-portrait photograph is captured by using the in-camera of the smartphone 100, a horizontally flipped (that is, mirroring) camera-through image is displayed in the preview area by default. Therefore, it is preferable that the same display is performed on the sub screen displayed on the sub display 10 according to the embodiment of the present disclosure from the viewpoint of the operability of the user.

Therefore, the sub display 10 performs similar display for the camera-through image based on the display control of the smartphone 100. However, each GUI displayed together with the camera-through image may be excluded from the target.

Specifically, when USB connection is established with the smartphone 100, and the display port is opened by the identification code “show”, the sub display 10 displays a sub screen in which a camera-through image is horizontally flipped by default as illustrated in the upper right of FIG. 12.

However, at this time, GUIs such as a high temperature detection mark, a remaining amount of the battery of the smartphone 100, a Rec status, and an audio level are not set as horizontally flipping targets. The focus frame is set as a horizontally flipping target.

Then, when the flip key 16 is pressed, the camera-through image and the focus frame are horizontally flipped as illustrated in the upper left of FIG. 12. However, also here, GUIs such as a high temperature detection mark, a remaining amount of the battery of the smartphone 100, a Rec status, and an audio level are not set as horizontally flipping targets.

This is because, when the high temperature detection mark, the remaining amount of the battery of the smartphone 100, the Rec status, the Audio level, and the like are horizontally flipped, the shape of each GUI, the direction of the indicator of each level, characters, and the like are reversed, meaning is less likely to be conveyed to the user, and usability is deteriorated.

Note that, when the flip key 16 is subsequently pressed, the sub screen is rotated by 180 degrees as illustrated in the lower left of FIG. 12. At this time, regarding the 180-degree rotation, each GUI other than the above-described horizontally flipping target is also the target.

Then, when the flip key 16 is further pressed, as illustrated in the lower right of FIG. 12, the camera-through image and the focus frame are horizontally flipped from the lower left sub screen. Here, the GUIs other than the above-described horizontally flipping target are also not the horizontally flipping target.

Then, when the flip key 16 is further pressed, the sub screen is rotated by 180 degrees and returns to the default display as illustrated in the upper right of FIG. 12.

Note that, in the description using FIG. 12, an example is described in which the sub screen is flipped or rotated by the flip key 16, but the present disclosure is not limited thereto, and for example, the sub screen may be flipped or rotated according to a sensor value of an acceleration sensor mounted on the smartphone 100, similarly to the case illustrated in FIG. 12.

In addition, for example, after the sub display 10 includes the acceleration sensor, the sub screen may be flipped or rotated according to the sensor value of the acceleration sensor of the sub display 10 as illustrated in FIG. 12.

Note that the input by the flip key 16 and the acquisition of the sensor value of the acceleration sensor can also be regarded as the occurrence of a predetermined event related to the horizontal flip or the vertical rotation of the sub screen.

5. Each Processing Sequence

Next, each processing sequence executed by the external display system 1 is described. For easy understanding of the description, contents of each processing sequence described below are provided in FIG. 13. FIG. 13 is an explanatory diagram of the contents of each processing sequence. An “explanatory diagram” in the drawing is a corresponding diagram when each processing sequence is described.

<5-1. Camera Application Start→Sub Display Connection>

FIG. 14 is a processing sequence in a case where the sub display 10 is connected after the camera application is started. The “Display Manager” in the drawing refers to a monitoring process of each display connected to the smartphone 100. A “USB Manager” refers to a monitoring process of a USB port, in the embodiment of the present disclosure, the communication unit 102. These monitoring processes correspond to middleware that resides and is executed in the control unit 18.

Also, “USB Device Connection” in the drawing indicates communication connection established between the smartphone 100 and the sub display 10. Also, “Presentation” corresponds to a part of the function of the application execution unit 106a and performs switching to the sub screen display for displaying the sub screen on the sub display 10.

As illustrated in FIG. 14, it is assumed that a camera application is first started (Step S101). It is assumed that the sub display 10 is connected via USB (Step S102).

Then, the USB manager detects the connection (Step S103) and notifies the camera application of the connection. The camera application that receives such a notification performs connection permission confirmation (Step S104).

FIG. 15 is a diagram illustrating a confirmation example of connection permission confirmation. The camera application displays a dialog as illustrated in FIG. 15 on the camera application screen of the display unit 103 and receives the confirmation of the user. If “Yes”, the sub display 10 is connected. If “No”, the sub display 10 is not connected. Here, it is assumed that “Yes” is input.

The description returns to FIG. 14. Then, the camera application prepares a sub display space (Step S105). The sub display space is a virtual space for displaying the sub screen and is also referred to as a “surface”.

The camera application generates the sub display space in Step S105 and thereafter outputs the sub display space to display the sub screen on the sub display 10 via the USB device connection.

Then, the camera application transmits the identification code “show” to the sub display 10 at a predetermined cycle (Step S106). When the identification code “show” is confirmed (Step S107), the sub display 10 opens the display port (Step S108).

Then, the Display Manager detects the opening of the port (Step S109) and notifies the camera application of the opening. The camera application that receives this notification starts display control of the sub screen (Step S110) and starts mirroring. Then, a sub screen is output.

In response to the output, the presentation is switched to the sub screen display (Step S111), and the sub screen is transmitted to the sub display 10 via the USB device connection. Note that the procedure after Step S105 can be performed by using, for example, an application programming interface (API) of an open graphics library (OpenGL) system.

<5-2. Sub Display Connection-Camera Application Start>

Next, FIG. 16 is a processing sequence in a case where the camera application is started after the sub display 10 is connected.

As illustrated in FIG. 16, first, it is assumed that the sub display 10 is connected via USB (Step S201). Then, the USB manager detects such connection (Step S202).

Then, it is assumed that the camera application is started (Step S203). Then, the camera application exchanges the presence or absence of connection with the USB manager and performs connection permission confirmation in a case where the sub display 10 is connected (Step S204). Step S204 is similar to Step S104 described above. Thereafter, a processing sequence similar to that in Step S104 is executed.

<5-3. Connected→Camera Application End>

Next, FIG. 17 is a processing sequence in a case where the camera application is ended while the sub display 10 is connected.

As illustrated in FIG. 17, during connection of the sub display 10, the camera application transmits the identification code “show” to the sub display 10 for each predetermined cycle (Step S301), and the sub display 10 confirms the identification code “show” (Step S302). Then, while “show” can be confirmed, the sub display 10 opens the display port.

Meanwhile, the camera application appropriately performs display control of the sub screen (Step S303), transmits the sub screen to the sub display 10, and the sub display 10 that receives the sub screen displays the sub screen (Step S304).

Then, for example, when receiving an end operation of the user, the camera application transmits an identification code “stop” to the sub display 10 (Step S305). When receiving and confirming such “stop” (Step S306), the sub display 10 closes the display port (Step S307).

Then, the Display Manager detects the closing of the port (Step S308) and notifies the camera application of the closing. The camera application that receives this notification ends the display control of the sub screen (Step S309), starts mirroring, and ends the camera application (Step S310).

<5-4. Connected→Sub Display Disconnection>

Next, FIG. 18 is a processing sequence in a case where the connection of the sub display 10 is disconnected while the sub display 10 is connected. Note that, since Steps S401 to S404 in FIG. 18 are similar to Steps S301 to S304 in FIG. 17, the processing sequence from Step S405 is described here.

As illustrated in FIG. 18, it is assumed that the connection of the sub display 10 is disconnected during the connection of the sub display 10 (Step S405). Then, the Display Manager detects the disconnection (Step S406) and notifies the camera application of the disconnection.

The camera application that receives such notification ends the display control of the sub screen (Step S407). Meanwhile, disconnection is also detected in the USB manager (Step S408), and the USB device connection also detects the disconnection (Step S409).

Then, the camera application stops transmission of the identification code (Step S410) and displays the camera-through image on the display unit 103 of the smartphone 100. That is, the camera application returns preview screen display on the display unit 103 of the smartphone 100 (Step S411).

<5-5. Connected→Sub Display Off/On>

Next, FIG. 19 is a processing sequence in a case where the sub display 10 is turned on/off while the sub display 10 is connected.

As illustrated in FIG. 19, it is assumed that the display unit 11 of the sub display 10 is turned off by the on/off switch 15 during the connection of the sub display 10 (Step S501).

Then, the Display Manager detects the display-off (Step S502) and notifies the camera application of the display-off. The camera application that receives this notification ends the display control of the sub screen (Step S503) and returns the preview screen display on the display unit 103 of the smartphone 100 (Step S504).

Meanwhile, the camera application transmits the identification code “show” to the sub display 10 at a predetermined cycle (Step S505).

Then, when the display unit 11 of the sub display 10 is turned on by the on/off switch 15 (Step S506), the Display Manager detects such display-on (Step S507) and notifies the camera application of the display-on.

The camera application that receives this notification starts display control of the sub screen (Step S508) and starts mirroring. Then, a sub screen is output.

In response to the output, the presentation is switched to the sub screen display (Step S509), and the sub screen is transmitted to the sub display 10 via the USB device connection.

<5-6. Connected→Pause/Resume of Smartphone>

Next, FIG. 20 is a processing sequence in a case where the smartphone 100 pauses/resumes while the sub display 10 is connected. Note that, since Steps S601 to S604 in FIG. 20 are similar to Steps S301 to S304 in FIG. 17, the processing sequence from Step S605 is described here.

As illustrated in FIG. 20, it is assumed that a Pause operation is performed in the smartphone 100 while the sub display 10 is connected (see “Pause” in the drawing). Then, the camera application ends the display control of the sub screen (Step S605) and transmits the identification code “stop” toward the sub display 10 (Step S606).

The sub display 10 receives and confirms such “stop” (Step S607) and closes the display port (Step S608).

Then, it is assumed that an operation of Resume or Unlock Screen is performed on the smartphone 100 in such a state (see “Resume or Unlock Screen” in the drawing). Then, thereafter, the processing sequence No. 2 illustrated in FIG. 13, that is, the processing sequence in a case where the camera application is started after the sub display 10 is connected, as illustrated in FIG. 16, is executed.

<5-7. Connected→External Power Supply Disconnection>

Next, FIG. 21 is a processing sequence in a case where the external power supply 70 is disconnected while the sub display 10 is connected. Note that, since Steps S701 to S704 in FIG. 21 are similar to Steps S301 to S304 in FIG. 17, the processing sequence from Step S705 is described here.

As illustrated in FIG. 21, it is assumed that the external power supply 70 connected to the sub display 10 is disconnected while the smartphone 100 is connected to the sub display 10 (Step S705). Then, the processing sequence No. 4 illustrated in FIG. 13, that is, the processing sequence in a case where the connection of the sub display 10 is disconnected while the sub display 10 is connected, as illustrated in FIG. 18, is executed.

Thereafter, the sub display 10 is restarted (Step S706). Then, thereafter, the processing sequence No. 1 illustrated in FIG. 13, that is, the processing sequence in a case where the sub display 10 is connected after the camera application is started, as illustrated in FIG. 14, is executed.

<5-8. Connected→Battery Shortage>

Next, FIG. 22 is a processing sequence in a case where the remaining amount of the battery of the smartphone 100 is insufficient while the sub display 10 is connected. Note that, since Steps S801 to S804 in FIG. 22 are similar to Steps S301 to S304 in FIG. 17, the processing sequence from Step S805 is described here.

As illustrated in FIG. 22, it is assumed that the camera application detects a battery shortage of the smartphone 100 while the sub display 10 is connected (see “Law Battery” in FIG. 22).

Then, the camera application performs end confirmation (Step S805). FIG. 23 is a diagram illustrating a confirmation example of end confirmation. The camera application displays a dialog as illustrated in FIG. 23 on the camera application screen of the display unit 103 and receives the confirmation of the user.

The description returns to FIG. 22. When receiving the confirmation of the user, the camera application transmits an identification code “stop” to the sub display 10 (Step S806). The sub display 10 receives and confirms such “stop” (Step S807) and closes the display port (Step S808).

Also, the camera application ends the display control of the sub screen (Step S809), starts mirroring, and ends the camera application (Step S810).

<5-9. Connected→Forced End of Camera Application>

Next, FIG. 24 is a processing sequence in a case where the camera application is forcibly ended while the sub display 10 is connected. Note that, since Steps S901 to S904 in FIG. 24 are similar to Steps S301 to S304 in FIG. 17, the processing sequence from Step S905 is described here.

As illustrated in FIG. 24, it is assumed that the camera application is unexpectedly forcibly ended while the sub display 10 is connected. In such a case, transmission of the identification code “show” is also forcibly ended, but the sub display 10 determines timeout of the identification code “show”, and when the identification code “show” is not arrived for 1000 msec, for example, the sub display 10 detects timeout (Step S905).

Also, when timeout is detected, the sub display 10 closes the display port (Step S906).

6. Modifications

Meanwhile, the above-described embodiments of the present disclosure can include some modifications.

<6-1. First Modification>

In the embodiment of the present disclosure described above, as illustrated in FIG. 1, a case where the sub display 10 functions as a camera through monitor for a self-portrait photograph function and is substituted for the display unit 103 of the smartphone 100 is described as an example, but the present disclosure is not limited thereto.

For example, the display unit 103 of the smartphone 100 may function as a display device for a main screen for a camera application, and the sub display 10 may be used merely as an auxiliary display device.

FIG. 25 is an explanatory diagram (part 1) of the first modification. Also, FIG. 26 is an explanatory diagram (part 2) of the first modification.

As illustrated in FIG. 25, the holding portion 50 may have a longer dimension than the case of FIG. 1. In addition, the smartphone 100 may be held at a position close to the distal end of the holding portion 50, and the sub display 10 may be held at a position close to the hand of the user.

Furthermore, at this time, the smartphone 100 may be arranged such that the display surface of the display unit 103 faces the user, similarly to the sub display 10. Therefore, in this case, the out-camera 101a of the smartphone 100 faces the subject with respect to the user.

In such a configuration, by raising the holding portion 50, the user can position the out-camera 101a at a position higher than usual and perform shooting other than a self-portrait photograph via the camera application by using the remote controller 50a (not illustrated) or the like.

Note that, by positioning the out-camera 101a at a position higher than usual, it is difficult for the user to see the display unit 103 of the smartphone 100. However, as illustrated in FIG. 25, by displaying the camera-through image supplementarily on the sub display 10 close to the hand, the user can perform shooting while confirming the camera-through image.

Incidentally, in the case of the configuration example illustrated in FIG. 25, the default display of the sub display 10 is different from that illustrated in FIG. 12 and is not for a self-portrait photograph, that is, the camera-through image, the focus frame, and the like are not horizontally flipped, as illustrated in FIG. 26.

As described above, by using the sub display 10 as an auxiliary display device in a case where it is difficult to see the smartphone 100, it is possible to enhance the convenience of the user including the ease of viewing and the like.

<6-2. Second Modification>

FIG. 27 is an explanatory diagram of the second modification. Similarly to the first modification, in the configuration in which the display surfaces of the smartphone 100 and the sub display 10 are arranged to face the user, the sub display 10 may be configured as a dedicated device that supplementarily displays, for example, a histogram of shoot data.

As described above, by using the sub display 10 as a display device that displays image information that cannot be displayed on the smartphone 100 having a limited display area in an expanded manner, it is possible to enhance the convenience of the user including the ease of viewing and the like.

<6-3. Third Modification>

Also, in addition to this, for example, the backlight of the sub display 10 may be configured to function instead of the flash light at the time of shooting a self-portrait photograph.

In addition, several patterns of effects at the time of shooting may be prepared and freely selected at the time of shooting.

In addition, the display unit 11 of the sub display 10 may be configured by a touch panel to be able to receive a touch operation, and a place touched by the user may be set as a target of touch tracking.

<6-4. Other Modifications>

Also, among the processes described in the above-described embodiments of the present disclosure, all or a part of the processes described as being performed automatically can be performed manually, or all or a part of the processes described as being performed manually can be performed automatically by a known method. In addition, the processing procedure, specific names, and information including various data and parameters disclosed in the document and the drawings can be arbitrarily changed unless otherwise specified. For example, the various types of information illustrated in each figure are not limited to the illustrated information.

In addition, each component of each device illustrated in the drawings is functionally conceptual and is not necessarily physically configured as illustrated in the drawings. That is, a specific form of distribution and integration of each device is not limited to the illustrated form, and all or a part thereof can be configured to be functionally or physically distributed and integrated in an arbitrary unit according to various loads, usage statuses, and the like.

In addition, the embodiments of the present disclosure described above can be appropriately combined in a region in which the processing contents do not contradict each other. Furthermore, the order of each step illustrated in the sequence diagram or the flowchart according to the present embodiment can be changed as appropriate.