118 is an example of the rewards system 104 of FIG. 1A or a component included therein and may receive data or information from content viewing tracker 114 and/or retrieve information from tracking database 120 to determine whether a reward should be awarded to a particular user account and/or a quantity of reward to be distributed to the user account, based on pre-established reward rules. Indications of the rewards may be stored to tracking database 120, which may also be used to store information about the amount of content the viewed in association with the user account. In some situations, a user account can be linked with multiple user devices which are permitted to stream content concurrently, so it is possible that over a defined period of time, the amount of rewards earned is multiples of the duration of time that has actually elapsed.

As mentioned above, the type of reward provided may be a credit that can be used toward services available via content provider system 110 for which an account owner would otherwise need to provide payment. For example, a particular piece of content may typically cost some monetary amount to stream. A reward provided via rewards engine 118 may permit the user to stream such content for free or for a reduced price. As another alternative, rewards may permit a user account to access a streaming channel (e.g., television channel that is available for streaming) for free for a period of time or at a reduced cost. In other embodiments, rewards may be redeemed for access to a gaming interface (e.g., an arcade interface) or particular games within the gaming interface. As yet another embodiment, rewards may be redeemable for merchandise or hardware available through the content provider that manages content provider system 110.

A key aspect to providing a user account with an award is being able to accurately track the amount of content actually viewed in association with the user account and/or the duration of time for which streaming content is being viewed in association with the user account. While content provider system 110 may be able to determine on its end whether streaming content is being transmitted to a streaming device that is linked with the particular user account, this output by content provider system 110 does not necessarily correspond to the streaming content being output by the streaming device nor to the streaming content actually viewed by the user using the streaming device.

Regardless of the form in which streaming device 140 takes, streaming device 140 may use the processing system 144 to execute a particular application (“app”) associated with content provider system 110. This application may be required to be executed in order to log in via a user account, access, and stream content from content provider system 110. Despite this application being required to be used in order to access and stream content from content provider system 110, the content provider may have little control over how streaming device 140 shuts down the application when the user no longer desires content to be streamed. For example, if a user provides an exit command to streaming device 140 signaling the users desire to exit the application, streaming device 140 may abruptly end the process associated with the application. No indication that the application has been closed or that streaming content has been stopped being output may be provided by streaming device 140 to content provider system 110. Therefore, for a time, content provider system 110 may continue transmitting streaming content to streaming device 140.

Additionally, some amount of the streaming content may be temporarily stored by streaming device 140 in content buffer 142 before being output. The amount of content stored to content buffer 142 may vary based on the connection speed between streaming device 140 and content provider system 110 and also based on the output resolution of the content. If the user provides an exit command or the application is otherwise terminated, streaming device 140 may discard content stored to content buffer 142 without having output such content for presentation. Therefore, for accuracy, it may be important to not credit the user account with having viewed such content, such as for the purposes of rewards.

Rather than relying on the amount or duration of content streamed by content provider system 110 to track the viewing of content, such as for providing viewing rewards, the application executed by processing system 144 may cause the streaming device 140 to periodically or occasionally generate status/heartbeat messages and transmit the status/heartbeat messages back to content provider system 110 via network 130. The content provider system 110 may determine the amount or duration of content that were actually output to the display device and viewed by the user, based on the information provided in the status/heartbeat messages.

The terms “status message,” “update message,” “check message,” “heartbeat message,” “status update message,” and other terms alike as interchangeably used here, in the context of content streaming and interaction with user devices, refers to a recurring and periodic signal sent from the user device to the service provider platform. The status/heartbeat message serves as a communication mechanism to indicate the continued presence and operational status of the user device. The status/heartbeat message may contain essential information about the user device's state, such as its identifier, connectivity status, operational status, playback status, streaming status, gameplay status, a status of a particular application executed on the user device, and other possibly relevant metrics. The periodic transmission of status/heartbeat messages allows the service provider platform to monitor and track the user's activity and engagement with the service provider platform through the user device. In the context of tracking user engagement with content streaming or gaming interfaces, status/heartbeat messages may also include information about the user's current activities, playback states, gameplay states, or other interaction patterns related to user engagement with the service provider platform.

In tracking and determining the reward for content streaming and viewing, since a heartbeat message requires that the application be executed in order to be created and transmitted, the information contained within the heartbeat message can be expected to be much more accurate than viewership data calculated exclusively by content provider system 110, for example, by the amount of content that has been transmitted from the content provider system 110 to the streaming device 140. An example of heartbeat message 160 is illustrated in FIG. 1B. Heartbeat message 160 can include: a header; and entries for one or more playback states. The header can indicate a unique identifier, details of the device being used for playback, and user details, such as an indication of a particular user profile active for a particular user account. The heartbeat message can include one or more playback states. Each playback state can indicate: a particular playback state (e.g., media plays, media paused, media buffering, play rate, stream resolution, output streaming rate, buffer rate, etc.), a duration of time in this particular state, a genre type (or asset type of an indication of a particular franchise, such as the movie franchise “Star Wars”); whether the content is an advertisement. Therefore, a heartbeat message may be transmitted less often since the message can relay details about multiple states which have occurred since the previous heartbeat message. As an example, the heartbeat could indicate that playback occurred for 27 seconds and the content was paused for 4 seconds.

The amount of time that content is indicated as being played back on the streaming device 140 may indicate a duration of time of the piece of streaming content that has been output, via output interface 148, for presentation either directly via streaming device 140 or an external device, such as display device 150. The playback length indicated in a playback state in the heartbeat message may exclude time during which a trick play mode was being used on streaming device 140, such as rewinding or fast forwarding. As an example, a heartbeat message may be transmitted periodically, such as once per 5 seconds or once per minute.

In some embodiments, the heartbeat message 160 may include a playback state that precisely indicates the current playback rate. This playback rate allows for the specification of the current rate at which the content is being consumed, whether it is at the normal playback rate (e.g., 1×), an accelerated rate (e.g., 2× or 3×), or any other playback rate selected by the user. The heartbeat message 160 may include additional information regarding the amount of content being output during accelerated playback, and a discounting mechanism may be applied to accurately reflect the actual amount of content streamed as if it were played under the normal playback rate. For instance, if the content is being played at an accelerated rate (e.g., 2× or 3×), the discounting mechanism adjusts the metrics to reflect the equivalent duration as if it were played at the normal or standard playback rate (e.g., 1×). This adjustment compensates for the increased speed and enables precise calculations of the user's engagement to ensure that the metrics align with the actual time spent on content consumption. Such discounting mechanism allows to adjust the content duration and user engagement in accordance with the chosen playback rate and provide a precise representation of the time spent by the user in consuming content at varying rates. The inclusion of such playback rate details in the heartbeat message 160 may contribute to a more granular and accurate assessment of user activity on viewing contents and user engagement with the streaming interface.

In some embodiments, a playback state can indicate a buffer length value. The buffer length value may be used to specify either the size of content buffer 142 or the duration of content currently stored to content buffer 142. The buffer length value may be used by content viewing tracker 114 to ensure that the user account is not credited with the output of content that has been buffered, but not yet output. As an example of how such data could be used, the buffer length value may be used instead of the playback length value. Using the buffer length value, content viewing tracker 114 can more accurately determine the duration of content that has actually been output for presentation by streaming device 140 via output interface 148. For example, when a piece of content ends, the duration of content stored to content buffer 142 can reach zero, since there is no portion of the content left to be output. In such a situation, heartbeat message 160 could indicate to content provider system 110 that the duration of content stored to content buffer 142 has reached zero or simply the corresponding playback state may not be included.

Based upon the information contained within heartbeat message 160 and subsequent heartbeat messages, content viewing tracker 114 may more accurately store information on the duration of content output in association with the user account, which can in turn allow rewards engine 118 to more accurately provide awards to user accounts at block 240. While rewards are an example, there are many other reasons why viewing via a user account may need to be accurately tracked, such as for monitoring the popularity of particular streaming content.

In some embodiments, the content viewing tracker 114 may generate an output or report indicating the total duration of output content and send a notification message that includes the output or report to the streaming device 140.

Multiple devices can be used to output streaming content in conjunction with a single account. For example, multiple members of a family may be permitted to access streaming content via a common account or linked accounts. Heartbeat-based tracking may be performed for all of these devices with points or value resulting from such tracking being aggregated such that a particular user or account can redeem the aggregated rewards. As an example, a component of content provider system 110, such as reward aggregator 119, can be used to combine rewards accumulated across linked accounts such that the aggregated rewards can be redeemed only once and/or by a particular account. It also may be possible that multiple devices are streaming content concurrently in association with a common account or linked accounts. This situation can result in rewards being accumulated at a higher rate (e.g., twice as fast, three times as fast) compared to a single stream being output for presentation.

Streaming device 140 may also be in communication with gaming host 170. Gaming host 170 may be an example of the game provider system 103 of FIG. 1A of a component thereof. Gaming host 170 may be accessible via the cloud and may be operated independently from content provider system 100. Through application executed by processing system 144, streaming device 140 may be directed to access, via network interface 146, gaming host 170 to access particular gaming functions via an arcade interface. In some embodiments, the gaming host 170 is an arcade provider system 180 or a component thereof (e.g., gaming server 182), which will be described with reference to FIG. 1C.

FIG. 1C illustrates another example of a communications system 100C, which is a variation of the communications system 100B. In the illustrated example, the communications system 100C includes content provider system 110, arcade provider system 180, network 130, and user device 190. Components similar to the communications system 100B as described above will not be repeated herein unless otherwise indicated.

The arcade provider system 180 is an example of the game provider system 103 of FIG. 1A. The arcade provider system 180 is responsible for delivering gaming content to user device 190, tracking user interaction and engagement with the arcade games provided by the arcade provider system 180, and determining and allocating rewards to user account associated with the user. In some embodiments, the arcade provider system 180 further includes, among others, a gaming database 181, a gaming server 182, a user engagement tracking engine 183 (or user engagement tracking device 183), a rewards engine 184, a network interface 185, and a tracking database 186.

The game database 181 stores arcade games and related information such as arcade game titles, descriptions, genres, ratings, and any other relevant metadata. Examples of arcade games include but are not limited to Tetris, Dr. Mario, Columns, Crossy Road, Frogger, Endless Ducker, Dino Jump, etc. The video and audio content or content segments related to the arcade games can be retrieved by the gaming server from the game database 181 and delivered to the user device 190 for the user to play or interact with. The gaming server 182 is responsible for hosting and facilitating single-player or multi-player gaming experiences. The gaming server 182 can manage and coordinate various aspects of single-player or multi-player games, allow multiple user devices to connect and interact, and enable multiple users to compete each other in real-time.

The user engagement tracking engine 183 is responsible for monitoring and tracking user activities and determining a level of user engagement with the arcade games output to the display. For example, the user activities may be indicated by a total duration of time for which the user actually played the arcade game, including the time spent navigating through menus, participating in gameplay, and interacting with in-game elements. The user engagement tracking engine 183 may also keep track of the total number of arcade games the user has successfully completed, monitor and record the total number of arcade games the user has won, track specific achievements or challenges completed by the user during gameplay, capture scoring data, and so on. The user engagement tracking engine 183 may generate user engagement data indicating a quantitative level of user engagement, which may be used for determining the amount of reward to be distributed to the user account associated with the user. The user engagement data may include various quantitative metrics reflecting the user's engagement with arcade games, including total playtime, the number of games completed, the number of victories, achievements unlocked, and other relevant gameplay statistics. The user engagement tracking engine 183 can assign numerical values or scores to different aspects of user engagement to create a quantitative representation of the user's interaction with the arcade games. The user engagement data may be stored in the tracking database 186.

The rewards engine 184 is responsible for determining the quantity of rewards based on the user interaction data generated by the user engagement tracking engine 183. The rewards engine 184 may utilize the quantified user engagement data and apply pre-established reward rules to determine the amount of reward to be distributed to the user. The reward determination process may take into account the metrics of the user engagement data including user's total duration of playtime, total number of games played, total number of achievements, total number of victories, and other relevant factors. The rewards engine 184 may further make final reward calculations based on pre-established reward rules and distribute the calculate reward to the user account.

The user device 190 is a variation of the streaming device 140 of FIG. 1A and include similar components described above. The user device 190 may include an application stored in memory 145 executable by processing system 144 to cause the streaming device 140 to periodically or occasionally generate heartbeat messages 165 and transmit the heartbeat messages 165 back to arcade provider system 180 via network 130. The arcade provider system 180 may determine the total duration of playtime for which the user actually played the game and other attributes for reward determination, based on the information provided in the heartbeat messages 165.

An example of heartbeat message 165 is illustrated in FIG. 1C. Heartbeat message 165 can include: a header; and entries for one or more gameplay states. Similar to the heartbeat message 160 for tracking the content actually viewed by the user, the header of the heartbeat message 165 may include, among others, unique identifier for the heartbeat message 165, user device information (e.g., user device ID), user details (e.g., user profile or user account ID), timestamp indicating when the heartbeat message was generated. The entries section includes one or more entries, each representing a specific gameplay state. The entries may convey real-time information about the user's interactions with the arcade game. For example, gameplay states may include media play state indicating the start of gameplay, when the gameplay is paused, a media buffering state indicating buffering events during gameplay, gameplay rate state specifying the current gameplay rate (normal rate, 2×, 3×), playing mode indicating the current gameplay mode (e.g., single-player, multiplayer), progress state indicating the user's progress within the arcade game, scoring of the arcade game, outcome of the arcade game, and so on.

In some embodiments, the heartbeat message 165 may include a gameplay state that precisely indicates a current controller state of a controller (e.g., the remote control 155), indicating whether the controller is active or in use. For example, when the user is operating the controller to play an arcade game, input events (e.g., user input signals) are generated by the user device 190 and transmitted to the gaming server 182. The input events may include signals indicating button presses, button movements, joystick movements, or other actions performed using the controller. The status of the controller may be recorded as a gameplay state included in the heartbeat message 165. By including the dedicated “controller in use” state in the heartbeat message 165, a more clear and accurate indication of when the user is actively engaged with the arcade game can be obtained. The “controller in use” state adds an additional layer of specificity to the overall gameplay tracking and enables the user engagement tracking engine 183 to precisely track the user's interaction with the arcade game.

In some embodiments, the user interface 147 of the user device 190 are bi-mode and can include a streaming interface 191 and an arcade interface 192, which can be switchable and/or simultaneously presented on the display device 150 in a split-screen manner or other suitable manners. User may have the option to switch between the streaming interface and the gaming interface, for example, by toggling between the two modes using the remote control 155. In some embodiments, the remote control 155 may have input functionalities to support both the streaming interface and the gaming interface. The remote control 155 may have dedicated controls or modes tailored for gaming interactions, such as buttons, joysticks, or triggers optimized for gaming scenarios. Simultaneously, the remote control 155 can switch or adapt to a different set of controls or modes suitable for navigating and interacting with streaming content. In the gaming mode, the remote control 155 provides the necessary inputs and controls for playing arcade games. In the streaming mode, the remote control 155 provides controls optimized for navigating streaming interfaces, such as menu navigation, selecting content, adjusting volume, and other streaming-related actions. Users may have the ability to manually switch between gaming and streaming modes on the controller, depending on their current activity.

In some embodiments, when both the streaming interface 191 and the arcade interface 192 are presented on the display device 152, both heartbeat message 160 and heartbeat message 165 may be used to respectively track the duration of streaming content and the duration of gameplay time. Using separate heartbeat messages allows for a comprehensive and detailed tracking of the duration of streaming content and gameplay time independently. In some embodiments, each one of the heartbeat messages 160 and 165 may further indicate a controller mode state indicating the whether the controller is in “streaming” mode or “gaming” mode. A combination of “controller in use” and “gaming mode” included in the heartbeat message 165 indicates that that the user is actually playing the arcade game.

In some embodiments, the set heartbeat messages 160 and the set of heartbeat messages 165 can be consolidated into one single set of heartbeat messages, and each of the consolidated heartbeat message may include a header, one or more playback states, and one or more gameplay states. The single set of heartbeat messages can be used to track the user engagement with both the streaming content and the game. The information from the single set of heartbeat messages can be used to more accurately determine the reward for the user account in association with the user device.

FIG. 2A illustrates an example screenshot of a user interface 200A displayed on a display area 201 of the display device 150. In the illustrated example, the user interface 200A is a dual-mode interface that includes a streaming interface 191 and an arcade interface 192 concurrently co-presenting on the display device 150 in a split-screen mode. The streaming content is output and presented in the streaming interface 191, the gaming content is output and presented in the arcade interface 192, concurrently with outputting the streaming content in the streaming interface 191. As illustrated, a television may be acting as the user device 190 or in communication with user device 190. Alternatively, a similar dual-mode interface may be presented via another form of streaming device, such as a tablet computer or gaming device.

A user, while streaming content is being output by a streaming device, may activate the arcade interface 192. In order to activate the arcade interface, the user may make a selection via an on-screen menu. Alternatively, on remote control 155 of user device 190, a dedicated button may be present that allows selection of an arcade mode and activation of the arcade interface 192. When activated, entering the arcade interface 192 may perform a “squeezeback.” A squeezeback results in streaming content being resized to occupy only a portion of the display area 201, thus allowing arcade interface 192 to be simultaneously co-present along with streaming content 206 being received from content provider system 110.

By default, when arcade interface 192 is activated, a portion of the user input device, such as remote control 155, may be used to navigate within arcade interface 192. As an example, a directional pad (“d-pad”) of the remote control 155 may be used for navigation within arcade interface 192, such as for interacting with the game or selecting a game 208 within arcade interface 192. D-pad input to user device 190 may not allow for exiting from arcade interface 192 (during gameplay, a user may be pressing the d-pad repeatedly and quickly, so an accidental input that exits gameplay may be desired to be avoided). Notably, the content for arcade interface 192 may be provided from a separate cloud-based source than streaming content 206. Therefore, while the executing application presents them concurrently, the application and underlying user device 190 may access the content of the game 208 and streaming content 206 from separate sources.

While navigating through arcade interface 192, the directional pad of the remote control 155 may be assigned to operate within only arcade interface 192, and a user may be required to press a dedicated exit button on the remote control 155 to in order to return to interaction with the streaming interface 191 and control of content streaming, such as what content is being streamed. A user may actuate a virtual button 209 to exit the arcade interface and return to the streaming interface 191. A user may select an arcade game and play the game, such as using the directional pad. In some embodiments, control may be more nuanced. For example, pressing a dedicated exit button for less than a duration of time on remote control 155 may not stop the game 208 currently being played but keep game 208 active; holding the dedicated exit button for more than the duration of time on remote control 155 can result in the game being paused, as detailed below, and allowing the user to interact with the streaming content 206 within the streaming interface.

When a user exits controlling game 208, such as by pressing an exit button on the remote control 155, the game 208 may be paused, which can include streaming device 140 transmitting a pause command to gaming host 170. When a user re-selects arcade interface 192, gameplay may automatically be resumed, such as a command being transmitted to gaming host 170 to resume gaming, or a subsequent user input may be required to unpause the game.

FIG. 2B illustrates another example screen shot of a user interface 200B displayed on a display area 201 of the display device 150. In the illustrated example, the user interface 200B is a dual-mode interface that includes a streaming interface 191 and an arcade interface 192 concurrently co-presenting on the display device 150 in a split-screen mode. The arcade interface 192 further includes a virtual panel 210 for audio output control. The virtual panel 210 provides options for the user to control audio output and specify whether the audio output via user device 190 is audio associated with streaming content 206 (through streaming interface 191) or game 208 (through arcade interface 192). The virtual panel 210 can provide more options for the user to control audio output, such as adjusting audio settings (volume, mute, audio source selection, etc.).

FIG. 3A is a flow diagram illustrating an example method 300A for tracking rewards. Method 300A may include one or more operations/steps. Few or additional operations/steps may be included. The operations of method 300A may be combined with operations of another method described herein in any suitable manner.

At 302, media content or multimedia content is transmitted from a service provider platform and received in a user device connected to the service provider platform via a communications network. The multimedia content may include streaming content, which is transmitted from a content provider system. The multimedia content may include game content, which is transmitted from a game provider system. One or more applications stored in the user device may be executed to receive, process, buffer, stream, play, and output the multimedia content. In some embodiments, the multimedia content is presented in a user interface on a display connected to the user device for the user to view and interact with the multimedia content.

At 304, user activities are tracked by the service provider platform to determine a user interaction level or a user engagement level. User activities may include a total duration or amount of content that has been output to the display and actually viewed/consumed by the user and a total duration of time for which the user actually played a game. In some embodiments, periodic or occasional heartbeat messages are used to track the user activities and user engagement level, as described below with references to FIGS. 3B-3D.

At 306, reward is determined by the service provider platform, based on the user interaction/engagement level and a pre-established reward rule. In some embodiments, the user interaction/engagement level is mapped to the reward, according to the pre-established reward rule. At 308, the reward is distributed/allocated to a user account associated with the user device.

FIG. 3B is a flow diagram illustrating another example method 300B for tracking rewards related to content streaming on a user device and consumption by a user, using heartbeat messages. Like method 300A, method 300B may include one or more operations/steps. Few or additional operations/steps may be included. The operations of method 300B may be combined with operations of another method described herein in any suitable manner.

At 322, a first application (e.g., streaming application) stored in a memory of a user device is executed to receive, process, buffer, stream, streaming content transmitted from a content provider system and received in the user device in communication with the content provider system via a communications network. The streaming content is buffered and then output for streaming, through a user interface, to a display connected to the user device.

At 324, a second application (e.g., tracking application) stored in the user device is automatically executed to cause the user device to periodically or occasionally generate a plurality of first heartbeat messages at a frequency and transmit the plurality of first heartbeat messages to the service provider system, upon initiation of the execution of the first application and during the execution of the first application. In some embodiments, the second application is automatically triggered and executed upon receiving the streaming content. Each one of the first heartbeat messages is timestamped and includes information indicating one or more playback states about the received streaming content that is being output for presentation. In some embodiments, the frequency is from once per one second to once per one minute. Other frequencies are also possible within the scope of the present disclosure.

In some embodiments, the first application and second application are the same application. For example, the first application may be a first unit of an application, and the second application may be a second unit of the same application. A unit within an application, as used herein, refers to a self-contained and modular component that is responsible for performing specific functionalities, tasks, or features. Each unit may be configured to operate as an independent component and to interact/collaborate with other units within the same application. In some embodiments, the application may include a first unit and a second unit. The first unit of the application is executable to receive, process, buffer, stream, and output streaming content transmitted from the content provider system. The second unit of the application is executable to cause the user device to periodically or occasionally generate a plurality of first heartbeat messages, during the execution of the first unit or during the execution of the application.

In some embodiments, each first heartbeat message includes a header and one or more playback states. The header includes information about the content item, the user device, and the user account. The one or more playback states include a first playback state indicating a current output streaming rate (e.g., bps), a second playback state indicating a duration of time in the current output streaming rate, and a third playback state indicating a current play rate (e.g., paused (play rate is zero), 1×, 2×, 3×). In some embodiments, the duration of time in the current play rate is the time interval calculated based on the frequency of the first heartbeat messages. For example, if the frequency of first heartbeat messages is once per 10 seconds, the time interval is 10 seconds.

The first heartbeat message may include additional playback states indicating other information related to user activities, user engagement with the streaming content, and user interaction with the user interface presenting the streaming content. For example, a playback state may indicate a current stream resolution. Higher resolutions (e.g., High Definition (HD), Ultra High Definition (UHD)) may result in larger file sizes for videos or images, and streaming higher resolution content may require more data to be transferred over the network to the user device and output to the display and thus may require a higher stream rate. Users may choose lower resolutions for faster streaming or opt for higher resolutions for enhanced visual fidelity. Accordingly, the resolution state may be a factor in calculating the total duration of consumed content.

The first heartbeat message may also include a playback state indicating a buffer state and a buffer rate may indicate a current stream resolution. Higher resolutions (e.g., High Definition (HD), Ultra High Definition (UHD)) may result in larger file sizes for videos or images, and streaming higher resolution content may require more data to be transferred over the network to the user device and output to the display and thus may require a higher stream rate. Users may choose lower resolutions for faster streaming or opt for higher resolutions for enhanced visual fidelity. Accordingly, the resolution state may be a factor in calculating the total duration of consumed content.

In some embodiments, the first heartbeat message further includes a playback state indicating a controller state of a controller (e.g., the remote control 155) operated by the user. The controller state may indicate whether the controller is “in use” or “idle” and provide a direct insight into the user's physical interaction with the output content. In some embodiments, when both a streaming interface and a gaming interface are activated and concurrently presenting on the display, the first heartbeat message may further include a playback state indicating whether the controller is in a “streaming mode” or a “gaming mode,” as described below.

At 326, a total duration of content of the content item that has been output and consumed by the user during execution of the first application (i.e., a total duration of consumed content) is determined, by the content service provider, based on the plurality of first heartbeat messages received in the content service provider. In some embodiments, a report is generated (e.g., by the content viewing track) to include the information about the total duration of output content, and the report is sent to and received by the user device.

In some embodiments, the first heartbeat messages are processed and analyzed to calculate the total duration of consumed content, based on a preestablished model. In some embodiments, the total duration of consumed content is calculated by aggregating the segment duration (hereinafter “segment”) of consumed content for each time interval of the plurality of first heartbeat messages. The segment of consumed content for each time interval can be calculated by multiplying the output streaming rate indicated by the preceding first heartbeat message and the duration of time interval. The total duration of consumed content may be the aggregation of the segment of consumed content for each time interval during a given period of time (e.g., when the player application is being executed). As noted above, the output streaming rate is zero when the playback state is “paused,” regardless of the amount of content being streamed in the buffer stage before being output to the display. In some embodiments, the total duration of consumed content excludes the amount of content being buffered but not yet output to the display nor presented in the user interface.

In some embodiments, a discounting mechanism may be applied to adjust the segment of content for a time interval based on the play rate indicated in a heartbeat message preceding the time interval. For example, a heartbeat message includes a playback state indicating that the current play rate is 3× (accelerated playing), the segment of output content for the time interval after the heartbeat message and before the next heartbeat message will be discounted by the play rate (e.g., the discounted duration is ⅓ of the calculated duration). The discounting mechanism allows the total duration of content consumption to accurately reflect the actual viewing time by taking into account variations in play rates. This approach contributes to precise tracking of user engagement and content consumption across different playback scenarios.

In some embodiments, the heartbeat message may further include playback states indicating whether the user is fast forwarding the content streaming. The playback state related to fast forwarding is identified within the information provided by the heartbeat messages. The playback state parameters associated with fast forwarding may include indicators such as “fast forward engaged” or a specific play rate indicating fast forward mode. The timestamp of the heartbeat message is recorded to establish the time when the user enters the fast-forwarding state. During the time interval between the current heartbeat message and the next one, the segment of output content may be adjusted based on the fast-forwarding state. If the user is fast forwarding, a similar discounting mechanism may be applied on the calculated duration to accurately represent the actual content viewed during fast forward.

In some embodiments, the heartbeat message may further include playback states indicating a current progress state (i.e., a playback position) of the content item. This information can be used to assess the practicality of the increase in content progress over a specific time interval. For example, a current heartbeat message may indicate a current playback position of the content item is 10%, and a preceding heartbeat message may indicate a playback position of 3%. A determination can be made on whether the increase from 3% to 10% in the time interval is practical within the time interval. If the increase is determined to be impractical or exceed expected rates of content progression, it indicates that the user may have entered a fast-forwarding mode during that time interval. This indication may be used to flag the time interval as a potential fast-forwarding scenario. During time intervals flagged as potential fast forwarding, a discounting mechanism may be applied to adjust the duration of output content. The duration is discounted to accurately represent the actual content viewed by considering the potential fast-forwarding activity.

It should be noted that the specific embodiments for tracking user engagement and calculating the total duration of consumed content as described herein are for illustrative purposes only, and other approaches may also be possible within the scope of the present disclosure.

At 328, a reward to be awarded to a user account associated with the user device is determined, for each content item, based on the total duration/amount of consumed content and a pre-established reward rule. The reward rule may specify a mapping correlation between the amount of reward and the amount of consumed content for each content item. At 330, when multiple content items are delivered to the user device and consumed by the user, the reward for each content item may be aggregated. In some embodiments, if the content has been output to multiple user devices associated with the user account and consumed by multiple users, respectively, the reward for each user may be aggregated for the common user account.

FIG. 3C is a flow diagram illustrating another example method 300C for tracking rewards related to gaming on a user device and game playing by a user, using heartbeat messages. Like method 300B, method 300C may include one or more operations/steps. Few or additional operations/steps may be included. The operations of method 300C may be combined with operations of another method described herein in any suitable manner.

At 342, a third application (e.g., arcade application) stored in a memory of a user device is executed to receive, process, buffer, stream, streaming content of a game transmitted from a game host and received in the user device in communication with the game host via a communications network. The content is output through a gaming interface, to a display connected to the user device for the user to play or interact with.

At 344, a fourth application (e.g., tracking application) stored in the user device is automatically executed to cause the user device to periodically generate a plurality of second heartbeat messages at a frequency and transmit the plurality of second heartbeat messages to the service provider system, upon initiation of the execution of the first application. In some embodiments, the fourth application is automatically triggered and executed upon receiving the content of the game. Each one of the second heartbeat messages is timestamped and includes information indicating one or more gameplay states about the received gaming content that is being output for presentation. In some embodiments, the frequency is from once per one second to once per one minute. Other frequencies are also possible within the scope of the present disclosure.

In some embodiments, the third application and fourth application are the same application, in a similar manner as first application and the second application. For example, the third application may be a third unit of an application, and the fourth application may be a fourth unit of the same application. The third unit of the application is executable to receive, process, buffer, stream, streaming content of a game transmitted from a game host. The fourth unit of the application is executable to cause the user device to periodically or occasionally generate a plurality of second heartbeat messages, during the execution of the first unit or during the execution of the application.

In some embodiments, the first, second, third, and fourth applications may be the same application. For example, the first, second, third, and fourth applications may be correspondingly the first, second, third, and fourth units within the same application excitable on the user device.

In some embodiments, each second heartbeat message includes a header and one or more gameplay states. The header includes information about the game, the user device, and the user account. The one or more gameplay states include a first gameplay state indicating a current state of a controller (e.g., a remote control) operated by the user as being active (e.g., in use) or idle (e.g., not in use), optionally a second gameplay state indicating a current mode of the controller as being a “streaming” mode or a “gaming” mode (e.g., an “arcade” mode), and a third gameplay state indicating a duration of time in the current state of the controller. In some embodiments, the duration of time in the current play rate is the time interval calculated based on the frequency of the second heartbeat messages. For example, if the frequency of second heartbeat messages is once per 10 seconds, the time interval is 10 seconds. The second heartbeat message may include additional playback states indicating other information related to user activities, user engagement with the game, and user interaction with the gaming interface presenting the game.

At 346, a total duration of playtime consumed by the user to play the game during execution of the third application (i.e., a total duration of consumed playtime) is determined, by the game host or a user engagement tracking device/engine of the game host, based on the plurality of second heartbeat messages received in the game host. In some embodiments, the second heartbeat messages are processed and analyzed to calculate the total duration of consumed playtime, based on a preestablished model. In some embodiments, the total duration of consumed playtime is calculated by aggregating the segment of consumed playtime for each time interval of the plurality of second heartbeat messages. The segment of consumed playtime for each time interval can be determined by counting the duration of the time interval with a preceding heartbeat message indicating a controller state as “in use” and in a “gaming” mode. As noted above, the time intervals having a preceding second heartbeat message indicating an inactive controller state or not in a “gaming” mode may be excluded from the calculation of the total duration of playtime. It should be noted that the example approach for calculating the total duration of consumed playtime as described herein is for illustrative purposes only, and other approaches may also be possible within the scope of the present disclosure.

At 348, a reward to be awarded to a user account associated with the user device is determined, for each game, based on the total duration of consumed playtime and a pre-established reward rule. The reward rule may specify a mapping correlation between the amount of reward and the amount of consumed playtime for each game. At 330, when multiple content items are delivered to the user device and consumed by the user, the reward for each game be aggregated.

FIG. 3D is a flow diagram illustrating another example method 300D for tracking rewards, using heartbeat messages. Method 300D may include one or more operations/steps. Few or additional operations/steps may be included. The operations of method 300D may be combined with operations of another method described herein in any suitable manner.

At 362, a user interface is activated on a user device. The user interface includes a streaming interface and an arcade interface concurrently presented on a display of the user device. In some embodiments, a first application (e.g., a media streaming application) is executed to receive, process, buffer, stream, streaming content of a content item transmitted from a content provider system and received in the user device in communication with the content provider system via a communications network. The streaming content is buffered and then output for streaming, in a streaming interface, on a display connected to the user device.

At 364, a second application (e.g., an arcade application) stored in the memory of the user device is executed to receive, process, buffer, stream, streaming content of a game transmitted from an arcade provider system and received in the user device in communication with the arcade provider system via a communications network. The gaming content is output through a gaming interface, to a display connected to the user device for the user to play or interact with. The first and second application are concurrently executed, and the streaming content and the gaming content are concurrently output and presenting on the display.

At 366, a third application (e.g., user interaction tracking application) stored in the user device to periodically generate a plurality of heartbeat messages at a frequency and transmit the heartbeat messages to the content provider system and the arcade provider system during the execution of the first application. Each heartbeat message is timestamped and includes a header, one or more playback states, and one or more gameplay states.

The header includes information about the content item, the game (if the gaming content of the game is output in the arcade interface), the user device, and the user account. The one or more playback states include a first playback state indicating a current output streaming rate (e.g., bps), a second playback state indicating a duration of time in the current output streaming rate, and a third playback state indicating a current play rate (e.g., paused (play rate is zero), 1×, 2×, 3×). In some embodiments, the duration of time in the current play rate is the time interval calculated based on the frequency of the first heartbeat messages. The one or more gameplay states include a first gameplay state indicating a current state of a controller (e.g., a remote control) operated by the user as being active (e.g., in use) or idle (e.g., not in use), optionally a second gameplay state indicating a current mode of the controller as being a “streaming” mode or a “gaming” mode, and a third gameplay state indicating a duration of time in the current state of the controller. In some embodiments, the duration of time in the current play rate is the same as the duration of time in the current gameplay state, which is the time interval calculated based on the frequency of the heartbeat messages.

At 368, a total duration of consumed content and a total duration of consumed playtime by the user during execution of the first application are determined, respectively, based on the playback states and gameplay states of the plurality of heartbeat messages.

At 370, a first determination and a second determination are made. The first determination is made, for each content item output in the streaming interface, on a first reward to be awarded to a user account associated with the user device, based on the total duration of the consumed content of the content item and a pre-established reward rule. The second determination is made, for each game output in the arcade interface, on a second reward to be awarded to the user account, based on the total duration of the consumed playtime and the pre-established reward rule. At 380, the first reward and the second reward are aggregated and distributed to the user account.

FIG. 3E is a flow diagram illustrating an example method 300E for playing an arcade game while streaming content is being output concurrently. Method 300E may include one or more operations/steps. Few or additional operations/steps may be included. The operations of method 300E may be combined with operations of another method described herein in any suitable manner.

At 382, a streaming interface of a user device is activated, upon a first user request or execution of a first application stored in the user device, to receive streaming content of a content item transmitted from a content provider system and output the streaming content in a streaming interface on a display connected to a user device.

At 384, an arcade interface is activated, upon a second user request or execution of a second application stored in the user device, to receive gaming content of an arcade game selected by the user and transmitted from an arcade provider system. The gaming content is output in the arcade interface on the display, concurrently with the outputting of the streaming content in the streaming interface. The streaming interface and the gaming interface are co-presenting concurrently on the display.

At 386, a user input selecting an “arcade” mode of a controller connected to the user device is received in the user device, and one or more input events to control the game within the arcade interface are received from the controller in the “arcade” mode, while the streaming content is being output in the streaming interface for presentation.