9202 who is interacting with a set-top box 9210 and television 9220 as he interacts with DVR content recording content and playing back recorded content using various modes including trick plays. The set-top box 9210 interacts with a Video Content Delivery System 9250 which delivers the content across a Network 9230.

It continues with Viewer Viewing VOD Content 9203 who is interacting with a set-top box 9210 and television 9220 as he interacts with VOD content playing the content using various modes including trick plays. The set-top box 9210 interacts with a Video Content Delivery System 9250 which delivers the content across a Network 9230.

It continues with Viewer viewing video content using tablet, smart phone, IP TV, or other video viewing device 9204 who is interacting with a variety of Video Viewing Devices 9212, including but not limited to tablet, smart phone, IP TV, PC, etc. The video viewing device interacts with a Video Content Delivery System 9250 which delivers the content across a Network 9230.

Video Content Delivery System 9250 then interacts with a Viewer Interaction Data, Data Collection System 9260 which collects all manner of viewer interaction data including Linear viewing including time-shifted linear viewing, Digital Video Recorder recording and playback/viewing, and Video on Demand viewing. The Viewer Interaction Data, Data Collection System 9260 then processes the data as needed to load it to a Media Measurement Data Base 100. The data in the Media Measurement Data Base 100 can then be used as input to my Aggregation Engine 200 as described in FIG. 2.

FIG. 2 illustrates an exemplary media measurement process from creation of the Video Viewing Activity Data File 130 to creation of the various aggregated video viewing activity files (Parts 220, 230, 240, 250) which can then be provided to downstream analytic processes as shown by Provide Files to Downstream Analytic Processes 210.

As noted previously, the video viewing activity may be sourced from a Media Measurement Data Base such as the one described in the Cable Television Laboratories, Inc. specification. The populating of the Media Measurement Database 100 is beyond the scope of this application and so only brief remarks will be made in reference to that. There are video viewing data collection systems that are commonly used in the industry for collecting channel tuning or video viewing activity data including switched digital video systems, set top box applications, internet protocol video viewing applications, and other video viewing applications. These systems enable the collection of the video viewing events which can be loaded to a Media Measurement Database 100. From such a database, Video Viewing Activity Data can be extracted in a format similar to that shown in FIG. 3 Video Viewing Activity Data File 130.

Proceeding with the review of FIG. 2, the process begins with Media Viewing Measurement Process Overview 110. The first step is to extract the video viewing events as per Extract Video Viewing Activity Data from Media Measurement Data Base 120. Those skilled in the art will have no difficulty creating a database query or similar process to extract data from a Media Measurement Database 100 or other source and making it available in a format similar to that defined in Video Viewing Activity Data File 130. The file structure is defined in FIG. 3 Video Viewing Activity Data File 130 which describes an exemplary format for the input video viewing activity data. As part of the extract process, the system may perform the following activities to prepare the data for processing:

- Discard tuning events having a duration less than a specified number of seconds
- Truncate tuning events having a duration greater than a specified number of seconds
- Map channels such that viewing of a high definition and a standard definition version of the same channel are assigned the same channel call sign

Other data preparation activities can be done according to business needs. Those with reasonable skill in the art will readily recognize how to perform these activities.

Proceeding with the review of FIG. 2, the Video Viewing Activity Data File 130 is then passed to a Data Explosion Process 140. In this process the individual tuning events are exploded such that there is one record created for every second of the tune duration represented in the video viewing activity record. Additionally, the detail keys and the tune-in datetime and tune-out datetime and tune duration can be discarded at this time because the MapReduce process will aggregate across those detail keys. In an alternative embodiment, any field that is not used in the aggregation process could be omitted from the Video Viewing Activity Data File 130 file. I have included these additional fields to provide a comprehensive picture recognizing that one can always drop the fields that they choose not to use.

The computer algorithm that the Data Explosion Process 140 runs to create the Video Viewing Detail File 150 is as follows:

Looping process to create the video viewing detail

records:

For each input record in Video Viewing Activity Data File 130

PERFORM VARYING SUB

FROM TUNE_IN_SECOND_OF_DAY 1090

BY 1

UNTIL SUB > TUNE_OUT_SECOND_OF_DAY 1110

MOVE GEOGRAPHIC_ID
1010
to GEOGRAPHIC_ID
1210

MOVE VIDEO_SERVER_ID
1020
to VIDEO_SERVER_ID
1220

MOVE VIDEO_CONTENT_ID
1030
to VIDEO_CONTENT_ID
1230

MOVE DEMOGRAPHIC_ID
1070
to DEMOGRAPHIC_ID
1240

MOVE SUB

to SECOND_OF_DAY_WHEN_TUNED 1250

MOVE 1

to COUNT_OF_1
1260

WRITE Video Viewing Detal File 150

End Loop

Note:

The following fields were optionally included in Video Viewing Activity Data File 130 for data validation purposes. During Data Explosion Process 140 they are dropped so that they do not pass forward to Video Viewing Detail File 150.

VIDEO_ASSET VIEWING_DEVICE_ID
1040

HOUSE_ID
1050

VIEWER_ID
1060

TUNE_IN_DATE_TIME
1080

TUNE_OUT_DATE_TIME
1100

TUNE_DURATION_SECONDS
1120.

The explosion process can be run in several ways to achieve the same result. I have included two alternative embodiments.

Alternative Embodiment #1

For each input record in Video Viewing Activity Data File 130

PERFORM VARYING SUB

FROM TUNE_IN_SECOND_OF_DAY 1090

BY 1

UNTIL SUB > (TUNE_IN_SECOND_OF_DAY
1090 +

TUNE_DURATION_SECONDS
1120)

MOVE GEOGRAPHIC_ID
1010
to GEOGRAPHIC_ID
1210

MOVE VIDEO_SERVER_ID
1020
to VIDEO_SERVER_ID
1220

MOVE VIDEO_CONTENT_ID
1030
to VIDEO_CONTENT_ID
1230

MOVE DEMOGRAPHIC_ID
1070
to DEMOGRAPHIC_ID
1240

MOVE SUB
to SECOND_OF_DAY_WHEN_TUNED 1250

MOVE 1
to COUNT_OF_1
1260

WRITE Video Viewing Detal File 150

End Loop

Alternative Embodiment #2

If the tune duration is provided, the looping construct can be done as follows:

For each input record in Video Viewing Activity Data File 130

PERFORM VARYING SUB

FROM TUNE_IN_DATE_TIME 1080

BY 1

UNTIL SUB > TUNE_OUT_DATE_TIME 1100

MOVE GEOGRAPHIC_ID
1010
to GEOGRAPHIC_ID
1210

MOVE VIDEO_SERVER_ID
1020
to VIDEO_SERVER_ID
1220

MOVE VIDEO_CONTENT_ID
1030
to VIDEO_CONTENT_ID
1230

MOVE DEMOGRAPHIC_ID
1070
to DEMOGRAPHIC_ID
1240

MOVE SUB

to SECOND_OF_DAY_WHEN_TUNED 1250

MOVE 1

to COUNT_OF_1
1260

WRITE Video Viewing Detail File 150

End Loop

Note: In this case, the SECOND_OF_DAY_WHEN_TUNED 1250 will represent a UNIX EPOCH time stamp.

Note: In each case the Video Viewing Detail File 150 records can be written directly to the Hadoop Distributed File System (HDFS) so that the video viewing detail records are ready for use by the MapReduce distributed computing framework.

Note: The Video Viewing Activity Data File 130 can be provided by the Extract 120 process in any computer readable format including, but not limited to, database tables, flat files, JSON messages, and XML messages. Alternatively, such video viewing events can be collected directly from the source without the need for a Media Measurement Database 100. In such a case, those events can still be provided as video viewing activity in a format similar to that shown in FIG. 3 for use by the Data Explosion Process 140.

For all of the above embodiments, at the completion of Data Explosion Process 140, one record has been written to the Video Viewing Detail File 150 for each second of the tune duration represented in the video viewing activity record. The Sample Data in FIG. 3 shows a non-limiting example of the input data for the Data Explosion Process 140. The Sample Data in FIG. 4 shows a non-limiting example of the data produced by the Data Explosion Process 140. The reader will note that FIG. 4 sample data contains one record for every second of the tuning activity represented in the input data.

Those skilled in the art will readily recognize that the Data Explosion Process 140 is suitable for running in parallel on multiple computers simultaneously with each process creating Video Viewing Detail File records that can be fed into the MapReduce Aggregation Engine 200.

Proceeding with the review of FIG. 2, the Video Viewing Detail File 150 data residing in HDFS is now ready for use by the MapReduce Aggregation Engine 200. The MapReduce Aggregation Engine 200 runs various word count algorithms against the incoming data. Each word count algorithm will aggregate the data to a separate level as shown in FIG. 2 (Parts 220, 230, 240, 250) with the details shown in FIGS. 5-8.

The MapReduce process can be coded in JAVA or in Pig. I have coded this in Pig. The code below can be used to create the four output files reviewed in the Drawings (FIGS. 5 to 8):

- I. Aggregated Video Viewing Geo+Server+Content+Demo File 220
- II. Aggregated Video Viewing Geo+Server+Content File 230
- III. Aggregated Video Viewing Content File 240
- IV. Aggregated Video Viewing File 250.

Using these four outputs, the reader will have a comprehensive set of aggregated video viewing metrics. The reader should recognize that the aggregation logic shown below provides several illustrations of what can be done. Additional aggregation combinations will be obvious to those skilled in the art.

The reader will note that I have used very descriptive names in the Pig Latin code below so as to convey the meaning of what is happening. Much shorter names could be used to produce the same result.

Creating the Aggregated Video Viewing Geo+Server+Content+Demo File 220

The Pig Latin coding to create the Aggregated Video Viewing Geo+Server+Content+Demo File 220 is shown next.

This summarization aggregates viewing activity for each combination of geographic identifier and server identifier and content identifier and demographic identifier for each second of the aggregation period. The result provides viewing metrics for each combination of geographic area and video server and content and demographic identifier as represented in the input data. As a nonlimiting example, a Video Content Identifier may be a channel call sign; this summary then provides a count of how many devices were tuned to that channel within each geographic area (a city or a region) and within each video server and for each demographic group. As an example, how many devices in the DENV Geo area served by SERVER-01 were tuned to ABC from Demo code 40-60k during each second of the time period. A second example, how many devices in the DENV Geo area served by SERVER-01 were tuned to Program Monday Night Football from Demo code 40-60k during cach second of the time period.

Video_Viewing_Detail_Data = LOAD ‘Video-Viewing-Detail-File’ 150 as

(GEOGRAPHIC_ID:chararray,
1210

VIDEO_SERVER_ID:chararray,
1220

VIDEO_CONTENT_ID:chararray,
1230

DEMOGRAPHIC_ID:chararray,
1240

Video_Viewing_Detail_Data = LOAD ‘Video-Viewing-Detail-File’ 150 as

(GEOGRAPHIC_ID:chararray.
1210

VIDEO_SERVER_ID:chararray,
1220

VIDEO_CONTENT_ID:chararray,
1230

DEMOGRAPHIC_ID:chararray,
1240

SECOND_OF_DAY_WHEN_TUNED:chararray,
1250

COUNT_OF_1:chararray);
1260

Aggregated_Video_Geo_Server_Content_Demo_Viewing =

GROUP Video_Viewing_Detail_Data

by (GEOGRAPHIC_ID,
1410

VIDEO_SERVER_ID,
1420

VIDEO_CONTENT_ID,
1430

DEMOGRAPHIC_ID,
1440

SECOND_OF_DAY_WHEN_TUNED); 1450

Count_of_Aggregated_Video_Geo_Server_Content_Demo_Viewing_by_Second =

FOREACH Aggregated_Video_Geo_Server_Content_Demo_Viewing

GENERATE group

as Aggregated_Video_Geo_Server_Content_Demo_Viewing,

COUNT (Video_Viewing_Detail_Data)

as AggrGeoServerContentDemoViewingThisSecond;

STORE

Count_of_Aggregated_Video_Geo_Server_Content_Demo_Viewing_by_Second

1460

INTO ‘Aggregated_Video_Viewing_Geo_Server_Content_Demo_File’; 220

Note:

A sample of the file created by the aggregation is shown in FIG. 5 Sample Data.

Creating the Aggregated Video Viewing Geo+Server+Content File 230

The Pig Latin coding to create the Aggregated Video Viewing Geo+Server+Content File 230 is shown next.

This summarization aggregates viewing activity for cach combination of geographic identifier and server identifier and content identifier for each second of the aggregation period. The result provides viewing metrics for each combination of geographic area and video server and content id as represented in the input data. As a nonlimiting example, a Video Content Identifier may be a channel call sign; this summary then provides a count of how many devices were tuned to that channel within each geographic area (a city or a region) and within each video server. As an example, how many devices in the DENV Geo area served by SERVER-01 were tuned to ABC during each second of the time period.

Video_Viewing_Detail_Data = LOAD ‘Video-Viewing-Detail-File’ 150 as

(GEOGRAPHIC_ID:chararray,
1210

VIDEO_SERVER_ID:chararray,
1220

VIDEO_CONTENT_ID:chararray,
1230

DEMOGRAPHIC_ID:chararray,
1240

SECOND_OF_DAY_WHEN_TUNED:chararray,
1250

COUNT_OF_1:chararray);
1260

Video_Viewing_Geo_Server_Content_Data =

FOREACH Video_Viewing_Detail_Data

GENERATE GEOGRAPHIC_ID,
1210

VIDEO_SERVER_ID,
1220

VIDEO_CONTENT_ID,
1230

SECOND_OF_DAY_WHEN_TUNED,
1250

COUNT_OF_1;
1260

Aggregated_Video_Geo_Server_Content_Viewing =

GROUP Video_Viewing_Geo_Server_Content_Data

by (GEOGRAPHIC_ID,
1610

VIDEO_SERVER_ID,
1620

VIDEO_CONTENT_ID,
1630

SECOND_OF_DAY_WHEN_TUNED); 1650

Count_of_Aggregated_Video_Geo_Server_Content_Viewing_by_Second =

FOREACH Aggregated_Video_Geo_Server_Content_Viewing

GENERATE group as Aggregated_Video_Geo_Server_Content_Viewing,

COUNT(Video_Viewing_Geo_Server_Content_Data)

as AggrGeoServerContentViewingThisSecond;

STORE Count_of_Aggregated_Video_Geo_Server_Content_Viewing_by_Second

1660

INTO ‘Aggregated_Video_Viewing_Geo_Server_Content_File’; 230

Note:

A sample of the file created by the aggregation is shown in FIG. 6 Sample Data.

Creating the Aggregated Video Viewing Content File 240

The Pig Latin coding to create the Aggregated Video Viewing Content File 240 is shown next. This summarization aggregates viewing across all geographic identifiers, all servers, and all demographic groups for each second of the aggregation period. The result provides viewing metrics for the content (channel) across all geographic areas, video servers, and demographic groups as represented in the input data. As a nonlimiting example, a Video Content Identifier may be a channel call sign; this summary then provides a count of how many devices were tuned to that channel during cach second of the viewing period.

Video_Viewing_Detail_Data = LOAD ‘Video-Viewing-Detail-File’ 150 as

(GEOGRAPHIC_ID:chararray,
1210

VIDEO_SERVER_ID:chararray,
1220

VIDEO_CONTENT_ID:chararray,
1230

DEMOGRAPHIC_ID:chararray,
1240

SECOND_OF_DAY_WHEN_TUNED:chararray,
1250

COUNT_OF_1; chararray);
1260

Video_Viewing_Content_Data =

FOREACH Video_Viewing_Detail_Data

GENERATE VIDEO_CONTENT_ID,
1230

SECOND_OF_DAY_WHEN_TUNED,
1250

COUNT_OF_1;
1260

Aggregated_Video_Content_Viewing =

GROUP Video_Viewing_Content_Data

by (VIDEO_CONTENT_ID,
1830

SECOND_OF_DAY_WHEN_TUNED); 1850

Count_of_Aggregated_Video_Content_Viewing_by_Second =

FOREACH Aggregated_Video_Content_Viewing

GENERATE group as Aggregated_Video_Content_Viewing,

COUNT (Video_Viewing_Content_Data)

as AggrContentViewingThisSecond;

STORE Count_of_Aggregated_Video_Content_Viewing_by_Second 1860

INTO ‘Aggregated_Video_Viewing_Content_File’; 240

Note:

A sample of the file created by the aggregation is shown in FIG. 7 Sample Data.

Creating the Aggregated Video Viewing File 250

The Pig Latin coding to create the Aggregated Video Viewing File 250 is shown next. This summarization aggregates viewing activity across all geographic identifiers, all servers, all content, and all demographic groups for each second of the aggregation period. The result provides viewing metrics across all geographic areas, video servers, content ids, and demographic groups as represented in the input data. As a nonlimiting example, this aggregation will provide insight into total viewing activity during each second of the measurement period. This is creating the denominator which can be used in calculations which measure the percentage of the total viewing audience that a particular piece of content earned.

Video_Viewing_Detail_Data = LOAD ‘Video-Viewing-Detail-File’ 150 as

(GEOGRAPHIC_ID:chararray,
1210

VIDEO_SERVER_ID:chararray,
1220

VIDEO_CONTENT_ID:chararray,
1230

DEMOGRAPHIC_ID:chararray,
1240

SECOND_OF_DAY_WHEN_TUNED:chararray,
1250

COUNT_OF_1:chararray);
1260

Video_Viewing_Data =

FOREACH Video_Viewing_Detail_Data

GENERATE SECOND_OF_DAY_WHEN_TUNED,
1260

COUNT_OF_1;
1250

Aggregated_Video_Viewing =

GROUP Video_Viewing_Data_by_SECOND_OF_DAY_WHEN_TUNED; 2050

Count_of_Aggregated_Video_Viewing_by_Second =

FOREACH Aggregated_Video_Viewing

GENERATE group as Aggregated_Video_Viewing,

COUNT (Video_Viewing_Data) as AggrViewingThisSecond;

STORE Count_of_Aggregated_Video_Viewing_by_Second 2060

INTO ‘Aggregated_Video_Viewing_File’; 250

Note:

A sample of the file created by the aggregation is shown in FIG. 8 Sample Data.

FIG. 3 illustrates an exemplary record layout for a Video Viewing Activity Data File 130 record formatted for use as input to the Data Explosion Process 140, according to one embodiment.