236 output the name of the recognized object.

The fourth recognition technology is a graphic recognition engine 238. The graphic recognition engine 238 locates and identifies graphic features in an image. The graphic features identified may include a dominant color, shapes, for example a square, distinct horizontal or vertical lines, and the like. The output of the graphic recognition engine 238 is a list of words describing the geographic features identified in the image portions of the media.

The output from any of the recognition engines or technologies used to extract image features may also be a null set. For example, the output from the logo recognition engine 232 may be blank when the logo recognition engine 232 did not locate a logo in the image portion of the unknown media.

The feature weighter 218 weights and combines the audio features and the video features into a list of features. The features may include a list of words, a list of keywords, a list of key phrases, a brand, a product name, a product type, the product manufacturer and the like. In some examples, the feature weighter 218 may use feedback between the audio features and the video features to weight the different features. In other examples, the feature weighter 218 may weight a feature based on the engine or technology used to extract the feature.

For example, the feature weighter 218 will give the highest weight to a feature extracted by the logo recognition engine 232, the next highest weight will be given to features extracted by the OCR engine 234, the next highest weight will be given to features extracted by the speech recognition engine 226 and the lowest weight will be given to features extracted by a graphic recognition engine 238. In some examples, the relative weights between the logo recognition engine 232, the OCR engine 234, the speech recognition engine 226 and the graphic recognition engine 238 are 10, 5, 2 and 1 respectively. The feature weighter 218 passes the list of weighted features to the database search engines 220.

The database search engines 220 use the list of weighted features from the feature weighter 218 to query a database of pre-tagged media, for example database 110. The database search engines may use one or more search engines to query the database of pre-tagged media. When multiple search engines are used, the different search engines may use different techniques to search the database. For example, one search engine may use a vector space model for document retrieval. Another search engine may use a probabilistic information retrieval technique.

In some examples, the queries use the weighted list of words from the feature weighter 218. In other examples, when detected, the categories may be used to narrow the search space. For example, the database search engines 220 may limit the search space to a company name, brand name, a manufacturer's name, a product name, a product type or the like.

The results from the database search engines 220 are passed to the tag assigner 222. The tag assigner 222 may display the results of the searches on the display 224, on a remote display (not shown), save the list for later use, or print the list on a printer (not shown). The results from the searches are a list of documents found in the database of pre-tagged media that are relevant to the unknown media. The list includes a relevancy score indicating how relevant the listed document is with respect to the unknown media. In addition, a list of tags for each of the identified document may also be displayed. A user can confirm that one or more of the tags from a relevant document apply to the unknown media. Once confirmed, the tags are assigned to the media and the media, along with the assigned tags, is stored into the database of pre-tagged media (e.g. database 110).

In some examples, the relevancy score is compared to a threshold, for example 85% relevant. When the relevancy score is above the threshold, the tag assigner 222 automatically assigns the tags from the relevant document to the unknown media and stores the now tagged media, along with the assigned tags, into the database of pre-tagged media (e.g. database 110). In some examples, the threshold is between 75% and 100%. Once the unknown media is tagged and stored in database 110, the media tracking entity 106 can track and assign impressions of the media to the company that is advertising the product in the media.

While an example manner of implementing the media tagger 108 of FIG. 1 is illustrated in FIG. 2, one or more of the elements, processes and/or devices illustrated in FIG. 2 may be combined, divided, re-arranged, omitted, eliminated and/or implemented in any other way. Further, the example media separator 212, an audio extractor 214, an image extractor 216, a feature weighter 218, database search engines 220, a tag assigner 222 and a display 224 and/or, more generally, the example media tagger 108 of FIG. 2 may be implemented by hardware, software, firmware and/or any combination of hardware, software and/or firmware. Thus, for example, any of the example media separator 212, an audio extractor 214, an image extractor 216, a feature weighter 218, database search engines 220, a tag assigner 222 and a display 224 and/or, more generally, the example media tagger 108 could be implemented by one or more analog or digital circuit(s), logic circuits, programmable processor(s), application specific integrated circuit(s) (ASIC(s)), programmable logic device(s) (PLD(s)) and/or field programmable logic device(s) (FPLD(s)). When reading any of the apparatus or system claims of this patent to cover a purely software and/or firmware implementation, at least one of the example, media separator 212, an audio extractor 214, an image extractor 216, a feature weighter 218, database search engines 220, a tag assigner 222 and a display 224 is/are hereby expressly defined to include a tangible computer readable storage device or storage disk such as a memory, a digital versatile disk (DVD), a compact disk (CD), a Blu-ray disk, etc. storing the software and/or firmware. Further still, the example media tagger 108 of FIG. 2 may include one or more elements, processes and/or devices in addition to, or instead of, those illustrated in FIG. 3, and/or may include more than one of any or all of the illustrated elements, processes and devices.

A flowchart representative of example machine readable instructions for implementing the media tagger 108 of FIG. 2 is shown in FIG. 3. In this example, the machine readable instructions comprise a program for execution by a processor such as the processor 612 shown in the example processor platform 600 discussed below in connection with FIG. 6. The program may be embodied in software stored on a tangible computer readable storage medium such as a CD-ROM, a floppy disk, a hard drive, a digital versatile disk (DVD), a Blu-ray disk, or a memory associated with the processor 612, but the entire program and/or parts thereof could alternatively be executed by a device other than the processor 612 and/or embodied in firmware or dedicated hardware. Further, although the example program is described with reference to the flowchart illustrated in FIG. 4, many other methods of implementing the example media tagger 108 may alternatively be used. For example, the order of execution of the blocks may be changed, and/or some of the blocks described may be changed, eliminated, or combined.

As mentioned above, the example processes of FIGS. 3, 4, and 5 may be implemented using coded instructions (e.g., computer and/or machine readable instructions) stored on a tangible computer readable storage medium such as a hard disk drive, a flash memory, a read-only memory (ROM), a compact disk (CD), a digital versatile disk (DVD), a cache, a random-access memory (RAM) and/or any other storage device or storage disk in which information is stored for any duration (e.g., for extended time periods, permanently, for brief instances, for temporarily buffering, and/or for caching of the information). As used herein, the term tangible computer readable storage medium is expressly defined to include any type of computer readable storage device and/or storage disk and to exclude propagating signals and to exclude transmission media. As used herein, “tangible computer readable storage medium” and “tangible machine readable storage medium” are used interchangeably. Additionally or alternatively, the example processes of FIGS. 3, 4, and 5 may be implemented using coded instructions (e.g., computer and/or machine readable instructions) stored on a non-transitory computer and/or machine readable medium such as a hard disk drive, a flash memory, a read-only memory, a compact disk, a digital versatile disk, a cache, a random-access memory and/or any other storage device or storage disk in which information is stored for any duration (e.g., for extended time periods, permanently, for brief instances, for temporarily buffering, and/or for caching of the information). As used herein, the term non-transitory computer readable medium is expressly defined to include any type of computer readable storage device and/or storage disk and to exclude propagating signals and to exclude transmission media. As used herein, when the phrase “at least” is used as the transition term in a preamble of a claim, it is open-ended in the same manner as the term “comprising” is open ended.

The program 300 of FIG. 3 begins where the media separator 212 of FIG. 2 accesses a new or unknown media (block 302). The media separator 212 separates the audio portion of the media from the image portions of the media. The media separator 212 passes the audio portion of the media to the audio extractor 214 of FIG. 2 and passes the image portions of the media to the image extractor 216 of FIG. 2. Flow continues in parallel at block 304 and 306.

The audio extractor 214 extracts audio features from the audio portion of the unknown media (block 304). The extracted audio features are passed to the feature weighter 218 of FIG. 2. In some examples, the type of engine or technology used to extract each audio feature is also passed to the feature weighter 218. Flow continues at block 308.

The image extractor 216 extracts image features from the image portion of the unknown media (block 306). The extracted image features are passed to the feature weighter 218. In some examples, the type of engine or technology used to extract each image feature is also passed to the feature weighter 218. Flow continues at block 308.

The feature weighter 218 weights and combines the audio features and the video features into a list of features (block 308). The features may include a list of words, a list of keywords, a list of key phrases, a category with an assigned value, for example, a brand, a product name, a product type, the product manufacturer and the like. In some examples, the feature weighter 218 may use feedback between the audio features and the video features to weight the different features. In other examples, the feature weighter 218 may weight a feature based on the type of engine or technology used to extract the feature. The feature weighter 218 passes the list of weighted features to the database search engines 220 of FIG. 2. Flow continues in block 310.

The database search engines 220 use the list of weighted features to query a database of pre-tagged media, for example database 110 (block 310). The database search engines may use one or more search engines to query the database of pre-tagged media. When multiple search engines are used, the different search engines may use different techniques to search the database. For example, one search engine may use a vector space model for document retrieval. Another search engine may use a probabilistic information retrieval technique.

In some examples, the queries use a list of words from the feature weighter 218. In other example, when detected, the categories may be used to narrow the search space. For example, the search engines may limit the search space to a brand name, a manufacturer's name, a product name, a product type or the like. The results from the search engines are passed to the tag assigner 222 of FIG. 2. Flow continues at block 312.

The tag assigner may display the results of the searches on display 224 of FIG. 2, on a remote display (not shown), save the list for later use (block 312). The results from the searches are a list of documents found in the database of pre-tagged media that are relevant to the unknown media. The list includes a relevancy score indicating how relevant the listed document is with respect to the unknown media. In addition, a list of tags for each of the identified document will be displayed. A user can confirm that one or more of the tags from a relevant document apply to the unknown media. Once confirmed, the tags are assigned to the media and the media, along with the assigned tags, is stored into the database of pre-tagged media (e.g. database 110).

In some examples, the relevancy score is compared to a threshold. When the relevancy score is above the threshold, the tag assigner automatically assigns the tags from the relevant document to the unknown media and stores the now tagged media, along with the assigned tags, into the database of pre-tagged media (e.g. database 110). In some examples the relevancy score threshold is between 75% and 100%, for example 85%. Once the unknown media is tagged and stored in database 110, the media tracking entity 106 can track and assign impressions of the media to the company that is advertising the product in the media.

FIG. 4 is a flowchart representative of example machine readable instructions for implementing the process in block 304 of FIG. 3 (extract audio features). The speech recognition engine 226 of FIG. 2 extracts a list of words spoken in the audio portion of the unknown media (block 402). Any speech to text recognition engine may be used, for example Google's speech to text recognition engine may be used. The output of the speech to text engine is a list of words spoken in the audio portion of the media. Flow continues at block