Camera for printing manipulated images | Patent Number 07460152
US 07460152 B2Kia Silverbrook
Simon Robert Walmsley
Simon Robert Walmsley
Paul Lapstun
Paul Lapstun
The present invention relates to a handheld camera which includes a sensor adapted to sense an image, an input for receiving predetermined data representing a respective form of data manipulation and a processing system. The processing system is adapted to obtain the image from the sensor and manipulate the image in accordance with the predetermined data to thereby generate a manipulated image.
- 1. A handheld camera, said camera comprising:na sensor adapted to sense an image;an input for receiving predetermined data representing a respective form of data manipulation, the predetermined data being disposed on a surface of the card; and,a processing system, the processing system being adapted to:obtain the image from the sensor,determine a card image in accordance with signals received from the input,rotate the card image in accordance with a skew of the card with respect to the input, andmanipulate the image in accordance with the predetermined data to thereby generate a manipulated image.
This is a Continuation of U.S. Ser. No. 09/113,057 filed on Jul. 10, 1998 now U.S. Pat. No. 6,850,274.
FIELD OF THE INVENTION
The present invention relates to a data processing method and apparatus and, in particular, discloses a process for the Utilisation of Image Texture Mapping in Photographs.
The present invention further relates to the field of image processing and in particular discloses a method for the creation of specialised effects in output images.
BACKGROUND OF THE INVENTION
Recently, a camera system has been proposed which provides for the creation of instant images on demand in a similar manner to an instant Polaroidâ„¢ camera. The system proposed includes means for insertion of a card to manipulate the printed image so as to produce interesting effects.
SUMMARY OF THE INVENTION
It is an object of the present invention to provide for the production of interesting texture effects in photographed images produced by a handheld camera device.
In accordance with a first aspect of the present invention, there is provided a handheld camera for the output of an image sensed by the camera, with the camera including:
- sensing means for sensing an image;
- texture mapping means for adding texturing effects to the sensed image to produce a textured image; and
- display means for displaying the textured image.
BRIEF DESCRIPTION OF THE DRAWINGS
Notwithstanding any other forms which may fall within the scope of the present invention, preferred forms of the invention will now be described, by way of example only, with reference to the accompanying drawings in which:
FIG. 1
illustrates an Artcam device constructed in accordance with the preferred embodiment;FIG. 2
is a schematic block diagram of the main Artcam electronic components;FIG. 3
is a schematic block diagram of the Artcam Central Processor;FIG. 3(
a) illustrates the VLIW Vector Processor in more detail;FIG. 4
illustrates the Processing Unit in more detail;FIG. 5
illustrates the ALU 188 in more detail;FIG. 6
illustrates the In block in more detail;FIG. 7
illustrates the Out block in more detail;FIG. 8
illustrates the Registers block in more detail;FIG. 9
illustrates the Crossbar1 in more detail;FIG. 10
illustrates the Crossbar2 in more detail;FIG. 11
illustrates the read process block in more detail;FIG. 12
illustrates the read process block in more detail;FIG. 13
illustrates the barrel shifter block in more detail;FIG. 14
illustrates the adder/logic block in more detail;FIG. 15
illustrates the multiply block in more detail;FIG. 16
illustrates the I/O address generator block in more detail;FIG. 17
illustrates a pixel storage format;FIG. 18
illustrates a sequential read iterator process;FIG. 19
illustrates a box read iterator process;FIG. 20
illustrates a box write iterator process;FIG. 21
illustrates the vertical strip read/write iterator process;FIG. 22
illustrates the vertical strip read/write iterator process;FIG. 23
illustrates the generate sequential process;FIG. 24
illustrates the generate sequential process;FIG. 25
illustrates the generate vertical strip process;FIG. 26
illustrates the generate vertical strip process;FIG. 27
illustrates a pixel data configuration;FIG. 28
illustrates a pixel processing process;FIG. 29
illustrates a schematic block diagram of the display controller;FIG. 30
illustrates the CCD image organization;FIG. 31
illustrates the storage format for a logical image;FIG. 32
illustrates the internal image memory storage format;FIG. 33
illustrates the image pyramid storage format;FIG. 34
illustrates a time line of the process of sampling an Artcard;FIG. 35
illustrates the super sampling process;FIG. 36
illustrates the process of reading a rotated Artcard;FIG. 37
illustrates a flow chart of the steps necessary to decode an Artcard;FIG. 38
illustrates an enlargement of the left hand corner of a single Artcard;FIG. 39
illustrates a single target for detection;FIG. 40
illustrates the method utilised to detect targets;FIG. 41
illustrates the method of calculating the distance between two targets;FIG. 42
illustrates the process of centroid drift;FIG. 43
shows one form of centroid lookup table;FIG. 44
illustrates the centroid updating process;FIG. 45
illustrates a delta processing lookup table utilised in the preferred embodiment;FIG. 46
illustrates the process of unscrambling Artcard data;FIG. 47
illustrates a magnified view of a series of dots;FIG. 48
illustrates the data surface of a dot card;FIG. 49
illustrates schematically the layout of a single datablock;FIG. 50
illustrates a single datablock;FIG. 51
andFIG. 52
illustrate magnified views of portions of the datablock ofFIG. 50
;FIG. 53
illustrates a single target structure;FIG. 54
illustrates the target structure of a datablock;FIG. 55
illustrates the positional relationship of targets relative to border clocking regions of a data region;FIG. 56
illustrates the orientation columns of a datablock;FIG. 57
illustrates the array of dots of a datablock;FIG. 58
illustrates schematically the structure of data for Reed-Solomon encoding;FIG. 59
illustrates an example Reed-Solomon encoding;FIG. 60
illustrates the Reed-Solomon encoding process;FIG. 61
illustrates the layout of encoded data within a datablock;FIG. 62
illustrates the sampling process in sampling an alternative Artcard;FIG. 63
illustrates, in exaggerated form, an example of sampling a rotated alternative Artcard;FIG. 64
illustrates the scanning process;FIG. 65
illustrates the likely scanning distribution of the scanning process;FIG. 66
illustrates the relationship between probability of symbol errors and Reed-Solomon block errors;FIG. 67
illustrates a flow chart of the decoding process;FIG. 68
illustrates a process utilization diagram of the decoding process;FIG. 69
illustrates the dataflow steps in decoding;FIG. 70
illustrates the reading process in more detail;FIG. 71
illustrates the process of detection of the start of an alternative Artcard in more detail;FIG. 72
illustrates the extraction of bit data process in more detail;FIG. 73
illustrates the segmentation process utilized in the decoding process;FIG. 74
illustrates the decoding process of finding targets in more detail;FIG. 75
illustrates the data structures utilized in locating targets;FIG. 76
illustrates the Lancos 3 function structure;FIG. 77
illustrates an enlarged portion of a datablock illustrating the clockmark and border region;FIG. 78
illustrates the processing steps in decoding a bit image;FIG. 79
illustrates the dataflow steps in decoding a bit image;FIG. 80
illustrates the descrambling process of the preferred embodiment;FIG. 81
illustrates one form of implementation of the convolver;FIG. 82
illustrates a convolution process;FIG. 83
illustrates the compositing process;FIG. 84
illustrates the regular compositing process in more detail;FIG. 85
illustrates the process of warping using a warp map;FIG. 86
illustrates the warping bi-linear interpolation process;FIG. 87
illustrates the process of span calculation;FIG. 88
illustrates the basic span calculation process;FIG. 89
illustrates one form of detail implementation of the span calculation process;FIG. 90
illustrates the process of reading image pyramid levels;FIG. 91
illustrates using the pyramid table for blinear interpolation;FIG. 92
illustrates the histogram collection process;FIG. 93
illustrates the color transform process;FIG. 94
illustrates the color conversion process;FIG. 95
illustrates the color space conversion process in more detail;FIG. 96
illustrates the process of calculating an input coordinate;FIG. 97
illustrates the process of compositing with feedback;FIG. 98
illustrates the generalized scaling process;FIG. 99
illustrates the scale in X scaling process;FIG. 100
illustrates the scale in Y scaling process;FIG. 101
illustrates the tessellation process;FIG. 102
illustrates the sub-pixel translation process;FIG. 103
illustrates the compositing process;FIG. 104
illustrates the process of compositing with feedback;FIG. 105
illustrates the process of tiling with color from the input image;FIG. 106
illustrates the process of tiling with feedback;FIG. 107
illustrates the process of tiling with texture replacement;FIG. 108
illustrates the process of tiling with color from the input image;FIG. 109
illustrates the process of applying a texture without feedback;FIG. 110
illustrates the process of applying a texture with feedback;FIG. 111
illustrates the process of rotation of CCD pixels;FIG. 112
illustrates the process of interpolation of Green subpixels;FIG. 113
illustrates the process of interpolation of Blue subpixels;FIG. 114
illustrates the process of interpolation of Red subpixels;FIG. 115
illustrates the process of CCD pixel interpolation with 0 degree rotation for odd pixel lines;FIG. 116
illustrates the process of CCD pixel interpolation with 0 degree rotation for even pixel lines;FIG. 117
illustrates the process of color conversion to Lab color space;FIG. 118
illustrates the process of calculation of 1/√X;FIG. 119
illustrates the implementation of the calculation of 1/√X in more detail;FIG. 120
illustrates the process of Normal calculation with a bump map;FIG. 121
illustrates the process of illumination calculation with a bump map;FIG. 122
illustrates the process of illumination calculation with a bump map in more detail;FIG. 123
illustrates the process of calculation of L using a directional light;FIG. 124
illustrates the process of calculation of L using a Omni lights and spotlights;FIG. 125
illustrates one form of implementation of calculation of L using a Omni lights and spotlights;FIG. 126
illustrates the process of calculating the N.L dot product;FIG. 127
illustrates the process of calculating the N.L dot product in more detail;FIG. 128
illustrates the process of calculating the R.V dot product;FIG. 129
illustrates the process of calculating the R.V dot product in more detail;FIG. 130
illustrates the attenuation calculation inputs and outputs;FIG. 131
illustrates an actual implementation of attenuation calculation;FIG. 132
illustrates an graph of the cone factor;FIG. 133
illustrates the process of penumbra calculation;FIG. 134
illustrates the angles utilised in penumbra calculation;FIG. 135
illustrates the inputs and outputs to penumbra calculation;FIG. 136
illustrates an actual implementation of penumbra calculation;FIG. 137
illustrates the inputs and outputs to ambient calculation;FIG. 138
illustrates an actual implementation of ambient calculation;FIG. 139
illustrates an actual implementation of diffuse calculation;FIG. 140
illustrates the inputs and outputs to a diffuse calculation;FIG. 141
illustrates an actual implementation of a diffuse calculation;FIG. 142
illustrates the inputs and outputs to a specular calculation;FIG. 143
illustrates an actual implementation of a specular calculation;FIG. 144
illustrates the inputs and outputs to a specular calculation;FIG. 145
illustrates an actual implementation of a specular calculation;FIG. 146
illustrates an actual implementation of a ambient only calculation;FIG. 147
illustrates the process overview of light calculation;FIG. 148
illustrates an example illumination calculation for a single infinite light source;FIG. 149
illustrates an example illumination calculation for a Omni light source without a bump map;FIG. 150
illustrates an example illumination calculation for a Omni light source with a bump map;FIG. 151
illustrates an example illumination calculation for a Spotlight light source without a bump map;FIG. 152
illustrates the process of applying a single Spotlight onto an image with an associated bump-map;FIG. 153
illustrates the logical layout of a single printhead;FIG. 154
illustrates the structure of the printhead interface;FIG. 155
illustrates the process of rotation of a Lab image;FIG. 156
illustrates the format of a pixel of the printed image;FIG. 157
illustrates the dithering process;FIG. 158
illustrates the process of generating an 8 bit dot output;FIG. 159
illustrates a perspective view of the card reader;FIG. 160
illustrates an exploded perspective of a card reader;FIG. 161
illustrates a close up view of the Artcard reader;FIG. 162
illustrates a perspective view of the print roll and print head;FIG. 163
illustrates a first exploded perspective view of the print roll;FIG. 164
illustrates a second exploded perspective view of the print roll;FIG. 165
illustrates the print roll authentication chip;FIG. 166
illustrates an enlarged view of the print roll authentication chip;FIG. 167
illustrates a single authentication chip data protocol;FIG. 168
illustrates a dual authentication chip data protocol;FIG. 169
illustrates a first presence only protocol;FIG. 170
illustrates a second presence only protocol;FIG. 171
illustrates a third data protocol;FIG. 172
illustrates a fourth data protocol;FIG. 173
is a schematic block diagram of a maximal period LFSR;FIG. 174
is a schematic block diagram of a clock limiting filter;FIG. 175
is a schematic block diagram of the tamper detection lines;FIG. 176
illustrates an oversized nMOS transistor;FIG. 177
illustrates the taking of multiple XORs from the Tamper Detect LineFIG. 178
illustrate how the Tamper Lines cover the noise generator circuitry;FIG. 179
illustrates the normal form of FET implementation;FIG. 180
illustrates the modified form of FET implementation of the preferred embodiment;FIG. 181
illustrates a schematic block diagram of the authentication chip;FIG. 182
illustrates an example memory map;FIG. 183
illustrates an example of the constants memory map;FIG. 184
illustrates an example of the RAM memory map;FIG. 185
illustrates an example of the Flash memory variables memory map;FIG. 186
illustrates an example of the Flash memory program memory map;FIG. 187
shows the data flow and relationship between components of the State Machine;FIG. 188
shows the data flow and relationship between components of the I/O Unit.FIG. 189
illustrates a schematic block diagram of the Arithmetic Logic Unit;FIG. 190
illustrates a schematic block diagram of the RPL unit;FIG. 191
illustrates a schematic block diagram of the ROR block of the ALU;FIG. 192
is a block diagram of the Program Counter Unit;FIG. 193
is a block diagram of the Memory Unit;FIG. 194
shows a schematic block diagram for the Address Generator Unit;FIG. 195
shows a schematic block diagram for the JSIGEN Unit;FIG. 196
shows a schematic block diagram for the JSRGEN Unit.FIG. 197
shows a schematic block diagram for the DBRGEN Unit;FIG. 198
shows a schematic block diagram for the LDKGEN Unit;FIG. 199
shows a schematic block diagram for the RPLGEN Unit;FIG. 200
shows a schematic block diagram for the VARGEN Unit.FIG. 201
shows a schematic block diagram for the CLRGEN Unit.FIG. 202
shows a schematic block diagram for the BITGEN Unit.FIG. 203
sets out the information stored on the print roll authentication chip;FIG. 204
illustrates the data stored within the Artcam authorization chip;FIG. 205
illustrates the process of print head pulse characterization;FIG. 206
is an exploded perspective, in section, of the print head ink supply mechanism;FIG. 207
is a bottom perspective of the ink head supply unit;FIG. 208
is a bottom side sectional view of the ink head supply unit;FIG. 209
is a top perspective of the ink head supply unit;FIG. 210
is a top side sectional view of the ink head supply unit;FIG. 211
illustrates a perspective view of a small portion of the print head;FIG. 212
illustrates is an exploded perspective of the print head unit;FIG. 213
illustrates a top side perspective view of the internal portions of an Artcam camera, showing the parts flattened out;FIG. 214
illustrates a bottom side perspective view of the internal portions of an Artcam camera, showing the parts flattened out;FIG. 215
illustrates a first top side perspective view of the internal portions of an Artcam camera, showing the parts as encased in an Artcam;FIG. 216
illustrates a second top side perspective view of the internal portions of an Artcam camera, showing the parts as encased in an Artcam;FIG. 217
illustrates a second top side perspective view of the internal portions of an Artcam camera, showing the parts as encased in an Artcam;FIG. 218
illustrates the backing portion of a postcard print roll;FIG. 219
illustrates the corresponding front image on the postcard print roll after printing out images;FIG. 220
illustrates a form of print roll ready for purchase by a consumer;FIG. 221
illustrates a layout of the software/hardware modules of the overall Artcam application;FIG. 222
illustrates a layout of the software/hardware modules of the Camera Manager;FIG. 223
illustrates a layout of the software/hardware modules of the Image Processing Manager;FIG. 224
illustrates a layout of the software/hardware modules of the Printer Manager;FIG. 225
illustrates a layout of the software/hardware modules of the Image Processing Manager;FIG. 226
illustrates a layout of the software/hardware modules of the File Manager;FIG. 227
illustrates a perspective view, partly in section, of an alternative form of printroll;FIG. 228
is a left side exploded perspective view of the print roll ofFIG. 227
;FIG. 229
is a right side exploded perspective view of a single printroll;FIG. 230
is an exploded perspective view, partly in section, of the core portion of the printroll; andFIG. 231
is a second exploded perspective view of the core portion of the printroll.DESCRIPTION OF PREFERRED AND OTHER EMBODIMENTS
The digital image processing camera system constructed in accordance with the preferred embodiment is as illustrated in
FIG. 1
. The camera unit 1 includes means for the insertion of an integral print roll (not shown). The camera unit 1 can include an area image sensor 2 which sensors an image 3 for captured by the camera. Optionally, the second area image sensor can be provided to also image the scene 3 and to optionally provide for the production of stereographic output effects.The camera 1 can include an optional color display 5 for the display of the image being sensed by the sensor 2. When a simple image is being displayed on the display 5, the button 6 can be depressed resulting in the printed image 8 being output by the camera unit 1. A series of cards, herein after known as “Artcards†9 contain, on one surface encoded information and on the other surface, contain an image distorted by the particular effect produced by the Artcard 9. The Artcard 9 is inserted in an Artcard reader 10 in the side of camera 1 and, upon insertion, results in output image 8 being distorted in the same manner as the distortion appearing on the surface of Artcard 9. Hence, by means of this simple user interface a user wishing to produce a particular effect can insert one of many Artcards 9 into the Artcard reader 10 and utilize button 19 to take a picture of the image 3 resulting in a corresponding distorted output image 8.
The camera unit 1 can also include a number of other control button 13, 14 in addition to a simple LCD output display 15 for the display of informative information including the number of printouts left on the internal print roll on the camera unit. Additionally, different output formats can be controlled by CHP switch 17.
Turning now to
FIG. 2
, there is illustrated a schematic view of the internal hardware of the camera unit 1. The internal hardware is based around an Artcam central processor unit (ACP) 31.Artcam Central Processor 31
The Artcam central processor 31 provides many functions which form the ‘heart’ of the system. The ACP 31 is preferably implemented as a complex, high speed, CMOS system on-a-chip. Utilising standard cell design with some full custom regions is recommended. Fabrication on a 0.25μ CMOS process will provide the density and speed required, along with a reasonably small die area.
The functions provided by the ACP 31 include:
1. Control and digitization of the area image sensor 2. A 3D stereoscopic version of the ACP requires two area image sensor interfaces with a second optional image sensor 4 being provided for stereoscopic effects.
2. Area image sensor compensation, reformatting, and image enhancement.
3. Memory interface and management to a memory store 33.
4. Interface, control, and analog to digital conversion of an Artcard reader linear image sensor 34 which is provided for the reading of data from the Artcards 9.
5. Extraction of the raw Artcard data from the digitized and encoded Artcard image.
6. Reed-Solomon error detection and correction of the Artcard encoded data. The encoded surface of the Artcard 9 includes information on how to process an image to produce the effects displayed on the image distorted surface of the Artcard 9. This information is in the form of a script, hereinafter known as a “Vark script†. The Vark script is utilised by an interpreter running within the ACP 31 to produce the desired effect.
7. Interpretation of the Vark script on the Artcard 9.
8. Performing image processing operations as specified by the Vark script.
9. Controlling various motors for the paper transport 36, zoom lens 38, autofocus 39 and Artcard driver 37.
10. Controlling a guillotine actuator 40 for the operation of a guillotine 41 for the cutting of photographs 8 from print roll 42.
11. Half-toning of the image data for printing.
12. Providing the print data to a print-head 44 at the appropriate times.
13. Controlling the print head 44.
14. Controlling the ink pressure feed to print-head 44.
15. Controlling optional flash unit 56.
16. Reading and acting on various sensors in the camera, including camera orientation sensor 46, autofocus 47 and Artcard insertion sensor 49.
17. Reading and acting on the user interface buttons 6, 13, 14.
18. Controlling the status display 15.
19. Providing viewfinder and preview images to the color display 5.
20. Control of the system power consumption, including the ACP power consumption via power management circuit 51.
21. Providing external communications 52 to general purpose computers (using part USB).
22. Reading and storing information in a printing roll authentication chip 53.
23. Reading and storing information in a camera authentication chip 54.
24. Communicating with an optional mini-keyboard 57 for text modification.
Quartz Crystal 58
A quartz crystal 58 is used as a frequency reference for the system clock. As the system clock is very high, the ACP 31 includes a phase locked loop clock circuit to increase the frequency derived from the crystal 58.
Image Sensing
Area Image Sensor 2
The area image sensor 2 converts an image through its lens into an electrical signal. It can either be a charge coupled device (CCD) or an active pixel sensor (APS)CMOS image sector. At present, available CCD's normally have a higher image quality, however, there is currently much development occurring in CMOS imagers. CMOS imagers are eventually expected to be substantially cheaper than CCD's have smaller pixel areas, and be able to incorporate drive circuitry and signal processing. They can also be made in CMOS fabs, which are transitioning to 12″ wafers. CCD's are usually built in 6″ wafer fabs, and economics may not allow a conversion to 12″ fabs. Therefore, the difference in fabrication cost between CCD's and CMOS imagers is likely to increase, progressively favoring CMOS imagers. However, at present, a CCD is probably the best option.
The Artcam unit will produce suitable results with a 1,500×1,000 area image sensor. However, smaller sensors, such as 750×500, will be adequate for many markets. The Artcam is less sensitive to image sensor resolution than are conventional digital cameras. This is because many of the styles contained on Artcards 9 process the image in such a way as to obscure the lack of resolution. For example, if the image is distorted to simulate the effect of being converted to an impressionistic painting, low source image resolution can be used with minimal effect. Further examples for which low resolution input images will typically not be noticed include image warps which produce high distorted images, multiple miniature copies of the of the image (eg. passport photos), textural processing such as bump mapping for a base relief metal look, and photo-compositing into structured scenes.
This tolerance of low resolution image sensors may be a significant factor in reducing the manufacturing cost of an Artcam unit 1 camera. An Artcam with a low cost 750×500 image sensor will often produce superior results to a conventional digital camera with a much more expensive 1,500×1,000 image sensor.
Optional Stereoscopic 3D Image Sensor 4
The 3D versions of the Artcam unit 1 have an additional image sensor 4, for stereoscopic operation. This image sensor is identical to the main image sensor. The circuitry to drive the optional image sensor may be included as a standard part of the ACP chip 31 to reduce incremental design cost. Alternatively, a separate 3D Artcam ACP can be designed. This option will reduce the manufacturing cost of a mainstream single sensor Artcam.
Print Roll Authentication Chip 53
A small chip 53 is included in each print roll 42. This chip replaced the functions of the bar code, optical sensor and wheel, and ISO/ASA sensor on other forms of camera film units such as Advanced Photo Systems film cartridges.
The authentication chip also provides other features:
1. The storage of data rather than that which is mechanically and optically sensed from APS rolls
2. A remaining media length indication, accurate to high resolution.
3. Authentication Information to prevent inferior clone print roll copies.
The authentication chip 53 contains 1024 bits of Flash memory, of which 128 bits is an authentication key, and 512 bits is the authentication information. Also included is an encryption circuit to ensure that the authentication key cannot be accessed directly.
Print-head 44
The Artcam unit 1 can utilize any color print technology which is small enough, low enough power, fast enough, high enough quality, and low enough cost, and is compatible with the print roll. Relevant printheads will be specifically discussed hereinafter.
The specifications of the ink jet head are:
Optional Ink Pressure Controller (Not Shown)
The function of the ink pressure controller depends upon the type of ink jet print head 44 incorporated in the Artcam. For some types of ink jet, the use of an ink pressure controller can be eliminated, as the ink pressure is simply atmospheric pressure. Other types of print head require a regulated positive ink pressure. In this case, the in pressure controller consists of a pump and pressure transducer.
Other print heads may require an ultrasonic transducer to cause regular oscillations in the ink pressure, typically at frequencies around 100 KHz. In the case, the ACP 31 controls the frequency phase and amplitude of these oscillations.
Paper Transport Motor 36
The paper transport motor 36 moves the paper from within the print roll 42 past the print head at a relatively constant rate. The motor 36 is a miniature motor geared down to an appropriate speed to drive rollers which move the paper. A high quality motor and mechanical gears are required to achieve high image quality, as mechanical rumble or other vibrations will affect the printed dot row spacing.
Paper Transport Motor Driver 60
The motor driver 60 is a small circuit which amplifies the digital motor control signals from the APC 31 to levels suitable for driving the motor 36.
Paper Pull Sensor
A paper pull sensor 50 detects a user's attempt to pull a photo from the camera unit during the printing process. The APC 31 reads this sensor 50, and activates the guillotine 41 if the condition occurs. The paper pull sensor 50 is incorporated to make the camera more ‘foolproof’ in operation. Were the user to pull the paper out forcefully during printing, the print mechanism 44 or print roll 42 may (in extreme cases) be damaged. Since it is acceptable to pull out the ‘pod’ from a Polaroid type camera before it is fully ejected, the public has been ‘trained’ to do this. Therefore, they are unlikely to heed printed instructions not to pull the paper.
The Artcam preferably restarts the photo print process after the guillotine 41 has cut the paper after pull sensing.
The pull sensor can be implemented as a strain gauge sensor, or as an optical sensor detecting a small plastic flag which is deflected by the torque that occurs on the paper drive rollers when the paper is pulled. The latter implementation is recommendation for low cost.
Paper Guillotine Actuator 40
The paper guillotine actuator 40 is a small actuator which causes the guillotine 41 to cut the paper either at the end of a photograph, or when the paper pull sensor 50 is activated.
The guillotine actuator 40 is a small circuit which amplifies a guillotine control signal from the APC tot the level required by the actuator 41.
Artcard 9
The Artcard 9 is a program storage medium for the Artcam unit. As noted previously, the programs are in the form of Vark scripts. Vark is a powerful image processing language especially developed for the Artcam unit. Each Artcard 9 contains one Vark script, and thereby defines one image processing style.
Preferably, the VARK language is highly image processing specific. By being highly image processing specific, the amount of storage required to store the details on the card are substantially reduced. Further, the ease with which new programs can be created, including enhanced effects, is also substantially increased. Preferably, the language includes facilities for handling many image processing functions including image warping via a warp map, convolution, color lookup tables, posterizing an image, adding noise to an image, image enhancement filters, painting algorithms, brush jittering and manipulation edge detection filters, tiling, illumination via light sources, bump maps, text, face detection and object detection attributes, fonts, including three dimensional fonts, and arbitrary complexity pre-rendered icons. Further details of the operation of the Vark language interpreter are contained hereinafter.
Hence, by utilizing the language constructs as defined by the created language, new affects on arbitrary images can be created and constructed for inexpensive storage on Artcard and subsequent distribution to camera owners. Further, on one surface of the card can be provided an example illustrating the effect that a particular VARK script, stored on the other surface of the card, will have on an arbitrary captured image.
By utilizing such a system, camera technology can be distributed without a great fear of obsolescence in that, provided a VARK interpreter is incorporated in the camera device, a device independent scenario is provided whereby the underlying technology can be completely varied over time. Further, the VARK scripts can be updated as new filters are created and distributed in an inexpensive manner, such as via simple cards for card reading.
The Artcard 9 is a piece of thin white plastic with the same format as a credit card (86 mm long by 54 mm wide). The Artcard is printed on both sides using a high resolution ink jet printer. The inkjet printer technology is assumed to be the same as that used in the Artcam, with 1600 dpi (63 dpmm) resolution. A major feature of the Artcard 9 is low manufacturing cost. Artcards can be manufactured at high speeds as a wide web of plastic film. The plastic web is coated on both sides with a hydrophilic dye fixing layer. The web is printed simultaneously on both sides using a ‘pagewidth’ color ink jet printer. The web is then cut and punched into individual cards. On one face of the card is printed a human readable representation of the effect the Artcard 9 will have on the sensed image. This can be simply a standard image which has been processed using the Vark script stored on the back face of the card.
On the back face of the card is printed an array of dots which can be decoded into the Vark script that defines the image processing sequence. The print area is 80 mm×50 mm, giving a total of 15,876,000 dots. This array of dots could represent at least 1.89 Mbytes of data. To achieve high reliability, extensive error detection and correction is incorporated in the array of dots. This allows a substantial portion of the card to be defaced, worn, creased, or dirty with no effect on data integrity. The data coding used is Reed-Solomon coding, with half of the data devoted to error correction. This allows the storage of 967 Kbytes of error corrected data on each Artcard 9.
Linear Image Sensor 34
The Artcard linear sensor 34 converts the aforementioned Artcard data image to electrical signals. As with the area image sensor 2, 4, the linear image sensor can be fabricated using either CCD or APS CMOS technology. The active length of the image sensor 34 is 50 mm, equal to the width of the data array on the Artcard 9. To satisfy Nyquist's sampling theorem, the resolution of the linear image sensor 34 must be at least twice the highest spatial frequency of the Artcard optical image reaching the image sensor. In practice, data detection is easier if the image sensor resolution is substantially above this. A resolution of 4800 dpi (189 dpmm) is chosen, giving a total of 9,450 pixels. This resolution requires a pixel sensor pitch of 5.3 μm. This can readily be achieved by using four staggered rows of 20 μm pixel sensors.
The linear image sensor is mounted in a special package which includes a LED 65 to illuminate the Artcard 9 via a light-pipe (not shown).
The Artcard reader light-pipe can be a molded light-pipe which has several function:
1. It diffuses the light from the LED over the width of the card using total internal reflection facets.
2. It focuses the light onto a 16 μm wide strip of the Artcard 9 using an integrated cylindrical lens.
3. It focuses light reflected from the Artcard onto the linear image sensor pixels using a molded array of microlenses.
The operation of the Artcard reader is explained further hereinafter.
Artcard Reader Motor 37
The Artcard reader motor propels the Artcard past the linear image sensor 34 at a relatively constant rate. As it may not be cost effective to include extreme precision mechanical components in the Artcard reader, the motor 37 is a standard miniature motor geared down to an appropriate speed to drive a pair of rollers which move the Artcard 9. The speed variations, rumble, and other vibrations will affect the raw image data as circuitry within the APC 31 includes extensive compensation for these effects to reliably read the Artcard data.
The motor 37 is driven in reverse when the Artcard is to be ejected.
Artcard Motor Driver 61
The Artcard motor driver 61 is a small circuit which amplifies the digital motor control signals from the APC 31 to levels suitable for driving the motor 37.
Card Insertion Sensor 49
The card insertion sensor 49 is an optical sensor which detects the presence of a card as it is being inserted in the card reader 34. Upon a signal from this sensor 49, the APC 31 initiates the card reading process, including the activation of the Artcard reader motor 37.
Card Eject Button 16
A card eject button 16 (
FIG. 1
) is used by the user to eject the current Artcard, so that another Artcard can be inserted. The APC 31 detects the pressing of the button, and reverses the Artcard reader motor 37 to eject the card.Card Status Indicator 66
A card status indicator 66 is provided to signal the user as to the status of the Artcard reading process. This can be a standard bi-color (red/green) LED. When the card is successfully read, and data integrity has been verified, the LED lights up green continually. If the card is faulty, then the LED lights up red.
If the camera is powered from a 1.5 V instead of 3V battery, then the power supply voltage is less than the forward voltage drop of the greed LED, and the LED will not light. In this case, red LEDs can be used, or the LED can be powered from a voltage pump which also powers other circuits in the Artcam which require higher voltage.
64 Mbit DRAM 33
To perform the wide variety of image processing effects, the camera utilizes 8 Mbytes of memory 33. This can be provided by a single 64 Mbit memory chip. Of course, with changing memory technology increased Dram storage sizes may be substituted.
High speed access to the memory chip is required. This can be achieved by using a Rambus DRAM (burst access rate of 500 Mbytes per second) or chips using the new open standards such as double data rate (DDR) SDRAM or Synclink DRAM.
Camera Authentication Chip
The camera authentication chip 54 is identical to the print roll authentication chip 53, except that it has different information stored in it. The camera authentication chip 54 has three main purposes:
1. To provide a secure means of comparing authentication codes with the print roll authentication chip;
2. To provide storage for manufacturing information, such as the serial number of the camera;
3. To provide a small amount of non-volatile memory for storage of user information.
Displays
The Artcam includes an optional color display 5 and small status display 15. Lowest cost consumer cameras may include a color image display, such as a small TFT LCD 5 similar to those found on some digital cameras and camcorders. The color display 5 is a major cost element of these versions of Artcam, and the display 5 plus back light are a major power consumption drain.
Status Display 15
The status display 15 is a small passive segment based LCD, similar to those currently provided on silver halide and digital cameras. Its main function is to show the number of prints remaining in the print roll 42 and icons for various standard camera features, such as flash and battery status.
Color Display 5
The color display 5 is a full motion image display which operates as a viewfinder, as a verification of the image to be printed, and as a user interface display. The cost of the display 5 is approximately proportional to its area, so large displays (say 4″ diagonal) unit will be restricted to expensive versions of the Artcam unit. Smaller displays, such as color camcorder viewfinder TFT's at around 1″, may be effective for mid-range Artcams.
Zoom Lens (Not Shown)
The Artcam can include a zoom lens. This can be a standard electronically controlled zoom lens, identical to one which would be used on a standard electronic camera, and similar to pocket camera zoom lenses. A referred version of the Artcam unit may include standard interchangeable 35 mm SLR lenses.
Autofocus Motor 39
The autofocus motor 39 changes the focus of the zoom lens. The motor is a miniature motor geared down to an appropriate speed to drive the autofocus mechanism.
Autofocus Motor Driver 63
The autofocus motor driver 63 is a small circuit which amplifies the digital motor control signals from the APC 31 to levels suitable for driving the motor 39.
Zoom Motor 38
The zoom motor 38 moves the zoom front lenses in and out. The motor is a miniature motor geared down to an appropriate speed to drive the zoom mechanism.
Zoom Motor Driver 62
The zoom motor driver 62 is a small circuit which amplifies the digital motor control signals from the APC 31 to levels suitable for driving the motor.
Communications
The ACP 31 contains a universal serial bus (USB) interface 52 for communication with personal computers. Not all Artcam models are intended to include the USB connector. However, the silicon area required for a USB circuit 52 is small, so the interface can be included in the standard ACP.
Optional Keyboard 57
The Artcam unit may include an optional miniature keyboard 57 for customizing text specified by the Artcard. Any text appearing in an Artcard image may be editable, even if it is in a complex metallic 3D font. The miniature keyboard includes a single line alphanumeric LCD to display the original text and edited text. The keyboard may be a standard accessory.
The ACP 31 contains a serial communications circuit for transferring data to and from the miniature keyboard.
Power Supply
The Artcam unit uses a battery 48. Depending upon the Artcam options, this is either a 3V Lithium cell, 1.5 V AA alkaline cells, or other battery arrangement.
Power Management Unit 51
Power consumption is an important design constraint in the Artcam. It is desirable that either standard camera batteries (such as 3V lithium batters) or standard AA or AAA alkaline cells can be used. While the electronic complexity of the Artcam unit is dramatically higher than 35 mm photographic cameras, the power consumption need not be commensurately higher. Power in the Artcam can be carefully managed with all unit being turned off when not in use.
The most significant current drains are the ACP 31, the area image sensors 2,4, the printer 44 various motors, the flash unit 56, and the optional color display 5 dealing with each part separately:
1. ACP: If fabricated using 0.25 μm CMOS, and running on 1.5V, the ACP power consumption can be quite low. Clocks to various parts of the ACP chip can be quite low. Clocks to various parts of the ACP chip can be turned off when not in use, virtually eliminating standby current consumption. The ACP will only fully used for approximately 4 seconds for each photograph printed.
