correct an old bug link
[imager.git] / design / represent.txt
CommitLineData
e4d18836
TC
1=head1 NAME
2
f6055a49 3 represent.txt - discuss image representation within Imager
e4d18836
TC
4
5=head1 SYNOPSIS
6
7 Virtual Images
8 Image Subsetting
9 Varying Bits/Sample
10 Paletted Images
11 Performance
faa9b3e7
TC
12 Robustness
13 XS Changes
e4d18836
TC
14
15=head1 DESCRIPTION
16
f6055a49
TC
17I'm going to try to explain what we can get from having a flexible
18representation of images within Imager.
e4d18836 19
f6055a49
TC
20The main idea is to have all, or almost all of Imager access the
21contents of an image through function pointers embedded in the i_img
22structure. This means that the underlying image data can be formatted
23to suit special purposes, including paletted images, images kept of
24disk (think of 64k x 64k RGB images), and virtual images (where
25there's no actual image data.)
26
b1e9fee6
TC
27=head1 IMAGE TYPES
28
f6055a49 29=head2 Paletted Images
e4d18836 30
f6055a49
TC
31This is the form we've discussed the most. The main advantage here is
32when the user is performing simple manipulations on the image data.
33One example that came up recently was when mjd was trying to combine
34several images into a single animated GIF file. If we were
35representing the image internally as paletted, and the GIF reader and
36writer knew how to do that we could have loaded the images into
37paletted images and then written them with no quantization required.
38
39Now we could get complicated and have writes with colours that don't
40exist in the image cause an extra entry be added to the palette, but
41this leads to complications as to when to convert the image to RGB.
42Some paletted formats support large palettes, so if we allowed a few
43hundred new colours to be drawn into the image and the tried to save
44to a format with only small palettes, not only has the user paid the
45performance cost in writing to the image (since paletted writes
46include a palette lookup), but also the hit in having to re-quantize
47the image anyway.
48
49So my idea was to have the paletted write functions be something like:
50
51 if (found entry in palette) {
52 save to the pixel
53 } else {
54 convert image to rgb
55 call rgb save function
56 }
57
58An initial implementation might only support 256 colour palettes, we
59might expand that later to support larger palettes.
60
61We could expose the quant.c functions to allow the user to create
62palettes (and possibly tune them), and to convert from RGB images to
63paletted images.
64
65For improved memory usage for large images we could also implement 4
66and 1 bit/pixel images. If we want good support for faxing this could
67be useful.
e4d18836 68
f6055a49 69=head2 Virtual Images
e4d18836 70
f6055a49
TC
71Another possible type of image is a B<virtual image> where the i_img
72that the user has, has no image data directly associated with it. The
73results of retreiving pixels would simply be the result of some
74function. Such virtualness could even be in terms of "virtual
75memory", so a 32-bit processor machine could work with 65536x65536x24
76bit images which doesn't even fit into the address-space of the 32-bit
77machine.
78
79One possible implementation of function based images would be through
80the use of the transform2() engine. This way the user could specify
81the function to be used to generate the virtual image. This is most
82useful for very large images, otherwise simply generating a new image
83using the transform2() function would be faster.
84
85=head3 Image Subsetting
86
87This would be mainly for when writing to an image, the user could
88supply another image where which pixels were non-black allowed writes
89to the corresponding pixels in another image. Since access is
90controlled via another image, we aren't limited to rectangular
91subsets.
92
93One simple example might be to create a mask image that has some text
94drawn in a large font. When a gradient or other effect is used it
95will fill the letters in the target image. A more subtle effect could
96be lightening, darkening or blurring through the image.
97
98One implementation consideration is that if calculating the pixel
99value is expensive the caller may want a method to check if given
100pixels are writable, to avoid that extra expense.
e4d18836
TC
101
102=head2 Varying Bits/Sample
103
b1e9fee6
TC
104The typical Imager image could stay at 8 bits/sample, but some
105applications may need to work with images with a higher or lower
106colour resolution. It may also be desirable to work with images where
107the sample values are floating point, eg. some FITS images.
108
109The problem with supporting floating point or a larger bit count is
110that we need an interface to get and put pixels/rows at the largest
111resolution for the high colour resolution. Since working at this
112colour resolution for lower colour resolution images would be
113inefficient, we also want another interface at some lower bit-count.
114To reduce maintenance costs we want to limit the number of interfaces.
115
116=head1 INTERFACE CONSIDERATIONS
117
118Common interfaces are those interfaces which available for every image
119type. These should include some common bits/sample, and some
120all-encompassing bits/sample, eg. 8 bits/sample for the common, and
121floating point for the all-encompassing.
122
123The idea is to make it possible for a given filtering function to have
124only one implementation at the all-encompassing bits/sample, while
125some other function has implementations at multiple bits/sample for
126improved speed.
127
128To reduce maintenance we want to reduce the number of common
129interfaces, both so that we don't have to write too much code for each
130image representation and so that we don't have someone trying to write
131five versions of the same code in an attempt to push efficiency to the
132limits.
f6055a49
TC
133
134Since we want some function pointers to only be available for some
135images (like 'getpixindex' against paletted images), callers need to
136be able to detect the type of image that they are working with.
e4d18836
TC
137
138=head2 Performance
139
140Why is performance last? Well, it needs to be considered in terms of
141the other topics above.
142
f6055a49
TC
143Perhaps some method to check the writability at given pixels could be
144used to avoid expensive pixel calculations.
145
b1e9fee6
TC
146We need to strike a balance between speed and maintainability.
147
148While most x86 processors have floating point performance almost as
149fast or faster than integer performance, some architectures don't,
150hence we need to support use of integer algorithms where appropriate.
e4d18836 151
b1e9fee6 152=head2 Robustness
e4d18836 153
b1e9fee6
TC
154By preferring access via the functions we can reduce the possibility
155of incorrect access to the image data.
156
2df3535a
AMH
157If an image interface function pointer isn't implemented, it should be
158set to NULL rather than being left uninitialized. This is so that use
159of the ununsed pointer results in an immediate crash rather than
160possibly calling the wrong function and crashing at some later point.
b1e9fee6
TC
161
162In a debug build of Imager the image interface functions should check
163that the correct type of image pointer is being passed in. This means
164that a caller cannot accidentally pass the wrong type of image pointer
165to an image access function.
166
167=head1 PROPOSED INTERFACE
168
169The basic interface would include:
170
171 typedef struct {
172 struct { unsigned char r,g,b,a; } rgb;
173 unsigned char channels[MAX_CHANNELS];
174 /* and others as we currently have */
175 } i_color;
b921d414 176
b1e9fee6
TC
177 typedef struct {
178 struct { double char r, g, b, a; } rgb;
179 double channels[MAX_CHANNELS];
180 /* and others as we currently have */
181 } i_fcolor;
b921d414 182
b1e9fee6
TC
183 typedef struct i_img_tag i_img;
184 typedef int (*i_f_ppix_t)(i_img *im, int x, int y, i_color *pix);
185 typedef int (*i_f_ppixf_t)(i_img *im, int x, int y, i_fcolor *pix);
186 typedef int (*i_f_plin_t)(i_img *im, int x, int r, int y, i_color *vals);
187 typedef int (*i_f_plinf_t)(i_img *im, int x, int r, int y, i_fcolor *vals);
188 typedef int (*i_f_gpix_t)(i_img *im, int x, int y, i_color *pix);
189 typedef int (*i_f_gpixf_t)(i_img *im, int x, int y, i_fcolor *pix);
190 typedef int (*i_f_glin_t)(i_img *im, int x, int r, int y, i_color *vals);
191 typedef int (*i_f_glinf_t)(i_img *im, int x, int r, int y, i_fcolor *vals);
b921d414 192
b1e9fee6
TC
193 typedef enum {
194 i_literal_type, /* keeps RGB values per pixel */
195 i_palette_type, /* keeps a palette index per pixel */
196 } i_img_types;
b921d414 197
faa9b3e7 198 /* interface functions */
b1e9fee6
TC
199 typedef int (*i_f_gpal_t)(i_img *im, int x, int r, int y, i_palidx *vals);
200 typedef int (*i_f_ppal_t)(i_img *im, int x, int r, int y, i_palidx *vals);
201 typedef int (*i_f_addcolor_t)(i_img *im, i_color *);
202 typedef int (*i_f_getcolor_t)(i_img *im, int i, i_color *);
203 typedef int (*i_f_colorcount_t)(i_img *im);
204 typedef int (*i_f_findcolor_t)(i_img *im);
b921d414 205
b1e9fee6
TC
206 typedef enum {
207 /* bits per sample, not per pixel */
208 /* a paletted image might have one bit perl sample */
209 i_8_bits = 8,
210 i_16_bits = 16,
211 i_double_bits = 64
212 } i_img_bits;
b921d414
AMH
213
214 typedef struct {
215 char *msg;
216 int code;
217 } i_errmsg;
218
b1e9fee6
TC
219 typedef struct {
220 char *name; /* name of a given tag, might be NULL */
221 int code; /* number of a given tag, -1 if it has no meaning */
222 char *data; /* value of a given tag if it's not an int, may be NULL */
223 int idata; /* value of a given tag if data is NULL */
224 } i_img_tag;
b921d414 225
b1e9fee6
TC
226 typedef struct {
227 int count; /* how many tags have been set */
228 int alloc; /* how many tags have been allocated for */
229 i_img_tag *tags;
230 } i_img_tags;
b921d414 231
b1e9fee6
TC
232 typedef struct {
233 int channels;
234 int xsize, ysize, bytes;
235 int ch_mask;
236 i_img_bits bits;
237 i_img_type type;
238 int virtual; /* image might not keep any data, must use functions */
239 void *idata; /* renamed to force inspection of existing code */
240 /* can be NULL if virtual is non-zero */
241 i_img_tags tags;
242
b921d414
AMH
243 i_errmsg error_stack[ERRSTK]; /* Store errors with image */
244
b1e9fee6
TC
245 /* interface functions */
246 i_f_ppix_t i_f_ppix;
247 i_f_ppixf_t i_f_ppixf;
248 i_f_plin_t i_f_plin;
249 i_f_plinf_t i_f_plinf;
250 i_f_gpix_t i_f_gpix;
251 i_f_gpixf_t i_f_gpixf;
252 i_f_glin_t i_f_glin;
253 i_f_glinf_t i_f_glinf;
254
255 /* only valid for type == i_palette_type */
256 i_f_gpal_t i_f_gpal;
257 i_f_ppal_t i_f_ppal;
258 i_f_addcolor_t i_f_addcolor;
259 i_f_getcolor_t i_f_getcolor;
260 i_f_colorcount_t i_f_colorcount;
261 i_f_findcolor_t i_f_findcolor;
262 } i_img;
263
264I'm using 8-bits for the base interface to remain compatible with
265existing code, and because it's a common sample size.
266
267I'm using double for the all-encompassing size since it's the biggest
268convenient type that supports floating point based-pixels.
269
270We might want to add functions to set/retrieve the whole palette at
271once, though setting the whole palette at once would make existing
272image data fairly useless.
e4d18836 273
faa9b3e7
TC
274=head1 XS CHANGES
275
276I had been considering moving the i_img object from being an
277Imager::ImgRef object to being an Imager object. But I don't see much
278point to it, so I'll leave it the way it is now.
279
e4d18836
TC
280=head1 AUTHOR
281
282Tony Cook <tony@develop-help.com>
283
b1e9fee6
TC
284=head1 HISTORY
285
28616May2001 - initially completed version, could use some polishing
b921d414 28716May2001 - Added i_error stack to the image structure.
faa9b3e7 28824May2001 - Added XS Changes section (TC)
b1e9fee6 289
e4d18836 290=cut