This post describes how to caliculate when flatten each pixel of XCF.
About flattening overview, you should read section 9 of xcf.txt by Henning Makholm.
You can find it in GIMP sourcecode's tar-ball.
I made an xcf viewer on iPad.
The flattening process in most modes are same as COMPOSITE functions in xcf.txt,
but are different in mode 5, 16 and 17. (Marked with "*").
It's why I wrote this.
http://ofo.jp/osakana/cgtips/blendmode.phtml (Japanese Page) was great help.
Variable letters
"a" is alpha value (premultiplied with general layer opacity) "r" is RGB's R "g" is RGB's G "b" is RGB's B "y" is grayscale intensity "i" is colormap indexed "x" is general value, means "a","r","g","b","y" or "i" in context. "1" as subscript means lower (closer to bottommost) layer "2" as subscript means higher layer "m" is blend mode
*
Auxiliary functions:
MIN(x1,...,xn) is the least value of x1...xn
MAX(x1,...,xn) is the largest value of x1..xn
MID(x1,...,xn) = (MIN(x1,...,xn)+MAX(x1,...,xn))/2
CLAMP(x) = if x < 0 then 0.0 else if x > 1 then 1.0 else x
BLEND(a1,x1, a2,x2) = (1-k)*x1 + k*x2
where k = a2/(1-(1-a1)*(1-a2))
*
Mode 0 and 1
Effective in RGB, grayscale and Indexed color model.
Mode 0 (Normal)
Grayscale: COMPOSITE(a1,y1, a2,y2,Normal)
= ( 1-(1-a1)*(1-a2), BLEND(a1,y1, a2,y2) )
RGB:
COMPOSITE(a1,r1,g1,b1, a2,r2,g2,b2,Normal)
= ( 1-(1-a1)*(1-a2), BLEND(a1,r1, a2,r2),
BLEND(a1,g1, a2,g2),
BLEND(a1,b1, a2,b2) )
Indexed color:
COMPOSITE(a1,i1, a2,i2,Normal) = if a2 > 0.5 then (1.0,i2) else (a1,i1)
Mode 1 (Dissolve)
COMPOSITE(a1,c1, a2,c2,Dissolve) = chose pseudo-randomly between
(1.0,c2) with probability a2
(a1,c1) with probability 1-a2
*
Mode 3~10 and 15~21
Effective in RGB and grayscale color model.
Common COMPOSITE function formula in mode 3~10 and 15~21.
Grayscale: COMPOSITE(a1,y1, a2,y2,m)
= ( a1, BLEND(a1,y1, MIN(a1,a2),f(y1,y2, m)) )
RGB:
COMPOSITE(a1,r1,g1,b1, a2,r2,g2,b2,m)
= ( a1, BLEND(a1,r1, MIN(a1,a2),f(r1,r2, m)),
BLEND(a1,g1, MIN(a1,a2),f(g1,g2, m)),
BLEND(a1,b1, MIN(a1,a2),f(b1,b2, m)) )
Function "f" in each mode.
Mode 3 (Multiply)f(x1,x2,3) = x1*x2Mode 4 (Screen)
f(x1,x2,4) = 1-(1-x1)*(1-x2)Mode 5 (Overlay) (*)
f(x1,x2,5) = if x1<0.5 then x1*x2*2 else 1 - 2*(1-x1)*(1-x2)Mode 6 (Difference)
f(x1,x2,6) = if x1 > x2 then x1-x2 else x2-x1Mode 7 (Addition)
f(x1,x2, 7) = CLAMP(x1+x2)Mode 8 (Subtract)
f(x1,x2, 8) = CLAMP(x1-x2)Mode 9 (Darken Only)
f(x1,x2, 9) = MIN(x1,x2)Mode 10 (Lighten Only)
f(x1,x2, 10) = MAX(x1,x2)Mode 15 (Divide)
f(x1,x2, 15) = CLAMP(x1/x2)Mode 16 (Dodge) (*)
f(x1,x2,16) = if x2=1 then 1 else CLAMP(x1/(1-x2))Mode 17 (Burn) (*)
f(x1,x2,17) = if x2=0 then 0 else CLAMP(1-(1-x1)/x2)Mode 18 (Hard Light)
f(x1,x2, 18) = if x2 < 0.5 then 2*x1*x2 else 1-2*(1-x1)(1-x2)Mode 19 (Soft Light)
f(x1,x2, 19) = (1-x2)*x1^2 + x2*(1-(1-x2)^2)Mode 20 (Grain Extract)
f(x1,x2, 20) = CLAMP(x1-x2+0.5)Mode 21 (Merge)
f(x1,x2, 21) = CLAMP(x1+x2-0.5)
*
Mode 11~14
Effective in RGB color model.
Common COMPOSITE function formula in mode 11~14.
COMPOSITE(a1,r1,g1,b1, a2,r2,g2,b2,m)
= ( a1, BLEND(a1,r1, MIN(a1,a2),r0),
BLEND(a1,g1, MIN(a1,a2),g0),
BLEND(a1,b1, MIN(a1,a2),b0) )
where (r0,g0,b0) = h(r1,g1,b1, r2,g2,b2, m)
Condtions determine (r0,g0,b0):
(r,g,b) has the _hue_ of (r',g',b')
if r' = g' = b' and r >= g = b
or there exist p and q such that p>=0 and r=p*r'+q and b=p*b'+q and g=p*g'+q
(r,g,b) has the _value_ of (r',g',b')
if MAX(r,g,b) = MAX(r',g',b')
(r,g,b) has the _HSV-saturation_ of (r',g',b')
if r' = g' = b' = 0 and r = g = b
or MIN(r,g,b) = MAX(r,g,b)*MIN(r',g',b')/MAX(r',g',b')
(r,g,b) has the _luminosity_ of (r',g',b')
if MID(r,g,b) = MID(r',g',b')
(r,g,b) has the _HSL-saturation_ of (r',g',b')
if r' = g' = b' and r = g = b
or MAX(r,g,b)-MIN(r,g,b) = MIN(MID(r,g,b),1-MID(r,g,b)) *
(MAX(r',g',b')-MIN(r',g',b'))/MIN(MID(r',g',b'),1-MID(r',g',b'))
I used auxiliary functions below:
INDEX_MIN(x1,x2,x3) returns 1 if MIN(x1,x2,x3)=x1 2 if MIN(x1,x2,x3)=x2 3 if MIN(x1,x2,x3)=x3 INDEX_MAX(x1,x2,x3) returns 1 if MAX(x1,x2,x3)=x1 2 if MXA(x1,x2,x3)=x2 3 if MAX(x1,x2,x3)=x3Mode 11: Hue (H of HSV)
h(r1,g1,b1, r2,g2,b2, 11) is
if r2=g2=b2 then (r1,g1,b1) unchanged
otherwise: the color that has
the hue of (r2,g2,b2)
the value of (r1,g1,b1)
the HSV-saturation of (r1,g1,b1)
actual calicuration is:
h(r1,g1,b1, r2,g2,b2, 11){
if r2=g2=b2 then return r1,g1,b1
else
RGB0[3] //An array contains r0,g0,b0; index begins with 1
max = INDEX_MAX(r2,g2,b2)
min = INDEX_MIN(r2,g2,b2)
rest = (remain of 1,2,3 after taken min and max)
RGB0[max] = MAX(r1,g1,b1)
RGB0[min] = RGB0[max]*MIN(r1,g1,b1)/MAX(r1,g1,b1)
p = (RGB0[max] - RGB0[min])/(MAX(r2,g2,b2)-MIN(r2,g2,b2))
q = RGB0[max] - p*MAX(r2,g2,b2)
RGB0[rest] = p*(middle value of r2,g2,b2) + q
return RGB0[1], RGB0[2], RGB0[3]
}
Mode 12: Saturation (S of HSV)
h(r1,g1,b1, r2,g2,b2, 12) is the color that has
the hue of (r1,g1,b1)
the value of (r1,g1,b1)
the HSV-saturation of (r2,g2,b2)
actual calicuration is:
h(r1,g1,b1, r2,g2,b2, 11){
if r2=g2=b2=0 then return MAX(r1,g1,b1),MAX(r1,g1,b1),MAX(r1,g1,b1)
else
RGB0[3] //An array contains r0,g0,b0; index begins with 1
max = INDEX_MAX(r1,g1,b1)
min = INDEX_MIN(r1,g1,b1)
rest = (remain of 1,2,3 after taken min and max)
RGB0[max] = MAX(r1,g1,b1)
RGB0[min] = RGB0[max]*Min(r2,g2,b2)/Max(r2,b2,g2)
p = (RGB0[max]-RGB0[min])/(MAX(r1,g1,b1)-MIN(r1,g1,b1))
q = RGB0[max] - p*MAX(r1,g1,b1)
RGB0[rest] = p*(middle value of r1,g1,b1) + q
return RGB0[1], RGB0[2], RGB0[3]
}
Mode 13: Color (H and S of HSL)
h(r1,g1,b1, r2,g2,b2, 13) is the color that has
the hue of (r2,g2,b2)
the luminosity of (r1,g1,b1)
the HSL-saturation of (r2,g2,b2)
actual calicuration is:
h(r1,g1,b1, r2,g2,b2, 11){
if r2=g2=b2 then return MID(r1,g1,b1),MID(r1,g1,b1),MID(r1,g1,b1)
else
RGB0[3] //An array contains r0,g0,b0; index begins with 1
max = INDEX_MAX(r2,g2,b2)
min = INDEX_MIN(r2,g2,b2)
rest = (remain of 1,2,3 after taken min and max)
mid1 = MID(r1,g1,b1)
mid2 = MID(r2,g2,b2)
diff = (mid1<1-mid1?mid1:1-mid1)*(MAX(r2,g2,b2)-MIN(r2,g2,b2)/(mid2<1-mid2?mid2:1-mid2)
RGB0[max] = mid1 + diff/2
RGB0[min] = mid1 - diff/2
p = (RGB0[max]-RGB0[min])/(MAX(r2,g2,b2)-MIN(r2,g2,b2))
q = RGB0[max] - p*MAX(r2,g2,b2)
RGB0[rest] = p*(middle value of r2,g2,b2) + q
return RGB0[1], RGB0[2], RGB0[3]
}
Mode 14: Value (V of HSV)
h(r1,g1,b1, r2,g2,b2, 14) is the color that has
the hue of (r1,g1,b1)
the value of (r2,g2,b2)
the HSV-saturation of (r1,g1,b1)
actual calicuration is:
h(r1,g1,b1, r2,g2,b2, 11){
if r1=g1=b1 and r1=0 then return r2,g2,b2
else if r1=g2=b1 and r1!=0 then return MAX(r2,g2,b2),MAX(r2,g2,b2),MAX(r2,g2,b2)
else
RGB0[3] //An array contains r0,g0,b0; index begins with 1
max = INDEX_MAX(r1,g1,b1)
min = INDEX_MIN(r1,g1,b1)
rest = (remain of 1,2,3 after taken min and max)
RGB0[max] = MAX(r2,g2,b2)
RGB0[min] = RGB0[max]*Min(r1,g1,b1)/Max(r1,b1,g1)
p = (RGB0[max]-RGB0[min])/(MAX(r1,g1,b1)-MIN(r1,g1,b1))
q = RGB0[max] - p*MAX(r1,g1,b1)
RGB0[rest] = p*(middle value of r1,g1,b1) + q
return RGB0[1], RGB0[2], RGB0[3]
}
