For writing of shader mode, a language to describe macOS/iOS custom filter based on OpenGL GLSL shader language - Core Image Kernel Language - is used. Here, the summary of shader mode description method and the grammar of the language are shown.
Least one function is described in shader field with grammar like C language. This function must be start with 'kernel' keyword and return color or position data. Because the function is called for each pixels on source image, a new image is generated as the result. The function is classified three types - color conversion, deformation, and general purpose - dependent on the return and argument types. In KansuImager, these are defined as named 'color', 'warp' and 'convert'.
kernel vec4 color(__sample s){
return s;
}
'color' function has an argument as '__sample' type to pass a color information at current position on original image. The return type is 'vec4' (four element vector) that returns converted color information. In the above example, the value of s is converted to vec4 value and returned as it is. That is, in this example, the original image is simply copied.
kernel vec2 warp(){
vec2 p=destCoord();
return p;
}
'warp' function has no argument. The return value is a position of original image. The pixel value at the position is copied to the current position on converted image. Normally, getting the current position with 'destCoord' function, calculating a position on original image with the current position, and the value is returned. In this example, because the before and after positions are coincident, the image do not change.
kernel vec4 convert(sampler s){
vec2 p=destCoord();
vec4 q=samplerTransform(s,p);
vec4 c=sample(s,q);
return c;
}
'convert' function is defined for simultaneously color and deformation conversion. It has a 'sampler' type argument that can be get information like pixel value at arbitrary position, image size or etc. The return value is a pixel value at current position.
In the example, getting the pixel position (p) on converted image, it is converted to coordinate (q) on sampler, and a pixel value (c) is get with 'sample' function from q. Since c is return as it is, the same image with original is generated.
Under Core Image Kernel Language，more arguments can be specified unlike above three functions, but in KansuImager, these three types are restricted to.
'main' function without arguments is described in OpenGL GLSL, and global variables are used to input or output information. However in Core Image Kernel Language, shader is defined with an user defined function with arguments.
As a KansuImager's own feature, value of parameter defined in pre-timing can be referred. As followings, when $ is attached at the head of parameter, the value is filled at the place as a real constant. When @ is attached at the head, the value is filled as a integer constant.
// A, N : parameter's name
float r=$A;
int n=@N;
for(int i=0; i<@N; i++) { ...
int - integer number float - real number
There are int as integer type and float as real type.
When describing a numerical value as a constant, please note that real numbers and integer are strictly distinguished by the presence or absence of a decimal point.
float x=0.0; // OK float y=0; // ERROR
Type cast is used to substitute value to different type variable.
int i=1; float x=float(i);
A variable is defined a constant with 'const'. Repeat count number of 'for' loop must be decided in compile time, the number is declared with 'const'.
const int n=5;
for(int i=0; i<n; i++){...
bool - true or false．
'bool' is used as type of boolean value.
bool fg=true; bool judge=(x>1.0);
vec2 - has x and y element vec3 - has x, y and z element vec4 - has x, y, z, and w element
As vector type (structure), vec2, vec3 and vec4 are defined according to the number of elemenets. vec2 is used to contain coordinate value in image. vec3 and vec4 are used to contain color information.
Moreover, the origin of coordinate system is placed at left-bottom corner. The right direction is X axis plus, and the upper direction is Y axis plus.
When vec3 or vec4 containts color information, x, y, z, and w elements correspond to red, green, blue, and alpha components respectively.
Initializing and accessing method for vector variable are shown in followings.
// initializing x as 1.0 and y as 2.0. vec2 v=vec2(1.0, 2.0); v.x=1.0; //substitution to x component v.y=2.0; //substitution to x component // initializing x,y, and z as 0.0 vec3 u=vec3(0.0); // initializing x,y, and z as value of // u, and w as 1.0 vec4 c=vec4(u,1.0);
Operations with vector versus vector can be calculated.
// getting current coordinate vec2 p=destCoord(); // right side position vec2 q=p+vec2(1.0, 0.0); // moreover, upper position. q+=vec2(0.0, 1.0);
__sample - r, g, b, a elements
__sample type (structure) is used for argument of color conversion function. This type has r, g, b, and a float elements, each mean red, green, blue, and alpha components respectively. The range of these values is from 0.0 to 1.0. 'a' means opacity, when the value is 0.0, it means perfect clear.
A value of __sample can be substituted to a vec4 variable.
// s is __sample variable． vec4 c=s; // subsitute to vec4 c.x=s.r; // substitute r component c=s.rgba; // subsitute to vec4 // subsitute after exchanging r // and b components c=s.bgra;
sampler type contains image information and is used to get information of input image. It used as an argument type of general purpose converting function.
To get a pixel value (color information) from a sampler variable, you must convert an image coordinate to a sampler's local coordinate and pass it to sample function.
// s is sampler variable. // get current location vec2 p=destCoord(); // convert to sampler coordinate vec4 q=samplerCoord(s,p); // get pixel value vec4 c=sample(s,q);
mat2, mat3, mat4, struct, and arrays types defined in GLSL can not be used in Core Image Shader Language. Please refer the more information as followings.
Core Image Kernel Language Manual
Like C language, 'if' sentence branches processings.
if(x<0.0){
...
} else if(x<1.0){
...
} else {
...
}
Iteration processing is operated with 'for' or 'while' sentence like C language. However, the number of repetitions must be decided in compile time. Therefore, a constant or a variable defined with 'const' is used for the end judgment statement.
'continue' and 'break' are not supported.
for(int i=0; i<5; i++){ ...
const int n=5;
for(int i=0; i<n; i++){ ...
while(x<5.0){ ...
const float x1=5.0;
while(x<x1){ ...
Here, some functions defined in Core Image Kernel Language are introduced. Refer Core Image Kernel Language Manual for the more details．
| destCoord() | current location on image |
| samplerTransform(s,p) | convert to sampler coordinate system |
| samplerCoord(s) | current location of sampler |
| samplerSize(s) | image size |
| sample(s,p) | pixel value at location p |
A location at target pixel on output image is returned.
vec2 p=destCoord();
A location is converted to sampler coordinate system.
// sampler s; vec2 p=destCoord(); vec2 q=samplerTransform(s,p);
A current pixel location on a sampler is getten.
// sampler s; vec2 p=samplerCoord(s);
Input image size is getten.
// sampler s; vec2 size=samplerSize(s);
By the way, since sampler variable is not getten with deformation or color converting function, samplerSize function cannot be used in these functions. Therefore, to get image size, define Pre-timing parameters and fill in shader code with $ or @ mark as followings.
#Example on text editor
#W,H:predefined variable
$Pre width W
$Pre height H
$Shader
kernel vec2 warp(){
vec2 size=vec2($width, $height);
...
A pixel value at location p on input image is getten.
// sampler s; vec2 p=samplerCoord(s); vec4 c=sample(s,p);
Here, some functions defined in OpenGL Shader Language (GLSL) are introduced. Please refer Core Image Kernel Language Manual or GLSL Reference . Please note that there are GLSL functions not supported by Core Image Kernnel Language.
GLSL supports many mathmatical functions like sin or cos. Please refer to the above manual for these functions. Here, useful functions related with vector are introduced.
| length(v) | length of vector |
| distance(p,q) | distance between tow positions |
| normalize(v) | normalization of vector |
| dot(u,v) | inner product of vector |
| equal(u,v) | judging that vectors are equal or not |
The length of vector is returned.
vec2 v=vec2(1.0,2.0); float L=length(v);
The distance between two points is returned.
vec2 p=destCoord(); float L=distance(p, vec2(100.0, 200.0));
The length of vector will be one.
vec2 p=destCoord(); vec2 n=normalize(p);
The inner (dot) product of vectors is carried out.
vec2 p=destCoord(); const float d=1.0/1.41421356; float L=dot(p, vec2(d,d));
Juding that vectors are equal or not.
if(equal(u,v)) { ...
An example of coding in Shader mode is shown. The code is a partial code described with text format.
The mirror image of the left side of image is copied to the right side.
1:$Pre width W
2:$Mode Shader
3:$Shader
4:kernel vec2 warp(){
5: vec2 p=destCoord();
6: const float x0=$width/2.0;
7: if(p.x>x0){
8: p.x=$width-p.x;
9: }
10: return p;
11:}
Line 1:
'width' parameter is defined with Pre direction.
The value points the value of predefined variable W (image width).
Line 2:
'Mode' is specified to 'Shader'.
Line 3:
Start of shader definition.
Line 4:
Header of deformation function 'warp'.
Line 5:
Getting the current location on converted image coordinate and it is contained to p.
Line 6:
The value of Pre-parameter 'width' is divided by 2.0 and is substituted to x0.
Line 7-9:
If X component p.x of converted position less than x0 that means that p.x is located on right side, p.x is changed to the distance from the right edge of image.
Line 10:
The vector p is returned.
The return value of 'warp' function specifies a location on input image. In above operations, when p is laid on the left side, since p is return as it is, the output image is equal to input image. When p is laid on the right side, since p is changed to the mirroring position, the value at the symmetrical position on left side of input image is copied.