For a project I’m currently working on I have to create a textured cube with clickable hotspots (aka a panorama) that can light up. Easy enough in Papervision, if not for the fact that the performance needs to be very high and oh yes one minor detail it has to be in AS2.
The client will be able to load 6 cubesides in an editor, draw hotspots on each side and press render, which will result in a 3d panorama, in which hotspots are going to light up as you mouse over them. Ah yes did I mention this was AS2?
Although this provides us with a huge range of items to discuss and play with, in this post I want to focus on the basics of one single subject: rendering a textured plane in 3d. I’ll get to the rotating/projecting etc bits later, can’t have too much coolness in one post ;).
I’ll try to refer to existing material to save myself some time, so I might be skipping through some stuff very quickly while focussing in on other.
As you might already know, Actionscript 2 doesn’t have 3d built in. It doesn’t have perspective distortion built in either. It does however allow you to scale, skew, translate and rotate clips using a so called transform matrix. We can use this fact to simulate/fake 3d. So how do you simulate 3d by skewing, scaling and rotating rectangular pictures? Seb Lee-Delisle explains the concepts very well. So before continuing read http://sebleedelisle.com/3d-example-files/3d-presentation-texture-maps/ first.
Welcome back. Understanding the concepts is a bit different from actually implementing it though, but it’s halfway there. Before I said that 3d is faked in flash by scaling, skewing and rotating clips, however it is usually created by matrix transforming bitmap fills instead of movieclips and using those bitmap fills on dynamically drawn triangles (using the flash drawing API). Nonetheless matrix transforming movieclips is a good way to start to learn this stuff and we’ll get to the actual triangle rendering later.
So before I get to explaining how a plane is split into rectangles, how rectangles are split into triangles, and how triangles are filled with a bitmap matching their orientation to produce the illusion of 3d perspective, we are first going to look into transforming movieclips to get a good feeling for the effect a transform matrix has on a movieclip.
I advice you not to look at the Flash Matrix API Docs, since aside from the fact that it contains errors, is pretty limited. Senocular has provided a much better resource at http://www.senocular.com/flash/tutorials/transformmatrix/, but it might be a bit overwhelming to read through. If you are short on time I would advise you to read at least these parts (but I’ll give a quick recap as well):
Matrices and the Transform Matrix, on what a matrix is and how matrix multiplication works
If you scroll down to ‘The ActionScript Matrix Class’ there is an interactive plaything just above that, or you can use the one below to play with the matrix interactively. Just note that there is much more information on the senocular page than what you need to know for this section.
The images in the example above are from the flash docs, where the red arrow points out the error in the documentation. So replace b with c, and SkY with SkX and then you are good to go. (So b represents SkY and c represents SkX).
If you have skipped the senocular post about the transform matrix, or glossed over, I'll just give you a quick recap/empirical take on the matrix.
First off, note that all the values in the numeric steppers are taken directly from the clip's transform matrix by reading out MC_rectangle.transform.matrix.a/b/c/d/tx/ty.
The rectangle has four points with the local coordinates P0(0,0), P1(100,0), P2(0,100) and P3(100,100). Note that the rectangle is located at position (300, 70) on the screen (believe me it's true ;)), so the global coordinates for those points are P0'(300,70), P1'(400,70), P2'(300,170) and P3'(400,170).
So how did the transform matrix turn those local coordinates into those global coordinates? (An easy question if you have read and understood the senocular page):
first when working with translating matrices we usually add a w component of 1 (unless it's a normal vector, but not for now)
then we perform all the calculations, for each point Pn we get Pn' by performing matrix multiplication
Transforming Pn to Pn'
TRANSFORMMATRIX x Pn = Pn'
a c tx PnX PnX'
b d ty * PnY = PnY'
0 0 1 1 PnW'
which is, written differently again using the definition of matrix multiplication (if you are used to matrices, you'll do this in your head instead of writing it out in cumbersome equations, but anyway):
So let's try this for our example, where we had P0(0,0), P1(100,0), P2(0,100) and P3(100,100), with a=1, d=1, b=c=0, tx=300, ty=70.
(a*PnX + c*PnY + tx, b*PnX + d*PnY + ty) becomes (PnX + 300, PnY + 70) if we fill in the a,b,c,d, tx & ty:
Just for practice and to get a good grip on the matrix, let's try a couple different ones. Note that what we have just tested is that applying the matrix to the local coordinates does indeed result in the clip as we see in on the screen. Looking at the meaning of the different elements we saw that the x and y scale where 1 (a and b), so there was no scaling, b & c are 0, so no skewing and tx and ty where 300 and 70 respectively, which did indeed result in the translation of each point by dx=300 and dy=70.
So we proved tx & ty directly affect the translation. Let's play with scaling now then. According to our matrix docs, a influences the x scaling and d the y scaling, so let's set a to 0.5 and d to 2. Our rectangle starts at (300,70) and has dimensions 100x100, so x scaling by 0.5 and y scaling it with 2 should give us a rectangle at (300,70) (since the translation doesn't change) with dimensions 50x200. So intuitively the transformed local coordinates should be:
P0(0,0), P1(50,0), P2(0,200) and P3(50,200)
and thus the transformed global coordinates (by translating them over 300x70):
P0(0+300,0+70), P1(50+300,0+70), P2(0+300,200+70) and P4(50+300,200+70) =
P0(300,70), P1(350,70), P2(300,270) and P3(350,270)
Well let's enter our a,b,c,d,tx,ty into our Pn' calculations formula's and see that these are indeed the coordinates we get by using this matrix:
a=0.5, b=c=0, d=2, tx=300, ty=70, so (a*PnX + c*PnY + tx, b*PnX + d*PnY + ty) becomes (0.5*PnY+300, 2*PnY+70).
We see the a & d elements indeed perform the scaling operation.
Last of the 3 basic operations is skewing, which is achieved by modifying the b & c elements. So refer to the gadget once again, and set the elements to
a = 1, b = 0, c = 1, d = 1, tx = 300, ty=70. You see visually that P2 is now aligned with P1 on the x coord (although by a trick of the eye it doesn't seem that way). Another thing to note is that skewing changes the x coordinates of points based on their difference in y coordinates. P0 and P1 have the same y coordinates, so their delta x remains unchanged. P2 and P3 have an y that is 100 more than P0 and P1 so their x's are moved 100 pixels * the c factor with respect the x's of P0 and P1.
The fact that the delta x of points with the same y doesn't change by skewing over x and the delta y of points with the same x doesn't change by skewing over y is an important one which we'll get back to later.
So intuitively where did our P0(0,0), P1(100,0), P2(0,100) and P3(100,100) end up? Looking in the interactive tool we can see P0 and P1 remained stationary while P2 and P3 where translated by 100 over x to (100,100) and (200,100) respectively. In global coordinates P0(300,70), P1(400,70), P2(400,170) and P3(500,170). Verifying this through (a*PnX + c*PnY + tx, b*PnX + d*PnY + ty), with a=1, b=0, c=1, d=1, tx=300 and ty=70 gives us (PnX + PnY + 300, PnY + 70):
As an exercise for you try to pick values which allow you to intuitively/visually see what the coordinates of the transformed points are, then use the (a*PnX + c*PnY + tx, b*PnX + d*PnY + ty) formula to verify that that is indeed correct.
One thing I haven't touched upon is the rotate method. Mathematically rotation is defined through a matrix using sinuses and cosinuses. I'm not going into details about that right now, if you want to know more about rotation take a look at http://en.wikipedia.org/wiki/Transformation_matrix. The only thing I'd like to show you is that it is indeed possible to rotate a movieclip by altering it's transformation matrix (which is actually exactly how flash does it when you set the _rotation property on a clip).
For example set the properties back to a=1, b=c=0, d=1, tx=300 and ty=70.
Now in order, keep clicking:
b until it is 1
c until it is -1
d until it is 0
a until it is 0
Voila rotation by 90 degrees! Although this might be hard to grasp if you have no experience with rotation matrices, and even harder to see how scaling and skewing accomplishes this, the easiest way to grasp this is to let go of the fact that a and d scale and b and c skew, since as you might have noticed after applying the above transforms a & d skew and b & c scale!
So it's easier to accept that a, b, c, d transform vertices and when modifying a single property in the identity matrix, a has the effect of ... b of .. etc, but might have other effects if any of the other properties have been modified. Back to rotation then, to understand how a-d influence the rotation of an object, you have to understand the formula's for rotating points in 2d space. A good overview (& proof!) is given in http://www.petesqbsite.com/sections/tutorials/tuts/relsoft3d/Chapter2/Chapter2.htm, "Section V. 2d Rotation". Once you understand that check out http://en.wikipedia.org/wiki/Transformation_matrix#Rotation which shows how to write these 2d rotation formulas in matrix form, and scroll a bit down to affine transforms, where you find the matrix we've been talking about here.
But ENOUGH about rotation since we are not going to use it anyway for now.
When we are going to draw triangles later, we are going to do so through the Flash Drawing API (moveTo, lineTo, etc). The texture of the triangles will be realized through bitmapfills (beginBitmapFill). We will have to match the bitmapfill for those triangles in a certain way in order to provide the illusion of 3d. If you wish you can refer back to http://sebleedelisle.com/3d-example-files/3d-presentation-texture-maps/ on what we are going to do exactly. The basics for that technique is being able to match a transform, or rather deduct a transform from a number of given vertices (points). It doesn't matter whether you are transforming a bitmapfill or a movieclip, let's just say bitmap equals movieclip at this point. Senocular has provided an interactive tool that demonstrates this principle, right at the bottom of http://www.senocular.com/flash/tutorials/transformmatrix/. You can see that we even only need 3 points to do so, which is great, since 3 points make a triangle which is exactly what we are going to need later on.
Although you can download the cube example from his page and check out the formula's, I'll try to explain and show to you what the formulas are and why.
Refer back to the matrix gadget above. Note that no matter what you change, P0 stays at the same location. This is logical looking at (a*PnX + c*PnY + tx, b*PnX + d*PnY + ty) since P0X and P0Y are zero, so P0 (0,0) = (a*0 + c*0 + tx, b*0 + d*0 + ty) = (tx, ty);
In other words:
matrix.tx = p0.x;
matrix.ty = p0.y;
Now change a, and look at P1' for example: P1' (100,0) = (a*P1X + c*P1Y + tx, b*P1X + d*P1Y + ty) = (100a + tx, 100b+ty).
Rewriting this gives us (using p0.x and p0.y as tx and ty in the p1 equations):
p1.x = 100a+p0.x
p1.y = 100b+p0.y
100a = p1.x-p0.x;
100b = p1.y-p0.y;
a = (p1.x-p0.x)/100;
b = (p1.y-p0.y)/100;
If we had used width and height instead of a 100 and 0, we would see that this is actually:
a = (p1.x-p0.x)/width
b = (p1.y-p0.y)/width
Intuitively this seems correct, if we look at A as the xscale factor again, imagine our original clip is a 100 pixls wide, but the distance between it's upperleft and upperright point is 200, then apparently it's scaled by a factor 2, so a = (200-0)/100 = 2.
B is the y skewing, as we mentioned earlier, coordinates with the same x will have the y displacement, coordinates with the same y will have the same x displacement. In other words, the difference in displacement on one axis is directly tied to the distance between those points on the other axis.
So if P1 has an x 100 higher than P0, it's y displacement will be b*100 (try it out in the interactive tool). Which is exactly the same as saying b = (p1.y-p0.y)/width. But more officially, let's derive the other formulas using P2.
As you have guessed by now, we can do the same for P2' (0,100):
Combine this with what we already had and the matrix becomes:
a = (p1.x-p0.x)/width
b = (p1.y-p0.y)/width
c = (p2.x-p0.x)/height
d = (p2.y-p0.y)/height
tx = p0.x
But wait you say, you are setting P1.y and P2.x to zero! Yes indeed, remember the tx and ty. When distorting a square clip, P0 will always be (tx, ty) and thus it's local coordinates (0,0) and hence P1.y and P2.x will always be zero (locally).
That settles it then! We've got the calculations for all matrix values based on three points P0, P1 and P2, which form a triangle. As mentioned before you can see this in action at the bottom of the senocular page if you'd like to play with it.
Now remember what I said about bitmapfills being transformed the same way while triangles are drawn and rendered using the drawing API. The example below shows this principle. Choose a bitmap and play with the points. Still in AS2, but easy to migrate to AS3. Note that the original bitmaps chosen on the left have different sizes. The equations don't become that different however, we simply plug in the bitmap's size in the equation instead of a movieclip size! Not so fundamentally different from transforming movieclips at all.
Although we could implement another cube here, I leave that as an exercise for you, take a look at senoculars cube and the bitmapfill method downloadable above and have a blast. The major difference is probably that the sides are no longer interactive, but we'll fix that in the future I promise ;).
Up to this point we've shown that we can deduct point number P3 based on P0, P1, P2 (see the source code of the example), and that we can generate a matrix based on three points, and that with it we can render a triangle with a bitmap. As shown in Seb Lee-Delisle's presentation we need partly independent triangles however, so I've updated the example slightly:
Now you can alter 4 handles. Things to note are that to keep the equations simple, we've split the rectangle in an upperright triangle and lowerright triangle so that they have P0 in common. In addition although we can now freely distort the square, this is only the first step. The rendered distortion bitmap is not in perspective, and you can see how awkward the shield and grid look. Of course now that the fourth handle is free and not locked to the position of P0, P1, and P2 you can do things like fold the triangles (when P1 to P2 don't intersect P0-P3 in the middle).
All that is content for another post though, so check back soon for the next article in this series on how to create a 3d panorama in ActionScript 2 (no worries I will migrate it to AS3 as last step). Next time we will look into not rendering a bitmap into a triangle, but rendering a specific part of a bitmap into a triangle, which is the first step on our way to texturemapping.