InnerDriveStudios.com

So I’ve been developing these actionscript 2 panorama’s, but had not tested them in player 8. *wrong*.
Apparently some version of the player 8 crash as soon as debug information is printed.

Some further research indicated that it is caused by a bug, through a combination of a dynamically created textfield on top of a clip with a scrollrect set, on which I draw using the drawing API.

The solution is in the CubeView class, instead of setting the scrollrect on the _canvas, set it on the _canvas._parent. I’m not going to update all downloads, but now you know how to fix it .

Just a tidbit of code, for a general matrix multiplication:

Although if you need real performance you will probably unroll the 3 loops.

Download the sources here: 3d Panorama v0.7 (243)

With respect to the last post/implementation, only a few minor changes were required to the demo we had so far. Basically making the hotspot light up is implemented by drawing on the plane’s material. That’s all there is to it. I’ve implemented this idea in the interactive material class.

Lots of room for optimization, but the basic principle remains the same.

Although the goal has never been to deliver a set of reusable classes/components, the basics are all there, and the possibilities are legion. And although the architecture of the code samples can be improved, and they are far from being a complete application, most of the principles and a lot of the sources can be reused and applied to your own panorama implementations. Also note that you might need to clean the code a bit (I saw some left over parameters sneaking around that are no longer used, hunt them down!)

So this concludes my panorama series, hope you enjoyed it. Onto the next project!

Below is a screenshot of the final version with real images and hotspots implemented. The image is a 3d image and was rendered by my colleague at TriMM, Sebastiaan Dorgelo.

Download the sources here: 3d Panorama v0.6 (201)

How you define your hotspots is up to you. For example you could write a tool which allows you to edit and generate a CubicPanorama. For a project for the dutch police academy I’m currently working on at TriMM, I am writing such an editor in Flex. Here is a screenshot of the work in progress:

Basically it allows you to select a cube side, import a picture for it and draw hotspots on it. The complete definition/model for the cube is then saved to xml, the polygon information with it.

In this demo I’m simply generating rectangular hotspots on the fly randomly as you can see when you reload the page. No matter whether you are using irregular shapes through xml or random rectangles the principle stays the same.

One important note though before I continue, the hotspots do not have to be visible and not rectangular either. The only reason they are here, is to keep it simple and to provide some visual clue of where you should point and click. In addition whether you show a handcursor or not is all up to your implementation.

But like I said the principle stays the same. In the last post we showed how to derive an x,y coordinate within a plane’s texture. But we do not have to use the plane’s texture, we can use any x,y lookup system. For example an array as a look up table. But wait, there is a better way, we simply use another bitmap as a lookup table.

So the basic idea is: for each cubeside, create another bitmap called lookupbitmap with the same size as the texture, and draw the hotspots for each plane onto the lookupbitmap with the hotspot’s ids as colorvalue.

From an architectural standpoint there are again multiple approaches of course. I introduced the CubicPanoramaModel a couple of examples ago, so if you do define your hotspots in xml, that would be a good place to store them. I decided to leave that as an exercise to the reader (you) however, and keep it simple.
We could implement a bitmap manager, material manager, hotspot manager or whatever. I decided to implement this feature in a subclass of a plane’s material and call it InteractiveMaterial. In this case, since it’s a bit hacked together and unoptimized, it’s called MockupInteractiveMaterial, since it is just there to demonstrate the principles.

The MockupInteractiveMaterial generates 5 random hotspots with an id of 1-5 for plane 0, 11-15 for plane 1, 21-25 for plane 2 etc. It paints these hotspots into it’s lookup table and onto it’s own material so you have a visual clue of where to point your mouse. You might even want to be able to toggle this hotspot visibility in which case you would have to retain a copy of the original material, before you ruin it with random rectangles.

There are a couple of things to keep in mind though:

• you will have to deal with overlapping hotspots. In this example I simply drew the hotspots into the lookup table using the lighten blend mode, which will give the hotspot with the highest id precedence.
• by default flash will draw everything anti-aliased, which cannot be switched off except by setting quality to low. Drawing a hotspot with an id of 10 will result in some pixels around the edges of the hotspot having id 9 which ruins the idea. The ‘trick’ is to draw your hotspot into an inbetween bitmap and threshold that bitmap on the id/pixel value that you want.

Checking which hotspot is under the mouse is now as simple as extending the interactive material with the following method:

Download the source here: 3d Panorama v0.5 (174)

In the previous post, we implemented a vector pointing at the cube plane under your mouse. In this post we will look at deriving the local x,y coordinate within that plane, and with it the local x,y coordinate of the pixel within the plane’s texture under your mouse. (Although for hotspots, any local coordinate system will do).

In order to do so, we need to project the vector at the plane it is pointing at. This results in a coordinate that lies within the plane. The coordinate comes from a set of coordinates that all lie within that plane, and this set can be mapped to a range representing the texture coordinates (or hotspot coordinates as we will see later). Although this might sound complicated, it is not so bad as it sounds.

Let’s review projection first.

The formula for projection was:

px = – (projectionplanedistance / original_z) * original_x
py = – (projectionplanedistance / original_z) * original_y

It doesn’t matter whether the original z lies in front or behind the projection plane.

My next two crappy sketches demonstrate this point, first behind the plane (I have left the y coordinate out of the images for clarity, so it is as if we are viewing the 3d scene from the top):

Our original point lies at an (x,z) of (1.5,-6) (in other words x,y,z = 1.5,0,-6).
Our projectionplane lies at a distance of 4 from the origin:

px = – ( 4 / -6 ) * 1.5) =>
px = 2/3 * 1.5
px = 1

Our projected point has an x of 1, at a z of -e.

A point in front of the projectionplane is handled the same way:

Our original point lies at an (x,z) of (1,-2) (in other words x,y,z = 1,0,-2).
Our projectionplane lies at a distance of 4 from the origin.

px = – ( 4 / -2 ) * 1) =>
px = 2

Our projected point has an x of 1, at a z of -e.

In a cube we can do the same, by projecting each vector at the plane it is pointing at no matter what the length of the original vector was:

The distance e we choose is arbitrary, it merely influences the range of the resulting projected values which have to be normalized/mapped to the texture coordinates anyway. However if we choose 1, it simplifies our projection formulas:

px = – (1 / original_z) * original_x
py = – (1 / original_z) * original_y

which is

px = -(original_x / original_z)
py = -(original_y / original_z)

Which will result in a mapping of the vector to a projected px and py within the range -1 to 1.

BUT which coordinate values we should use as x, y and z’s for our projection formulas depends on the orientation of the plane we are projecting our vector at, take a look at the following image:

The projection formulas are based on the projection of coordinates having a negative z with respect to the plane we are projection points on. So for plane 0 we get what you would expect:

px = -(original_x / original_z)
py = -(original_y / original_z)

And in plane 0, z is negative, so this is the same as:

px = (original_x / Math.abs(original_z))
py = (original_y / Math.abs(original_z))

The only reason I’m writing it that way is that we already calculated the maximum absolute values of x,y,z to determine which plane we were pointing at. We see the original upperleft corner in plane 0 has an x,y value of -1,-1 to a lowerright x,y value of 1,1, so there is nothing extra we have to do to map these coordinates from topleft to lowerright to a range of -1 to 1.

For plane 1, we see that the negative z value we need for our projection formulas is actually a positive x value, and that the local x,y is represented by a pair of y,z coordinates. So we negate our x value and fill in the rest:

px = -(original_y / -original_x)
py = -(original_z / -original_x)

which is

px = (original_y / original_x)
py = (original_z / original_x)

Since the topleft y,z = -1,-1 to bottomright y,z 1,1 is already mapped correctly no further action is required.

(If you are confused about the planes and their corner coordinates, check out the previous post which displays the cube with all it’s coordinates per plane.)

For plane 2, we see the negative required z value is actually a positive z value, and the local x,y pair is actually a local x,y pair, so we get:

px = -(original_x / -original_z)
py = -(original_y / -original_z)

which is

px = (original_x / original_z)
py = (original_y / original_z)

BUT WAIT! We see that the upperleft coordinate of the plane is not -1,-1. It’s 1,-1. And the lowerright is not 1,1 but it is -1,1. In other words the x range has been flipped. To correct this we need to negate the px:

px = -(original_x / original_z)
py = (original_y / original_z)

We can do this for each plane, but I will spare you the pain, since I already did that in the source code.

Now we have the calculations that map our mouse position within a plane to a local x,y coordinate with x and y both in the range -1, 1. We can map this to a texture pixel coordinate. Assuming our projected local x and y are represented by planeX and planeY we get:

texture_x = (plane_x/2) * texture_size + (texture_size/2);

which is the same as:

texture_x = ((plane_x+1) * texture_size) /2;

and ofcourse for y:

texture_y = ((plane_y+1) * texture_size) /2;

The example at the top of the page demonstrates these principles. Check out the updated checkPlane method. Note that the texture size is 400 x 400 pixels.

Finally in our next post we will implement the hotspot detection and then we have only one post to go I think which demonstrates hotspots in a 3d panorama that light up as you mouse over them and are correctly transformed in perspective along with the rest of the cube.

Download the source here: 3d Panorama v0.4 (97)

In order to be able to detect hotspots under the mouse, the first thing we need to do is find some way to detect which plane the mouse is currently over and what the local x,y coordinates of the mousecursor are in the local space of that plane.

Instead of providing all the theoretical background I want to try and make it conceptually clear how we can do this. So let go of the mouse coordinates for a moment and take a look at our untransformed starting cube we discussed in a previous post:

We can see that a cube is nothing more than a space enclosed by 6 planes. We can even specify the equations for these planes:

The plane numbers correspond to how we have defined our planes in the panoramas these past examples. Contained within this cube is the set of all (x,y,z) coordinates with an absolute value for x, y and z less or equal to 1. In addition for the coordinates that lie exactly within a plane of the cube, ’special’ equations hold. For example for plane 0 we could say that Math.abs(x) <= 1 AND Math.abs (y) <= 1, which is the same as saying that for all points in plane 0, Math.abs(x) <= Math.abs(z) AND Math.abs (y) <= Math.abs(z).

But this is true for plane 2 as well, so how do we differentiate between these two planes? Simple, check the sign of z. If z < 0 and Math.abs(x) <= Math.abs(z) and Math.abs (y) <= Math.abs (z), or even shorter, if z < 0 and Math.abs(x) >= z and Math.abs (y) >= z, we are dealing with points in plane 0.

It gets even better, if the planes are closer or further away from the origin (0,0,0) and in effect the cube is smaller or larger, this will still hold. In order words seeing the z instead of -1 and 1 in the example equations above means that these equations hold for planes parallel to our planes as well. And we can write these kind of equations for each plane. So given any point you can tell in which plane (or rather quadrant) the point lies by looking at “the sign of the coordinate having the largest absolute value” (from page 142 from Mathematics for 3d Game Programming and Computer Graphics).

Visually this means that given any point contained in the cube you cannot only tell in which plane it lies, but in which quadrant as well. In other words if you view a cube as 6 pyramids put together, you know in which ‘pyramid’ a point lies by looking at “the sign of the coordinate having the largest absolute value”.

(A cube’s content split into 6 pyramids)

So the next problem presents itself: when we are pointing at a certain pixel in our cube, what is it’s coordinate? We can calculate the coordinate by pointing a vector in the direction of our mouse pointer and see where it intersects one of the 6 planes. But how do we calculate the direction of the vector based on the position of the mouse pointer?

Remember the information we discussed in an earlier post about relating the field of view, wall size and wall distance (projection plane distance)?:

projectionplane_distance = (wallsize/2) / tan (fov_in_radians * 0.5)

Visually:

The area within the rectangle on the wall, is the area of our 3d space actually visible on the screen.

(Note that we are using the same field of vision horizontally as we are vertically, like it has been in our panorama’s all this time)

What this did in effect, was given a wall size (stage size in pixels), and a field of vision, calculate the projection plane distance.

Once we have a projection plane distance, we could pick a point in the visible area of our 3d space, in other words a point on our stage, in other words at a certain distance in pixels from the center of the wall/stage, and calculate what the angle of a vector towards that point on the wall would be:

First for the angle alpha which represents the angle between a vector pointing straight at the screen and the mouse x position:

projectionplane_distance = x_offset_from_center / tan (alpha) =>
projectionplane_distance * tan (alpha) = x_offset_from_center =>
tan (alpha) = x_offset_from_center / projectionplane_distance =>
alpha = atan (x_offset_from_center / projectionplane_distance)

The x_offset_from_center is the difference between the center of the stage and the mouse x in pixels.

Of course the mouse position does not only present an offset across the x axis but across the y axis as well:

We can see the same formula applies, however instead of the projectionplane_distance we need to use pythagoras on the plane distance and the mouse y offset from the center, to calculate the length of our adjacent, which gives us the following y angle:

beta = atan (y_offset_from_center / sqrt (projectionplane_distance^2 + x_offset_from_center^2))

So now, given any vector that goes through our camera and points at the center of the screen, we can create a vector that points at the pixel under our mouse by rotating it by the calculated angles.

rotatedvector = [0 0 z].rotateY (alpha).rotateX(beta)

Visually:

Pick whatever you want for z for example [0 0 -1] or [0 0 -10] or whatever. As long as x and y are zero and we are pointing away from the camera.

So now we have rotated vector, what can we do with it?

If we view our rotated vector as a point, we can determine in what quadrant of the cube we are. However we are faced with another problem. The principle that “the sign of the coordinate having the largest absolute value” determines that quadrant only works for an unrotated cube. And we are constantly rotating our cube as we are looking around in our panorama. So we have to ‘undo’ the rotation of the cube on our rotated vector, which gives us the vector we would have had if we were able to point at the same spot in the unrotated cube and use that one to determine in which quadrant (is quadrant actually the right term or should it be pyramid?) our vector lies:

The example in this post demonstrates that principle. You can look around in our cube, in which I have replaced the shrine images by images containing the plane number, and while you are doing that a textfield displays in which plane the calculations think you are currently moving your mouse.

Next time we will see how we can process this info to find the x and y coordinate local to that plane’s texture, which brings us one step closer to our hotspot implementation.

Download the source here: 3d panorama v0.3 (123)

There is an issue with distributing source code for your demo’s. Besides demonstrating what you want to demonstrate, you give people a look at how you program. But the focus is not how I program, it’s demonstrating the concepts of creating interactive panorama’s with hotspots that light up. This means I have to find some middle ground in which the code is kept to the bare minimum without hardly any framework classes or complex program structures, while keeping the example readable and extensible.

Therefore I decided to open this post with a little information on application structures. Probably one of the hardest parts of programming is rising to/with the correct level of complexity. You want to be prepared for anything, but not over complicate things (KISS).

So what do you choose for your application structure? You can choose one of the many frameworks, or you can roll with your own. And after that, do you apply that framework to any application you write?

Personally I love to use a setup which allows me to refactor fast, so I can keep the application as simple as possible and then go more complex when needed. This might not always be preferable when working with larger teams, but for these tutorials, for me it is a nice way to work.

For example:
super simple concept demonstration => some quick & dirty code on a timeline
simple concept demonstration => single class demo without any clear division of responsibilities
concept demonstration => couple classes, mainapplication and subparts start to take shape
more formal simple single view application => mainapplication, mainmodel, mainview (with possibly other names like we have in this example)
large scale multiview application => mainapplication, startup tasks, mainmodel, mainview, submodels, subviews etc. I could write a book about this but I won’t anytime soon.

In the more simpler variants, models might initialize themselves and they might use simple callbacks (like in this example). In more complex variants, models might be deserialized from xml, but queueloaders and services might take care of loading other required data, and everything might communicate through events instead of callbacks, maintaining some sense of progress and reporting errors when necessary.

If the application gets really complex, I usually employ a main application model, through which all the application parts are accessible. For people familiar with PureMVC’s ApplicationFacade, it shares similarities (although the way I build stuff everything is strongly type-checked and I almost never employ singletons. What I am doing in these examples with arrays without accessor functions is something that would be a nono in large applications). If the application gets even more complex, the application gets divided into sub applications / modules / application cores etc and the process repeats itself.

All in all there is a pretty seamless way to scale up when required, and when you know ahead of time that you will be building a monstrosity there is always the option of scaling up from the very beginning. The main thing is that the complexity of the architecture is usually matched by the complexity of what you want to achieve (warranted by).

So back to these tutorials, I’m trying to find a middle ground between demonstrating the principles, keeping a decent structure which can be extended, not over complicating things, not adding a bunch of library classes to bulk load content etc etc. And some days I succeed better at that than other days .

Just remember the focus is on concepts, not on application structure nor holy code conventions wars. And that most of this is written after 9:30 PM when the gremlins have gone to bed .

All that said, since we will be adding hotspots and light up features in the coming examples, this example prepares the way to add those things later. I’ve scaled up the example’s complexity a bit and replaced the DistortedPlane class by a less general CubicPanoramaPlane class.

All images are now loaded externally, and the example can no longer be run from the Flash IDE since there is no longer a fla required (although it is very simple to create one yourself). Instead you should download the latest FlashDevelop version.

As mentioned before, there is no errorhandling, no progress display and probably lots of other stuff missing. Feel free to add it, but it is not part of what I am trying to demonstrate here.

Download the sources for this panorama (fully documented): 3d Panorama v0.2 (125)

In the previous panorama post I mentioned a number of issues that would have to be fixed. For this demo I used another image from http://www.flickr.com/photos/heiwa4126/, who has a set of amazing images, so it’s really worth to go there and take a look (the other images in the flickr equirectangular group are very good as well btw).

So in no particular order, what did we add/fix in this version?

Clipping
The clipping issue was described in the previous post.

Seams
The seams were a surprise to me. I thought the render process caused the seams, and that I would have to fix it by making the planes slightly overlap. I even implemented it first, so if you ever need it, here it is:

however, that didn’t fix it, it only made things worse. Then I discovered that the seams were caused by the bitmapfill repeat parameter which was set to true, when smoothing was turned on for the planes. If you remember, this setting was switched on to prevent drawing lag when points crossed the diagonal of drawn triangle. Since that will never happen when rendering the inside of a cube with our field of view setting, we can safely turn it off. And poof ! No more seams.

(Fluid) motion
I updated the mouse and key movement and added mass. The slight delay after you press a key is now gone as well. See the render function for more info on this.

Correct rotation
The rotation is correct. In order to implement that, we needed to keep the original points untransformed, and store a transformed copy of those points as well. Using two rotation matrices, one for x and one for the y direction, we can correctly transform these points each render pass (but only if we have moved).

Field of vision

I implemented a somewhat more scientific way of calculating the projection plane distance based on a field of view parameter. Please refer to the sources for more info.

What’s next?

We fixed a number of minor other stuff as well and cleaned the code a bit, so we are slowly getting to where we want to be. In the next version I will refactor the DistortedPlane class into a more specific custom PanoCubeSide class to make the performance even better, although I think that for actionscript 2 it is pretty solid already.

In addition we will look at externalizing the images and the implementation of hotspots in the next version!

You may or may not have noticed, but the clipping procedure in the DistortedPlane class we’ve been using can be improved, since it allows for false negatives. This is demonstrated when turning around fast in the panorama from the previous post: white triangles (or whatever your background color is) will pop up.

What is going on?

Basically our clipping method tests whether one of the four corner points of the quad-to-render is within the clipped region. If true, render, if false, skip. That test is very easy to understand: “hey part of me is within the visible region, so render me!” Unfortunately it is also very wrong.

Take a look at the following image:

You see the first two cases are fine, however the last case is wrong. It is a false negative, our clipping test tells us we can skip it, but visually we can see it should have been drawn.

Logical, since we tested for the corner points only.

What we should have done is test whether any part of the quad-to-draw (referred to as quad in the rest of this post) is intersecting our clipping rectangle. One way to do that is to write an intersection test for the quad with the non distorted clipping rectangle, for example through a separating axis test.

Problem is that we don’t want to do that for each quad since our panorama will slow to a crawl. So what we will be doing instead is a bounding box test.

Bounding box test

What is the bounding box for a quad? Assuming we have the four points UL, UR, LL, LR that we’ve been using in each post, the upperleft and lowerright coordinates of the bounding box around these points are given by (in actionscript pseudocode):

Visually:

If we replace our clipping test with a bounding box intersection test between the clipping rectangle and a quad’s bounding box we will get the following results:

So we see that with a bounding box test we get the opposite from what we had with the ‘clipping-rectangle-contains-at-least-one-quad-vertex’ test: we might get false positives. In other words now and then a quad might be rendered which is outside the clipping rectangle. We don’t care. It won’t happen a lot and it’s better than not rendering false negatives.

But how do we implement a bounding box or in other words a rectangle to rectangle intersection test?
There are different options, one simple option would be to create two rectangle instances representing our rectangles and call the intersect method on them. This is bad for two reasons:

a) we would have to create new objects (although we could reuse these objects)
b) we would have to sort our points so that we know which point is upperleft and which point is lowerright in order to create these rectangles right (which is sloooow).

So assuming we roll our own intersection test, how does the intersection test work without sorting the points?
Intersection tests might be hard to understand (or not depending on your math skill), a lot of the explanations I’ve found online talk about overlapping coordinates, which gets hard to visualize mentally fast if we can’t even be sure which point is which.

I find this the easiest way to look at the problem: when is there no intersection?

Seeing our clipping plane has x boundaries clipLeft and clipRight and y boundaries clipTop and clipBottom, we can safely say there is absolutely no possibility of an intersection, if:

• all the points of our quad have an x lower than clipLeft OR
• all the points of our quad have an x higher than clipRight OR
• all the points of our quad have an y lower than clipTop OR
• all the points of our quad have an y higher than clipBottom (clipTop being smaller than clipBottom)

Take a look at the following image for a visual explanation, this image shows all the equations where the quad does not intersect the clipping rectangle:

So when DOES the quad intersect the clipping rectangle? Answer: in all other cases: not all x’s are to the left or right of the clipping rectangle and neither are all y’s: the quad’s bounding box must be hitting some part of the clipping rectangle!

For our DistortedPlane we replace our faulty clipping test with:

Note that if we had sorted our points this test would reduce to a much simpler rectangle-rectangle intersection test which can be found all over google. The same principle applies but since the coordinates are sorted, we no longer have to test ALL coordinates but only specified coordinates based on which side the intersection test is being performed:

but all the min/max operations are killing so we’ll do it the other way.

Next time an updated 3d Panorama in Actionscript 2 with a lot of issues fixed!

Basically it is nothing more than just another cube like we had in the previous example, but now the cube is centered around our camera at (0,0,0) and we are looking at the planes on the inside of the cube.

Getting some test images

I tried to find some test images that were not copyrighted, searching google up and down for ’sample cube faces’ and those kind of search terms. After some long hard searching I realized that searching for ‘equirectangular’ was a better option and I stumbled on this flickr group.

You have to check the source license appended to each image, so that you don’t use copyrighted material. In this post I will be using this image:
http://www.flickr.com/photos/heiwa4126/2507985087/, which requires attribution, so there it is .

Converting equirectangular images to cube faces

Converting the test image to cube faces send me on another long google search spree, but as it turned out you can use Pano2VR (http://gardengnomesoftware.com/pano2vr.php) for that. Simply create a new project, select the equirectangular image and select ‘Convert to cubefaces’.
I’ve processed the images afterwards to resize them and am using 600×600 sized images for this first test, which is still way bigger than required for the example, but we might be zooming in later so I’ll probably leave it at that.

The CubicPanorama

In the previous examples I had a couple of classes, but the examples were constantly implemented in a main class. Since we want this thing to be reusable and refactorable, we’ll create a new class called CubicPanorama, and we will keep refactoring this example as we go along.

Specifying corners point and projection

In the previous example I used some arbitrary coordinates for the cube’s corner points. Then I adjusted my focal point and projection plane until I got something that looked acceptable. Since the cube is now going to be centered on the camera (or the other way around, depending how you look at it), and we might be using images of different sizes for the panorama we need to do this a little bit less ad-hoc.

So although we might pick anything, a fairly common choice is to simply use -1 and 1 to base the corner points on, so the points would become:

(-1,-1,-1), (1,-1,-1), (-1,1,-1), (1,1,-1), (-1,-1,1), (1,-1,1), (-1,1,1), (1,1,1).

See the image below for a visual explanation:

And now we have to map the different distorted planes to these points. By referring to the image above this is fairly easy. I’ll get to the projection bit shortly.

Rotating points

In the previous examples I implemented a rotation method on the points, but there is a quicker/better way. Instead of rotating each point individually, we can use one rotated matrix and multiply each point-as-a-vector with that matrix. But which matrix? We need a matrix that preserves the original coordinate system, and there is a very simple matrix that does just that: the identity matrix. So by simple rotating the identity matrix and multiplying that result with all our points-as-vectors, we end up with the rotated points.

Projecting the planes

To project the planes, we need to calculate the projection plane distance. To understand how this works, take a look at:

http://www.panohelp.com/lensfov.html

3 things are related in calculating the field of view:

• the distance from ‘wall’ (projection plane)
• the ‘wall size’
• the field of view

If you look at the explanation in the link given above you can see that:
tan (fieldofview/2) = (wallsize/2)/projectionPlaneDistance

In other words:
projectionPlaneDistance = (wallsize/2)/tan(fieldofview/2);

Our wallsize is the panorama (viewport) size, and the fov needs to be in radians. Now each photo represents one cube side, which corresponds with an fov of 90, so using an fov of 90 here will map those photos exactly to the corner points of the planes. In other words point (-1,-1,-1) should be projected at (-viewportwidth/2, -viewportheight/2):

projectionPlaneDistance = (wallsize/2)/tan(fieldofview/2); =>
projectionPlaneDistance = (wallsize/2)/tan(90/2 (in radians)); =>
projectionPlaneDistance = (200/2)/tan(45 in radians); =>
projectionPlaneDistance = 100/1 = 100

So px and py become (-100/-1)*-1, (-100/-1)*-1 is (-100, -100) which is correct. If we had used for example coordinates like (-2,-2,-2) we would get (-100/-2)*-2, (-100/-2)*-2 which is exactly the same.

Anyway an fov of 50-60 is more realistic, so we’ll be using that.

Issues (Stuff to fix)

In the next version we will look at a number of improvements:

Fluid motion (mass)

The motion is ugly, besides switching to a mouse for the next version, we need something commonly known as ‘mass’, which is used in almost all the panoramas out there I think. In other words when you stop applying force either by keys or mouse, the panorama has a certain momentum while will decrease due to the application of an opposite force called friction. Simply put, it will slowly stop moving instead of abruptly.

Correct rotation

We mistakingly apply a rotation to the cube in it’s currently rotated state, we can see this is wrong, since we can end up lying on our side. What we should do is first rotate the cube left and right (around the y axis) and then up and down (around the x axis). However this requires us to maintain the original coordinates, and store the left-right/up-down angle and apply the whole rotation each time we change our viewing direction.

Seams

If you look closely you see seams in the rendered image (for example disable the grid and look up a bit until you see a small discontinuity in the rendering).

Better clipping and positioning

I wanted to show how the clipping and rendering works. Now that we’ve seen it, the next example will use a scrollrect to mask the panorama, which means we can no longer use a centered clip (scrollrect doesn’t work with negative offsets), but will have to a clip with x,y at 0,0.

Here is all the code for this example: 3d Panorama v0.1 (128).

Onwards to version 0.2!