| Subcribe via RSS

AS 3.0 Augmented Reality with Blender and PV3D

December 10th, 2009 Posted in ActionScript 3.0, Flex

Introduction

In this tutorial you will learn how to create a simple AR(Augmented Reality) scene, at first appearance this code is a little hairy but you don’t have to understand every part of it to use it.

Requirements


1. Web Camera


2. Adobe Flash CS4

Try or Buy

3. Flex Builder

Try or Buy

4. Blender

Download

5. PV 3D

Download

6. Augmented Reality

Download

7. Marker File

Download

8. Pattern

Download

Pre-Requesites

It would be useful for you if you are familiar with PV3D. It would be good to know 3d modeling with Blender and other 3d modeling program

Save a Pattern

First of all, this is not a tutorial for beginers, but I will try to explain very well all the steps that must be taken to obtain the final result.

Well, download the Marker File. You will see a AIR file, this AIR file will help ous to save a pattern for our Augmented Reality project. Instal the Marker File…on the desktop you will see a ARToolKit Marker Generator AIR file.

Open it, in the same time you Web Camera will start, than you need to save a pattern, you will see a button named Save Pattern and same red lines walking there, those lines will try to find a pattern. Well, now you need to show a paper with a pattern.

A exemple of pattern:

While, you show the paper, on the camera you will see that the red linis are around the pattern.Like in this picture:

In that moment click the save button and save the pattern. In this tutorial my pattern is named pattern.pat.

That was the first step.

3D Model

For designers, this is maybe the most important step, but I’m not a designer ao something like that. I’m only a ActionScript developer, and I will analyze the step from my point of view. For this step you can use Blender, Swift 3D or another modeling program, but unse only the program able to export in Collada. First if you are a beginer in 3d (like me), you can draw a cube or something. If you are use Blender, and have already the made 3d model. Go to File->Export->Collada 1.4(dae).

After exporting you will have a file named something.dae. This file contains our 3d model that we use in PV3D.

If you are useing Swift 3D, the first thing again is draw you model, and export to PV3D. In Swift 3D, we use other option caled Export Selected object To Papervision3D…. Like in this picture:

After that you will have a .swf file , a .fla file, and the most important thing a .dae file. Well now we have the pattern and the 3d model. Is time to do same develop work with ActionScript 3.0 .

Final step 2

Make a new project

If you use Flex Builder just create a new ActionScript 3.0 Project and called augmentedReality. If you unse Flash CS3 or CS4 create a Flash File(ActionScript 3.0), you can give what name you want. At this file create a class called augmentedReality.as. The Flash File and the class must be in the same folder.

For Flex:


And if you are useing flash, there is a simple pictutre to help you:


Well, now we are ready to write the code.

Final step 3.

Start to write the code 

1
2
3
4
5
6
7
8
9
10
11
12

package
{
	import flash.display.Sprite;
 
	public class augmentedReality extends Sprite
	{
		public function augmentedReality()
		{
 
		}
	}
}

Here we have the augmentedReality class copy this in Flash, if you are useing flash.

Same imports thing

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32

 
package
{
    import flash.display.BitmapData;
    import flash.display.Sprite;
    import flash.events.Event;
    import flash.media.Camera;
    import flash.media.Video;
    import flash.utils.ByteArray;
 
    import org.libspark.flartoolkit.core.FLARCode;
    import org.libspark.flartoolkit.core.param.FLARParam;
    import org.libspark.flartoolkit.core.raster.rgb.FLARRgbRaster_BitmapData;
    import org.libspark.flartoolkit.core.transmat.FLARTransMatResult;
    import org.libspark.flartoolkit.detector.FLARSingleMarkerDetector;
    import org.libspark.flartoolkit.pv3d.FLARBaseNode;
    import org.libspark.flartoolkit.pv3d.FLARCamera3D;
    import org.papervision3d.objects.parsers.Collada;
    import org.papervision3d.render.BasicRenderEngine;
    import org.papervision3d.scenes.Scene3D;
    import org.papervision3d.view.Viewport3D;
 
    [SWF(width="640", height="480", frameRate="30", backgroundColor="#FFFFFF")]
 
    public class augmentedReality extends Sprite
    {
         public function augmentedReality()
	   {
 
         }
     }
}

Well, here we need the Papervision 3d library, and the Augmented Reality library.

Private Variables and Embled files

Now we need to embled the pattern and a most important file the camera_para.dat This file is in the Augmented Reality lib . You don’t need to know want is in this file just embled this file. 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55

package
{
    import flash.display.BitmapData;
    import flash.display.Sprite;
    import flash.events.Event;
    import flash.media.Camera;
    import flash.media.Video;
    import flash.utils.ByteArray;
 
    import org.libspark.flartoolkit.core.FLARCode;
    import org.libspark.flartoolkit.core.param.FLARParam;
    import org.libspark.flartoolkit.core.raster.rgb.FLARRgbRaster_BitmapData;
    import org.libspark.flartoolkit.core.transmat.FLARTransMatResult;
    import org.libspark.flartoolkit.detector.FLARSingleMarkerDetector;
    import org.libspark.flartoolkit.pv3d.FLARBaseNode;
    import org.libspark.flartoolkit.pv3d.FLARCamera3D;
    import org.papervision3d.objects.parsers.Collada;
    import org.papervision3d.render.BasicRenderEngine;
    import org.papervision3d.scenes.Scene3D;
    import org.papervision3d.view.Viewport3D;
 
    [SWF(width="640", height="480", frameRate="30", backgroundColor="#FFFFFF")]
 
	public class augmentedReality extends Sprite
	{
		[Embed(source="pattern.pat", mimeType="application/octet-stream")]
		private var pattern:Class;
 
		[Embed(source="camera_para.dat", mimeType="application/octet-stream")]
		private var params:Class;
 
		private var FlarParams:FLARParam;
		private var FlarCode:FLARCode;
		private var vid:Video;
		private var camera:Camera;
		private var FlarCamera:FLARCamera3D;
		private var bmd:BitmapData;
		private var FlarRaster:FLARRgbRaster_BitmapData;
		private var FlarDetector:FLARSingleMarkerDetector;
		private var container:FLARBaseNode;
		private var viewPort:Viewport3D;
		private var scene:Scene3D;
		private var renderEngine:BasicRenderEngine;
		private var TransMatResult:FLARTransMatResult;
 
            public function augmentedReality()
		{
			setFLAR();
			setCamera();
			setBitmap();
			setPV3D();
			addEventListener(Event.ENTER_FRAME, loop);
		}
     }
}

In the constructor we have same function that we will have to explain, in the following steps.

We have the setFLAR(); function. In this function we will load the camera_para, and the pattern.

In the setCamera(); function, we will set the option for the Web Camera.. the resolution, the fps.

Well in the setBitmap(); function, we will do a bitmap data for our FLARRgbRaster_BitmapData. And we will draw the bitmap to the video.

And here at the setPV3D(); function, we will add the 3d model.

Private Methods

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46

            private function setFLAR():void
		{
			FlarParams = new FLARParam();
			FlarParams.loadARParam(new params() as ByteArray);
			FlarCode = new FLARCode(16,16);
			FlarCode.loadARPatt(new pattern());
		}
 
		private function setCamera():void
		{
			vid = new Video(640,480);
			camera = Camera.getCamera();
			camera.setMode(640,480,31);
			vid.attachCamera(camera);
			addChild(vid);
		}
 
		private function setBitmap():void
		{
			bmd = new BitmapData(640,480,true);
			bmd.draw(vid);
			FlarRaster = new FLARRgbRaster_BitmapData(bmd);
			FlarDetector = new FLARSingleMarkerDetector(FlarParams,FlarCode,80);
		}
 
		private function setPV3D():void
		{
			scene = new Scene3D();
			FlarCamera = new FLARCamera3D(FlarParams);
			container = new FLARBaseNode();
			scene.addChild(container);
 
			var obj:Collada = new Collada("tank.dae");
			container.addChild(obj);
 
			renderEngine = new BasicRenderEngine();
			TransMatResult = new FLARTransMatResult();
 
			FlarCamera.z = 0;
 
			viewPort = new Viewport3D();
			addChild(viewPort);
 
			obj.rotationY += 20;
			obj.rotationX += 10;
		}

Well,for a better understanding about this private methods. I will explain again what deals they have.

1
2
3
4
5
6
7

            private function setFLAR():void
		{
			FlarParams = new FLARParam();
			FlarParams.loadARParam(new params() as ByteArray);
			FlarCode = new FLARCode(16,16);
			FlarCode.loadARPatt(new pattern());
		}

In setFLAR we have a FlarParams instance, and we load the params in this instance. And because the params are a binary file, we load this params as a ByteArray. And in the FLARCode instance we load the pattern file, and give a resolution, in this case 16×16.

1
2
3
4
5
6
7
8

private function setCamera():void
		{
			vid = new Video(640,480);
			camera = Camera.getCamera();
			camera.setMode(640,480,31);
			vid.attachCamera(camera);
			addChild(vid);
		}

The setCamera function set up the option for our camera. Well, I set up the resolution 640×480, but if you have a more performance camera, give a larger resolution. And I think 31 fps is good for all kind of camera. And after that we attach the camera on the Video instance. And of course we addChild the vid(Video).

Well, after you set this part of code, if you run you project, ypu will see, samething like that:

And if you click Allow, your camera will start.

1
2
3
4
5
6
7

private function setBitmap():void
		{
			bmd = new BitmapData(640,480,true);
			bmd.draw(vid);
			FlarRaster = new FLARRgbRaster_BitmapData(bmd);
			FlarDetector = new FLARSingleMarkerDetector(FlarParams,FlarCode,80);
		}

Well, now we have the setBitmap, here we create a BitmapData, give the same resolution as the camera. And we “draw” the vid that contains all the information about our camera.After that we have the FlarRaster and the FlarDetector. Well the FlarDetector, is a detector, which will see the pattern on the stage… 80 reprents the marker width.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

private function setPV3D():void
		{
			scene = new Scene3D();
			FlarCamera = new FLARCamera3D(FlarParams);
			container = new FLARBaseNode();
			scene.addChild(container);
 
			var obj:Collada = new Collada("tank.dae");
			container.addChild(obj);
 
			renderEngine = new BasicRenderEngine();
			TransMatResult = new FLARTransMatResult();
 
			FlarCamera.z = 0;
 
			viewPort = new Viewport3D();
			addChild(viewPort);
 
			obj.rotationY += 20;
			obj.rotationX += 10;
		}

And finaly the setPV3D function. In this function I add the 3d model to the FLARBaseNode.

Final Code

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119

package
{
    import flash.display.BitmapData;
    import flash.display.Sprite;
    import flash.events.Event;
    import flash.media.Camera;
    import flash.media.Video;
    import flash.utils.ByteArray;
 
    import org.libspark.flartoolkit.core.FLARCode;
    import org.libspark.flartoolkit.core.param.FLARParam;
    import org.libspark.flartoolkit.core.raster.rgb.FLARRgbRaster_BitmapData;
    import org.libspark.flartoolkit.core.transmat.FLARTransMatResult;
    import org.libspark.flartoolkit.detector.FLARSingleMarkerDetector;
    import org.libspark.flartoolkit.pv3d.FLARBaseNode;
    import org.libspark.flartoolkit.pv3d.FLARCamera3D;
    import org.papervision3d.objects.parsers.Collada;
    import org.papervision3d.render.BasicRenderEngine;
    import org.papervision3d.scenes.Scene3D;
    import org.papervision3d.view.Viewport3D;
 
    [SWF(width="640", height="480", frameRate="30", backgroundColor="#FFFFFF")]
 
	public class augmentedReality extends Sprite
	{
		[Embed(source="pattern.pat", mimeType="application/octet-stream")]
		private var pattern:Class;
 
		[Embed(source="camera_para.dat", mimeType="application/octet-stream")]
		private var params:Class;
 
		private var FlarParams:FLARParam;
		private var FlarCode:FLARCode;
		private var vid:Video;
		private var camera:Camera;
		private var FlarCamera:FLARCamera3D;
		private var bmd:BitmapData;
		private var FlarRaster:FLARRgbRaster_BitmapData;
		private var FlarDetector:FLARSingleMarkerDetector;
		private var container:FLARBaseNode;
		private var viewPort:Viewport3D;
		private var scene:Scene3D;
		private var renderEngine:BasicRenderEngine;
		private var TransMatResult:FLARTransMatResult;
 
		public function augmentedReality()
		{
			setFLAR();
			setCamera();
			setBitmap();
			setPV3D();
			addEventListener(Event.ENTER_FRAME, loop);
		}
 
		private function setFLAR():void
		{
			FlarParams = new FLARParam();
			FlarParams.loadARParam(new params() as ByteArray);
			FlarCode = new FLARCode(16,16);
			FlarCode.loadARPatt(new pattern());
		}
 
		private function setCamera():void
		{
			vid = new Video(640,480);
			camera = Camera.getCamera();
			camera.setMode(640,480,31);
			vid.attachCamera(camera);
			addChild(vid);
		}
 
		private function setBitmap():void
		{
			bmd = new BitmapData(640,480,true);
			bmd.draw(vid);
			FlarRaster = new FLARRgbRaster_BitmapData(bmd);
			FlarDetector = new FLARSingleMarkerDetector(FlarParams,FlarCode,80);
		}
 
		private function setPV3D():void
		{
			scene = new Scene3D();
			FlarCamera = new FLARCamera3D(FlarParams);
			container = new FLARBaseNode();
			scene.addChild(container);
 
			var obj:Collada = new Collada("tank.dae");
			container.addChild(obj);
 
			renderEngine = new BasicRenderEngine();
			TransMatResult = new FLARTransMatResult();
 
			FlarCamera.z = 0;
 
			viewPort = new Viewport3D();
			addChild(viewPort);
 
			obj.rotationY += 20;
			obj.rotationX += 10;
		}
 
		private function loop(e:Event):void
		{
			bmd.draw(vid);
 
			try
			{
				if(FlarDetector.detectMarkerLite(FlarRaster, 80) && FlarDetector.getConfidence() > 0.5)
				{
					FlarDetector.getTransformMatrix(TransMatResult);
					container.setTransformMatrix(TransMatResult);
					renderEngine.renderScene(scene, FlarCamera, viewPort);
				}
			}
 
			catch(e:Error){}
		}
	}
}

And the final result must be something like that,my 3d model is a thank, and look how he is:

That was the Augmented Reality tutorial.

Cya next time

Tags: , , , , ,
This entry was posted on Thursday, December 10th, 2009 at 8:12 PM and is filed under ActionScript 3.0, Flex. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.

17 Responses to “AS 3.0 Augmented Reality with Blender and PV3D”

  1. Jane Says:

    February 25th, 2010 at 9:25 PM

    Hi! I’m Jane. I really like you tutorial,I would like to do something like this in my design project .But there is a problem in the steps of “setFLAR(); function. In this function we will load the camera_para, and the pattern.”

    Actually I had put the” camera_para,pattern,material jpg and pat1.pat” in the src folder, but why it stills have the problems?

    This is my screenshot of my flex :) Would you help me to check it?

    http://img534.imageshack.us/img534/5969/2010022631129.png

    Thank you!

    Jane
    Greeting from Hong Kong :)


  2. Horatiu Condrea Says:

    February 25th, 2010 at 9:31 PM

    Hei Jane!

    Watch carefully, somewhere you forgot to close a private function. Look again on your code


  3. Jane Says:

    February 26th, 2010 at 4:41 AM

    O i see, thanks a lot!

    And I got another problem

    http://img690.imageshack.us/img690/9797/20100226103411.png

    Description Resource Path Location Type
    1018: Duplicate class definition: augmentedReality. augmentedReality.as augmentedReality/src line 36 Flex Problem


  4. Horatiu Condrea Says:

    February 26th, 2010 at 4:26 PM

    You have 2 classes with the same name


  5. Tom Says:

    March 18th, 2010 at 2:31 AM

    Hi, I’m getting 3 separate errors for everything I import.
    I’ve set up the source path correctly for the actionscript project and all the files are there in the right place. I’m stumped. Any thoughts?

    Example errors:
    Type was not found or was not compile-time constant: FLARParam
    Definition org.libspark.flartoolkit.core.param:FLARParam could not be found
    Call to a possibly undefined method FLARParam

    Thanks


  6. Horatiu Condrea Says:

    March 18th, 2010 at 5:26 PM

    Hello,

    All the 3 errors are because you don’t import org.libspark.flartoolkit.core.param.FLARParam;. Or maybe this class does not exist in folder.
    Check again…


  7. Tom Says:

    March 18th, 2010 at 9:36 PM

    Thanks for getting back to me so quickly, I’ve checked again and I have imported it and the class is there in the folder…

    Here a screen shot of it:

    http://img532.imageshack.us/img532/5530/screenshot20100318at193.png

    Thanks for your help.


  8. Horatiu Condrea Says:

    March 18th, 2010 at 9:57 PM

    Try to put all classes in the “src” folder. The errors are because the AS don’t “see” that org folder.


  9. Tom Says:

    March 18th, 2010 at 10:42 PM

    I’ve done that and now I have a whole new bunch of errors.

    http://img9.imageshack.us/img9/6789/screenshot20100318at203.png


  10. Horatiu Condrea Says:

    March 18th, 2010 at 10:58 PM

    Here is my project: http://cid-5a9c086cdf2e94fa.skydrive.live.com/self.aspx/.Public/augmentedReality.rar

    Look and see what is wrong in your code!


  11. Mack Brosnan Says:

    March 24th, 2010 at 11:26 AM

    Hi,

    I want to know more about Augmented Reality.Kindly send me any link.

    In Brief,

    We are a England, UK based an integrated Web solutions company with our head office in Surrey and development center in India with primary focus on Website Designing & Development (ASP, ASPdotNet, Java ,Perl, joomla and PHP development)

    We have our competency in CMS (Joomla, Modx, Mambo and other quality Content Management System) and e-commerce website.

    Please visit our website http://www.sushayne.com

    We wish you the best of luck and looking forward to a long and healthy business relationship with you and your company.

    Kind Regards,
    Mack Brosnan


  12. Horatiu Condrea Says:

    March 24th, 2010 at 3:59 PM

    Augmented Reality class is quite complex, and many don’t understand this class. But if you want to read something. Take a look here:http://www.howstuffworks.com/augmented-reality.htm

    Good luck with the company.And of course if you need an AS 3.0 developer, contact me!


  13. Site Promotion Guidebook Says:

    March 31st, 2010 at 12:05 PM

    Interesting post, having a little trouble accessing the RSS feed. I would quite like to subscribe to your blog. Will try again tomorrow, hopefully it is fixed by then – Otherwise send me an e-mail and I might be able to help. Thanks, the guys from Site Promotion Guidebook


  14. Agnes Says:

    July 2nd, 2010 at 6:35 PM

    Hi

    I tried out your tutorial but i got a problem that the augmented reality image doesn’t appear.. What could be the reason? There are no errors. But the augmented reality image just didn’t came out.


  15. Horatiu Condrea Says:

    July 3rd, 2010 at 4:36 PM

    Hello,

    Maybe the Blender model is to large, has a resolution too high! Try to adjust the Z axe.


  16. Agnes Says:

    July 7th, 2010 at 6:11 AM

    Hi

    Thanks for the reply. It solved my problem. Is it possible for you to do a simple tutorial on how to add a button into the video and had an action with a click on the button? Example, once i clicked on the button, the 3D model changed into another model. It would be great if you could do a tutorial on that. :D Thanks alot!!


  17. galih Says:

    July 19th, 2010 at 4:12 AM

    hi,
    i want to know about the collada’s file.
    is collada’s file (.dae) that is exported from blender, an animation?
    and when it’s imported by papervision3d, can it move on (animated) otomatically while executed by papervision3d?


Leave a Reply

Get Adobe Flash playerPlugin by wpburn.com wordpress themes