Augmented reality for version 7 Tutorial

From AlternativaPlatform Wiki

Jump to: navigation, search


Contents


In this tutorial we will make simple demo in FlashDevelop IDE using Alternativa3D v7 and third party trackers.

It is assumed you already have FD set up and Alternativa3D downloaded.

So, let's make the simplest demo. We will make the duck follow the marker in webcam video on the screen.

While it seems hard to code at first, special libs (trackers) make it as easy as using the mouse.

Consider regular button for a moment. Interaction with the button is reflected in mouse events series mouseOver, when mouse pointer enters the button, mouseMove, when the mouse is moved around, mouseOut, when mouse pointer leaves the button.

Special libs allow you to work with webcam video in the same way. Webcam video behaves similar to the button, when the user holds the marker up to camera, the lib finds it and sends us the "marker found" event. From there on, the lib tracks the marker and sends "marker moved" events. Finally, when the user hides the marker, we get "marker lost" event.

[edit] Using qtrack

Ok, let's open up FD and make empty AS3 project. Copy Alternativa3D swc there and add to the project.

Now download qtrack tracker (very simple, free and newbie-friendly tracker :) Unzip the duck, swc and Alternativa3D example, copy to the same folder. Add swc to the project, mark the example to compile. Select FP10 target in project settings, enter swf name and specify how to open it (external player, if you have debugger installed). Ok, we're done.

Let's go through the code (see comments): 

package {
import alternativa.engine3d.alternativa3d;
import alternativa.engine3d.core.Camera3D;
import alternativa.engine3d.core.Object3DContainer;
import alternativa.engine3d.core.Sorting;
import alternativa.engine3d.core.View;
import alternativa.engine3d.loaders.Parser3DS;
import alternativa.engine3d.materials.TextureMaterial;
import alternativa.engine3d.objects.Mesh;
import flash.display.Bitmap;
import flash.display.Sprite;
import qtrack.QuadMarker;
import qtrack.QuadTracker;
import qtrack.QuadTrackerEvent;
 
[SWF (width=640, height=480)]
public class Alternativa3D_example extends Sprite {
 
	// marker image file
	[Embed(source='duck/duck.gif')]
	private var MarkerImage:Class;
 
	// duck model
	[Embed (source='duck/duck.3ds', mimeType='application/octet-stream')]
	private var ModelData:Class;
	[Embed (source='duck/duck.jpg')]
	private var ModelTexture:Class;
 
	// important variables
	private var camera:Camera3D;
	private var model:Mesh;
	private var tracker:QuadTracker;
 
	public function Alternativa3D_example () {
		// create the camera and place it into empty container
		camera = new Camera3D;
		camera.view = new View (640, 480);
		(new Object3DContainer).addChild (camera);
 
		// create duck material
		var material:TextureMaterial = new TextureMaterial (Bitmap (new ModelTexture).bitmapData);
 
		// parse duck model
		var parser:Parser3DS = new Parser3DS;
		parser.parse (new ModelData, "", 0.1);
 
		model = Mesh (parser.objects [0]);
		model.sorting = Sorting.AVERAGE_Z;
		model.setMaterialToAllFaces (material);
 
		// update camera internal variables (focal length)
		camera.render ();
 
		// create tracker instance
		tracker = new QuadTracker (
			// pass the marker,
			new QuadMarker (new MarkerImage),
			// webcam index,
			0,
			// video dimensions,
			640, 480,
			// camera focal length
			camera.alternativa3d::focalLength
		);
 
		// subscribe to events
		tracker.addEventListener (QuadTrackerEvent.MARKER_FOUND, onMarkerFound);
		tracker.addEventListener (QuadTrackerEvent.MARKER_LOST, onMarkerLost);
		tracker.addEventListener (QuadTrackerEvent.MARKER_MOVE, onMarkerMove);
 
		// add everything to the screen
		addChild (tracker);
		addChild (camera.view);
 
		// run
		tracker.start ();
	}
 
	private function onMarkerFound (e:QuadTrackerEvent):void {
		// marker found - add the duck to the scene
		camera.parent.addChild (model);
	}
 
	private function onMarkerLost (e:QuadTrackerEvent):void {
		// marker lost - remove the duck from the scene
		camera.parent.removeChild (model);
		camera.render ();
	}
 
	private function onMarkerMove (e:QuadTrackerEvent):void {
		// marker moved - move the duck
		model.matrix = tracker.getPose3D ();
		camera.render ();
	}
}
}

To test the demo you will need to print duck/duck.gif

Download sources

[edit] Using FLARManager

If you need to use more advanced trackers, check out FLARManager. FLARManager is a wrapper around different trackers (at the moment - FLARToolkit, flare*tracker and flare*nft) that simplifies and unifies common tasks.

FLARManager comes with its own examples, but we shall change the above code to work with FLARManager. Download FLARManager to new folder from svn (using tortoise svn), create empty AS3 FlashDevelop project in the same folder, copy our example to source folder, Alternativa library and duck folder. Set project settings as above and add correct source path (to src folder).

Now change the code like this: 

package {
import alternativa.engine3d.core.Camera3D;
import alternativa.engine3d.core.Object3DContainer;
import alternativa.engine3d.core.Sorting;
import alternativa.engine3d.core.View;
import alternativa.engine3d.loaders.Parser3DS;
import alternativa.engine3d.materials.TextureMaterial;
import alternativa.engine3d.objects.Mesh;
import com.transmote.flar.camera.FLARCamera_Alternativa3D;
import com.transmote.flar.FLARManager;
import com.transmote.flar.marker.FLARMarker;
import com.transmote.flar.marker.FLARMarkerEvent;
import com.transmote.flar.tracker.FLARToolkitManager;
import com.transmote.flar.utils.geom.AlternativaGeomUtils;
import flash.display.Bitmap;
import flash.display.Sprite;
import flash.events.Event;
import flash.geom.Rectangle;
 
[SWF (width=640, height=480)]
public class Alternativa3D_example extends Sprite {
 
	// duck model (change paths)
	[Embed (source='../duck/duck.3ds', mimeType='application/octet-stream')]
	private var ModelData:Class;
	[Embed (source='../duck/duck.jpg')]
	private var ModelTexture:Class;
 
	// important variables (change qtrack to FLARManager)
	private var camera:FLARCamera_Alternativa3D;
	private var model:Mesh;
	private var manager:FLARManager;
 
	public function Alternativa3D_example () {
		// load project configuration (see below) and specify the tracker (FLARToolkit)
		manager = new FLARManager ("../duck/duck.xml", new FLARToolkitManager, stage);
		manager.addEventListener (Event.INIT, onManagerInit);
	}
 
	private function onManagerInit (e:Event):void {
		manager.removeEventListener (Event.INIT, onManagerInit);
 
		// use special FLARManager camera
		camera = new FLARCamera_Alternativa3D (manager, new Rectangle (0, 0, 640, 480));
		camera.view = new View (640, 480);
		(new Object3DContainer).addChild (camera);
 
		var material:TextureMaterial = new TextureMaterial (Bitmap (new ModelTexture).bitmapData);
 
		var parser:Parser3DS = new Parser3DS;
		parser.parse (new ModelData, "", 0.1);
 
		model = Mesh (parser.objects [0]);
		model.sorting = Sorting.AVERAGE_Z;
		model.setMaterialToAllFaces (material);
 
		// subscribe to FLARManager events
		manager.addEventListener (FLARMarkerEvent.MARKER_ADDED, onMarkerFound);
		manager.addEventListener (FLARMarkerEvent.MARKER_REMOVED, onMarkerLost);
		manager.addEventListener (FLARMarkerEvent.MARKER_UPDATED, onMarkerMove);
 
		// add everything to the screen
		addChild (Sprite (manager.flarSource));
		addChild (camera.view);
	}
 
	private function onMarkerFound (e:FLARMarkerEvent):void {
		camera.parent.addChild (model);
	}
 
	private function onMarkerLost (e:FLARMarkerEvent):void {
		camera.parent.removeChild (model);
		camera.render ();
	}
 
	private function onMarkerMove (e:FLARMarkerEvent):void {
		model.matrix = AlternativaGeomUtils.convertMatrixToAlternativaMatrix(
			e.marker.transformMatrix, manager.mirrorDisplay);
		camera.render ();
	}
}
}

Careful reader would note that we never specified the marker, video dimensions, camera parameters etc. FLARManager reads these settings from configuration file. In this example we will use default configuration file with only minor changes (and copy corresponding files to duck folderй): 

<flar_config>
	<!-- image source settings -->
	<flarSourceSettings
		sourceWidth="640"
		sourceHeight="480"
		displayWidth="640"
		displayHeight="480"
		framerate="30"
		trackerToSourceRatio="0.5" />
 
	<!-- FLARManager settings -->
	<flarManagerSettings
		mirrorDisplay="true"
		smoothing="4" >
		<smoother className="FLARMatrixSmoother_Average" positionToRotationRatio="0.5" />
		<thresholdAdapter className="DrunkHistogramThresholdAdapter" speed="0.3" />
	</flarManagerSettings>
 
	<!-- selected tracker settings -->
	<trackerSettings>
 
		<!-- camera parameters file -->
		<flarToolkitSettings
			cameraParamsFile="../duck/FLARCameraParams.dat" >
 
			<!-- markers -->
			<!-- we use single marker, the rest are omitted -->
			<patterns resolution="8" patternToBorderRatioX="50" patternToBorderRatioY="50" minConfidence="0.5" >
				<pattern path="../duck/patt001.pat" size="2" />
			</patterns>
 
		</flarToolkitSettings>
 
	</trackerSettings>
 
</flar_config>

To test the demo you will need to print duck/patt001.png

Download sources

You can work with FLARManager supported trackers directly. However in this case you will have to write far more code. To give you an idea, let's redo our demo with IN2AR, not supported by FLARManager at the moment.

[edit] Using IN2AR

So, let's download IN2AR and try to modify our example code to work with it. All preparations are similar to what we did above, so let's jump to code modifications: 

package {
import alternativa.engine3d.alternativa3d;
import alternativa.engine3d.core.Camera3D;
import alternativa.engine3d.core.Object3DContainer;
import alternativa.engine3d.core.Sorting;
import alternativa.engine3d.core.View;
import alternativa.engine3d.loaders.Parser3DS;
import alternativa.engine3d.materials.TextureMaterial;
import alternativa.engine3d.objects.Mesh;
import apparat.memory.Memory;
import flash.display.Bitmap;
import flash.display.BitmapData;
import flash.display.Sprite;
import flash.events.Event;
import flash.geom.Matrix3D;
import flash.media.Camera;
import flash.media.Video;
import flash.utils.ByteArray;
import flash.utils.Endian;
import ru.inspirit.asfeat.ASFEAT;
import ru.inspirit.asfeat.calibration.IntrinsicParameters;
import ru.inspirit.asfeat.detect.ASFEATReference;
import ru.inspirit.asfeat.event.ASFEATDetectionEvent;
 
/**
 * 3D model example, Alternativa3D.
 */
[SWF (width=640, height=480)]
public class Alternativa3D_example extends Sprite {
 
	// use the marker shipped with in2ar
	[Embed(source = '../assets/def_img_data.ass', mimeType='application/octet-stream')]
	private var MarkerImage:Class;
 
	[Embed (source='../duck/duck.3ds', mimeType='application/octet-stream')]
	private var ModelData:Class;
	[Embed (source='../duck/duck.jpg')]
	private var ModelTexture:Class;
 
	private var camera:Camera3D;
	private var model:Mesh;
 
	// in2ar tracker and additional variables
	private var tracker:ASFEAT;
	private var found:Boolean;
	private var spoon:BitmapData;
	private var video:Video;
 
	public function Alternativa3D_example () {
		camera = new Camera3D;
		camera.view = new View (640, 480);
		(new Object3DContainer).addChild (camera);
 
		var material:TextureMaterial = new TextureMaterial (Bitmap (new ModelTexture).bitmapData);
 
		// parse duck model with larger scale
		// (according to marker dimensions)
		var parser:Parser3DS = new Parser3DS;
		parser.parse (new ModelData, "", 10);
 
		model = Mesh (parser.objects [0]);
		model.sorting = Sorting.AVERAGE_Z;
		model.setMaterialToAllFaces (material);
 
		// create tracker instance (in2ar loads in parts,
		// so we wait for init event)
		tracker = new ASFEAT;
		tracker.addEventListener (Event.INIT, onLibraryExtracted);
	}
 
	private function onLibraryExtracted (e:Event):void {
		tracker.removeEventListener (Event.INIT, onLibraryExtracted);
 
		// in2ar requires some explicit settings
		var maxPoints:int = 300;
		var maxMarkers:int = 1;
		var maxTransformError:Number = 10 * 10;
 
		// in2ar uses apparat - so configure it 1st
		var memory:ByteArray = new ByteArray;
		memory.endian = Endian.LITTLE_ENDIAN;
		memory.length = tracker.lib.calcRequiredChunkSize (640, 480, maxPoints, maxMarkers);
		memory.position = 0;
		Memory.select (memory);
 
		// now configure the tracker
		tracker.lib.init (memory, 0, 640, 480, maxPoints, maxMarkers, maxTransformError, stage);
 
		// add the marker
		tracker.lib.addReferenceObject (ByteArray (new MarkerImage));
 
		// let the tracker know it was the only one
		tracker.lib.setSingleReferenceMode (true);
 
		// sync camera focal length to tracker parameters
		var params:IntrinsicParameters = tracker.lib.getIntrinsicParams ();
		camera.fov = 2 * Math.atan2 (
			Math.sqrt (640 * 640 + 480 * 480), // screen diagonal
			params.fx + params.fy // double focal length
		);
 
		// subscribe to single event
		tracker.lib.addListener (ASFEATDetectionEvent.DETECTED, onDetected);
 
		// in2ar does not process webcam video on its own
		spoon = new BitmapData (640, 480);
		var cam:Camera = Camera.getCamera ();
		cam.setMode (640, 480, 30);
		video = new Video (640, 480);
		video.attachCamera (cam);
		addEventListener (Event.ENTER_FRAME, updateAndProcessVideoFrame);
 
		// add stuff to the screen
		addChild (video);
		addChild (camera.view);
	}
 
	private function updateAndProcessVideoFrame (e:Event):void {
		found = false;
 
		// feed video snapshot to the tracker
		spoon.draw (video); tracker.lib.detect (spoon);
 
		// if the tracker did not find a thing, remove the duck
		if (!found && (model.parent != null)) onMarkerLost ();
	}
 
	private function onDetected (e:ASFEATDetectionEvent):void {
		var markerInfo:ASFEATReference = e.detectedReferences [0];
 
		// if the duck is not added yet, add it
		if (model.parent == null) {
			onMarkerFound ();
		}
 
		// update duck transformation
		var R:Vector.<Number> = markerInfo.rotationMatrix;
		var t:Vector.<Number> = markerInfo.translationVector;
		onMarkerMove (new Matrix3D (new <Number> [
			+R[0], +R[3], +R[6], 0,
			-R[1], -R[4], -R[7], 0,
			-R[2], -R[5], -R[8], 0,
			+t[0], +t[1], +t[2], 1
		]));
 
		found = true;
	}
 
	private function onMarkerFound ():void {
		camera.parent.addChild (model);
	}
 
	private function onMarkerLost ():void {
		camera.parent.removeChild (model);
		camera.render ();
	}
 
	private function onMarkerMove (matrix:Matrix3D):void {
		model.matrix = matrix;
		camera.render ();
	}
}
}

To test the demo you will need to print assets/def_img.jpg

Download sources

Retrieved from "http://wiki.alternativaplatform.com/Augmented_reality_for_version_7_Tutorial"
Personal tools
Namespaces
Variants
Actions
Navigation
Category
Toolbox
In other languages