1. Alice

http://www.developer.com/java/other/article.php/3673761/Introduction-to-Alice-Programming.htm#Objects_in_3D_space

No not that Alice, but it sure is a bit of a Wonderland it introduces you to. This is a link to a particular place on the site, which is closest to where ’tilt, roll and yaw’ are mentioned. However it is well worth having a look at the rest of this site. It is about a programming environment or program providing a lot of classes for 3D programming, written in Java. Geared towards teaching students about programming, it is graphically compelling.

2. 3D input devices

http://xahlee.org/3d/3D_inputs.html

No 3D programming can be really fully interactive without the use of controllers which provide three dimensional data. Wii controllers and iPods/iPhones are definitely not the only kids on the block, as this article demonstrates clearly.

3. Virtual Wiiality Redux

http://joelclemens.colinr.ca/thesis/components.html

The title is really pun intended, as the article explains why these people have used the Sony Playstation 3 rather than the Nintendo Wii for motion and position control in a 3D virtual reality world. Lots of detailed explanations with video and pictures as well as downloadable code for USB/Bluetooth communication. An alternative for what we did with the Wii Remote, be it an older implementation and possibly not as well supported any longer.

And of course there is lots more out there if you want to look for it beyond the obvious…

Chris

Using the code and samples you can find at

http://penguinsflash.wordpress.com/

you can simulate a device which can feed back tilt, roll and yaw information. It’s all you need to mathematically determine motion in a three dimensional world in any direction. And if you don’t know what these are, there is a very good graphical explanation about it at that WordPress site too. Even the avid remote control plane enthusiast might have an interest in checking it out.

You can use any of the provided examples to control any Object property in Flash/ ActionScript 3, just by feeding the right variables the Penguin object provides from a constantly generated XML file. Look for variables such as rollX and rollY in the sample code and use them with your own objects.

The fastest way to get a good grip on what you can do with these variables is to trace them and then use the TiltEmulator to look at the extremes in both directions by making the device picture tilt left to right or roll up and down. This will help you determine what factors you want to build into your own code, to manipulate the date into a range suitable for the property you want to control. Or in practical terms you could use the range data with the stageWidth and stageHeight to move something within the confines of the stage, or turn the data into a 0 to 1 range for example for a control of the alpha property of a MovieClip object.

The possibilities are endless.

And if you get hold of a device which can send movement data (i.e. from accelerometers…) such as Nintendo Wii controllers, Sony Playstation 3 controllers, iPods/iPhones as well as the software to communicate this data with your computer, you can really experiment with tilt, roll and yaw to your heart’s content…

Here are some screenshots of an application built for use with the Penguin technology and the tiltemulator as used with it.

a spinning cube, with its rotation direction and speed controlled with the

tiltemulator software.

Enjoy!

Chris

1. FlashKit

Sometimes when we are coding, we just need some sound files which we can use. Sometimes we need some effects. And of course we like to have it all in a nice Flash format, apart from the .wav and .mp3 files. Well ‘Flashkit’ may be old (remember Macromedia?), but provides the goods:

Sound Loops – Flash Kit, A Flash Developer Resource for Macromedia …

www.flashkit.com/loops/

Sound FX – Flash Kit, A Flash Developer Resource for Macromedia …

www.flashkit.com/soundfx/

2. Peachpit

Peachpit on the other hand is about providing a good learning experience:
Peachpit: Controlling Flash Audio Volume with Sound Objects > What …

www.peachpit.com › Articles › Design › Adobe Creative Suite

Nothing more than the basics. But well explained, with code and screenshots provided in a 3 page article.

3. SoundManager 2: Javascript Sound For The Web

www.schillmania.com/projects/soundmanager2/

Interesting to see how through using JavaScript to access a Flash video (!?) file this code provides a lot of functionality.

Inside <head> on your HTML page you insert this one line:

<script type=”text/javascript” src=”soundmanager2.js”></script>

A site well designed to get you started quickly and with a good understanding of how to set it up. Documentation and demonstrations galore! All just a free download away.

Enjoy,

Chris

Woodpecker is another one of Dan Zen’s feathered friends. Look up ‘Flash Feathers’ if you don’t understand what this is about ( flashfeathers.wordpress.com/).

At least half of the music applications (i.e. MP3 player software, sound editors…) show one or more bars moving with the frequency or amplitude of the sound. Woodpecker gives you a tool to add that kind of dynamic content to your Flash development. It is easy to use and adds very little overhead. Latency doesn’t seem to be a problem.

But more than that, Woodpecker provides the sound related variables which apart from use in the typical bar displays (bar as in ‘equalizer’ type bar, not the liquid providing kind), also can be used to change the properties of just about any MovieClip you can think of.

Other Flash feathers tended to lead to applications completely centered on the functionality of the particular Flash feather application. Woodpecker lends itself also very well to using it to add a bit of dynamics (pizzazz…) to applications that are not in the first place about animating with sound.

Let your imagination run wild with Woodpecker!

Chris

Only one of them is a practical example of code which is supposed to allow us to do similar things as in Robin, and that is Hemlock.

The other four are mostly papers on multi-user technology, some quite interesting because they date far back, yet touch on the core issues of multi-user applications.

HEMLOCK – Build Multi-User, real time web applications

http://hemlock-kills.com

This is a downloadable system, with added example code and some screenshots of examples (Mac and Unix like systems, last updated June last year)

DONATIEN – Towards an easy way to build interactive multi-user applications

www.sics.se/pepito/D6.1/donatien.ppt

PowerPoint presentation from 2002, from professor Donatien Grolaux at the Catholic University of Leuven (UCL), Brussels

ECSW – A General Multi-User Undo/Redo Model

http://www.ecscw.org/1995/15.pdf

Yes the ’1995′ in the URL does stand for its year of creation. A paper presented at the Fourth European Conference on Computer – Supported Cooperative Work (ECSW), in Stockholm, Sweden. A topic which obviously is of great importance for multi-user applications.

DAO – Configuring Communication Middleware to Support Multi User Object-Oriented Environments

http://www.gsd.inesc-id.pt/~ler/reports/doa02.pdf

Sep 01, 2002 – paper presented by professors from the University of Lisboa, Portugal – intersting for bringing together the OO and the Multi User aspects in MOOs.

FROM MOOs TO… – Moving from MOOs to Multi-User Applications

http://research.microsoft.com/en-us/um/people/sdrucker/papers/moving.pdf

Paper from Steven M. Drucker, for Virtual Worlds Group, Microsoft Research, Microsoft Corporation.
DHTML – HTML 4.0 – CSS 1.0

As said earlier, from the above links, if interested in Robin driven applications, Hemlock is probably your best bet for an easy alternative. Note however the obvious difference Robin provides: Flash technology means development in Adobe Flash and end result running in web browsers and thus independent of operating system.

Enjoy!

Chris – 01/26/2010

In this blog I illustrated how I approached the development of a Flash application for use with Robin:

Working with Robin

To work with Robin it helps to clearly divide the code up in code dealing with what needs to happen on each of the PCs and code that deals with the communication of data between the PCs.

On each machine three things need to happen:

1. Initialization when the program is started, which takes care of:

- adding the PC to the group of PCs using the application (including making the distinction between joining existing group or being the first one on it);
- put the avatars for the own PC and the other PCs if any, on the local PC screen;
- send out the initial coordinates and avatar number for the local PC avatar.

2. If and when data is received from the other PCs, show the avatars in their changed positions.

3. We need to be able to move the local avatar with the mouse on the local PC and when we do its number and new position needs to be sent out to the other PCs.

Example Code

The example code that follows comes from a program I called Cats. It allows for 5 ‘players’. The ‘avatars’ are small jpg pictures we put in the library of the .fla file. From there we drop them in 5 consecutive frames of a MovieClip. This allows for using them separately with gotoAndStop(frame number) statements.

In the initialization code I first check whether there is data out there. If not my avatar number is 0. If there is, the number of used avatars is determined and our avatar automatically gets as number the number of avatars in use. This also points to the fact that the first element in the returned data array is element 0. Worth remembering as the frames for the avatars start counting from 1. Because we send the avatar numbers as part of the data and don’t use the position in the data array as the avatar number it doesn’t really matter though.

Cat.as file:

package cat {

import flash.display.MovieClip;
import flash.display.Sprite;
import flash.events.*;
import flash.geom.Rectangle;
import fl.transitions.Tween;
import fl.transitions.easing.Regular;

import com.danzen.utilities.Robin;

public class Cat extends MovieClip {

private var myRobin:Robin;
private var cats:Array = [];
private var myCatNr:Number = 0;
private var nrOfCats:Number = 0;

public function Cat() {

trace (“hi from Cat”);

myRobin = new Robin(“RobinCat”,5,true);
myRobin.addEventListener(Event.CONNECT, init);
myRobin.addEventListener(DataEvent.DATA, receiveData);
myRobin.addEventListener(Event.CLOSE, closeData);

} // constructor Cat

private function init(e:Event) {

trace (“Connected!”);
trace (“In init()”);

if (!(myRobin.getProperties(“x”))) {
trace(“Nothing there yet!”);
} else {
myCatNr = nrOfCats = myRobin.getProperties(“x”).length;
} // if
initCats();
} // init

private function initCats() {

trace (“Connected!”);
trace (“In initCats()”);

for (var i = 0; i<nrOfCats; i++) {
cats[i] = new mcCats();
cats[i].x = myRobin.getProperties(“x”)[i];
cats[i].y = myRobin.getProperties(“y”)[i];
cats[i].frame = myRobin.getProperties(“frame”)[i];
addChild(cats[i]);
cats[i].gotoAndStop(myRobin.getProperties(“frame”)[i]);
} // for

cats[myCatNr] = new mcCats();
cats[myCatNr].x = mouseX;
cats[myCatNr].y = mouseY;
cats[myCatNr].frame = myCatNr + 1;
addChild(cats[myCatNr]);
cats[myCatNr].ButtonMode = true;
cats[myCatNr].gotoAndStop(myCatNr+1);

myRobin.setProperties({x:cats[myCatNr].x, y:cats[myCatNr].y,

frame:cats[myCatNr].frame});

cats[myCatNr].addEventListener(MouseEvent.MOUSE_DOWN, dragCat);

} // init

private function receiveData(e:DataEvent) {

trace(“Receive Data.”);
nrOfCats = myRobin.getProperties(“x”).length;
trace(“Received nrOfCats : ” + nrOfCats);

for (var i = 0; i<nrOfCats; i++) {

if (i!=myCatNr) {
cats[i].x = myRobin.getProperties(“x”)[i];
cats[i].y = myRobin.getProperties(“y”)[i];
cats[i].frame = myRobin.getProperties(“frame”)[i];
cats[i].gotoAndStop(myRobin.getProperties(“frame”)[i]);
trace(“Received from record nr.” + i + ” – x = ” + cats[i].x + ” – y = “  +                  cats[i].y + ” – frame = ” + cats[i].frame);
} else {
trace(“i = myCatNr”);
} // if

} // for

} // receive data

private function closeData(e:DataEvent) {

nrOfCats = myRobin.getProperties(“x”).length;

for (var i:uint=0; i<nrOfCats; i++) {

cats[i].visible = false;

} // for

} // close data

private function dragCat(e:MouseEvent) {
cats[myCatNr].addEventListener(MouseEvent.MOUSE_MOVE, moveCat);
cats[myCatNr].addEventListener(MouseEvent.MOUSE_UP, dropCat);
cats[myCatNr].startDrag(false,new Rectangle(cats[myCatNr].width/2,

cats[myCatNr].height/2,

stage.stageWidth-cats[myCatNr].width/2,

stage.stageHeight-cats[myCatNr].height/2));
} // dragCat

private function moveCat(e:MouseEvent) {
myRobin.setProperties({x:cats[myCatNr].x,

y:cats[myCatNr].y,

frame:cats[myCatNr].frame});
} // moveCat

private function dropCat(e:MouseEvent) {
cats[myCatNr].removeEventListener(MouseEvent.MOUSE_MOVE,

moveCat);
cats[myCatNr].removeEventListener(MouseEvent.MOUSE_UP,

dropCat);
cats[myCatNr].stopDrag();
} // dropCat

} // class

} // package

The corresponding .fla file only contains one movieclip with 5 frames, each one containing one picture (the ‘avatar’).

This program still needs some further fine tuning and debugging, but illustrates clearly the aforementioned approach to using Robin. For getting the Robin multiuser server set up and running please consult the documentation at the Robin WordPress blog.

Links

Robin is PHP based for its socket control. Find the documentation and code for running Robin at:

http://robinflash.wordpress.com/

If Robin wets your appetite for more cutting edge accessible systems, then also have a look at the other systems Dan Zen gives us in Flash Feathers at:

http://flashfeathers.wordpress.com/

For other multiuser systems you might want to have a look at :

Palabre,  a python socket server for flash multiuser applications (Palabre – Flash Xml Multiuser Socket Server 0.5). For info:

http://script.wareseeker.com/free-flash-xml-multiuser-socket-server/

For an xSocket,  Java  and Flash based Multiuser Chat Server, look here:

http://giantflyingsaucer.com/blog/?p=224

Pleasant coding!

Chris – 01/26/2010

So it is definitely a good project subject.

There are a lot of different aspects to 3D vision. All our club members got involved in searching the web and other publications and we got an appointment with Mr. Song at s415 in the SCAET building.  I put a summary of my understanding of what I found on the web and heard from Mr. Song in an email to the rest of the group. An email I copied lower down the blog.

My own particular interest goes to ‘stereoscopy’, which basically is dedicating a separate image to each eye, taken with cameras separated at approximately eye distance between left and right eye. Blame it on my ViewMaster having being one of my favourite possessions as a kid.

I firmly believe that combining new technologies (better monitors, cameras, …) with older technology (optical) can still be worth our while.

Therefore I started experimenting with two web cams used simultaneously, creating two images on a monitor. The challenge is to create an optical system that allows you to look with one eye at the one picture, and the other at the other picture. I tried this with mirrors but haven’t been able so far to make something well enough aligned and stable enough to be used with real success.

The promise of a system like this is that the optical system is independent of the picture generation. So as better material becomes available (monitor resolution and size, camera resolution, remote controlled moving cameras, …) what you see gets better with the same optical set up.

Apart from this I also tried out polarized glasses in front of a projector and wearing polarized glasses. This did not work. I found out on the internet why not. Most projection screens mess up the polarization. It takes a special screen to ot mess up the polarization.

All great fun.

Chris

Dec 16th, 2009

3D Vision
10/27/2009
Meeting with Mr. Song, at Sheridan, S415

Mr. Song gave examples of different techniques:

1. stereoscopic images (example ViewMaster, late 19th century cards
with two pictures (left eye/right eye…))

Pros:   simple, cheap;
Cons:    no moving images, can only be viewed by one viewer;

2. holograms – used to be monochromatic, now full colour and high
resolution

Pros:    high resolution, now in full colour, no special glasses
needed;
Cons:   needs external light source on it, expensive and complicated
to produce;

3. multiple images broken apart and arranged in lines to compose a
picture seen through special screen. This technique is used in many
different ways:

a. printed material with plastic grooved lens (parallel lines)
attached to it (example of dog on notebook cover)(static image);

Pros: simple, cheap, no special glasses needed;
Cons: one viewer at a time, limited viewing angle;

b. plastic grooved lens in front of iPod (moving image, animation)
(example shown on Mr. Song’s iPod), no gap between iPod screen and
lens;

Pros: relatively simple and cheap material, no special glasses
needed;
Cons: limited viewing angle, one viewer at a time, lowered
resolution

c. sophisticated bigger lenses attached to computer monitors/TVs and
showing more than two views, with gap between screen and lens;;

Pros: wider viewing angle, no special glasses needed, more than one
viewer;
Cons: expensive screens, expensive and complicated production with
multiple cameras;

4. projected images with different colour filters or polarization
filters for each eye or synchronized shutter techniques by which the
viewer sees alternating images in rapid succession;

Pros: can be seen by many viewers simultaneously, great for special
effects;
Cons: coloured lenses do not allow for normal colours;
polarized lenses mean loss in luminosity;

Some of the points to note:

1. 3D content production is greatly helped by taking into account how
the human brain perceives 3D: changed focus (depth of field) and size
for different distances, shadows, straight lines seperating for
instance  walls, ceilings and floors (perspective clues) …

2. Because most of these techniques are based on two or more images
being shown in the same space and in the same time as we perceive the
final composed image, resolution is typically divided by the number of
images shown simultaneously and refresh frequency needs to be a
multiple of the frequency typically used for non 3D imaging. For
instance the iPhone has a resolution of 320 x 480, which gets reduced
to 160 x 480 (no vertical duplication of image).

3. Several different aspects of 3D vision are used either seperately
or combined: the different viewpoint of left and right eye and the
different views a viewer expects when he/she moves. This also opens up
the possibility of simulated 3D where viewer position is tracked to
show different images on the screen (for example through the use of
sensors as found in the Wii controller or the small sensors Mr. Song
showed us).

Conclusion:

Most of the discussed techniques require substantial resources. The
cheapest to further experiment with may well be stereoscopic vision
(dual image) and simulated 3D (Wii controller).

Chris Rogiers – 10/28/2009

Have a look at it. Try it!

If you are in awe for the picture manipulation in the CSI labs, but realize it is probably all simulation, try DODO. Real multi-touch, real usable interface building.

From a hardware point of view you can really do this on the cheap. From a software point of view it does really require minimal additional code to get something fully functional.

The first way to try this is with a simple carboard box, a web cam in the back of it and a plastic screen at the front of it. The translucency, reflectiveness and darkness (lightness) of the plastic (or other material of your choice) determines whether you are better off using your fingers touching the screen (dark on light) or laserpointers/flash lights (light on dark) as cursors. Translucency also determines whether you see some of the background and use it in your application.

A second way to try this (courtesy of Nitin) is by replacing the box with a simple light coloured screen and then use light coloured objects with a black dot at the end (i.e. customized chopsticks or stirring sticks) as cursors against it. This leaves you easier adjustment possibilities than the closed box concept (camera distance, manual focus…). Just don’t all go out raiding your local Chinese or coffee shop at once…

Near instant gratification, yet enough opportunity for further exploration.

Chris

Starting from the gooseexample.fla provided by Dan Zen in the multimedia pioneering course, I developed a gooseexample3.fla with a matching Goose Example3.as.

At first it all looks a little daunting, but once you start editing the example files it soon becomes obvious that it takes a relatively little amount of work, in PhotoShop or GIMP and in Flash with ActionScript 3.0, to come up with a completely different program look and feel.

In my case I made a ‘Tug of War’ program where two or more users can put a cursor on the screen at one computer with their mouse at  different other computers. For ‘Tug of War’ they can grab either the left or right hand side of a rope and try to move it their way. Past a marker on the left or right makes the corresponding counter run up. A reset button allows for repositioning the rope in the middle and putting both counters back at 0.

Graphics and code for moving the rope were easily developed. It took me a while to realize that I could not directly read the position of the rope because I had no function that fed back that info in either one of the aforementioned files. But one morning when I woke up, and thought about the program I worked on the previous day, EUREKA, I thought of the ENTER_FRAME event and used it to call an updateScore method I wrote, to constantly detect where the registration point x of the rope object was versus the the x coordinates of both markers. The scores were adjusted accordingly based on some ‘if’ statements.

During development I tested the programs with three computers. The Robin server was running in Apache 2.2 on a machine running Ubuntu 8.04 desktop version with the example program, all accessed in Firefox 3. The nodes for control, via http://www.danzen.com/node.html, were running on two separate computers. One was also running Ubuntu 8.04 desktop version, the other Ubuntu 9.04 desktop version and Firefox was used on both.

That is how I showed myself that everything was functional.

The one thing to remember when doing that with a typical LAMP setup (Linux, Apache, mySQL, Perl & Python) is that our server root directory is sitting in /var/www/ which is only accessible for writing to by the root user (use sudo in Ubuntu for temporary root privileges) . Also remember that you have to set user and group owners of all files you copy from your Windows based Flash development environment to www-data. Apart from that, it is pretty much straight sailing.

Good fun! I really can recommend looking into all the class packages as provided on www.danzen.com. Although the examples most of us built are little more than pleasant exercises of exploration, the multitouch functionality of the provided classes could really be used for professional applications.

And there is no doubt that the computing world is moving towards multitouch,  first on the small screens (i.e. iPod), and now also on bigger screens (Windows 7). Be it not quite as shown on CSI and NCIS yet… Dan just gives us the tools for early adoption with simple means!

All you need to get started with Goose can be found at

http://gooseflash.wordpress.com/

And if you need to find some good design ideas for the sites on which you play with Goose,  you might want to have a look at

http://veerle.duoh.com/

Happy experimenting!

Chris

Yes, today time was up for the delivery of a vision based Flash project that uses web cams as a tool to test out interactivity.

As suggested in class earlier today, typical issues for us neophytes were mostly related to stopping and starting. So, yes, with the code we learned today in class, I am sure I could have presented a more elaborate and/or refined end product. On the other hand nothing beats some self study to come up with our own solutions.

The starting point obviously was the set of  .as files under the Ostrich banner Dan Zen provided, as available on www.danzen.com. These provide everything needed for the processing of movement as detected by the camera. Well not really,  as the camera does not detect anything, but the programs based on the provided classes do detect changes in the images by looking at grouped differentials between subsequent moments in time.

Of all the classes provided I considered using the Ostrich button and Ostrich cursor classes. The former defining an area on the screen and the latter providing the coordinates of where movement is detected. Which brings with it the possibility of the one (movement location – cursor) to interact with the defined button location on screen.

The example of use of this movement detection system that I came up with is a simple ‘look for the secret location’ exercise and show some score which relates to the amount of  ‘moving around’ before hitting the target.

The layout was kept simple. Rather than using all the available ‘real estate’ for the camera image, a wide rim was left on the right and bottom. The stage being set at 640 x 480, I chopped of 140 on the right and 80 at the bottom, thus leaving ample space for a title (static), short indication of what it is about (static),  a visual indication (asterisk) when the secret target gets hit the first time and an indication of how often the system saw your movement cursor on the squares which were not the target.

The rim leaves an area of 500 x 400 for the image, convenient size for a 5 x 4 matrix of 100 x 100 squares. One of which will be the target square.

The Ostrich camera and cursor classes were needed, but the use of  the Ostrich button class was found to be optional.

Basically I could have defined an Array of 20 button instances which would have required calculating the coordinates of said buttons.  Not being fully familiar with the Array class yet nor wanting to circumvent that with 20 blocks of code,  I applied the required calculations to convert the coordinates detected by the cursor to determine (0,0) to (5,4) block  X, Y coordinates within the 20 block matrix.

The secret target block is chosen by using the random() function of the flash.Math import to to elect a set of block coordinates  [(0,0) to (5,4)].

Every scan the system sees coordinates which don’t result in the same block coordinates as those of the target,  a counter is incremented. When the target block is ‘moved over’ that counter does not further increment. A big asterisk is then shown and the final counter tally.

All this worked as expected,  regarding detection of cursor and  checking against target block.

The main issue I struggled with is the start and stop as well as the representation of the score.

Two problems arose:

1. It all starts immediately, cursor sometimes is sitting on the target, hence immediate stop with a zer0 score.

2. Initially the score counter kept running and the system detecting more hits and the score showing increasing results over and over again on top of each other creating an illegible block blob.

Solutions:

1. Not incrementing score when detecting cursor on target within the first 200 scans, but incrementing a variable (started)  and only when that one is above 200 the score counter starts counting and as long as it is less than 200,  hitting the target is not reported as the first hit.

2. The continuing counting and showing after the first hit really was the result of an oversight, and was easily resolved with a variable ‘shown’ which is initiated at zero, which circumvents the line that shows the ‘misses’ counter at first correct hit if  ‘shown’  is zero  and sets ‘shown’ at 1 to prevent further showing.

That way it looks half decent.

The main things I would change and add, given more time, is some implementations of ‘start’ and ‘stop’ for the whole process. Now the start is basically the starting of the .swf file, and thus very immediate, although it was improved to some extent with the 200 scan delay. There is no ‘stop’, so it just sits there after the first hit and keeps scanning. No way of restarting (yet) ,  for instance with a button or key press. All you can do is close the window and if you wish start the program again.

The code itself can do with a bit of a clean-up, to remove code that does not do anything, from some past experimenting and also to add some additional comments.

Apart from the course material covered before today, the following resources were used:

‘Essential ActionScript 3.0′, published by O’Reilly,

ISBN-10: 0-596-52694-6,

information of which can be found at the publisher’s web site and at http://www.chapters.ca. Note that the O’Reilly site offers quite a lot of content online in the form of freely provided chapters as a way of advertising.

Amongst other things, the use of the random() function and the use of TextField and TextFormat , was learned from the aforementioned book.

That’s all Folks… for now.