From Dream to Reality in 2011?

[lboucher]

Hi All

Well my 2010 show is barely 75% up, I am starting a new job in 2 week, we will have to move within a few months, I have a wife, an 18 month old and a really annoying dog. Also working on 2 droid apps without even owning the hardware. (Much thanks to everyone who is testing or sent in their show!)

And just like the rest of you I am already thinking forward to 2011.

So I thought I would post my next idea and see if anyone wants to tackle it as I feel I am stretched too thin and I am sure there are people out there more capable than I.

So to start with the Dream:

For the past year and a half the dream in the back of my head was to find some way,some how, to allow a person to walk up to my house, wave thier hand, and have the corresponding lights they are pointing to turn on. Magic! Well today at work while talking about the droid app, a coworker suggested the exact answer I have been looking for.

Reality:
Microsoft has been so kind to recently mass produce the most affordable, easy to access, stereo vision system that I know of, and of course very smart people have already hacked it. http://idav.ucdavis.edu/~okreylos/ResDev/Kinect/index.html

Using this single sensor and a (sure to be) complex input plugin for vixen it should definitly be possible to track a persons movement to turn on the lights. Imagine, literally standing in your front yard like a conducter, and literally sequencing by waiving your hands at the lights. (Fantasia anyone)

Of course we will need some way to layout all the lights in the show in 3d. (Maybe some prancer code could be useful?) and people will of course want fading control. Not sure how to do that yet.

Who likes this idea??

[Livermore-Dad]

I'm new to the game but a friend and I discussed this. I stated many have hacked the WII controller (IR) for various things and with the new Microsoft Kinect (no IR), I had the same thoughts as you did. Allow a pedestrian, take center stage and be able to turn on various lights effects with just movement.

I'll take a look at your link as well

[Auggie2612]

I'm new to the game but a friend and I discussed this. I stated many have hacked the WII controller (IR) for various things and with the new Microsoft Kinect (no IR), I had the same thoughts as you did. Allow a pedestrian, take center stage and be able to turn on various lights effects with just movement.

I'll take a look at your link as well

The Kinect popped into mind when reading the 1st post.. They hacked the Guitar hero guitar too make the light's blink.. Only a matter of time for the Kinect to start showing up in the shows..

[StandingInAwe85]

FYI - The Wii controller is bluetooth; with an IR reciever in front of it so that it can identify the position of the TV (or in reality the "sensor bar" above/below the tv). I used the Wiimote and vixen triggers to control the effects in my halloween display this year... if you get a program called glovepie (don't ask) it's very easy to do as it will map the buttons and movements of the wii controller to keys on the keyboard (and then use the kiwi trigger plugins).

[Livermore-Dad]

how funny, the link included was for the kinect. People are already hacking this, and not even owning an Xbox. 3D Camera on the cheap, that sounds bard arse.

So we all seem to be thinking of the same thing.. Cool!

[lboucher]

Ya really neat people are thinking the same thing. Now if we could all just mind meld our skills together we could bust this out in no time.

[ptone]

So to start with the Dream:

For the past year and a half the dream in the back of my head was to find some way,some how, to allow a person to walk up to my house, wave thier hand, and have the corresponding lights they are pointing to turn on. Magic! Well today at work while talking about the droid app, a coworker suggested the exact answer I have been looking for.

Reality:
Microsoft has been so kind to recently mass produce the most affordable, easy to access, stereo vision system that I know of, and of course very smart people have already hacked it. http://idav.ucdavis.edu/~okreylos/ResDev/Kinect/index.html

Using this single sensor and a (sure to be) complex input plugin for vixen it should definitly be possible to track a persons movement to turn on the lights. Imagine, literally standing in your front yard like a conducter, and literally sequencing by waiving your hands at the lights. (Fantasia anyone)

Of course we will need some way to layout all the lights in the show in 3d. (Maybe some prancer code could be useful?) and people will of course want fading control. Not sure how to do that yet.

Who likes this idea??

A worthy dream - and the hacking work with kinect is very cool. I wonder whether it needs a backplane for reference? Looks like it does work outside - here is a cool example of combining kinect and projection mapping (elsewhere I've lobbied for projection mapping being a far better and cheaper alternative to large pixel matrixes).

-Preston

[Livermore-Dad]

A worthy dream - and the hacking work with kinect is very cool. I wonder whether it needs a backplane for reference? Looks like it does work outside - here is a cool example of combining konect and projection mapping (elsewhere I've lobbied for projection mapping being a far better and cheaper alternative to large pixel matrixes).

-Preston

I was going to include that statement in my post as well, but the cost of projectors powerful enough seems to be cost prohibitive.

But yes the 4D displays that they are doing after mapping buildings is fantastic! I started looking into this, but the cameras and computer time just to get the house/location into the system in 3D was a bit of a problem and again not even talking about how to get a project that can black out a house.

[lboucher]

Looks like straight stereo imaging to me, so you shouldnt need a backplane. Low lighting/constantly flashing lighting may introduce alot of noise.

[plasmata]

The only issue I see with Kinect is that light shows are run in the dark. I wonder how well it will work in those (low light) conditions.

[ptone]

The only issue I see with Kinect is that light shows are run in the dark. I wonder how well it will work in those (low light) conditions.

I wondered that at first too - but the depth sensing is all IR:

http://www.youtube.com/watch?v=E-x8Mn1l3rA

-Preston

[Auggie2612]

I hate too sound or even look like a dumba**... But all the little dots or points of light... Is that coming from the Kinect in the video above... Could type more but I'll wait for some responses

[ptone]

CURSE YOU BOUCHER :p

I was all set to get some productive work in another area of my project and you have me off chasing this distraction and trying to get one on Ebay before the holiday crazy's drive these sky high. (I ended up getting outbid).

My primary excitement was discovering how quickly the hack project has gotten organized. And most importantly to me, there are already some early Python bindings!

https://github.com/kelvie/libfreenect/commit/7221fa0adab12a44f4fc01a8bf19055406cd80f8

This means that if I could get my hands on the hardware, I'm pretty sure I could get this up and running with my new software for 2010 - heck I bet I could get it working tonight in a crude form. Not sure I'm going to push hard for that though, as there are lots of things on my plate these days to get done.

Basically my software represents lights, groups, chases etc as objects in code, that are then triggered or modified by a set of signals. Currently these signals are being generated through MIDI events, but it would be trivial to add a signaling pathway from the depth image of a Kinect - I wrote the code lighting code specifically abstracted from the triggering code.

Anywhooo - I'll certainly post if I can get something put together. I'm also thinking of something for Halloween 2011 with this.

-Preston

[budude]

I wondered that at first too - but the depth sensing is all IR:

http://www.youtube.com/watch?v=E-x8Mn1l3rA

-Preston

Does the person on this video have a brother that saw a double rainbow? :rolleyes::p - seems to be blown away by little dots of light...

[lboucher]

As I said, people way more capable than I will take this idea, and blow the rest of us out of the water. (Now you have set the bar, Preston, don't disappoint!)


CURSE YOU BOUCHER :p

I was all set to get some productive work in another area of my project and you have me off chasing this distraction and trying to get one on Ebay before the holiday crazy's drive these sky high. (I ended up getting outbid).

My primary excitement was discovering how quickly the hack project has gotten organized. And most importantly to me, there are already some early Python bindings!

https://github.com/kelvie/libfreenect/commit/7221fa0adab12a44f4fc01a8bf19055406cd80f8

This means that if I could get my hands on the hardware, I'm pretty sure I could get this up and running with my new software for 2010 - heck I bet I could get it working tonight in a crude form. Not sure I'm going to push hard for that though, as there are lots of things on my plate these days to get done.

Basically my software represents lights, groups, chases etc as objects in code, that are then triggered or modified by a set of signals. Currently these signals are being generated through MIDI events, but it would be trivial to add a signaling pathway from the depth image of a Kinect - I wrote the code lighting code specifically abstracted from the triggering code.

Anywhooo - I'll certainly post if I can get something put together. I'm also thinking of something for Halloween 2011 with this.

-Preston

[ptone]

Now you have set the bar, Preston, don't disappoint!)

Well I went out and picked up a kinect today! So we will see how far I can get. Out on the town being social for now (babysitter on duty)

-P

[ptone]

OK - got the kinect talking to my Mac - now to get it to do something meaningful in the light controlling software. Attached pic is of depth map with my son on my lap.

-P

[ptone]

OK, took some work with others to get the python binding to libfreenect working, but once I did,it only took 10 minutes to get this basic proof of concept working:

http://www.youtube.com/watch?v=8kiUw0-NKhY

There is of course a lot more interesting stuff to build onto this.

Note - I'm probably not going to post much here on this thread - as really I have no intention of doing this as a vixen plugin. Also on DIYC there is no forum section for software other than vixen :(

So I may continue this on the aus forums - not sure, I'll post a link here for those that want to follow along.

-Preston

[lboucher]

Dude you freakin rock, I cannot believe someone actually controlled a light with the Kinect in under a week.

Please keep us posted.