Interactivity. Kinect and Watchout?

[Lloyd Stewart]

I would like to put together a way for video characters, rear projected onto a screen via Watchout, to interact with real people. 

 

I found this helpful thread relating to sensors:

 

http://forum.dataton.com/topic/994-wo-sensor/?hl=sensor

 

And I found this thread dealing a bit with Kinect:

 

http://forum.dataton.com/topic/436-watchout-kinect/

 

Kinect, and other motion controllers, like the ones for Wii and Playstation, according to the Wikipedia entry, seem to be more versatile and even include speech recognition, though, not being a gamer, I have no idea how well they do what they do.

 

But what I was envisioning was having certain human movements or even human speech be interpreted by Kinect and then, using "glue code" that takes what Kinect has recognized, "adapts the data, and forwards relevant information to Watchout using its network protocol" (Mike's words) which would then trigger an auxiliary timeline in Watchout where a video character's response would be played back accordingly.. 

 

Once the "video character" responded, the video would continue with that video character just standing there waiting for a response until a human actually did respond, at which point another auxiliary time line would be triggered in the same way, replacing the running one, as a response to the human, etc, etc.  So I'd film perhaps hundreds, or maybe eventually thousands, of possible responses that might possibly be triggered.  

 

So my questions are:

1. In general, is this reasonable, or have I, as usual, strayed far off base again?

2. Is there a limit to auxiliary time lines?

3. Has anyone tried this who might suggest any pointers?   Perhaps a better way?

4. Does the use of auxiliary time lines seem like a reasonable way for a video character to respond? (Confession: I've never needed to use auxiliary timelines or even Midi so all I know about them is what I've read in the manual and on the forum)

 

Input is most appreciated.  Thanks!

 

Lloyd

 

 

 

[NNinja]

I can answer you on aux timelines - we have done several live events with about 100 timelines - we usually use them as cues for specific moments in the scenario (new stage, new speaker, diff videos, etc)

 

Depending on the reaction of the video character - maybe you can also use tween tracks like position instead of only aux timelines

[jfk]

... So my questions are:

1. In general, is this reasonable, or have I, as usual, strayed far off base again?

 

it is reasonable, albeit formidable - not for neophytes.

 

2. Is there a limit to auxiliary time lines?

 

No, not in the WATCHOUT software.

 

There are practical hardware limits on how many that can concurrently be active,

to some extent you will need to manually manage that.

Response time is affected by programming technique.

i.e. if you can wake up (pause) a subset of aux timelines prior to the selection,

the paused (and therefore cached) aux timelines can respond close to instantly.

The unused paused timelines need to be managed after the selection as well.

If you are sending a play to a stopped aux timeline,

it takes time to cache media before it is available, adjust accordingly.

 

3. Has anyone tried this who might suggest any pointers?   Perhaps a better way?

 

Yes and NNinja's live tween suggesting is solid.

 

Also, consider incorporating conditional layer(s).

This can be useful in reducing the number of timelines,

while still tailoring responses on fixed for the run variables like gender, age group, day of week, etc, etc.

without creating a unique aux timeline for each one.

 

4. Does the use of auxiliary time lines seem like a reasonable way for a video character to respond? (Confession: I've never needed to use auxiliary timelines

 

Yes. This is an advanced technique and requires the wo programmer to manage resources

so you don't bog it down over the run.

 

or even Midi so all I know about them is what I've read in the manual and on the forum)

 

You probably would not use MIDI for this type of thing,

you can do anything MIDI can (and much more) with IP.

If you are going to the trouble to create the custom "digital glue"

to interface such a device, might as well use IP for easier access to the full feature set.

[Lloyd Stewart]

Most encouraging Jim and NNinja.  Thanks so much!

 

We begin this little journey tomorrow. 

 

Lloyd

[fdoeve]

Not really a direct response to the thread... But i didnt see the use of starting a whole new topic for this question...

 

When thinking about using a Kinect sensor with Watchout, is there a certain type i should look for? because i see the "standard" Kinect for Xbox 360 and i see Kinect for PC...

Would either one work?

And Does it get its power from the usb or would i need a seperate power supply?

 

Thanx!

[NNinja]

I have experience with Kinect for PC (the old one, not ver2)  but I think there are adapters on the market that allow using Xbox Kinect with a PC

[cowboyclint]

The Kinect for XBox ships with a power/USB breakout, so you can get straight USB to the computer and the wall wart into the wall.

[Mike Fahl]

I assume you've already seen this:

 

   http://academy.dataton.com/recipe/kinect-input

 

Uses the original Kinect, and includes all you need to get going with some basic interaction.

 

Mike

[fdoeve]

I did see and read the recipe, just wondered if it was the original or PC kinect... so it is the original and that's good, because it's easier to come by :-)