| Create Digital Noise Forum Index » DIY, Programming, Installations, and Interactive Art » live camera capture control surface for performance |
|
Page 1 of 1 |
|
| Author |
Message |
| funnelbc |
 Posted: Fri Apr 07, 2006 9:26 pm |
 |
|
Joined: 03 Feb 2006
Posts: 48
Location: Melbourne, Australia
|
I had expressed interest in this in the comments of the CDM main site aaages ago, but I went looking for it and couldn't find it.
I'm after information and links on systems that use video cameras to film a surface in real time, and allow you to have real word objects on that surface and be able to assign that object parameters as an lfo/sample/effects.. And then you move them about on the surface in realtime.
The version I saw of this kind of system had the surface divided into quadrants and there was a sweeping bar that went around and it would hit each item as a trigger. Moving the objects in and out towards the center would affect the parameters of the objects.
I'd be very keen to find out more about this kind of control surface if anyone has some handy links that would be great. There is a term that escapes me that is used to describe this kind of thing.
Cheers folks  |
_________________
Buy Me a Pony! |
|
| Back to top |
  
|
| Jaymis |
| Posted: Fri Apr 07, 2006 11:04 pm |
|
|
Site Admin
Joined: 04 Feb 2006
Posts: 260
Location: Brisneyland, Australia
|
That sounds like Audiopad, except for the quadrants bit. Audiopad uses antennas and (I think) RF tagged objects rather than a camera for capturing motion. I think this tends to be a better option because it has less lag and is accurate even if someone's hand is over the object being tracked.
Physical interface? I think the designers just tend to make up their own terms. |
|
|
| Back to top |
|
| Guest |
| Posted: Sat Apr 08, 2006 2:44 pm |
|
|
|
|
There have been a lot of these projects, Nat. Chris O'Shea had actually started an open sourced project along these lines:
http://www.chrisoshea.org/projects/sonicforms/
. . . but I don't think he's had much time to work on it lately. That should get you started.
Basically, it comes down to:
1. you need a surface with a projector in it for visual feedback (or if you don't care about visual feedback, just a surface)
2. you need a camera at a fixed distance
3. you need a light source, preferably IR as you won't get interference from the projection
4. you need a way of processing the image using blob detection. This is readily available in libraries for Processing, the cv.jit extension for Jitter, etc. Processing has the advantage of being free. To get really fancy, you recognize certain shapes or use RFID tags to differentiate the objects from one another.
5. whatever you want to control
Interestingly, there has been so much focus on implementing 1-4 that I've rarely seen anyone do 5 terribly well. (Well, 1-4 are a lot of work!) I've done a bit of work with motion tracking, but not with blob detection and discrete objects because I've been working with dancers onstage.
It would be great to get an open sourced approach to this.
Maybe what we need here is CDM Labs, where we get a group of us to work on this and blog our results! I'm game!  |
|
|
| Back to top |
|
| PeterKirn |
| Posted: Sat Apr 08, 2006 2:49 pm |
|
|
Site Admin
Joined: 03 Feb 2006
Posts: 822
Location: New York, NY
|
Okay, that was me in case you hadn't guessed. Now I seem to be having issues getting accidentally logged out, which other people had complained about . . . will have to pay more attention and see if I can watch the cookies and figure out why that's happening.
Peter |
|
|
| Back to top |
|
| PeterKirn |
| Posted: Sat Apr 08, 2006 2:50 pm |
|
|
|
Site Admin
Joined: 03 Feb 2006
Posts: 822
Location: New York, NY
|
Oh, also, I've covered quite a lot of things related to this . . . which you'll be able to easily find thanks to the tags on the new site. Watch for the "tangible" and "tables" tags.  |
|
|
| Back to top |
|
| thesimplicity |
| Posted: Sun Apr 09, 2006 1:06 am |
|
|
Joined: 07 Apr 2006
Posts: 122
Location: Tucson, AZ
|
I built an interface a while back that functions along similar lines in Processing. It used color detection to determine the movement of objects across a grid... it was actually quite easy to build, basically a glorified eye-mouse. It had an 'initialize' button (spacebar) that I would press to grab the starting positions, then any movement would be in relation to that on a set scale. So I'd have all these different colored pebbles on a gridded surface (a Go board) that I would move like X/Y controllers. I didn't use it for audio (it was a poetry engine, actually, where movement would alter nouns) but it would probably be pretty easy to set it up with Sonia. If you poke around the Processing forums there are a lot of projects just like this with source code. I'd offer you my own source, but it's far from what I'd call 'stable.'  |
|
|
| Back to top |
|
| rylos |
| Posted: Mon Apr 10, 2006 12:14 am |
|
|
Joined: 10 Apr 2006
Posts: 2
Location: missouri
|
| Real time control sure beats using menus. I took the easy way out and built a bunch of touch-sensitive strips. |
_________________
Scrounge and Ye Shall Find |
|
| Back to top |
|
| thompsotd |
| Posted: Mon Apr 10, 2006 11:47 pm |
|
|
|
Joined: 07 Apr 2006
Posts: 87
Location: West Palm Beach
|
Anonymous wrote: It would be great to get an open sourced approach to this.
If you're on Windows, EyesWeb is a really nice open-source package. Very thorough with its own MIDI, OSC, and audio objects in addition to all of the video tracking stuff. Also supports FreeFrame plugins. I keep a Windows book around just to use it occasionally and transmit data via OSC back to Max on my mac.
I don't play with it much anymore because I've started to use Jitter. The developers (at the Engineering School of the University of Genoa) use it mostly for research projects, but also for artistic things. I know they have talked (off record) about porting to OS X, and the chief holdup was that EyesWeb's extensive use of Intel libraries. But, I would be surprised to see a mac or linux port anytime soon, unless someone else took it on.
If you're on windows, it's definitely worth a download. |
|
|
| Back to top |
|
| chrisoshea |
| Posted: Tue Apr 11, 2006 10:41 am |
|
|
|
Joined: 11 Apr 2006
Posts: 10
|
Anonymous wrote: There have been a lot of these projects, Nat. Chris O'Shea had actually started an open sourced project along these lines:
. . . but I don't think he's had much time to work on it lately. That should get you started.
....
Maybe what we need here is CDM Labs, where we get a group of us to work on this and blog our results! I'm game!
Yes its true, I am mega busy so no time to update the Sonicforms site, although expect a revamp some time after the Cybersonica festival.
By the way Jaymis, physical interface isnt a made up term
I've used pretty much all modular tracking software now, by far the best one for you to use is ReacTivision. Its the reason I stopped working on an open source system, as there is a team of people working on this one and have recently released it.
http://www.iua.upf.es/mtg/reacTable/?software
If you do create any sketches or soundtoys using a tangible interface, post them on the Sonicforms website and it'll be added to the gallery section for the updated site. |
_________________
http://www.pixelsumo.com
http://www.chrisoshea.org |
|
| Back to top |
|
| Jaymis |
| Posted: Tue Apr 11, 2006 11:23 am |
|
|
|
Site Admin
Joined: 04 Feb 2006
Posts: 260
Location: Brisneyland, Australia
|
chrisoshea wrote: By the way Jaymis, physical interface isnt a made up term
Obviously  What I meant is that people making cutting edge projects tend to make up cool names for what they're doing as they go along, so it's often hard to find similar concepts because each developer has their own nonclamenture. All terms were made up once upon a time. Most of the ones I use in my work didn't exist 5-10 years ago. |
|
|
| Back to top |
|
| madhatter |
| Posted: Wed Apr 19, 2006 7:43 am |
|
|
|
Joined: 19 Apr 2006
Posts: 47
Location: Bristol, UK
|
thompsotd wrote: Anonymous wrote: It would be great to get an open sourced approach to this.
If you're on Windows, EyesWeb is a really nice open-source package.
Cool! Thanks for that, will certainly give it a try. |
_________________
Weird Electronic Noises
Velkro Smaak on MySpace.com |
|
| Back to top |
 
|
| PeterKirn |
| Posted: Thu Jun 08, 2006 12:38 pm |
|
|
|
Site Admin
Joined: 03 Feb 2006
Posts: 822
Location: New York, NY
|
I'm surprised we never got into RFID tags or (better yet) pattern recognition on the objects themselves here . . . we'll have to get this thread going again.
Also, for motion analysis that isn't concerned only with objects on tables, I'm starting to really like Myron / JMyron. I thought it was always "a Director thing" and I don't like working in Director much, but it's migrating in other directions -- Java, Processing, and even Jitter. (And, happily, better Windows and Linux support!)
http://webcamxtra.sourceforge.net/
. . . even the "webcamxtra" name is likely to get dropped. |
|
|
| Back to top |
|
|
