To register and login, use your Google, Twitter, Facebook, LinkedIn, or OpenID credentials.

This is allowing us to stop most spam registrations. We've deleted most of the spam accounts that got through, and we're closely watching for more.

Integrating libpd with the Unity Game Engine

arcadiadivinearcadiadivine Posts: 3
edited November -1 in Pd Everywhere
I'm still relatively new to this, so you'll have to forgive my ignorance, but how would I integrate libpd with the Unity Game Engine? More specifically the free version. Is there any specific process involved? Would it even be possible with the free version of Unity? Here is a link to the website. http://unity3d.com/unity/licenses
«1

Comments

  • pbrinkmannpbrinkmann Posts: 685 ✭✭
    FRACT uses libpd with Unity 3.5 (fractgame.com). Henk Boom, one of the developers, says he'll write a blog post about it soon.
  • Cool, that's a relief knowing it can be integrated into Unity 3.5. Do you have any tips or suggestions on integrating libpd into the free version of Unity 3.5? I recently bought the Android and iOS basic licenses because I eventually want to develop for those platforms. Well actually I got them for free because of the sale Unity Technologies was having, but that's not the point of the conversation. I figured it was either libpd or Csound. I'd prefer to use libpd because it doesn't seem like it takes nearly as much effort as integrating Csound into Unity 3.5. Like I said before though, do you have any tips or suggestions on integrating libpd with Unity 3.5? Any amount of help would be greatly appreciated!
  • pbrinkmannpbrinkmann Posts: 685 ✭✭
    I have personally never looked at Unity. All I know about libpd and Unity is what I learned from Henk Boom's tweets (@henkboom). I'm looking forward to reading his blog post.
  • reakinatorreakinator Posts: 301
    Wow, I didn't realize FRACT used libpd, I've been following it's progress and wondered what they used for sound synthesis - cool!

    @John Colgrove, can you find any documents at tying into Unity's audio layer through it's plug-in interface?  I can only find this little bit of info from the Unity manual:
    http://unity3d.com/support/documentation/Manual/Plugins.html

    So, there is a C plug-in interface, and you need to find out how it provides and audio callback, where at you could tie in libpd with libpd_process_float() and friends.

    Or, you can wait for this anticipated blog post. :)
  • pbrinkmannpbrinkmann Posts: 685 ✭✭
    Yes, FRACT uses libpd: http://createdigitalmusic.com/2012/02/fract-3d-adventure-game-played-with-synths-and-sequencers-myst-meets-music-making/

    From what I gather from Henk's tweets, there are concurrency issues to consider when using the C plugin interface, and so it may be a bit tricky to get right.
  • pbrinkmannpbrinkmann Posts: 685 ✭✭
    Thanks for linking to the Unity manual, Rich! I took a look, and it appears that we really ought to add C# bindings for libpd. We already have a current pull request (https://github.com/libpd/libpd/pull/6) that adds support for DLLs to the build system, so that should help a bit. Any volunteers?
  • joewhite4joewhite4 Posts: 34
    @John Colgove coincidentally i'm just on my way back from the Amsterdam MusicHackDay where I was working on integrating ZenGarden as a plugin in Unity. I've got it running but have some issues at the moment with opening patches on iOS.

    As Peter pointed out, Henk has already integrated libpd in FRACT. Hopefully he'll get his blog post up soon and I'd also like to document the process I went through. In the meantime here's some info that you might find helpful. 

    - a C/++ library has to be integrated as a plugin to work in Unity
    - The plugin is just a simple C#/JS script
    - Using plugins requires Unity Pro UNLESS you're building for iOS or Android
    - The iOS/Android add-ons are currently free (until April I think) so get them quick!
    - The library just needs a C interface, I would say it doesn't seem neccessary for C# bindings in libpd, but I could be wrong.
  • curbycurby Posts: 2
    @John Colgrove I've also been tinkering with this and had moderate success using Unity to run patches using libpd as a plugin on Android.

    I say moderate success in that I've gotten libpd to initialise, load a test patch without error and output audio on my android phone. The audio is mostly correct, however there are some artifacts and the audio seems to jump a bit.

    For those wondering the method I've used to tie libpd into Unity's audio layer is using the AudioClip.PCMReaderCallback >> http://unity3d.com/support/documentation/ScriptReference/AudioClip.PCMReaderCallback.html
    I'm using this callback ( which from the little to no documentation about it, have assumed is a buffer request from the audio card via FMOD ), to 'pump' the libpd process function.  However I'm unsure of how 'in synch' this is with libpd (it seems not very) and whether it's being called at the correct times.  Does a buffer request from the audio card seem like a viable method to call libpds process functions?

    Also as an aside I just wanted to say a big thanks to all who've contributed to the libpd project, it's really inspiring work!
  • pbrinkmannpbrinkmann Posts: 685 ✭✭
    @Luke,
    Yes, that's exactly what the process functions are for. There shouldn't be any timing issues because libpd's only measures time in terms of the number of samples requested via libpd_process*. The artifacts you're experiencing are probably due to mismatched buffers; make sure that Unity and libpd agree on buffer size, endianness, and the alignment of channels.
  • curbycurby Posts: 2
    @Peter OK, thanks for verifying.  I'll take another look at those buffers.
  • berengerberenger Posts: 56
    actually Unity for android and iOs is free until 8th of april : https://store.unity3d.com/index.html

    so get it while it's hot and follow this topic !
  • now, is there any c# file around? otherwise i would set one up...
  • pbrinkmannpbrinkmann Posts: 685 ✭✭
    Several people have asked about C# support, but so far nobody has done it, at least as far as I know. It would certainly be good to have.
  • yes, i have working c# bindings now... will fork the repo in github and open a pull request.

    i just have a little problem. i am trying to set the printhook too but it looks like its not called by libpd. for that i have created a setter method and a test method in the C code because .Net/Pinvoke can't set dll fields. the test method works, but when i load a patch with prints i don't get anything...
  • pbrinkmannpbrinkmann Posts: 685 ✭✭
    Thanks, Tebjan! I left some preliminary feedback at GitHub.
  • pbrinkmannpbrinkmann Posts: 685 ✭✭
    By the way, about having libpd call methods in C#, it appears that COM Interop is the way to go: http://msdn.microsoft.com/en-us/library/zsfww439.aspx
  • Very interested about this, too. This could open up heaps of new opportunities for in game music!
  • pscmtlpscmtl Posts: 23
    Hi!
    1) I have compiled libpd using MinGW:
    make csharplib = libs/libpdcsharp.dll!
    2) Now in SharpDevelop I have compiled csharp/LibPDBinding.csproj = bin/Debug/LibPDBinding.dll!
    3) In SharpDevelop I have compiled csharptest/LibPDBindingTest.csproj = LibPDBindingTest.dll!
    Now I am looking for help: Anyone successfully use this in Unity or even a simple C# audio demo? I see that tebjan have a folder called csharp in samples: https://github.com/tebjan/libpd/tree/master/samples/csharp but it doesn't work with LibPDBinding.dll
    Thanks!
  • pscmtlpscmtl Posts: 23
    Hi again!
    I think the first step is a simple console application (no sound) then I will try using the C# PortAudio or NAudio. Right now I have an error:

    System.AccessViolationException: Attempted to read or write protected memory. This is often an indication that other memory is corrupt.
       at LibPDBinding.LibPD.openfile(String basename, String dirname)
       at LibPDBinding.LibPD.OpenPatch(String filepath) in C:\MinGW\msys\1.0\home\psc\libpd\csharp\Native\LibPDNativeMethods.cs:line 143

    I am using SharpDevelop.
    Thanks!
  • pscmtlpscmtl Posts: 23
    Resolved the read or write protected memory error: when using OpenPatch you need to add ./ if the patch is in the same directory of your executable:

    OpenPatch(@".\test_csharp.pd");

    Now trying to import those DLL in Unity...
  • hey listers, new guy here - greetings!

    has anyone got libPD working with Unity3D Free? alpha build, beta build, whatever. i know tebjan created a version of libpd with C# bindings, but i've no idea as to what to do to make it work on a Mac, as there is no documentation.

    i've heard precious little since then. patrick sebastien got it to barely work on PC i believe. anyone else have any idea? i've been in contact with someone who claims it's working fine in a prototype form, but hasn't released anything so far to check out, and i know Henk Boom uses it in Fract but i haven't heard anything there either. he was supposed to detail a blog about it 'soon' back in March when i last read about it.

    anyway, any information appreciated. thanks for any help!

  • rrodkinrrodkin Posts: 5

    I too have been waiting for the much-anticipated post from Henk, but I suspect that with new fatherdom and game dev, it won't be coming any time soon.

    Good news, though, I think. After yet one more google, I found this post, and looking around here, it doesn't look like anyone has picked it up. I haven't tried ANY of this out yet, so caveat emptor, but it does look promising, and at least offers some transparency into a workable solution:

    http://blog.tridek.com/post/29834155461/using-libpd-with-unity3d-on-mobile-devices-part-1

    I also noticed that a poster in this thread -- filippo.peccoz -- is involved with the tridek project, and has posted an article as well, which looks very interesting:

    http://blog.tridek.com/post/29055895421/welcome-to-trideks-audio

    @Filippo: Any news or insights re: the above? Tridek looks like a pretty cool project, would love to hear more.

    Thanks!

  • rrodkinrrodkin Posts: 5

    Arrrgh. Looks like I was a bit over-eager -- according to the Tridek blog post, they've only got Unity and libpd running via TCP in Edit mode . . . which is essentially the same method as using OSC -- all well and good until runtime.

    I suppose they'll need to figure it out sooner or later, and it will be fun and interesting to observe . . . and I wish them best of luck!

    In the meantime, looks like we're back to waiting for Mr. Boom!

  • rrodkinrrodkin Posts: 5

    WOW! that is quite a gift . . . thank you!!

  • pscmtlpscmtl Posts: 23
    edited August 2012

    np but it's still a work in progress! it would be very nice if people could fork and pull request so that we have a solid integration. i am also looking for a way to use audio input (soundcard input) and compiling for android and iOS. i have finally managed to use statically linked external (look at my fork of libpd).

    the main issue is that pd / libpd can only be use in a single instance, meaning that we have to switch~ patch to avoid glitches and i am have issue with expr~ / sig~ timing.

  • bzgebbzgeb Posts: 1

    I wrote a post about embedding libPD into a Unity iOS app http://bronsonzgeb.blogspot.ca/2012/08/embedding-pure-data-into-unity-ios-app.html which also covers how to embed libPd into an iOS app. I wasn't aware that Patrick Sébastien had created libpd4unity at the time.

  • the biggest thing about Patrick's approach is he compiled the wrapper/DLL for the libpd base in Windows and the Unity project he uses crashes hard and completely on a Mac. it looks like yours was made for a Mac, so i will definitely give this one a try when i have a moment.

    i like filippo's approach in terms of designing or prototyping, but my idea would be to run a separate and more generally functional GUI sending commands to libpd to dynamically create Audio Sources and manipulate audio parameters, rather than using the PD canvas to do it. my idea is to be able to use this GUI with other game engines and APIs.

    i am curious if at some point you'll be able to run multiple instances of libpd. i suspect it will be a processor hog for smaller devices, but i'd be happy to be proven wrong.

  • metaphysicianmetaphysician Posts: 4
    edited August 2012

    @rrodkin - i think you should take a look at the post again from filippos crew (it's sebastien who is programming it). it looks like they do have it working on Android and iOS, they simply prototype in PD/OSC. and yes there's only one instance of libpd. they have directions on what exact code to modify/adapt in iOS Xcode and Android.

Sign In or Register to comment.