Audio Reactivity and RealSense Integration

Hey Smudgy,

The first thing that I noticed is that we used SR435 during development, so I was thinking that SR300 might not be supported. But then, you are saying that the camera itself works when you use it in other sample projects? Which sample project was that? I am not aware of any that use RealSense depth feed (but then again, I might be wrong).

In short, can you please confirm that you can see the depth feed from your RealSense in Layer Layouts in a sample project (or a project built from scratch)?

Thanks!
Mitja

Hey Mitja,

Thank you, as ever, for your help.

Did the attached project not come through, please find attached now. This project uses
both the depth field and RGB streams.

The attached project yields the following resultant outputs:

Hope you can help more?

What would have I have tio do with the project to convert it for use for the SR300 camera, please?

Kind regards,

Smudgy

RealSense.la (8.54 KB)

Hi,

We are going to have to have a more in-depth look at this, I am afraid. From the initial looks of it, things get a bit sketchy when opening projects saved with different RealSense camera.

At the moment, the only thing I would recommend is for you to recreate the project from scratch.

I’ll update this thread with new findings.

Thanks,
Mitja

Thank you Mitja,

I’ll recreate once I’m home later and let you know the result.

All the best,

Smudgy

Hey Mitja,

I hope you’re well… Happy Friday!

I have rebuilt the project from scratch (please find attached) and still the issue persists. How does your project receive the data from the webcam and what possible differences are there between the SR300 and D435 in relation to the datastream inputs into LA?

I only recently purchased my BlasterX Senz 3D Webcam (Intel SR300), though if I need to get the D435/D435i (which one would you choose?), I will; I really want to get my head round this and do it all properly and if it means upgrading the webcam, then so be it.

I’m interested in purchasing a suitable LA license and have requested the associated prices for each type.

Have a great weekend!

Kind regards,

Dominic

TheDepthsOfYou313_Rebuild.la (19.1 KB)

Hey Dominic,

Happy Friday to you too!

I’ve opened your file and it works on my side with D435

I’m afraid if SR300 doesn’t work with Lightact then there’s not much we can do at this point (aside from you sending your camera over so we can integrate it and send it back :slight_smile: ) I am just curious why it seems that it works with other projects (if I understood you correctly). If yes, then at which point does it stop working? Please note that knowing this won’t really help us fix the issue without having the camera here, so if you want to be up and running asap I am afraid you’ll have to buy a new camera. Sorry about that.

We are using D435, so if you want to be 100% certain, you’ll probably want to use the same one.
IMG_20190125_145810

I know @Jason_Ellemor (let’s see if this ‘@’ thing works) has been using RealSense with Lightact successfully. Jason, which RealSense are you using if you don’t mind me asking?

Thanks!
Mitja

Thank you Mitja =].

Only 3 hours of shift left… and counting.

This is most bizarre. There’s two LA projects it works for, the one named ‘RealSense’ of I have attached already and your other example project, ‘Multiverse’, where you wave your hands in front of the webcam and the small squares on a network grid light up.

TheDepthOfYou project fails when on Devices (F7) window, when the camera RealSense SR300 #Serial Number input is selected.

When the new webcam comes, I’ll send the SR300 to you to see if you can get it working.

Oh well that was $189 dollars down the drain for the SR300. Just about to order the D435i, so I’ll give you an update when it arrives and I have it setup.

All the best and enjoy the weekend =].

1 Like

I am using Real Sense D435 without any issues. I’ve found that you can have issues if you disconnect the real sense camera during a project. Try opening the realsense depth viewer (or other tools) to make sure the camera is working, then kick off LightAct and it will generally work without an issue

1 Like

Thank you Jason for the response.

I have just ordered the D435i, so this will just work as you and LA say.

The bizarre thing is I have two other LA projects that use the SR300 and they both work fine. Also does the Intel Viewer and other apps that use the camera.

I have an up to date system and have USB3.1 ports, so when the USB connection error comes on in the LA project upon selecting the camera device input, it really puzzles me.

Hey Mitja,

I hope you’re ok and all is well?!

Ok, finally got my D435i and setting it up, calibration completed successfully.

Opened the LA project, TheDepthsOfYou and saved as TheDepthsOfYou 313.
Opened the UE4 project map, depthsOfYou and hit run, though nothing appears on planes.

On opening Devices window:

If I hit ‘Refresh List’, the camera doesn’t show.

The D435i works with the Intel RealSense Viewer V2.17.1:

Any ideas, please?

All the best,

Smudgy

Hey,

Hmm that’s weird. Just to be sure, can you try to:

  1. Restart your pc

  2. Start Lightact and open Devices window. Don’t open the Intel real viewer just yet.

  3. Insert RealSense Reader node and see if you see your RealSense in the list?

Sometimes I got somewhat similar symptoms on a relatively old PC with what I guess were older USB ports/controllers so the RealSense connection was sometimes a bit dodgy. What helped me on that particular pc was to get an additional PCIe expansion slot. I’m not saying this is the way you should go though - just stating my experience.

Let me know how it goes please.

Best regards,
Mitja

Hey M,

Tis weird too!

The camera doesn’t show up in list at all.

Other programs can use it though.

So I followed steps 1 to 3 and nada.

My machine is new with USB3.1, type C ports, the mind boggles!

Kind regards,

Smudgy

Hey M,

Tis weird too!

The camera doesn’t show up in list at all.

Other programs can use it though.

So I followed steps 1 to 3 and nada.

Please see attached machine spec.

Kind regards,

Smudgy

(Attachment Zeus_11-02-2019.nfo is missing)

Hey Mitja,

In relation to your D435 camera settings for your project, are they like this:

.

And are your drivers like this:
image

image

This is so confusing, I really would just like to develop as opposed to fix things, lol.

All the best,

Smudgy

1 Like

Hey Mitja,

Would it be possible to for you to test the D435 using the latest version of firmware (5.11.01.00) and SDK (V2.18.1), from Intel, please?

Hey Smudgy,

Yes, I’ll look into it now and let you know.

The firmware number is the same but I’ll update the SDK version now. I was using 2.17.

image

image

I’ll let you know the news once I install SDK 2.18.

Cheers,
Mitja.

Hello again,

‘Unfortunately’ it worked even with SDK 2.18. on my end.
image

image


Procedure with SDK 2.18.1


Procedure with SDK 2.17.0

It seems the only difference now between our setups is the fact that I am using D435 and you are using D435i. I looked a few posts above, and it seems Jason was also using D435. As I don’t have it here atm, its a bit hard to test, but I’ll give it a try.

If you could send it over, we’ll be happy to have a look at it though. :slight_smile:

One more thing to try: could you please install SDK 2.17.0 and give it a go?

Thanks!
Mitja

One more thought: I noticed you are using USB 3.0 whereas we are using USB 3.2.
image

On one of the computers I do have USB 3.0 and I remember I had some problems with large bandwidth devices such as RealSenses and Kinects (although, as if on purpose, I can’t reproduce it on USB 3.0 today either).

Is there any chance you could try it out on a computer with a USB 3.2 port?

Lastly, I am going to send you testing a Lightact.exe file in a separate message. While I don’t expect this exe to fix your issues, it might give us some clues on what’s happening.

Thanks,
Mitja

Hey Mitja,

Most confused now, as it works with the latest SDK 2.18.1; I did try the 2.17.1 and nada, still the same.

This is bizarre too:
image

You have 3.2 and I have 3.0, even though I have USB3.1, Type C ports. I states it’s VR Ready and that’s the one I’m using currently, mind you, I have tried the non VR Ready ports too.

Noticed too, that you have Driver Version: 10.0.17134.1, whereas I have 10.0.17763.1.

Please find attached a copy of my System Info, maybe this may shed some light on things:
ZEUS_12-02-2019_SystemInfo.txt (2.1 MB)

All the best.

Well, that’s good. :slight_smile: These confusing sometimes-it-works-sometimes-it-doesn’t happened on our end with one of the older computers where the USB controller (I guess) is not the latest. On this computer I also have to wait a bit longer (5 seconds max though) for the RealSense list to refresh.

I hope you have plain sailing from now on, Dominic! :+1: Thanks for sticking with it!