Solutions to runtime pass sound data to pygame(Urgent)

Recently I’ve started developing with a UE plugin called Carla, which is a simulator for traffic and for developers to implement their auto-driving algorithm on.

This plugin has a server which is based on UE and a client interface which is based on pygame. And it has a built in interface to pass the camera screen from UE to pygame, but there isn’t an interface for runtime-passing the sound data from UE to pygame, which is my goal. But I couldn’t find any documentation for getting the sound data(and after the attenuation I wish) from a soundcue. Any solutions or suggestions are welcome, thanks!

Sounds interesting. Why is a python client used rather than a UE4 client though? This seems like an odd decision (although presumably done so clients are as lightweight as possible?).

I’ve had a very quick look at Carla and it seems as though they have a camera which captures the scene, writes the pixels to a buffer and streams them, which is then presumably read by the pygame implementation. (see carla/PixelReader.h at 661bc87a0cad9f18c8f66280fedb12bff633dc12 · carla-simulator/carla · GitHub )

Unfortunatly this doesn’t make it easy to capture the sounds. You’d need to write your own system which samples the sound at any given location and then streams it in a similar way. Unreal might have some sort of audio listener built in, but I’ve never come across it. The idea of streaming an image in the way done by Carla is quite unusal, the idea of streaming sound is even more unusual In a traditional game, you wouldn’t stream sounds from the server, you’d just have the client play them in reponse to a message from a server.

Is there a reason you need to persist with the idea of using pygame? Could you just run as a UE client instead?

Thanks for your reply. I think they did so because as you said so they hope to make the clients as light as possible. Besides maybe they didn’t even plan to make passing sounds available.

After thinking of your suggestion, I think running a client as UE is actually a thing. However because we have multiple subprojects and therefore multiple teams of developers working on Carla simulator(which my group is the only one to have use of sound data to analyze the current traffic), I have to make nearly 99% sure that this is gonna work.

Another thing is that Carla client has lots of built-in functions such as impact sensors or crossing-line sensor(when the car contacts the white line or whatnot on the road). If I change the client from the default pygame to my own UE client, it also means that I have to give up those built-in functions. Anyway thanks for your quick reply again! Still any suggestions or discussions are welcomed!