Crash when Deserializing TArray into Json

Development Programming & Scripting Multiplayer & Networking
, , , , , ,
1

So I am currently creating functions that will allow me to take in an array of bytes through a Network Socket, deserialize it back into Json, and then Parse said Json. However, when I try to use the deserialize Function, the entire Engine Crashes, and I am given a link to send a crash report.Here’s the .rar file to my crash report.

The Code Chunk that is most relevant is below:

TArray<uint8> ReceivedData;
	uint32 Size;
	uint8 receivedData[100];
	int32 Read = 0;

		while (Socket->HasPendingData(Size))
		{
			ReceivedData.SetNumUninitialized(FMath::Min(Size, 65507u));

			Read = 0;
			Socket->Recv(ReceivedData.GetData(), ReceivedData.Num(), Read);

		}

		if (ReceivedData.Num() <= 0)
		{
			VShow("Error, the Server isn't streaming data");
		}
		FString output = StringFromBinaryArray(ReceivedData);
		FString debugData = BytesToString(receivedData, Read);
		Print("Got Message" + debugData);
		TSharedPtr<FJsonObject> JsonObject;
		TSharedRef< TJsonReader<> > Reader = TJsonReaderFactory<>::Create(debugData);
		//If the line below this is commented out, the program doesn't crash
                bool result = FJsonSerializer::Deserialize(Reader, JsonObject);
		if (!result)
			Print("Error, Json Deserialization Failed");

My Implementation in my Blueprint currently Connects to the other IP Address, delays for a second, then calls the above function. I know that the Socket works fine, as I am getting a connection, along with the data.

The Json being sent is being sent like this, according to the developers of the Software I am trying to access:

{"animationValues":
{
	"mouth_rightMouth_stretch":0.0000000000000000,
	"mouth_leftMouth_narrow":0.00000000000000000,
	"mouth_up":0.0000000000000000,
	"mouth_leftMouth_stretch":0.0000000000000000,
	"mouth_rightMouth_narrow":0.00000000000000000,
	"mouth_down":0.00000000000000000,
	"mouth_upperLip_left_up":0.0000000000000000,
	"mouth_upperLip_right_up":0.0000000000000000,
	"mouth_lowerLip_left_down":0.0000000000000000,
	"mouth_lowerLip_right_down":0.0000000000000000,
	"mouth_leftMouth_frown":0.0000000000000000,
	"mouth_rightMouth_frown":0.0000000000000000,
	"mouth_leftMouth_smile":0.00000000000000000,
	"mouth_rightMouth_smile":0.00000000000000000,
	
	"eyes_lookRight":0.0000000000000000,
	"eyes_lookLeft":0.00000000000000000,
	"eyes_lookDown":0.0000000000000000,
	"eyes_lookUp":0.00000000000000000,
	"eyes_leftEye_blink":0.00000000000000000,
	"eyes_rightEye_blink":0.00000000000000000,
	"eyes_leftEye_wide":0.0000000000000000,
	"eyes_rightEye_wide":0.0000000000000000,
	
	"brows_leftBrow_up":0.0000000000000000,
	"brows_leftBrow_down":0.00000000000000000,
	"brows_rightBrow_up":0.0000000000000000,
	"brows_rightBrow_down":0.00000000000000000,
	"brows_midBrows_up":0.0000000000000000,
	"brows_midBrows_down":0.00000000000000000,
	
	"jaw_open":0.0000000000000000,
	"jaw_left":0.0000000000000000,
	"jaw_right":0.00000000000000000,
	
	"mouth_phoneme_oo":0.0000000000000000,
	"mouth_right":0.0000000000000000,
	"mouth_left":0.00000000000000000,
	"mouth_phoneme_mbp":0.0000000000000000,
	"mouth_phoneme_ch":0.0000000000000000
	
},

"headRot":[0.0,0.0,0.0]}

I have no reason to assume there is anything wrong on their side, as they offer a plugin to do the same job. However, it costs $1500, so that’s out of the question. I don’t really see how I can be doing anything wrong in this situation, as the code compiles correctly without any errors.

2

Can I bump this? Its really important that I get this resolved quickly

3

Hi IrishMTS,

Please do not bump threads that have not gone unanswered for at least 4 days. This will give us time to take a look at the question and determine what may be occurring. If after 4 days you have not heard from anyone, please feel free to bump the thread once.

I am assigning a member of our support team to this bug so we can assist you further.

4

Perfect, sorry.

5

What is the callstack of the crash?

6

Where would I find that information?

7

In Visual 's “Call Stack” window, in the CrashReporter dialog window (if shown), or in the game’s log file (please attach to this thread!)

8

This is the .log file from the crash report. This is the log from the directory (ProjectName\Saved\Logs)

9

Can you reattach the second file? It seems like it didn’t make it through.

10

Is it working now?

11

Yes, but these don’t look like the right logs. A new log file is created each time you run the Engine (Editor or Game). Can you please reproduce your crash and then send the newest log file in /ProjectName/Saved/Logs ?

12

That should be it there, must have clicked the wrong one by accident

13

Hi gmpreussner. I’ve been playing around with perhaps using a different library to deserialise the Json, and while I was playing around seeing how I could fit it in, I found that I got the same crash, and from the looks of things, the same fatal error:

LogPhysics:Warning: PHYSX: …..\PhysX\src\NpScene.cpp (2946) 8 : PxScene::unlockWrite() called without matching call to PxScene::lockWrite(), behaviour will be undefined.

I’ve attached the function as it is now, and as can be seen, the deserialising functions have been commented out. I’m not actually sure anymore whats causing the crash, but I know that if this function isn’t called, the crash doesn’t occur. Here’s the new log file

void ANetworkPlayerController::ReceiveData()
{
	TArray<uint8> ReceivedData;
	uint32 Size;
	//uint8 receivedData[1000];
	int32 Read = 0;

		while (Socket->HasPendingData(Size))
		{
			ReceivedData.SetNumUninitialized(FMath::Min(Size, 65507u));

			Read = 0;
			Socket->Recv(ReceivedData.GetData(), ReceivedData.Num(), Read);

		}

		if (ReceivedData.Num() <= 0)
		{
			VShow("Error, the Server isn't streaming data");
			//return;
		}
		FString output = StringFromBinaryArray(ReceivedData);
		const char *char_pointer2 = reinterpret_cast<char*>(ReceivedData[0]); 
		VShow("Is it working?", char_pointer2);
		UE_LOG(LogTemp,Log,TEXT("%d"),ReceivedData[0]);
		//TSharedPtr<FJsonObject> JsonObject;
		//TSharedRef< TJsonReader<> > Reader = TJsonReaderFactory<>::Create(output);
		//bool result = FJsonSerializer::Deserialize(Reader, JsonObject);
		//if (!result)
		//	Print("Error, Json Deserialization Failed");
}
14

The crash is in PhysX and has nothing to do with your code. It seems that you have a memory corruption somewhere. How is StringFromBinaryArray() implemented?

15

Its a simple reinterpret_cast, just the one line essentially:

FString ANetworkPlayerController::StringFromBinaryArray(const TArray<uint8>& BinaryArray)
{
	//Create a string from a byte array!
	std::string cstr(reinterpret_cast<const char*>(BinaryArray.GetData()), BinaryArray.Num());
	return FString(cstr.c_str());
}

What’s the next stage? Is this something that can be fixed at my end?

16

You need to figure out which part of your code is corrupting memory. What does VShow() do?

Also, why are you copying the string data? TJsonReader can process the data directly. Create a FMemoryReader archive around the TArray and then create a Json reader from it. Pseudo-code:

FMemoryReader MemoryReader(ReceivedData.GetData());
TJsonReader<> JsonReader = TJsonReader<>::Create(&MemoryReader);
17

VShow is a function that just sets up ClientMessage in a particular way, nothing particularly unusual there.

Having set up the code using an FMemoryReader archive, the code doesn’t crash, but I think that’s more because the deserialize function now returns false. Would you have any idea why that might be? I unfortunately don’t know whether thats due to my code, or due to something on the other end, but I have been assured that all works fine on the server program.

The New code:

void ANetworkPlayerController::ReceiveData()
{
	TArray<uint8> ReceivedData;
	uint32 Size;
	//uint8 receivedData[1000];
	int32 Read = 0;

		while (Socket->HasPendingData(Size))
		{
			ReceivedData.SetNumUninitialized(FMath::Min(Size, 65507u));

			Read = 0;
			Socket->Recv(ReceivedData.GetData(), ReceivedData.Num(), Read);

		}

		if (ReceivedData.Num() <= 0)
		{
			VShow("Error, the Server isn't streaming data");
			//return;
		}
		FMemoryReader MemoryReader(ReceivedData);
		TSharedRef< TJsonReader<> > Reader = TJsonReader<>::Create(&MemoryReader);
		bool result = FJsonSerializer::Deserialize(Reader, JsonObject);
		if (!result)
			Print("Error, Json Deserialization Failed");
}
18

false usually indicates that the Json string is malformed. You can validate it here: http://jsonlint.com/

If the string is OK, you’ll have to step through the code to see what’s happening.

19

I’m wondering, whats the likelihood that I am taking too small a packet size in as the minimum before reading it? Is there anything in that function that might suggest that the Json is getting cut off in the middle somewhere?

20

Okay, I spoke too soon. Opened it up today and its still crashing. Is Buffer Overflow something that I may have to be worried about? Does the string that I’m receiving cause memory corruption perhaps?