x

Search in
Sort by:

Question Status:

Search help

  • Simple searches use one or more words. Separate the words with spaces (cat dog) to search cat,dog or both. Separate the words with plus signs (cat +dog) to search for items that may contain cat but must contain dog.
  • You can further refine your search on the search results page, where you can search by keywords, author, topic. These can be combined with each other. Examples
    • cat dog --matches anything with cat,dog or both
    • cat +dog --searches for cat +dog where dog is a mandatory term
    • cat -dog -- searches for cat excluding any result containing dog
    • [cats] —will restrict your search to results with topic named "cats"
    • [cats] [dogs] —will restrict your search to results with both topics, "cats", and "dogs"

[4.7.3] Saving a UAsset with a byte array expands 32x

I have a fixed size byte array as a uproperty which is some opaque platform agnostic data that I want to serialize into a uasset.

 UPROPERTY()
 uint8    Bytes[BYTE_COUNT];

This issue is that the built in Serializer emits 32+ bytes per entry which turns my ~2 MB of data into ~64 MB on disk as well as takes forever to save.

This is pretty crazy, realistically the serializer should handle uint8/int8 arrays (especially static ones) as a blob. It's totally unnecessary to serialize all these tags for every single element.

Product Version: Not Selected
Tags:
more ▼

asked Apr 08 '15 at 12:08 AM in Bug Reports

avatar image

kylawl
279 14 20 36

avatar image Tim C ♦♦ STAFF Apr 27 '15 at 08:59 PM

Hi kylawl,

Sorry for the delayed response to your post. Are you adding this array to an existing class, or did you create a unique class just for this array? Would you be able to provide the code for the class where this array is located?

Tim

avatar image kylawl Apr 27 '15 at 09:51 PM

Hi Tim, no problem.

The class doesn't need to be anything special, the following will do.

 UCLASS()
 class UOpaqueBytesObject : public UClass
 {
     GENERATED_BODY()
 
     enum { ArrayLength = 1024*1024 }; // 1MB
 
     UPROPERTY()
     uint8 OpaqueByteArray[ArrayLength];
 };

In Engine/Source/Runtime/CoreUObject/Private/UObject/Class.cpp around line 1241, the serializer serializes each element with a bunch of tag info that bloats small data elements.

There are actually huge gains to be made here. All primitive types and even the UE4 math types such as FVector could be condensed into far more optimized blobs.

The amount effort needed to condense this data down was madness. In addition to implementing the custom serialization for all properties in the class, we had to implement this other bizarre Import/ExportCusomProperties stuff in order to make copy and paste work. Why that doesn't use the same serialization system is beyond me?

(comments are locked)
10|2000 characters needed characters left
Viewable by all users

1 answer: sort voted first

Hi kylawl,

After looking into this further, we were able to confirm the results that you are seeing, and agree that there are some possibilities to make some significant gains in performance here. We have logged a ticket to have this area investigated further (UECORE-157).

In the meantime, it was suggested that you could still accomplish what you are trying to do by using a TArray reserved for the appropriate size that you need, instead of using a native array. This would require an extra heap allocation, but should still provide some significant savings overall.

Tim

more ▼

answered May 01 '15 at 08:46 PM

(comments are locked)
10|2000 characters needed characters left
Viewable by all users
Your answer
toggle preview:

Up to 5 attachments (including images) can be used with a maximum of 5.2 MB each and 5.2 MB total.

Follow this question

Once you sign in you will be able to subscribe for any updates here

Answers to this question