I have a fixed size byte array as a uproperty which is some opaque platform agnostic data that I want to serialize into a uasset.
UPROPERTY()
uint8 Bytes[BYTE_COUNT];
This issue is that the built in Serializer emits 32+ bytes per entry which turns my ~2 MB of data into ~64 MB on disk as well as takes forever to save.
This is pretty crazy, realistically the serializer should handle uint8/int8 arrays (especially static ones) as a blob. It’s totally unnecessary to serialize all these tags for every single element.
Sorry for the delayed response to your post. Are you adding this array to an existing class, or did you create a unique class just for this array? Would you be able to provide the code for the class where this array is located?
The class doesn’t need to be anything special, the following will do.
UCLASS()
class UOpaqueBytesObject : public UClass
{
GENERATED_BODY()
enum { ArrayLength = 1024*1024 }; // 1MB
UPROPERTY()
uint8 OpaqueByteArray[ArrayLength];
};
In Engine/Source/Runtime/CoreUObject/Private/UObject/Class.cpp around line 1241, the serializer serializes each element with a bunch of tag info that bloats small data elements.
There are actually huge gains to be made here. All primitive types and even the UE4 math types such as FVector could be condensed into far more optimized blobs.
The amount effort needed to condense this data down was madness. In addition to implementing the custom serialization for all properties in the class, we had to implement this other bizarre Import/ExportCusomProperties stuff in order to make copy and paste work. Why that doesn’t use the same serialization system is beyond me?
After looking into this further, we were able to confirm the results that you are seeing, and agree that there are some possibilities to make some significant gains in performance here. We have logged a ticket to have this area investigated further (UECORE-157).
In the meantime, it was suggested that you could still accomplish what you are trying to do by using a TArray reserved for the appropriate size that you need, instead of using a native array. This would require an extra heap allocation, but should still provide some significant savings overall.