Using UE4 memory allocation in third party libs

Hi guys,

I have a problem and need community help. I’m trying to use third party library (in particular google protobuf), but have an issue with iOS shipping build like described here https://answers.unrealengine.com/questions/869991/weird-behavior-with-stdstringstream-on-iphone-xs.html . According to workaround I decided to replace STL allocator and use UE allocation (FMemory::Malloc FMemory::Free etc.), but have an issue with compiling protobuf library - I have to link implementation of UE memory functions, how can I do this ? Does UE4 have compiled static libs or can I get it from source ?

Thanks a lot for your assistance! This issue is a real pain :frowning: So as I understood you don’t link protobuf library to your Plugin and just use generated protobuf code in your plugin ? I’m not expert of protobuf, but do this generated files contatin all necessary code ? I thought that they need some code from protobuf library

Hi, You don’t need to rebuild the Protobuf library. it would be a tough work since you’re linking Unreal’s library with Protobuf so I’m not sure that it is a good way to fix this.

My Team made a script (.sh/.bat) to replace the std containers that are in generated code. (in pb.cc) You can also see that script files too in the repository.

If you’re using Cmake or similar build system, You may have to add “add_custom_command()” and make it executes every time when you generate these pb.cc files.

Yes. The only thing you have to do is, replacing containers in generated files(not Protobuf source code). Please make sure that you need a wrapping allocator before replacing(you can see the allocator class in my team’s repository too).

Hm… Just tried as you described - replace containers in generated files, but hten when I tried to build the plugin got linker errors that it can’t find protobuf function implementations :frowning: Maybe I need to geenerate protobuf files with some options ? Will be appreciate for your help.

Can I get your error message or piece of code that you replaced? It will be helpful to find out the solution.

Like an example of errors

Error LNK2019 unresolved external symbol “public: __cdecl google::protobuf::internal::LogMessage::LogMessage(enum google::protobuf::LogLevel,char const *,int)” (??0LogMessage@internal@protobuf@google@@QEAA@W4LogLevel@23@PEBDH@Z) referenced in function “public: __cdecl google::protobuf::internal::ArenaImpl::ArenaImpl(struct google::protobuf::ArenaOptions const &)” (??$?0UArenaOptions@protobuf@google@@@ArenaImpl@internal@protobuf@google@@QEAA@AEBUArenaOptions@23@@Z) totalpoker D:\projects\totalpoker\Intermediate\ProjectFiles\poker-engine.lib(message_utils.obj) 1
Error LNK2019 unresolved external symbol “public: __cdecl google::protobuf::internal::LogMessage::~LogMessage(void)” (??1LogMessage@internal@protobuf@google@@QEAA@XZ) referenced in function “public: __cdecl google::protobuf::internal::ArenaImpl::ArenaImpl(struct google::protobuf::ArenaOptions const &)” (??$?0UArenaOptions@protobuf@google@@@ArenaImpl@internal@protobuf@google@@QEAA@AEBUArenaOptions@23@@Z) totalpoker D:\projects\totalpoker\Intermediate\ProjectFiles\poker-engine.lib(message_utils.obj) 1
Error LNK2019 unresolved external symbol “public: class google::protobuf::internal::LogMessage & __cdecl google::protobuf::internal::LogMessage::operator<<(char const *)” (??6LogMessage@internal@protobuf@google@@QEAAAEAV0123@PEBD@Z) referenced in function “public: __cdecl google::protobuf::internal::ArenaImpl::ArenaImpl(struct google::protobuf::ArenaOptions const &)” (??$?0UArenaOptions@protobuf@google@@@ArenaImpl@internal@protobuf@google@@QEAA@AEBUArenaOptions@23@@Z) totalpoker D:\projects\totalpoker\Intermediate\ProjectFiles\poker-engine.lib(message_utils.obj) 1

Something like this

Error LNK2019 unresolved external symbol “public: __cdecl google::protobuf::internal::LogMessage::LogMessage(enum google::protobuf::LogLevel,char const *,int)” (??0LogMessage@internal@protobuf@google@@QEAA@W4LogLevel@23@PEBDH@Z) referenced in function “public: __cdecl google::protobuf::internal::ArenaImpl::ArenaImpl(struct google::protobuf::ArenaOptions const &)” (??$?0UArenaOptions@protobuf@google@@@ArenaImpl@internal@protobuf@google@@QEAA@AEBUArenaOptions@23@@Z)

There are more then 100 such errors (145 to be precise), seems it depends on proto library, and looking into source code of generated files they have include to proto library and I believe they should depends on library code.

Just replace std::string from generated files to custom type with custom allocator, and replaced plugin dependency in Build.cs from protobuf.lib

Hm… and one more question - I can see some protobuf source code in your plugin source directory, why did you put it there ? Seems it’s for resolving necessary funtions ?

ok, maybe I’m wrong :frowning: which Prptobuf version do you have anyway? Please note that this plugin only supports version2 currently.

Could you try with version2.6? I’m not sure version 3.6 will work because my team did not test it. As you can see those commits that are posted, we didnt touch any source code.

@gigone.lee Could you assist please ?

If ‘Protobuf source code’ means a bunch of pb.cc files, then yes. because iFun server needs the same protocol to receive the message from the client.

Since I’m not working as a client programmer, but As far as I know, We don’t manage the Protobuf library source code in the repository.

Could you double check that you replace the std containers that are using std::allocator, to the custom allocator that is using Unreal’s function?

You also need to add your custom allocator header in the Protobuf header so they can be compiled(Sorry I think I didn’t mention this before)

We added the funapi_std_allocator.h file in protobuf/stubs/common.h like the bellow code.

#define GOOGLE_PROTOBUF_COMMON_H__

#include <funapi_std_allocator.h>

First of all thanks for your help! No I mean folder “google” with some protobuf sources in your plugin - “engine-plugin-ue4-experimental\Plugins\Funapi\Source\Funapi\Private\google”, seems protobuf compiling with your plugin somehow…

Yes, I could see this line and made this change and in my version, but I beleieve it’s not the reason of linker errors

We use 3.6.1, so you’ve added proto sources to your plugin, right? If yes then one more question - how did you compile it ? Because I tried to add proto source to Plugin, but got a lot of errors regarding c++17 features, how did you switch UE4 to compile c++17 code ?

I tried to add your proto sources and compile proto with protoc from 2.6, but again I got compilation errors regarding c++17 feature, couldn’t find any useful information how to add c++ 17 feature compilation in google :frowning:

okay. let me have some time to find out the issue. Any detilas(like protobuf version, build environment) will be helpful. :slight_smile:

Thanks a lot!!! Will wait for any news :slight_smile: