x

Search in
Sort by:

Question Status:

Search help

  • Simple searches use one or more words. Separate the words with spaces (cat dog) to search cat,dog or both. Separate the words with plus signs (cat +dog) to search for items that may contain cat but must contain dog.
  • You can further refine your search on the search results page, where you can search by keywords, author, topic. These can be combined with each other. Examples
    • cat dog --matches anything with cat,dog or both
    • cat +dog --searches for cat +dog where dog is a mandatory term
    • cat -dog -- searches for cat excluding any result containing dog
    • [cats] —will restrict your search to results with topic named "cats"
    • [cats] [dogs] —will restrict your search to results with both topics, "cats", and "dogs"

Weird behavior with std::stringstream on iPhone XS

Hello, I found something very strange bug with std::stringstream on iPhone XS. I'm still putting in work to reproduce on other iPhone devices but I found it impossible to reproduce on other devices and platforms. the followings are the detailed environment.

UE4 version : 4.21.1

target device : iPhone XS

iOS version : 12.1.1

Xcode version: 10.1

The test scenario is very simple, just put these 4 lines (or maybe 3 lines) into any project and run on iPhone XS.

 std::stringstream stream;
 // just put random long string into the stringstream value.
 stream << "5sdocmoiatjiladmfc9euhxqincabiughjijdfjafdsjh9rpewiojdsicpklnjijowfvookjkfas";
 stream << "5sdocmoiatjiladmfc9euhxq57456456456incabiughjijdfjafdsjh9rpewiojdsicpklnjijowfvookjkfas";
 auto temp_string = stream.str(); // <<-- crash point.

The problem occurs in the line that I marked, the application keeps crashing on iPhone XS, right after calling the destructor of std::string that was created temporary while copying the value into the temp_string variable.

I found someone already struggled with the same issue before and fixed in the previous versions(https://answers.unrealengine.com/questions/664905/unreal-crashes-on-two-lines-of-extremely-simple-st.html). I'm guessing this bug is related to new/delete symbols in the linker script but I have no ideas.

Product Version: UE 4.21
Tags:
more ▼

asked Jan 30 '19 at 02:14 PM in C++ Programming

avatar image

gigone.lee
96 1 3 3

(comments are locked)
10|2000 characters needed characters left
Viewable by all users

5 answers: sort voted first

I confirmed that the problem can happen anytime when the std containers that are allocated by libc++ is deallocated in iOS. I doubt that this won't cause any problem in Android OS because they use libstd++ instead of libc++.

I think the bug is somewhere between allocator and deallocator. at least one of these methods is using std::allocator/deallocator instead of FMemory::Malloc() and FMemory::Free().

My team solved this problem a month ago by wrapping the std containers so they can use FMemory::Malloc() and FMemory::Free() properly. You don't need to rebuild the whole engine code.

Of course, you can see my team's code since it is opened to anyone. (https://github.com/iFunFactory/engine-plugin-ue4/commit/bc6faababee22025f5fc8b9107e855b0bc769f1e#diff-984a672cbc3814300082a9dd8e3b3c81)

My team tested it on iPhone XS and the other devices after the code is merged, No more crashes :) I assume that you can use this workaround in the production environment until the Unreal Engine(or iOS) fix this.

  • Please note that this problem also can happen in the third-party libraries like JSON or Protobuf, If you're struggling with that, you may have to make a script file that replaces these containers after your code is generated, or after you update the package.

more ▼

answered Mar 20 '19 at 02:41 PM

avatar image

gigone.lee
96 1 3 3

avatar image kolobok1386 Mar 27 '19 at 08:23 AM

Thanks a lot for your help, could you check my question please here https://answers.unrealengine.com/questions/885467/using-ue4-memory-allocation-in-third-party-libs.html ? Could you clarify how you build protobuf libs ? I tried your project from github and when try to compile under Win10 get a lot of errors about c++17 features support in UE4, couldn't find how to resolve them :(

(comments are locked)
10|2000 characters needed characters left
Viewable by all users

c_str() does not return a string but a char const*. The buffer it points to is only valid during the lifetime of the std::string and even may be invalidated by subsequent operations on the string (s.a. docs). Knowing this I would consider it a rather expected behaviour to get access violations when trying to access this buffer after the string is deleted. That this is working on other platforms because the memory usually is not overwritten a few cycles later does not neither mean it is guaranteed to nor that it should. To work around this just keep the string as long as you need it. This really shouldn't mess up your memory footprint, if it does there probably is another issue elsewhere.

more ▼

answered Jan 30 '19 at 05:25 PM

avatar image

j.mueller RFG
304 3 10 10

avatar image gigone.lee Jan 31 '19 at 12:32 AM

Yes. a temporal string that is created by str(), should be released after putting in the 'temp_string'. but it still crashes even when I use str() instead of str().c_str(). The string should be deep-copied, not referred in this time. it only happens on iPhone XS. I'm going to dig deeper.

Currently, I think working around this is not an option because I already found there are several potential crash points like this in a lot of the third party libraries. I can't just fix them all.

avatar image j.mueller RFG Jan 31 '19 at 11:50 AM

This sounds indeed weird. Can you also provide a complete code sample of the version using a deep copy and a stack trace of the crash(es)? When in doubt of what types are used it might help to drop the auto keyword and fall back to static type usage. I also would advice to perform some static code checking on your code. (I personally like to use Cppcheck to sanitize my code.)

Hard to believe it is such a common practice in 3rd party libs to use buffers of already deleted objects, because this is invalid behaviour on any platform (even if it might work in many cases) and there are established techniques around to deal with this sort of problems (e.g. smart/weak pointers).

avatar image gigone.lee Feb 01 '19 at 12:07 AM

Sorry for confusing. I mean, They use str(), not c_str(). The most cases are copying string that is created from std::stringstream, I didn't see any c_str() code. As you know, Copying a string from std::stringstream is a common practice, and It doesn't seem harmful to me.

The problem always happens when std::stringstream::str() is deallocated. even when I use std::copy() and std::move() with std::stringstream::str(). so I think any code will make the problem. I already removed auto keyword to make sure there's no problem on the left side.

I heard iPhone XS is very sensitive at managing memory than other devices. Not sure whether it is true.

I'm going to update my post with call stack and some other things that might be useful, but You can reproduce it with the above code.

avatar image j.mueller RFG Feb 01 '19 at 10:58 AM

They use str(), not c_str(). [...] Copying a string from std::stringstream is a common practice, and It doesn't seem harmful to me.

D'accord. Copying around strings should be safe to do in general and that is why I guess that the access violation is going on elsewhere even if it may be raised when the string is deallocated. Have you tried to run a sample program that does nothing else except for the lines from the sample above (not using c_str() ) to isolate the problem? I would be quite confident this alone should work as expected. Please provide the code you are testing as [complete and minimal][1] as possible. [1]: https://stackoverflow.com/help/mcve
(comments are locked)
10|2000 characters needed characters left
Viewable by all users

we have the same problem, and not fixed

more ▼

answered Mar 04 '19 at 12:22 PM

avatar image

abeyondcd
1

(comments are locked)
10|2000 characters needed characters left
Viewable by all users

We solved a similar problem by replacing Binned memory allocator (default choice on IOS build) with the ANSI one. you can do this by modifying line 210 of ApplePlatformMemory.cpp (4.20 version).

more ▼

answered Mar 05 '19 at 01:49 AM

avatar image

Clones1201
6 1 1

avatar image kolobok1386 Mar 20 '19 at 12:32 PM

Do we need to rebuild whole engine from source code after that ? Or we can set definition of FORCE_ANSI_ALLOCATOR variable by compilation parameters?

avatar image gigone.lee Mar 20 '19 at 02:42 PM

Please see my answer. You don't need to rebuild the whole source code.

avatar image kolobok1386 Apr 04 '19 at 07:08 AM

@Clones1201 Could you confirm please that allocator change solved your issues with iPhone XS ?

avatar image Clones1201 Apr 04 '19 at 08:19 AM

yes, I can confirm this, that allocator change currently still in our code.

avatar image kolobok1386 Apr 04 '19 at 09:01 AM

Thanks a lot! And can you describe please how did you replace it ? Did you rebuild UE from source code with your changes ?

avatar image Clones1201 Apr 04 '19 at 09:02 AM

yes, we are using an engine built from source

(comments are locked)
10|2000 characters needed characters left
Viewable by all users

I have a similar problem。 I Build a plugin as static lib use in unreal 4.2.1. have used google proto buf use xcode 9.4 and crash On the iphone A12 CPU series. like XR, XS Max Got a lot of Log : LogMemory: Warning: Attempting to free a pointer we didn't allocate! i check the code. protobuf generate .h files and include inline function use delete. it likes this delete will be use FMemory::Free() so.i add c++ flag -fno-inline-functions to rebuild lib. it's no log any more

next I Got a Crash. Dump Stack is std::basic_string::~basic_string() It's a struct member function that returns std::string definition : std::string ToString() const; implement :

     std::string tagCSHead::ToString() const {
         std::stringstream oss;
         oss << "CSHead"
             << ", crypto_type: " << crypto_type
             << ", compression_type: " << compression_type
             << ", pkg_size: " << pkg_size
             << ", version: " << (uint16_t)version
             << ", msg_type: " << (uint16_t)msg_type
             << ", flag: " << (uint16_t)flag
             << ", cmd: " << cmd
             << ", seq: " << seq
             << ", session_id: " << session_id
             << ", service_name_size: " << (uint16_t)service_name_size
             << ", service_name: " << service_name
             << ", target: " << target
             << ", data_size: " << data_size;
 
         return oss.str();
     }

 printf("send internal, cshead:%s", a_stCSHead.ToString().c_str());

it will print right log and then crash... then i test sprintf .then it works...

  std::string tagCSHead::ToString() const {
         char szBuff[4096];
         sprintf(szBuff, "CSHead, crypto_type: %u, compression_type:%u, pkg_size:%u, version:%u, msg_type:%u, flag:%u, cmd: %u, seq:%u, session_id:%llu, service_name_size:%u, service_name:%s, target:%u, data_size:%u"
                 , crypto_type, compression_type, pkg_size, (uint16_t)version, (uint16_t)msg_type, (uint16_t)flag, cmd, seq, session_id, (uint16_t)service_name_size, service_name, target, data_size);
         
         return szBuff;
  }

so ..I Guess it Should be related to the implementation of std::stringstream or Xcode stl library conflicts with A12?

more ▼

answered Jul 22 '19 at 05:12 AM

avatar image

peterqyu
1 1

(comments are locked)
10|2000 characters needed characters left
Viewable by all users
Your answer
toggle preview:

Up to 5 attachments (including images) can be used with a maximum of 5.2 MB each and 5.2 MB total.

Follow this question

Once you sign in you will be able to subscribe for any updates here

Answers to this question