CSV still does not export properly using blueprint structs to create data tables. JSON seems to work fine. I would like this to be fixed so people don’t have to dig through C++ to edit struct data. Any data table with a nested struct will not export or import correctly. I rather use csv since it’s easier to work with in spreadsheet.
Exporting as a CSV will not export string, text, structs correctly…
You can NEVER import structs using blueprint structs. Works with C++ structs only.
Often writes CSV incorrectly, having multiple rows on single lines.
Importing will say successful even when it isn’t. Sometimes the entire database will be gone.
I ran some tests with data tables and exporting/importing CSV files, and I am not sure that I am clear about where you are seeing things going wrong.
I created a struct matching the example that you provided and made a data table from it. After adding several rows and setting the values, I exported the data table to a CSV file and opened it in Excel. All of the data appeared to have been exported correctly.
I then renamed the original data table in the Editor, and imported the CSV file I had just created in the previous step. Comparing the old and new data tables showed all of the data was the same.
For the multiple rows on a single line, do you mean that if the data table has, for example, three rows. When it is exported to CSV and opened in Excel, all three rows will appear on a single line?
I didn’t see the import fail when I was testing. In what situation do you typically see this process fail but still indicate that it succeeded?
This is what I always get. You don’t even need excel.You can just open a text editor, like notepad, to see it isn’t exporting correctly… You can also look at the link I posted to see the CSV syntax how it suppose to export opposed to how it actually exports.
The bug where it doesn’t separate the lines are rare but still happens. Below is from the current version of UE4.
EDIT:
Just to be clear about one thing. IF you make data table struct in C++ and then export, it will export correctly. It is only blueprint structs with nested structs that it will not work. The first snippet will NOT work even left as is. The values will always be reset to default AND that is only if the reference link to that nested struct doesnt break. NOT to mention, EVEN IF you format the nested struct correctly in CSV, IT STILL does not import. The values will ALWAYs be default.
Reimporting a CSV with a minor error, such as adding an extra quotation/comma, will import successfully but can actually delete the entire database.
There is also issues with the C++ version, where struct/enum references will break, variables will not update after being compiled, unless you restart engine. Both options have issues. Even people who submit things to market place will refuse to use nested structs in a datatable.
The variable names with the extra hexadecimal strings appended are unfortunately expected in this case. This is because the CSV export process just runs ExportText on the property and gets the actual variable name instead of the “pretty” display name that is shown in the Editor (The JSON exporter uses a different process and does use the display name shown in the Editor). The only display names exported for CSV are for the variables in the top-level struct that the data table is based on. You don’t see this happen with C++ structs because the variables in those structs most likely have much simpler names than the ones given to Blueprint variables by the Editor.
I still have not seen an instance where I changed a value in the CSV file, then imported that into the Editor and had the new change not appear in the Editor, or get reset to defaults. I am not sure what I am doing differently from you in that regard. I exported the CSV and opened it (in either Excel or Notepad), edited a few of the values, saved the CSV, then imported it into the project and replaced the existing data table. When I opened the data table, the changes that I had made in the CSV were shown in the data table.
I also tried adding an extra comma or double quote to see if I could get it to delete all of the data from the data table. In the case of adding a comma, when I imported the CSV the Editor displayed a message saying that that row had more cells than properties, then it wouldn’t display that row. That would be expected in this case. Removing the extra comma allowed the row to display normally again. Adding an extra double quote sometimes showed the same behavior as an extra comma depending on where the extra double quote was placed, and other times showed a message saying that there were too few cells, and then some items in the row showed incorrect data. This would also be expected.
We do also still have some open tickets related to hot reloading C++ structs and data tables.
Ever since data tables where added I always had problems with nested structs. The above “hexadecimal” string will also break reference at times if the struct is changed. Meaning you have to delete the nested struct in re add it, in which case would delete the data for that struct in the data table.
I don’t see why structs have this problem since text and some other variables export similar and then can reimport with the proper CSV format. The way I see it, it should import and export with proper CSV syntax like the C++ version. It shouldn’t be hard change this…
One of my problems, with the way it is, is that it isn’t practical to work with in CSV. It’s a mess, hard to read, and buggy. The reason I wanted to use blueprint structs is because it’s easier to make changes and easier for other people to go in and make changes. At this point, I am better off programming json database in .net.
I guess I am not clear on what you mean when you say it is not exporting with the proper CSV syntax. The format that I expect to see exported for variables in the nested struct is VariableName=Value. When I export the data table to a CSV file, I get something like this: VarThree_7_E6CA59C444A026B56B78BA9BEAB11F88=42.000000
That is what I would expect to see in this instance. VarThree is the variable’s display name that the Editor shows when I am working with that struct. However, the actual name of the variable is VarThree_7_E6CA59C444A026B56B78BA9BEAB11F88. That is the name that the underlying code for the Blueprint data table uses to identify that variable.
I agree that these variable names are not easy to read, and can make working with the CSV file somewhat difficult. It gets a little easier once you become familiar with the format for the names, but they are still cumbersome and ugly. Unfortunately the CSV exporter retrieves the actual variable name for variables in nested structs instead of the display name. As I mentioned, the JSON export process is somewhat different, and that does retrieve the display names, which makes it a bit easier to work with in this case.
I did notice some odd behavior when making changes to the struct that is nested in the data table. Sometimes it would mark the containing struct and data table dirty, sometimes it wouldn’t. Sometimes it would automatically update the containing struct and data table, sometimes it wouldn’t. And sometimes it did appear to clear the data in that struct from the data table (it did not seem to affect any other data in the data table, though). I’ll have to see if I can narrow down specifically what changes cause what results.
I did notice some odd behavior when
making changes to the struct that is
nested in the data table. Sometimes it
would mark the containing struct and
data table dirty, sometimes it
wouldn’t. Sometimes it would
automatically update the containing
struct and data table, sometimes it
wouldn’t. And sometimes it did appear
to clear the data in that struct from
the data table (it did not seem to
affect any other data in the data
table, though). I’ll have to see if I
can narrow down specifically what
changes cause what results
This what I mean when the nested struct loses reference. This is not what I was talking about when the database gets wiped. The database gets wiped only on importing if something went wrong, (even if import was successful). The behavior of this is exactly as you described. Had this happen often in one of my projects and would also break pins connected to nodes in actual blueprints as well. It was a headache to have to recheck everything.
When using a comma separated files to store data when making other applications, I always kept the syntax a certain way so it would easily be able to format and assign it to memory. However, the way unreal engine imports the CSV, it literally checks the string that gets formatted and then associated with both a variable name and column name, (so it seems). What I was suggesting here, is not actually match it literal variable name but rather assign it to the name we actually named the variable or just assign variables in order as they appear in the struct. Either way, there shouldn’t be a problem unless you write the CSV file wrong to begin with. I know this may be change epic may not be willing to make. But it would be more of a convenience on the user end.
Also couldn’t add this due to text limit. The hexadecimal variable may been my mistake. They seem to work correctly with 4.17. I may have been confused when trying to format it the way it was in C++. However, I will still refuse to use them since it will confuse other people. I will probably make my own database application to write to JSON for the future.
Sorry for not getting back to you sooner. I have still not seen the issue where importing a CSV file sometimes corrupts the data table. Is this something that you are still running into?
I have entered a request to investigate the possibility of using friendly names for variables in a nested struct when exporting a data table as a CSV file (UE-56202). I also have a few other issues that I am currently investigating related to this that I will be entering reports about soon.
Not recently. I been mainly just keeping everything in engine. Using CSV would be easier for managing databases, though. These structs is the main reason I stopped using CSV.
