Libgdx runtime questions

[denisk20]

You're welcome; LibGdx json parser is not that bad, it just parses json into a DOM tree, which is easy to use but is not really suitable for big data sets. Look for some streaming, event-based library. Gson has these capabilities:
https://sites.google.com/site/gson/streaming
JavaEE JsonParser is pretty good too (you can grab its sources from Glassfish server for example). I wouldn't recommend using whole Jackson here because it's pretty big.
Maybe it's worth re-packaging relevant code from one of these libraries to minimize the resulting application size - they all have permissive licenses (Apache2 for Gson and Jackson and CDDL|GPL for Glassfish: https://glassfish.java.net/public/CDDL+GPL.html).

[chong]

Well I think one other way around it is to put in the binary format feature available already for Unity. The Unity load times are almost instantaneous now with the new binary format plus the memory footprint is also much lower.

(In UE4 which is C++, the json library used is also much more efficient in terms of both speed and memory. With LibGDX, I had unfortunately made the mistake of using the default Json parser since I assumed it was suitable for the task of reading in large json files.)

However, this will take a bit of time since it requires writing a new binary reader for the Java runtime. When I get the cycles I will take a look at it.

Cheers,
Chong

[denisk20]

Thanks! Support of binary format in libgdx would be a fantastic feature, looking forward to it!

[chong]

Hello,

Good news. After a rather intensive day of coding, I have preliminary support for the new fast binary flat data format. You can sync up now and have a look. I will be posting more in depth documentation soon.

Essentially, you should:

1) Go to the: https://github.com/kestrelm/CreatureToo ... atData/Bin page and grab either the Windows or Mac binary converter. You will run this program on an input json file and it will convert it into a flat binary file.

2) Grab the new runtimes: https://github.com/kestrelm/Creature_LibGDX

3) How to use the code for loading:

Code: Select all

		String baseResourcePath = "C:\\Mydata\\";
		String flatFilename = baseResourcePath + "character_raptor_data.bytes";
		String pngFilename = baseResourcePath + "character_raptor_img.png";

		// Load flat data
		CreatureFlatDataJava.rootData flat_data = CreatureModuleUtils.LoadCreatureFlatData(flatFilename);
		
		// Create creature
        Creature new_creature = new Creature(flat_data);
        
        // Create animations
        CreatureAnimation new_animation1 = new CreatureAnimation(flat_data.dataAnimation(), "default");

... and then continue as before. Again, I am rather tired now so I will polish it up and do some real documentation soon.

Cheers,
Chong

[denisk20]

Hi,
That was pretty fast!
I'm currently getting an error when running binary converter on Mac:

Code: Select all

$ ./CreatureTools/FlatData/Bin/Mac/CreatureFlatData libgdx-creature/android/assets/Ue4_export/character_raptor_data.json libgdx-creature/android/assets/Ue4_export/character_raptor_data.bin
Assertion failed: (false), function operator[], file /Users/jychong/Projects/EngineAppMedia/ExtensionsGitHub/CreatureTools_Distro/FlatData/rapidjson/document.h, line 834.
Abort trap: 6

Should be easy to fix.
FlatBuffers is an awesome library, great job!

[chong]

Hello,

Oh that's most probably because the exported file is of an older format. Let me take a look and re-export.

Cheers,
Chong

[chong]

Hello,

You can re-download again at the link:
http://creature.kestrelmoon.com/Ue4_export.zip

The documentation has also been updated:
http://www.kestrelmoon.com/creaturedocs ... times.html

I have updated the json file as well as packaged up the binary file for you (with a .bin extension)
Give it a go.

My current memory profile tells me the memory usage is now much better than before. It also loads very fast, at least on my machine.

Cheers,
Chong

[denisk20]

A couple of things:
1. I'm currently getting all kinds of errors with new runtimes: wrong package names, non-existing fields, referencing HashMap.Entry instead of Map.Entry as the primary ones. I did my best to correct as many errors as I can but there are some errors in generated flatbuffer mappings which I can't resolve. Take a look at exported error report from IntelliJ:

Code: Select all

/Users/me/libgdx-creature/core/src/MeshBoneUtil/CreatureModuleUtils.java
    Error:Error:line (263)java: cannot find symbol
  symbol:   method ReadRestParentMat()
  location: variable cur_node of type CreatureFlatDataJava.skeletonBone
    Error:Error:line (265)java: cannot find symbol
  symbol:   method ReadLocalRestStarPt()
  location: variable cur_node of type CreatureFlatDataJava.skeletonBone
    Error:Error:line (266)java: cannot find symbol
  symbol:   method ReadLocalRestEndPt()
  location: variable cur_node of type CreatureFlatDataJava.skeletonBone
    Error:Error:line (267)java: cannot find symbol
  symbol:   method ReadChildren()
  location: variable cur_node of type CreatureFlatDataJava.skeletonBone
    Error:Error:line (375)java: cannot find symbol
  symbol:   method ReadWeights()
  location: variable w_node of type CreatureFlatDataJava.meshRegionBone
    Error:Error:line (495)java: cannot find symbol
  symbol:   method ReadStartPt()
  location: variable bone_node of type CreatureFlatDataJava.animationBone
    Error:Error:line (496)java: cannot find symbol
  symbol:   method ReadEndPt()
  location: variable bone_node of type CreatureFlatDataJava.animationBone
    Error:Error:line (584)java: cannot find symbol
  symbol:   method ReadLocalDisplacements()
  location: variable mesh_node of type CreatureFlatDataJava.animationMesh
    Error:Error:line (590)java: cannot find symbol
  symbol:   method ReadLocalDisplacements()
  location: variable mesh_node of type CreatureFlatDataJava.animationMesh
    Error:Error:line (672)java: cannot find symbol
  symbol:   method ReadLocalOffset()
  location: variable uv_node of type CreatureFlatDataJava.animationUVSwap
    Error:Error:line (673)java: cannot find symbol
  symbol:   method ReadGlobalOffset()
  location: variable uv_node of type CreatureFlatDataJava.animationUVSwap
    Error:Error:line (674)java: cannot find symbol
  symbol:   method ReadScale()
  location: variable uv_node of type CreatureFlatDataJava.animationUVSwap
/Users/me/libgdx-creature/core/src/MeshBoneUtil/Creature.java
    Error:Error:line (153)java: cannot find symbol
  symbol:   method ReadPoints()
  location: variable flat_mesh of type CreatureFlatDataJava.mesh
    Error:Error:line (156)java: cannot find symbol
  symbol:   method ReadIndices()
  location: variable flat_mesh of type CreatureFlatDataJava.mesh
    Error:Error:line (159)java: cannot find symbol
  symbol:   method ReadUvs()
  location: variable flat_mesh of type CreatureFlatDataJava.mesh
/Users/me/libgdx-creature/core/src/MeshBoneUtil/Tuple.java

2. There are 2 copies of flatbuffer core classes in the project

3. What about the above error with Mac json->bin converter? Does it occur because of a hardcoded path?

======================

It seems that you're manually copying source files to the project from some external location. Why don't you want to create a structured working project with an example similar to my test project? It would save tons of time to you and to any person trying to get up and running with the runtimes. If you fix the above errors and I get it running I can create a pull request with a nice self-contained project instead of a bunch of source files which needs to be copied into a test project by hands in order to give it a try.

[chong]

Hello,

On the issue of the converter:

Are you still getting the same error? I updated the packaged json file and ran it on my mac. There is no hard-coding involved and it went through. The .bin file is generated from the mac converter actually. The previous error with the converter happened because the json file from the old zip was exported a while back ( the format was slightly different)

Did you try running it on the newly packaged json file from the link?

I will take a look at packaging up the project for libGDX and resolving the errors for you right now. I am grabbing the Gradle integration stuff for Eclipse.

Cheers

[denisk20]

Sorry I tried it with old json - I thought it's an issue with the converter itself. With new json the conversion works properly.