IPB Style© Fisana

Jump to content


Android port


  • Please log in to reply
227 replies to this topic

#81 Ykkrosh

Ykkrosh

  • WFG Programming Team

  • Primus Pilus
    (4,869 posts)

Posted 01 February 2012 - 10:41 AM

About SDL: since a few days, the Hg version of SDL 1.3 is now called SDL2 instead, and has removed compatibility with the SDL1.2 API. I guess we need to either use 1.2, or a slightly older Hg version of 1.3, or update our code to be compatible with SDL2 (probably better in the long term but I haven't tested it to see how much we'd have to change).

Incidentally, don't remember if this has been mentioned before, but it's impossible (or at least discouraged and unportable and seemingly undocumented) to use GLES with pure C++ - you have to use some Java to set up EGL windows before the C++ can render into it. SDL provides the necessary Java code, and SDL's android-project can build it: when you run "ndk-build" it builds libSDL2.so and libmain.so, and when you run "ant debug" it compiles the Java code and packages everything into an .apk file. (Then you do an "adb push whatever.apk /data/app" to install it onto the device). I imagine it would be easier to write a custom script to perform those build steps, instead of trying to integrate our existing code into ndk-build, but I don't know exactly what's required.

Unfortunately the Android emulator doesn't seem to do GLES2 - I can compile and run a simple SDL test application but it just prints "E/libEGL ( 504): called unimplemented OpenGL ES API" to logcat. At least it's possible to test that the application starts up without linker errors or Java exceptions etc, but for any graphical output it'll need to be run on real hardware.
Philip Taylor [aka Ykkrosh]

Wildfire Games Programmer
Contact me: philip@wildfiregames.com

#82 Guest_afeder_*

Guest_afeder_*

  • Guests

Posted 01 February 2012 - 12:21 PM

View Posthistoric_bruno, on 01 February 2012 - 01:25 AM, said:

Have you updated the source code recently? ogl_shader was removed in 10984 and there's been a few more recent changes that might help. That still leaves a lot of errors :(
They are numerous in quantity, but similar in quality. With a good plan, it shouldn't be impossible to get rid of them.

Updated output from latest svn

View PostYkkrosh, on 01 February 2012 - 10:41 AM, said:

Incidentally, don't remember if this has been mentioned before, but it's impossible (or at least discouraged and unportable and seemingly undocumented) to use GLES with pure C++ - you have to use some Java to set up EGL windows before the C++ can render into it. SDL provides the necessary Java code, and SDL's android-project can build it: when you run "ndk-build" it builds libSDL2.so and libmain.so, and when you run "ant debug" it compiles the Java code and packages everything into an .apk file. (Then you do an "adb push whatever.apk /data/app" to install it onto the device). I imagine it would be easier to write a custom script to perform those build steps, instead of trying to integrate our existing code into ndk-build, but I don't know exactly what's required.
There's two isolated steps to SDL's approach, as far as I can tell: 1) you build the native shared library component with ndk-build and 2) you build the Java and package it with the native shared library with Ant. So, I imagine we can simply use the toolchain you suggested to do step 1 and use Ant as normal for step 2. Ant "won't care" how the native shared library came about.

#83 Ykkrosh

Ykkrosh

  • WFG Programming Team

  • Primus Pilus
    (4,869 posts)

Posted 02 February 2012 - 11:14 AM

View Postafeder, on 01 February 2012 - 12:21 PM, said:

With a good plan, it shouldn't be impossible to get rid of them.
I think a possibly sensibly plan would be to focus on getting just the GUI to work for now (plus the build system and anything else required), and #ifdef out any problematic non-GUI code, so we can have a runnable starting point before porting all the other graphics code. (I was attempting to start with the GUI here, getting rid of some of the fixed-function code). That will probably involve more changes to the CShaderProgram API so that it can use glVertexAttribPointer instead of the not-in-GLES2 glColorPointer etc, and the rest is probably just tedious rearrangement of code to use the new APIs.

Quote

There's two isolated steps to SDL's approach, as far as I can tell: 1) you build the native shared library component with ndk-build and 2) you build the Java and package it with the native shared library with Ant. So, I imagine we can simply use the toolchain you suggested to do step 1 and use Ant as normal for step 2. Ant "won't care" how the native shared library came about.
I know very little about ndk-build and ant, but that sounds reasonable... If I understand correctly we could probably use SDL's normal ndk-build step, with its jni/src/Android.mk's YourSourceHere.c just being a simple file that defines SDL_main() to load libpyrogenesis.so and call a startup function inside that shared library, and then we can build libpyrogenesis.so independently using the game's build system, and then get ant to work as with normal SDL but copy libpyrogenesis.so into the package too, or something like that.

Incidentally, about the libraries bundled with the game: there's probably no point porting NVTT to Android, since it's used for S3TC texture compression and most mobile GPUs don't support S3TC; I think we should disable that code and load uncompressed textures instead. (Then, if not everybody has got fed up with Android, experiment with ETC1 to save VRAM, and/or JPEG to save download size, etc.)
Philip Taylor [aka Ykkrosh]

Wildfire Games Programmer
Contact me: philip@wildfiregames.com

#84 Guest_afeder_*

Guest_afeder_*

  • Guests

Posted 02 February 2012 - 01:23 PM

View PostYkkrosh, on 02 February 2012 - 11:14 AM, said:

I know very little about ndk-build and ant, but that sounds reasonable... If I understand correctly we could probably use SDL's normal ndk-build step, with its jni/src/Android.mk's YourSourceHere.c just being a simple file that defines SDL_main() to load libpyrogenesis.so and call a startup function inside that shared library, and then we can build libpyrogenesis.so independently using the game's build system, and then get ant to work as with normal SDL but copy libpyrogenesis.so into the package too, or something like that.
Yes, that might work. There is a related section in the NDK documentation: docs/PREBUILT.html.

#85 Guest_afeder_*

Guest_afeder_*

  • Guests

Posted 03 February 2012 - 01:10 PM

View PostYkkrosh, on 02 February 2012 - 11:14 AM, said:

I think a possibly sensibly plan would be to focus on getting just the GUI to work for now (plus the build system and anything else required), and #ifdef out any problematic non-GUI code, so we can have a runnable starting point before porting all the other graphics code. (I was attempting to start with the GUI here, getting rid of some of the fixed-function code). That will probably involve more changes to the CShaderProgram API so that it can use glVertexAttribPointer instead of the not-in-GLES2 glColorPointer etc, and the rest is probably just tedious rearrangement of code to use the new APIs.
So, the main obstacle I am facing now is SpiderMonkey. ENet already builds and I can #ifdef out NVTT and FCollada. But I'm not sure how to build SpiderMonkey and can't seem to get any response from the official mailing lists. It has a --with-android-toolchain configure option, I'm just not sure what other options to combine it with. Any suggestions appreciated.

#86 Ykkrosh

Ykkrosh

  • WFG Programming Team

  • Primus Pilus
    (4,869 posts)

Posted 03 February 2012 - 02:23 PM

Have you seen this? I guess something similar should work for us, though I don't know whether the 1.8.5 release is compatible enough or whether we should use a newer Hg revision. With 1.8.5 we don't need --enable-threadsafe or --with-nspr-* (and don't need to compile NSPR); don't know about newer revisions. Also I presume we can target android-10 (2.3.3+), not android-8 (2.2+), but I guess that won't matter much.
Philip Taylor [aka Ykkrosh]

Wildfire Games Programmer
Contact me: philip@wildfiregames.com

#87 Pedro Falcão

Pedro Falcão

  • Community Members
    PipPipPipPipPip

  • Centurio
    (651 posts)

Posted 03 February 2012 - 03:12 PM

Age of Empires 3 Mobile:Posted Image
Pedro Falcão
Latin: Petrus Falco; Literally means 'Stone Hawk'.
English equivalent: ' Peter ';


Undergraduate Computer Scientist by UFCG
Shotokan Karate Adept, 3rd Kyu (Green Belt) & Muay Thai initiate

#88 Guest_afeder_*

Guest_afeder_*

  • Guests

Posted 04 February 2012 - 07:44 PM

View PostYkkrosh, on 03 February 2012 - 02:23 PM, said:

Have you seen this? I guess something similar should work for us, though I don't know whether the 1.8.5 release is compatible enough or whether we should use a newer Hg revision. With 1.8.5 we don't need --enable-threadsafe or --with-nspr-* (and don't need to compile NSPR); don't know about newer revisions. Also I presume we can target android-10 (2.3.3+), not android-8 (2.2+), but I guess that won't matter much.
There seem to be some kind of incompatibility issue with android-ndk-r5-crystax-2 (it searches the wrong paths). Someone from Mozilla suggested that I build Fennec and use the libraries that it generates, but that results in a similar error.

Would it be feasible to port the parts of the code using wstrings to some other Android-compatible representation? The wstrings are the only thing preventing me using the latest official NDK, which SpiderMonkey seems set up for.

Edit: I'm going to try if I can use another toolchain in parallel specifically to build Spidermonkey. Theoretically, it should not be a problem since the NDK API is intended to be stable across releases.

Edited by afeder, 05 February 2012 - 12:21 AM.


#89 Ykkrosh

Ykkrosh

  • WFG Programming Team

  • Primus Pilus
    (4,869 posts)

Posted 05 February 2012 - 12:02 PM

View Postafeder, on 04 February 2012 - 07:44 PM, said:

Would it be feasible to port the parts of the code using wstrings to some other Android-compatible representation? The wstrings are the only thing preventing me using the latest official NDK, which SpiderMonkey seems set up for.
If the only thing missing from the official NDK is the definition of the std::wstring type, then we could easily define the type ourselves (exactly like source/ps/utf16string.h except with wchar_t). But it sounds like the NDK is probably missing most of the functions for operating on wchar_t too, in which case that won't be sufficient. Changing the game to not use wstring/wchar_t should be hypothetically feasible (it may be best to store everything as UTF-8 encoded strings internally, and just convert to/from UTF-16 when interacting with Win32 APIs) but probably a huge amount of work - I expect it'd be much easier to sort out the build system issues instead of changing all the code.
Philip Taylor [aka Ykkrosh]

Wildfire Games Programmer
Contact me: philip@wildfiregames.com

#90 Guest_afeder_*

Guest_afeder_*

  • Guests

Posted 06 February 2012 - 03:25 AM

View PostYkkrosh, on 03 February 2012 - 02:23 PM, said:

With 1.8.5 we don't need --enable-threadsafe or --with-nspr-* (and don't need to compile NSPR);
How do you prevent it from building with NSPR? update-workspaces.sh seems to run configure with --with-system-nspr, but that just causes it to link in the NSPR in my absolute /usr/lib. If configure it without any nspr options at all, I get this:

/home/afeder/android/js-1.8.5/js/src/jscpucfg.cpp:48:21: fatal error: prtypes.h: No such file or directory

Edited by afeder, 06 February 2012 - 05:35 AM.


#91 Ykkrosh

Ykkrosh

  • WFG Programming Team

  • Primus Pilus
    (4,869 posts)

Posted 06 February 2012 - 12:40 PM

View Postafeder, on 06 February 2012 - 03:25 AM, said:

update-workspaces.sh seems to run configure with --with-system-nspr
Where does it do that? libraries/spidermonkey/build.sh only mentions --with-system-nspr in a commented-out line.

Quote

/home/afeder/android/js-1.8.5/js/src/jscpucfg.cpp:48:21: fatal error: prtypes.h: No such file or directory
That's in a "#if defined(CROSS_COMPILE) && !defined(FORCE_BIG_ENDIAN) && !defined(FORCE_LITTLE_ENDIAN)" section. I presume CROSS_COMPILE is being defined when building for Android, triggering the error. It looks like defining FORCE_LITTLE_ENDIAN (simplest hack is probably to stick it at the top of jscpucfg.cpp) should avoid that dependency on NSPR, hopefully.
Philip Taylor [aka Ykkrosh]

Wildfire Games Programmer
Contact me: philip@wildfiregames.com

#92 Ykkrosh

Ykkrosh

  • WFG Programming Team

  • Primus Pilus
    (4,869 posts)

Posted 06 February 2012 - 08:16 PM

View PostYkkrosh, on 01 February 2012 - 10:41 AM, said:

Unfortunately the Android emulator doesn't seem to do GLES2 - I can compile and run a simple SDL test application but it just prints "E/libEGL ( 504): called unimplemented OpenGL ES API" to logcat.
My test code didn't work on real hardware either since I had the SDL 2.0 init code wrong and it was defaulting to GLES1, but it worked after I fixed that. (Then the emulator fails on application startup because it can't find a usable surface type). The Java code in android-project needs modification to support useful things like 24-bit RGB output (instead of default 16-bit) and antialiasing (4xAA is "nearly free on Mali hardware (around 2% performance drop)"), but that's easy enough.
Philip Taylor [aka Ykkrosh]

Wildfire Games Programmer
Contact me: philip@wildfiregames.com

#93 Guest_afeder_*

Guest_afeder_*

  • Guests

Posted 06 February 2012 - 09:05 PM

View PostYkkrosh, on 06 February 2012 - 12:40 PM, said:

Where does it do that? libraries/spidermonkey/build.sh only mentions --with-system-nspr in a commented-out line.
Indeed, I read that line incorrectly.

Quote

That's in a "#if defined(CROSS_COMPILE) && !defined(FORCE_BIG_ENDIAN) && !defined(FORCE_LITTLE_ENDIAN)" section. I presume CROSS_COMPILE is being defined when building for Android, triggering the error. It looks like defining FORCE_LITTLE_ENDIAN (simplest hack is probably to stick it at the top of jscpucfg.cpp) should avoid that dependency on NSPR, hopefully.
Yes, that worked. Now Spidermonkey builds too Posted Image

The full diff of changes/hacks I had to make is here.

#94 Guest_afeder_*

Guest_afeder_*

  • Guests

Posted 10 February 2012 - 12:15 AM

Any idea why this fails:

afeder@ubuntu:~/android/toolchain/sysroot$ ls $PKG_CONFIG_SYSROOT_DIR/usr/local/lib/pkgconfig/mozjs185.pc
/home/afeder/android/toolchain/sysroot/usr/local/lib/pkgconfig/mozjs185.pc

afeder@ubuntu:~/android/toolchain/sysroot$ pkg-config mozjs185 --cflags
Package mozjs185 was not found in the pkg-config search path.
Perhaps you should add the directory containing `mozjs185.pc'
to the PKG_CONFIG_PATH environment variable
No package 'mozjs185' found
According to the man page of pkg-config, /usr/local/lib/pkgconfig is supposed to be one of the default search paths.

Edit: I guess pkg-config doesn't actually search the sysroot, it just prepends it to the absolute paths Posted Image
Edit2: I fixed this by using the PKG_CONFIG_LIBDIR variable.

Edited by afeder, 11 February 2012 - 06:11 AM.


#95 Guest_afeder_*

Guest_afeder_*

  • Guests

Posted 11 February 2012 - 06:32 AM

These are the files causing errors in the latest error output:

/source/simulation2/components/CCmpRallyPointRenderer.cpp
/source/simulation2/components/CCmpTerritoryManager.cpp
/source/ps/Profiler2GPU.cpp
/source/ps/ProfileViewer.cpp
/source/ps/CLogger.cpp
/source/ps/Util.cpp
/source/ps/CConsole.cpp
/source/ps/GameSetup/HWDetect.cpp
/source/ps/GameSetup/GameSetup.cpp
/source/maths/Brush.cpp
/source/maths/BoundingBoxAligned.cpp
/source/graphics/Material.cpp
/source/graphics/ShaderTechnique.cpp
/source/graphics/Camera.cpp
/source/graphics/ParticleEmitter.cpp
/source/graphics/ObjectEntry.cpp
/source/graphics/TerritoryTexture.cpp
/source/graphics/LOSTexture.cpp
/source/graphics/Sprite.cpp
/source/graphics/ShaderProgramFFP.cpp
/source/graphics/CinemaTrack.cpp
/source/renderer/DecalRData.cpp
/source/renderer/ParticleRenderer.cpp
/source/renderer/HWLightingModelRenderer.cpp
/source/renderer/TerrainRenderer.cpp
/source/renderer/ShadowMap.cpp
/source/renderer/PlayerRenderer.cpp
/source/renderer/ModelRenderer.cpp
/source/renderer/RenderModifiers.cpp
/source/renderer/PatchRData.cpp
/source/renderer/OverlayRenderer.cpp
/source/renderer/SkyManager.cpp
/source/renderer/InstancingModelRenderer.cpp
/source/renderer/Renderer.cpp
/source/renderer/FixedFunctionModelRenderer.cpp
/source/renderer/TerrainOverlay.cpp
/source/renderer/VertexBuffer.cpp
/source/renderer/TransparencyRenderer.cpp
/source/tools/atlas/GameInterface/ActorViewer.cpp
/source/tools/atlas/GameInterface/Handlers/TerrainHandlers.cpp
/source/gui/CGUI.cpp
/source/gui/GUIRenderer.cpp
/source/gui/MiniMap.cpp
/source/lib/res/graphics/unifont.cpp
/source/lib/res/graphics/ogl_tex.cpp
Which of these would characterize as being "non-GUI"? And how do you suggest I remove them from the build system?

#96 Ykkrosh

Ykkrosh

  • WFG Programming Team

  • Primus Pilus
    (4,869 posts)

Posted 11 February 2012 - 11:20 AM

r11039/r11040 should have fixed the build errors in source/gui/. (At least they work for me with Mesa's GL ES on Linux). I still need to add GLSL versions of the GUI shaders, but that probably should be enough to get the GUI displaying. I think the rest of the GL code can get either fixed easily (translating to the new shader API) or temporarily #if'd out, but that'll depend on the precise details of each case.
Philip Taylor [aka Ykkrosh]

Wildfire Games Programmer
Contact me: philip@wildfiregames.com

#97 Guest_afeder_*

Guest_afeder_*

  • Guests

Posted 11 February 2012 - 02:35 PM

Great. But how do I practically go about preventing the Premake system from compiling those that I #ifdef out? Where are the input files for the compiler generally listed?

#98 Ykkrosh

Ykkrosh

  • WFG Programming Team

  • Primus Pilus
    (4,869 posts)

Posted 11 February 2012 - 03:03 PM

It will still need to compile all of the source files, since it needs most of the definitions of classes and methods etc even if they're not working yet, else it'll fail later with linker errors. Generally just small sections of code inside the methods should be disabled, like this.

The source files aren't actually listed anywhere - build/premake/premake4.lua lists directories, and it searches for all *.cpp inside them.
Philip Taylor [aka Ykkrosh]

Wildfire Games Programmer
Contact me: philip@wildfiregames.com

#99 Guest_afeder_*

Guest_afeder_*

  • Guests

Posted 11 February 2012 - 03:07 PM

Got it.

#100 Guest_afeder_*

Guest_afeder_*

  • Guests

Posted 11 February 2012 - 03:15 PM

I'll be trying to commit any hacks I make here as I go along: https://github.com/afeder/0ad-android




0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users