IPB Style© Fisana

Jump to content


Build environment and deployment on the Mac

Started by Yves , Jan 06 2012 03:19 AM

  • Please log in to reply
189 replies to this topic

#61 historic_bruno

historic_bruno

  • WFG Programming Team

  • Primus Pilus
    (1,930 posts)

Posted 19 February 2012 - 06:19 AM

Ran into a few tricky problems while working on the script. I couldn't figure out why it kept linking to the wrong version of libpng (the one bundled with OS X Lion, not from my script). It turns out a bunch of system search paths get added before any of the game's bundled library paths, and they'll be searched in that order. So I'm thinking we should move those to the end of the search list. I don't know if this causes problems on other platforms, but it shouldn't? Seems like all that's needed is factoring out the few offending lines into their own function and calling it in the correct order.

Also Premake needs a "sysroot" option to handle SDK builds, otherwise you have to define environment variables like CPPFLAGS every time you build, which is less than ideal. This would also help the Android cross-platform build if I'm not mistaken. For OS X specifically, we need to handle the minimum required version parameter, as that tells the compiler how to link certain parts of the API depending on which version of OS X introduced them. Perhaps a "min-osx-version" option, or it could be done with an environment variable, but an option is more explicit and better documented. Both "sysroot" and "min-osx-version" would be set by update-workspaces.sh for the convenience of OS X devs (defaulting to /Developer/SDKs/MacOSX10.6.sdk and 10.6 respectively)

View PostGeneral Octavian, on 19 February 2012 - 05:59 AM, said:

Will this build allow for atlas and sound available on the mac version?
Atlas - yes, sound - no. It doesn't change the sound problems, but if there's no other solution forthcoming we could always add OpenALsoft to the script.
Ben Brian [ aka historic_bruno ]

Wildfire Games Programmer
Contact me: ben@wildfiregames.com

#62 General Octavian

General Octavian

  • Community Members
    Pip

  • Discens
    (67 posts)

Posted 19 February 2012 - 06:21 AM

Atlas is a improvement but I'd much rather have sound... the game seems boring without some nice audio you have created ^_^ I do play with some of my own music but its not the same!

#63 Ykkrosh

Ykkrosh

  • WFG Programming Team

  • Primus Pilus
    (4,869 posts)

Posted 19 February 2012 - 02:46 PM

View Posthistoric_bruno, on 18 February 2012 - 10:53 PM, said:

osx (should it be named osx86_64? since it only supports Intel 64-bit)
(I don't think it should - if we add other OS X platforms later, they should probably be added to the same build script (to avoid redundancy) inside libraries/osx/, and they don't need a new top-level directory. Platform-specific compiled output could perhaps go in libraries/osx/x86_64/ if there's a real desire to support multiple platforms in parallel in a single checkout, but that sounds quite a theoretical concern to me.)

Quote

"linux" will contain only headers.
That doesn't sound right - it needs to contain the source for (if I remember correctly) nvtt, fcollada, spidermonkey, enet, and probably valgrind (only headers), and shouldn't contain anything else. It would be nice if we didn't duplicate the forked source in both libraries/osx/ and libraries/linux/ - maybe put them into a single libraries/sources/ (along with any shared-by-Linux-and-OSX custom build scripts for each).

Quote

The way of making this clear, I believe, is to have an empty "libraries" directory in SVN, with a readme that explains the further process. You will need to do a separate checkout of the libraries for your OS of choice (separate from the rest of the source code).
Could it be made part of the existing update-workspaces.{sh,bat} step? That can automatically do a "svn co" to get or update the appropriate libraries/{linux,osx} (and libraries/sources/{nvtt,...}), and then (on Linux/OSX) automatically run the library build script, so we don't need to worry about users getting it wrong or forgetting to update when we change the libraries.

(In that case, update-workspaces should probably specify an explicit SVN revision of the libraries that it wants, and we should update that specification when updating the libraries, so that people can still build old versions of the game successfully with the contemporary libraries, instead of it always requesting the latest versions. Also, need some alternative for Linux release downloads so that they bundle everything necessary, since package build scripts shouldn't perform their own network access.)

(update-workspaces should quite possibly be moved and/or renamed, perhaps to "build-setup" in the root directory or something like that, since currently it's weirdly hidden away.)

Quote

OS X [...] Currently it only builds static libs, I see this posing the least difficulties in creating a portable version of the game.
Linux packagers generally insist on dynamic linking for various reasons, but OS X development never seems to bother with anything like that, so static linking on OS X sounds reasonable and least problematic. (Static linking also means bigger executables (which I wouldn't like on Windows since it would probably make the autobuilds much bigger), and seemingly much worse linking times (which I wouldn't like on Linux since I compile the game myself on Linux and am impatient) ((unless I'm misinterpreting the cause of the slow linking of Android builds)), but those don't seem particularly important on OS X.)
Philip Taylor [aka Ykkrosh]

Wildfire Games Programmer
Contact me: philip@wildfiregames.com

#64 historic_bruno

historic_bruno

  • WFG Programming Team

  • Primus Pilus
    (1,930 posts)

Posted 19 February 2012 - 10:57 PM

By the way, my test build works on both Lion and Snow Leopard, the only thing I don't know is if it will work on Mountain Lion ;)

View PostYkkrosh, on 19 February 2012 - 02:46 PM, said:

That doesn't sound right - it needs to contain the source for (if I remember correctly) nvtt, fcollada, spidermonkey, enet, and probably valgrind (only headers), and shouldn't contain anything else. It would be nice if we didn't duplicate the forked source in both libraries/osx/ and libraries/linux/ - maybe put them into a single libraries/sources/ (along with any shared-by-Linux-and-OSX custom build scripts for each).
You're quite right about that, and I also like the idea of a shared directory, so I'll work on that :)
Ben Brian [ aka historic_bruno ]

Wildfire Games Programmer
Contact me: ben@wildfiregames.com

#65 historic_bruno

historic_bruno

  • WFG Programming Team

  • Primus Pilus
    (1,930 posts)

Posted 20 February 2012 - 12:38 AM

Output of otool -L for pyrogenesis, test, ActorEditor, libAtlasUI, libCollada:

Spoiler

Notice the minimal number of dylibs they link against :P What I don't know is if Apple plans to maintain compatibility with the 10.6 SDK going forward, nothing they do would surprise me, we'll just have to wait and see. It would be annoying if e.g. Mountain Lion ships without some system dylib we depend on.
Ben Brian [ aka historic_bruno ]

Wildfire Games Programmer
Contact me: ben@wildfiregames.com

#66 Geek377

Geek377

  • Web Development Team

  • Primus Pilus
    (1,718 posts)

Posted 20 February 2012 - 05:12 AM

@Ben, don't worry about Mountain Lion just yet. Save that for summer :P
The Geek [ aka Geek377]
Wildfire Games Webmaster
Contact me: Posted Image Posted Image

#67 historic_bruno

historic_bruno

  • WFG Programming Team

  • Primus Pilus
    (1,930 posts)

Posted 23 February 2012 - 02:25 AM

Build script for OS X libraries, but note you can't really run it because paths have changed:
Spoiler
By default, it builds against OS X 10.6 SDK with minimum required OS X version 10.6 (what this means: only system libraries available on Snow Leopard or later are linked. Further, you will likely get compile errors if you try building on 10.5. Changing these variables and it might work. However, the minimum version also tells the compiler how to link system functions: for 10.6 or earlier functions they are linked at compile-time, i.e. "strong linking", but 10.7 and later functions (assuming you changed the SDK) are linked at run-time, i.e. "weak linking". We don't handle that distinction yet, nor do we have much cause to, we'll just need to be aware of this).



Because I was so tantalizingly close, I made a quick app bundle script which does all the things needed for a release. I'm sure it can be improved, but at least this way we might have something in place for Alpha 9 like we originally planned:

(note: this script won't work for you, because I've changed libraries paths and Premake scripts, it's just to show what's involved :P Very much subject to change)
Spoiler

As you can see the steps to creating a bundle are:
  • Clean and generate workspaces with Premake
  • Build game
  • Confirm tests succeed
  • Create standard app bundle directories
  • Run archive builder on public mod (precaches XML, textures, and COLLADA models)
  • Copy files into the bundle (shared libs into Contents\Frameworks, data into Contents\Resources, binary into Contents\MacOS) - for the reasons see Apple's Bundle programming guide
  • Create Contents\Info.plist using PlistBuddy - this is just an XML property list file that contains info about the bundle

Almost everything is done on the command line with the exception of creating the 0ad icon (icns seems to be the only format OS X recognizes and I created it using http://iconverticons.com from our crappy low resolution 0ad.png) and compressing the bundle into a dmg. I'll investigate whether/how these steps can also be performed on the command line. Final package ends up being around 375MB and I don't know anywhere to upload this for testing :( I tried copying the bundle from Lion to different locations on Snow Leopard, and it ran perfectly including Atlas.

As far as making the bundle paths work, I've got a patch that adds some sysdep code so we can detect if the binary is in a valid bundle (tests if the bundle identifier is "com.wildfiregames.0ad") and a few functions to get the internal Resources and Frameworks paths (unrelated to previous paths patch). Paths are not hardcoded into the build unlike most of the solutions used previously.
Ben Brian [ aka historic_bruno ]

Wildfire Games Programmer
Contact me: ben@wildfiregames.com

#68 Echelon9

Echelon9

  • Donator

  • Discens
    (54 posts)

Posted 23 February 2012 - 02:39 AM

Exciting steps!I think we can get the SVN or release version in the plist file as well.

#69 General Octavian

General Octavian

  • Community Members
    Pip

  • Discens
    (67 posts)

Posted 23 February 2012 - 03:31 AM

Is sound working in this build?

#70 historic_bruno

historic_bruno

  • WFG Programming Team

  • Primus Pilus
    (1,930 posts)

Posted 23 February 2012 - 03:36 AM

View PostYkkrosh, on 19 February 2012 - 02:46 PM, said:

Could it be made part of the existing update-workspaces.{sh,bat} step? That can automatically do a "svn co" to get or update the appropriate libraries/{linux,osx} (and libraries/sources/{nvtt,...}), and then (on Linux/OSX) automatically run the library build script, so we don't need to worry about users getting it wrong or forgetting to update when we change the libraries.

(In that case, update-workspaces should probably specify an explicit SVN revision of the libraries that it wants, and we should update that specification when updating the libraries, so that people can still build old versions of the game successfully with the contemporary libraries, instead of it always requesting the latest versions. Also, need some alternative for Linux release downloads so that they bundle everything necessary, since package build scripts shouldn't perform their own network access.)

(update-workspaces should quite possibly be moved and/or renamed, perhaps to "build-setup" in the root directory or something like that, since currently it's weirdly hidden away.)
That's a bit more complicated than what I've got working now, but it sounds like it would make the build process slightly less painful. That would be more of an incremental improvement than an initial change. I don't think it will ever be foolproof, compiling software is hard and at least some technical knowledge is needed :P

Quote

Linux packagers generally insist on dynamic linking for various reasons, but OS X development never seems to bother with anything like that, so static linking on OS X sounds reasonable and least problematic. (Static linking also means bigger executables (which I wouldn't like on Windows since it would probably make the autobuilds much bigger), and seemingly much worse linking times (which I wouldn't like on Linux since I compile the game myself on Linux and am impatient) ((unless I'm misinterpreting the cause of the slow linking of Android builds)), but those don't seem particularly important on OS X.)
I haven't noticed linking taking longer than usual, though I do run the build with "-j5" :) The binary size is about 14MB using the static libs, so not huge but not small either.

View PostEchelon9, on 23 February 2012 - 02:39 AM, said:

I think we can get the SVN or release version in the plist file as well.
Yes, the version keys should be strings formatted like "X.Y.Z", but I have no idea how we want to format release numbers.

For anyone interested, Apple has good documentation on the Info.plist keys here and a list of recommended keys here - I added the recommend ones.

View PostGeneral Octavian, on 23 February 2012 - 03:31 AM, said:

Is sound working in this build?
Sound is a totally separate issue from building and deploying bundles for OS X :) Nothing works in this bundle that wouldn't already work without a bundle. Not saying it won't be fixed, but you have to take one thing at a time.
Ben Brian [ aka historic_bruno ]

Wildfire Games Programmer
Contact me: ben@wildfiregames.com

#71 Geek377

Geek377

  • Web Development Team

  • Primus Pilus
    (1,718 posts)

Posted 23 February 2012 - 03:42 AM

View Posthistoric_bruno, on 23 February 2012 - 02:25 AM, said:

Almost everything is done on the command line with the exception of creating the 0ad icon (icns seems to be the only format OS X recognizes and I created it using http://iconverticons.com from our crappy low resolution 0ad.png) and compressing the bundle into a dmg. I'll investigate whether/how these steps can also be performed on the command line. Final package ends up being around 375MB and I don't know anywhere to upload this for testing :( I tried copying the bundle from Lion to different locations on Snow Leopard, and it ran perfectly including Atlas.

Bro, please let me get my hands on that! I don't care how big the file size may be, Geek needs updates! Like I've said before, I would take testing 0 AD over doing my boring history homework any day. (It's a win-win situation :))In the meantime, I have a pretty decent icon file.
The Geek [ aka Geek377]
Wildfire Games Webmaster
Contact me: Posted Image Posted Image

#72 gudo

gudo

  • Community Members
    PipPipPip

  • Duplicarius
    (228 posts)

Posted 23 February 2012 - 05:24 AM

.torrent? Or, you could break it into smaller archive files and upload to zippyshare.com. You don't need an account, but file size is limited to 200MB.

Actually, now that I think about, would distributing the various alphas as a .torrent be possible? (with the current DL server as a web seed) Seems a good idea all around in terms of bandwidth and speed.

Edited by gudo, 23 February 2012 - 05:28 AM.

Did you contribute to 0 AD?
Make sure you're in the credits!

AI Players
JuBot | RootBot | SplitBot | qBot | Marilyn | arBot

#73 historic_bruno

historic_bruno

  • WFG Programming Team

  • Primus Pilus
    (1,930 posts)

Posted 23 February 2012 - 05:40 AM

View PostGeek377, on 23 February 2012 - 03:42 AM, said:

Bro, please let me get my hands on that! I don't care how big the file size may be, Geek needs updates! Like I've said before, I would take testing 0 AD over doing my boring history homework any day. (It's a win-win situation :))In the meantime, I have a pretty decent icon file.
Nice icon! I'm using it now :D In return I'll provide you with an app bundle: http://hotfile.com/d...12-osx.dmg.html

(Edit: Fixed app bundle with correct icon, new link)
Ben Brian [ aka historic_bruno ]

Wildfire Games Programmer
Contact me: ben@wildfiregames.com

#74 wraitii

wraitii

  • WFG Programming Team

  • Primus Pilus
    (1,018 posts)

Posted 23 February 2012 - 06:34 AM

That's great, historic_bruno. Very nice work :)
Just one question: you seem to export the collada models directly, ie not use the library... Any reason for it?

Edit: just tried it. Downloaded, started from the dmg (which sometimes can cause bugs), and everything worked fine on 10.7.2 on a MacBook Air. You've really done a great work.

Edited by wraitii, 23 February 2012 - 07:43 AM.

Wraitii
Wildfire Games Programmer, AI developer, auxiliary map designer, dealing with anything water.
Contact me: wraitii@wildfiregames.com
Also the world's only three-dimensional poodle.

#75 historic_bruno

historic_bruno

  • WFG Programming Team

  • Primus Pilus
    (1,930 posts)

Posted 23 February 2012 - 08:55 PM

View Postwraitii, on 23 February 2012 - 06:34 AM, said:

Just one question: you seem to export the collada models directly, ie not use the library... Any reason for it?
There's no technical reason, but that's how Alpha 9 data will be packaged :) If you think about it, it doesn't really make sense to require conversion of archived COLLADA documents at runtime.
Ben Brian [ aka historic_bruno ]

Wildfire Games Programmer
Contact me: ben@wildfiregames.com

#76 wraitii

wraitii

  • WFG Programming Team

  • Primus Pilus
    (1,018 posts)

Posted 23 February 2012 - 09:22 PM

Allright then. Indeed, it's more logical that way.

To do some "idiotic stuff", I've tried renaming the app bundle... Still works.

Atlas starts, but I get a wxWidgets error when I try to move around :"../src/osx/cocoa/combobox.mm(114): assert "item >= 0 && item < [m_comboBox numberOfItems]" failed in SetSelectedItem(): Inavlid item index."
Wraitii
Wildfire Games Programmer, AI developer, auxiliary map designer, dealing with anything water.
Contact me: wraitii@wildfiregames.com
Also the world's only three-dimensional poodle.

#77 historic_bruno

historic_bruno

  • WFG Programming Team

  • Primus Pilus
    (1,930 posts)

Posted 23 February 2012 - 10:09 PM

View Postwraitii, on 23 February 2012 - 09:22 PM, said:

To do some "idiotic stuff", I've tried renaming the app bundle... Still works.
Excellent, it's good to test these things :)

Quote

Atlas starts, but I get a wxWidgets error when I try to move around :"../src/osx/cocoa/combobox.mm(114): assert "item >= 0 && item < [m_comboBox numberOfItems]" failed in SetSelectedItem(): Inavlid item index."
Oh, that's not good. What action causes the error? I tried opening different maps and using various tabs/controls but couldn't reproduce it. If you have gdb, can you try something like this to capture the call stack:

$ gdb 0ad.app/Contents/MacOS/pyrogenesis

(gdb) run -editor

*crash*

(gdb) bt full

Ben Brian [ aka historic_bruno ]

Wildfire Games Programmer
Contact me: ben@wildfiregames.com

#78 historic_bruno

historic_bruno

  • WFG Programming Team

  • Primus Pilus
    (1,930 posts)

Posted 24 February 2012 - 02:56 AM

View Postwraitii, on 23 February 2012 - 09:22 PM, said:

Atlas starts, but I get a wxWidgets error when I try to move around :"../src/osx/cocoa/combobox.mm(114): assert "item >= 0 && item < [m_comboBox numberOfItems]" failed in SetSelectedItem(): Inavlid item index."
OK this is reproducible even without an app bundle, I believe the cause has been found and will be fixed in SVN.
Ben Brian [ aka historic_bruno ]

Wildfire Games Programmer
Contact me: ben@wildfiregames.com

#79 General Octavian

General Octavian

  • Community Members
    Pip

  • Discens
    (67 posts)

Posted 24 February 2012 - 03:05 AM

I downloaded the app and atlas works well and the game runs well :)

#80 historic_bruno

historic_bruno

  • WFG Programming Team

  • Primus Pilus
    (1,930 posts)

Posted 24 February 2012 - 09:17 PM

View PostGeek377, on 20 February 2012 - 05:12 AM, said:

@Ben, don't worry about Mountain Lion just yet. Save that for summer :P
I couldn't help it, I "peeked" and here's what I found out :P

  • Mountain Lion won't be including X11, if an app requires X11 to run, the user is prompted to visit Apple's website with instructions on how to install XQuartz
  • Apple is deprecating more unspecified parts of Carbon (C/C++ API for OS X, also primarily 32-bit), obviously pushing people into the Objective-C/C++ 64-bit Cocoa API
  • There's a new feature called Gatekeeper, it blocks apps from running if they don't have a special signature from Apple, you have to be an approved Mac developer to acquire one, assuming that means paying at least $99/yr. Modifying a trusted app makes it untrusted and the OS X firewall will block it.

:(
Ben Brian [ aka historic_bruno ]

Wildfire Games Programmer
Contact me: ben@wildfiregames.com
Back to Game Development & Technical Discussion


1 user(s) are reading this topic

0 members, 1 guests, 0 anonymous users

  1. Wildfire Games Community Forums
  2. → 0 A.D.
  3. → Game Development & Technical Discussion
  4. IMPORTANT! Please read before posting!