IPB Style© Fisana

Jump to content


Build environment and deployment on the Mac


  • Please log in to reply
189 replies to this topic

#41 historic_bruno

historic_bruno

  • WFG Programming Team

  • Primus Pilus
    (1,924 posts)

Posted 04 February 2012 - 09:54 PM

About prebuilt dependencies: I have a few concerns about how this is progressing:

  • By getting rid of the Mac Ports requirement, we lose the nice pkg-config tool, which is what non-Windows builds currently use to get include and lib paths for most dependencies. For some this is not a problem, because they are simpler and self-contained, but for example SDL and wxWidgets need many frameworks and other dependencies as well as compiler directives which need to be included based on the build configuration.
  • Luckily we have sdl-config and wx-config, but it should be noted those tools only return absolute paths from where they were installed (with make install). So my dependencies build script runs make install as a final step for each of them (to a nearby directory so sudo is not required), but that means the results are not really reusable across systems (imagine you have an SVN checkout in /path/to/ps/ but I have it in /some/other/path/to/ps/, well then my bundled wx-config and sdl-config wouldn't work for you).
  • Normally these files go in standard OS-specific locations, so it's the same for each system. So Philip suggested that the dependencies build script should get run every time a new developer wants to build the game on OS X (or more specifically, every time they want to build in a new location).
  • I feel that defeats the concept of prebuilt dependencies, the goal should be to offer them in SVN so all that needs to be done is a checkout of the libraries package for your OS and the game's source. We already do this for Windows, except wxWidgets but I believe we can even include that, since only Windows developers would be forced to download it.
  • The only solution I can think of to fix the lack of pkg-config and the absolute paths of e.g. sdl-config, is to hardcode their results in extern_libs4.lua for OS X, for developers who don't want to build the libraries.
  • So that means we would need two build paths for OS X alone: one for developing with the downloaded prebuilt dependencies (default), and one for those who use the rebuild script (maybe because they can't use the others) or Mac Ports/Homebrew or some other alternative. I guess a flag passed to update-workspaces.sh would select the second case.
  • The worst part about that is the maintenance required, whenever the OS X dependencies changed, they would need to be rebuilt and e.g. sdl-config run to retrieve the necessary info, which would be placed by hand in extern_libs4.lua. That introduces a lot of potential for error, but maybe acceptable if it rarely happens.

Any other ideas? Blender SVN offers prebuilt static libs for multiple versions of OS X, Linux 32/64-bit, and Windows 32/64-bit, and they never ask you to build any of them, the result is an otherwise daunting and time-consuming process becomes much simpler. Of course they don't use Premake, but I assume they had other similar concerns in their build system :)

As far as the time it takes to rebuild the dependencies with my current script, I would say maybe 5 minutes on my PC using 4 parallel build tasks, and considerably longer on older hardware, that doesn't include time required to download the tarballs if necessary (I do a test in the script, if the file doesn't exist, it assumes it should be downloaded first - still need to test that actually works).
Ben Brian [ aka historic_bruno ]

Wildfire Games Programmer
Contact me: ben@wildfiregames.com

#42 historic_bruno

historic_bruno

  • WFG Programming Team

  • Primus Pilus
    (1,924 posts)

Posted 04 February 2012 - 10:03 PM

As far as data paths go, expect a patch for #1145 very soon, taking care of both Windows and OS X at the same time.
Ben Brian [ aka historic_bruno ]

Wildfire Games Programmer
Contact me: ben@wildfiregames.com

#43 Echelon9

Echelon9

  • Donator

  • Discens
    (54 posts)

Posted 05 February 2012 - 06:42 AM

View Posthistoric_bruno, on 04 February 2012 - 10:03 PM, said:

As far as data paths go, expect a patch for #1145 very soon, taking care of both Windows and OS X at the same time.
Sorry, but I really don't like the idea of using ~/Documents on Mac as the file path location of saved games and screenshots. It just doesn't tie into the way any other (broad statement: but I haven't seen any examples) games or Mac apps utilise the file system.

Mac is often about hiding away the complexity of the filesystem (at least for novice, casual users) and providing a nice interface to underlying files and features. Accessing save games makes most sense *within* the 0AD application, I really don't care where they are on the physical file system most of the time. Storing them in Documents is too in the users face, as that location is primarily for user created documents, not outputs of an application or game.

I'm strongly of the view that all generated data should be within ~/Library/Application Support or ~/Libraries/Caches.

#44 k776

k776

  • Open Source Development Manager

  • Centurio
    (658 posts)

Posted 05 February 2012 - 07:04 AM

I agree with Echelon. Saved games should be in ~/Library/Application Support/0ad

Only screenshots should make it into the Documents folder (~/Documents/0ad/)

Same goes for windows. %appdata%/0ad for saved games, %localdata%/0ad for screenshots (I think it's localdata for My Documents on Windows)






Kieran P [ aka k776 ]
Wildfire Games Open Source Development Manager
Contact me: kieran@wildfiregames.com


#45 Yves

Yves

  • WFG Programming Team

  • Triplicarius
    (582 posts)

Posted 05 February 2012 - 10:18 AM

View Posthistoric_bruno, on 04 February 2012 - 10:03 PM, said:

As far as data paths go, expect a patch for #1145 very soon, taking care of both Windows and OS X at the same time.
It's good to see some progress, but it's also frustrating that you have done the same thing I've worked on the last two weekends and was close to getting a patch too.
One purpose of this thread and the Wiki article was to avoid such a situation.
I said I'd do this task and also updated the wiki article accordingly in this post.


About the savegame location. That has been discussed in this thread too.
Savegames are data the user could like to access for making backups or sharing it between computers/people.
At least on Windows some games I know place savegames in mydocuments. My opinion is that they belong into ~/Documents on OSX.


EDIT - Some feedback to the patch:

Game data root:
Your patch doesn't change the behaviour as specified in the table. However, it's still possible to define INSTALLED_DATADIR to make it work in a bundle.
My approach was doing some basic validation of the paths at runtime which is checking if the directory exists in this case. If it doesn't exist, the default directory executable_path/../data/ is used.
However the question is if this path is even needed at all if we can have separate paths for its subdirectories (like mods).

Mod-paths:
I've had a quick discussion with Philip about this topic but didn't update the wiki-article.
We sorted out that we currently have at least three kinds of mod-directories.
1. Deployedmod: Usually public.zip which contains the "default mod" we deploy with the game.
2. mods: the directory containing all other mods.
3. usermod: the directory where atlas saves maps etc...
(4. downloadedmods) there's another potential directory for automatically downloaded mods... but current it should be sufficient to just have "mods" instead.

The reason to have a separate "Deployedmod" directory is to allow us including it in a simple app-bundle.
We can't have the mods-directory in the app bundle.


Directory validation:
That's another problem I tried solving with my path. Currently the game just crashes in most cases if some required directories don't meet the requirements (don't exist, wrong permissions etc...).

User-customizations:
A nice feature would be allowing user-customized paths for all those directories... but that's just nice to have.


My approach

My approach was bit more radical.
I've created an interface-class for well-known system paths and derived an OS-specific class from it for each OS.
class WellKnownSystemPaths
{
public:

virtual ~WellKnownSystemPaths();

// Directory containing the game's data (e.g. binaries/data).
// Required access: Read
	virtual const OsPath DeployedDataDir() = 0;	
// Directory where the main executable is started from.
// Required access: Read
	virtual const OsPath ExecutableDir() = 0;
// Directory containing the bundled libraries
// Required access: Read
	virtual const OsPath CacheDir() = 0;
// Directory containing User-Data. Userdata is basically all data whose creation is triggered
// by the user and which the user might also want to access from outside of the game.
// Required access: Read, Write
	virtual const OsPath UserDataDir() = 0;
// Directory containing the users customized settings for the game
// Required access: Read, Write
	virtual const OsPath UserConfigDir() = 0;
	
private:
};
I've also added a new class representing a game directory and containing all properties and functions such a directory could have.

The definition of this class:
// Validation flags
#define GAMEDIRECTORY_VALIDATE_EXISTANCE	1
#define GAMEDIRECTORY_VALIDATE_WRITE		2

class GameDirectory
{
public:
	// The user is responsible for checking the return-value and acting appropriately!
	bool ValidatePathsAndChoose();
	
	void SetUserPath(OsPath UserPath) { m_Paths[DirectorySources.USER] = UserPath; }
	void SetOSDefaultPath(OsPath OSDefaultPath) { m_Paths[DirectorySources.OS_DEFAULT] = OSDefaultPath; }
	void SetGameDefaultPath(OsPath GameDefaultPath) { m_Paths[DirectorySources.GAME_DEFAULT] = GameDefaultPath; }
	void SetValidationBitmask(Byte ValidationBitmask) { m_ValidationBitmask = ValidationBitmask; }
	
	// Can return undefined results if ValidatePathsAndChoose isnt used properly!
	OsPath GetPath() { return m_Paths[m_ChosenDirectorySource]; }
	
private:
	// Directory sources in order of priority
	enum m_DirectorySources { USER, OS_DEFAULT, GAME_DEFAULT };
	const int m_DirectorySourcesNbrOfElements = 3;
	OsPath m_Paths[m_DirectorySourcesNbrOfElements];
	Byte m_ValidationBitmask;
	DirectorySources m_ChosenDirectorySource;
	
	bool Validate(OsPath Path);
}

As you can see each GameDirectory can contain three paths (USER, OS_DEFAULT, GAME_DEFAULT).
It also contains a validation bitmask specifying the requirements of that directory.

ValidatePathsAndChoose() validates each directory according to its requirements specified in ValidationBitmask in the given order (User, OS, Game).
It will use the first valid path it finds.


A nice aspect of this solution it that you can define all the directories at one place in a consistent way:
	#ifdef USER_CONFIG_DIR
	m_Paths[m_GameDirectoriesEnum.CONFIG].SetUserPath(OsPath(STRINGIZE(USER_CONFIG_DIR))/"");
	#endif
	m_Paths[m_GameDirectoriesEnum.CONFIG].SetOSDefaultPath(WellKnownPaths->UserConfigDir()/subdirectoryName/"");
	m_Paths[m_GameDirectoriesEnum.CONFIG].SetValidationBitmask(GAMEDIRECTORY_VALIDATE_EXISTANCE | GAMEDIRECTORY_VALIDATE_WRITE);



I don't know if this approach is better and I can't provide a patch yet because it's not yet working.
Now it probably doesn't make sense working on it as you already have a working patch.

#46 Ykkrosh

Ykkrosh

  • WFG Programming Team

  • Primus Pilus
    (4,869 posts)

Posted 05 February 2012 - 12:53 PM

View Posthistoric_bruno, on 04 February 2012 - 09:54 PM, said:

Philip suggested that the dependencies build script should get run every time a new developer wants to build the game on OS X (or more specifically, every time they want to build in a new location). I feel that defeats the concept of prebuilt dependencies, the goal should be to offer them in SVN so all that needs to be done is a checkout of the libraries package for your OS and the game's source.
I think the important goal is that building the game should be easy and automatic, not that it should be fast. For first-time developers, the time-consuming part will be downloading the game's code and data, and waiting extra tens of minutes for building won't be a major issue. For repeat developers, they'll only need to rebuild the libraries every few months when changes are made, so slowness won't matter.

On Windows we provide prebuilt libraries because there's no easy automatic way to compile them on the user's machine, and because Windows allows prebuilt libraries to be portable, but it's a bit of a mess since we end up with outdated libraries and mismatched compiler versions. On Linux we build from source because that can be easy and automatic, and prebuilt libraries wouldn't be portable. OS X seems much more like Linux in this.
Philip Taylor [aka Ykkrosh]

Wildfire Games Programmer
Contact me: philip@wildfiregames.com

#47 historic_bruno

historic_bruno

  • WFG Programming Team

  • Primus Pilus
    (1,924 posts)

Posted 05 February 2012 - 10:22 PM

View PostEchelon9, on 05 February 2012 - 06:42 AM, said:

Sorry, but I really don't like the idea of using ~/Documents on Mac as the file path location of saved games and screenshots. It just doesn't tie into the way any other (broad statement: but I haven't seen any examples) games or Mac apps utilise the file system.
The advantage of having screenshots in a hidden folder would be...? :)

If it was so important to Apple that apps use Application Support for *all* data, they shouldn't hide it by default, now they are forcing us to consider alternatives. Also if we were writing a native Mac app, I would agree, we would use the standard file open/save UI. But we're not going to have an OS-specific UI in game AFAIK, the path will be chosen by the game but it's the user who takes the action to create the file. Saved games is a debate worth having, but if there's no strong resistance to having them in My Documents on Windows, then I don't see the problem on OS X either. We can't pretend "OS X users don't care about paths but Windows users do" because that's a false statement.

Everyone can have a preference about this, which is why something like what Yves suggests might work.

View PostYves, on 05 February 2012 - 10:18 AM, said:

It's good to see some progress, but it's also frustrating that you have done the same thing I've worked on the last two weekends and was close to getting a patch too.
One purpose of this thread and the Wiki article was to avoid such a situation.
I said I'd do this task and also updated the wiki article accordingly in this post.
Oh. I think I did read that post, but interpreted it as working on bundle stuff, my patch doesn't have anything to do with bundles. Also I looked on Trac and didn't see a ticket and there was little activity on the forum or IRC, so I just started working on it. I wasn't trying to duplicate effort or step on toes. I don't think our work is totally incompatible, and also it was good to get some experience with Cocoa and Objective-C++ (and I discovered some unrelated bugs meanwhile) :)

I mostly started working on it because I knew it would only take a few days, and then we would have code to discuss what's right or wrong, like we are now so I guess it worked :) No need to throw away what you've been working on.

Quote

Game data root:
Your patch doesn't change the behaviour as specified in the table. However, it's still possible to define INSTALLED_DATADIR to make it work in a bundle.
INSTALLED_DATADIR should be an absolute path, so that doesn't sound like a good idea for bundles which are portable by nature. The paths I changed have nothing to do with bundles, but will be the same for SVN vs. releases.

Quote

My approach was doing some basic validation of the paths at runtime which is checking if the directory exists in this case. If it doesn't exist, the default directory executable_path/../data/ is used.
However the question is if this path is even needed at all if we can have separate paths for its subdirectories (like mods).
I don't understand. Can't we just use CFBundleGetMainBundle or something to see if we're a bundle or not and go with that? I haven't tried it to know if it works reliably (we could also check if there's a .app in the path as an additional validation, which is an "ugly" hack we use somewhere else, but it might work). My feeling is we should use it anyway to get bundle info, like the bundle name, so we can set the paths accordingly (e.g. ~/Library/Application Support/BundleName/). If I'm not mistaken, the VFS already checks if full paths exist or not and creates them accordingly, of course that doesn't help reading the root game data if it's missing, but that's something either the engine (with simple logic) or the installer should get right, rather than being left to user decisions or compile-time options.

Quote

Mod-paths:
I've had a quick discussion with Philip about this topic but didn't update the wiki-article.
We sorted out that we currently have at least three kinds of mod-directories.
1. Deployedmod: Usually public.zip which contains the "default mod" we deploy with the game.
2. mods: the directory containing all other mods.
3. usermod: the directory where atlas saves maps etc...
(4. downloadedmods) there's another potential directory for automatically downloaded mods... but current it should be sufficient to just have "mods" instead.

The reason to have a separate "Deployedmod" directory is to allow us including it in a simple app-bundle.
We can't have the mods-directory in the app bundle.
Yeah, my patch doesn't do anything about mods, and honestly the VFS is in such a state currently that it's not even worth thinking of multiple mods IMO (in fact I recall recent discussions about only allowing a single mod to be mounted as a simplifying case). But if we do, all that needs to happen is, for example: (1) is already mounted in VFS root (readonly), (2)/(4) m_GameData / "mods"/* get mounted in VFS root (readonly), and (3) m_UserData / "mods"/* get mounted in VFS root with read/write permission. Am I missing something?

Quote

Directory validation:
That's another problem I tried solving with my path. Currently the game just crashes in most cases if some required directories don't meet the requirements (don't exist, wrong permissions etc...).
Permissions is a tricky issue and one we try to avoid so far, but it's easy to fix in this case. With sensible locations for all the game's data, there's no need to worry about permissions, because the current user will have full permission for their home directory (where all the runtime data should be created) and read permission elsewhere. And there's no need to worry about whether they exist, because if they are writable runtime data paths, then the VFS will create them for us. The only thing that absolutely has to exist in advance and be readable is the data and binaries bundled with the game, in which cases are these missing or not accessible?

Quote

I don't know if this approach is better and I can't provide a patch yet because it's not yet working.
Now it probably doesn't make sense working on it as you already have a working patch.
It's always worth working on something if it can be done better, because nobody has exclusive rights to change a part of the game's source code :)

View PostYkkrosh, on 05 February 2012 - 12:53 PM, said:

I think the important goal is that building the game should be easy and automatic, not that it should be fast. For first-time developers, the time-consuming part will be downloading the game's code and data, and waiting extra tens of minutes for building won't be a major issue. For repeat developers, they'll only need to rebuild the libraries every few months when changes are made, so slowness won't matter.
That would simplify things quite a bit, right now the simplest case is downloading libraries/osx from SVN, and running the single ./rebuild-libraries.sh script, which downloads, builds, and installs everything, or we can choose to provide the source tarballs, but then it could be argued that doesn't help much because it adds to SVN checkout/update time and doesn't reduce build time, but does increase maintenance.
Ben Brian [ aka historic_bruno ]

Wildfire Games Programmer
Contact me: ben@wildfiregames.com

#48 historic_bruno

historic_bruno

  • WFG Programming Team

  • Primus Pilus
    (1,924 posts)

Posted 05 February 2012 - 11:02 PM

Argument against saved games in (My) Documents: the files have no value without the rest of the game. Screenshots have some intrinsic value, whereas saved games might as well be deleted with the rest of the game data, if it's deleted. Also, I notice Steam used to store all data in Documents, but then a bunch of people complained, so they changed to Application Support.

Related question: should we have an "uninstaller" for OS X? Otherwise we leave a bunch of files behind after the bundle is deleted. The same question could be posed for Windows and Linux.
Ben Brian [ aka historic_bruno ]

Wildfire Games Programmer
Contact me: ben@wildfiregames.com

#49 fcxSanya

fcxSanya

  • WFG Programming Team

  • Centurio
    (646 posts)

Posted 06 February 2012 - 06:14 AM

View Posthistoric_bruno, on 05 February 2012 - 11:02 PM, said:

Argument against saved games in (My) Documents: the files have no value without the rest of the game. Screenshots have some intrinsic value, whereas saved games might as well be deleted with the rest of the game data, if it's deleted.
The game can be easily redownloaded/reinstalled, but you can't get your savegames back once you deleted them. In my experience games either don't delete saved games or explicitly ask about this in the uninstaller dialog. Our savegames probably not so worth as savegames of some RPG with tens of hours invested to gameplay, but still, we can get some unhappy users if they will reinstall the game by some reason and notice that they lost all their savegames.
Alexander Olkhovskiy [ aka fcxSanya ]
Wildfire Games Programmer
E-mail: fcxsanya at wildfiregames dot com

#50 Yves

Yves

  • WFG Programming Team

  • Triplicarius
    (582 posts)

Posted 06 February 2012 - 09:16 PM

Quote

Oh. I think I did read that post, but interpreted it as working on bundle stuff, my patch doesn't have anything to do with bundles. Also I looked on Trac and didn't see a ticket and there was little activity on the forum or IRC, so I just started working on it. I wasn't trying to duplicate effort or step on toes.
No problem, it's not your fault. I guess my current situation (very little time available during the week but a lot time needed to get all the required background knowledge) doesn't fit well for this kind of task and how we are working on it.
Collaboration means being able to give feedback or answer to feedback within a relatively short period of time, being available on irc. etc...
I started working on this because I remember that about a year ago we already discussed about Mac deployment but nobody wanted to work on it and nothing really happened since then.
Now it looks like there's actually some progress and people working on it. I don't want to constrict that progress by taking tasks and not "allowing" other people who can do it faster to work on it. But I also don't want to spend my time on something I don't know if it will be used even if the quality of the final result is good.
At the moment I have even a bit less time than normal and next weekend I'll have to work at least on Saturday. Working with the required pace on this task would be quite hard in that situation. For those reasons I decided to stop or at least interrupt my work on Mac deployment. I'll think about this decision again in about a month, but probably it will already be done by then. :)

I hope nobody misunderstands that in any way. :)
I'll still participate in the discussing when I have time to and I will probably do some smaller tasks in the meantime... and would be happy to get feedback to the minimap-patch. ;)

#51 historic_bruno

historic_bruno

  • WFG Programming Team

  • Primus Pilus
    (1,924 posts)

Posted 06 February 2012 - 11:05 PM

View PostYves, on 06 February 2012 - 09:16 PM, said:

would be happy to get feedback to the minimap-patch. ;)
That's on my TODO list actually, there's so many patches needing review :( Actually I've looked at it a few times, but as it's a long patch, it needs a long review.
Ben Brian [ aka historic_bruno ]

Wildfire Games Programmer
Contact me: ben@wildfiregames.com

#52 stwf

stwf

  • WFG Programming Team

  • Sesquiplicarius
    (162 posts)

Posted 10 February 2012 - 06:53 PM

Hello All,
I made it my goal to get you a double clickable app for 0AD on the Mac, and I am almost there. It all compiles into a framework in XCode, just need to finish getting the linking to work...
.. It has been a pain.

I am going the route of a hand built XCode project, so I'm not sure it fits in with your vision should how it works. Last I read you seemed to want to automate creation of the project? Anyway I could just maintain this myself in github and use it to provide you with an app that will run on a clean system, my goal!

As for the storage issue, things that the app absolutely needs should go inside the app. Things it creates and manipulates itself should go in application support, and things the user can manipulate should go in docs.

So a good app would have a basic maps, themes, scenarios bundled inside it. Space willing all maps should go in there. This way it can be drag copied to a new machine and still work. If that takes up too much space you'll need an installer to put large maps, music, etc inside Application Support.

It would be nice to have an optional 0AD folder inside the Documents folder where I could add themes/mods. I'd like to see saved games there too, so I could manipulate, copy and archive them myself.

Edited by stwf, 10 February 2012 - 06:54 PM.


#53 Guest_afeder_*

Guest_afeder_*

  • Guests

Posted 11 February 2012 - 12:04 AM

A little note about Android, just to make sure people have it in mind if a port to this platform materializes:

1) At least one of the bundled libraries (spidermonkey) need to be patched, if using the approach I've come up with so far. I don't know how this would fit into the plans. Probably the patch can be included in the SVN and then update-workspaces.sh can apply it if instructed to target Android...

2) The filesystem on Android is different from any other platform, so all the paths need to be rethought. As far as I understand now, all apps can access the SD card (path: "/sdcard"). It might be a good idea to designate this as the place to put mods - then anyone can distribute mods through Android Market and have them install themselves in this shared space. Due to space requirements, things like game assets might also be stored on the SD card. Other things, like config and savegames, might be placed in the game app's "special data directory" - this is a directory which no other apps can access, residing in the device's internal flash memory, which is erased when the app is uninstalled.

Edited by afeder, 11 February 2012 - 12:07 AM.


#54 Echelon9

Echelon9

  • Donator

  • Discens
    (54 posts)

Posted 11 February 2012 - 03:13 AM

stwf said:

1328899996[/url]' post='233922']
Hello All,
I made it my goal to get you a double clickable app for 0AD on the Mac, and I am almost there. It all compiles into a framework in XCode, just need to finish getting the linking to work...
.. It has been a pain.

I am going the route of a hand built XCode project, so I'm not sure it fits in with your vision should how it works. Last I read you seemed to want to automate creation of the project? Anyway I could just maintain this myself in github and use it to provide you with an app that will run on a clean system, my goal!
A hand-rolled Xcode4 project file already exists to build an OAD .app bundle.
However, as you noted it needs to be kept updated when source code files are added or deleted vs the automated Premake solution that works nicely on Windows, but poorly on Xcode project files.
http://www.wildfireg...ndpost&p=233049

Edited by Echelon9, 11 February 2012 - 03:16 AM.


#55 stwf

stwf

  • WFG Programming Team

  • Sesquiplicarius
    (162 posts)

Posted 11 February 2012 - 04:44 AM

View PostEchelon9, on 11 February 2012 - 03:13 AM, said:

A hand-rolled Xcode4 project file already exists to build an OAD .app bundle.
However, as you noted it needs to be kept updated when source code files are added or deleted vs the automated Premake solution that works nicely on Windows, but poorly on Xcode project files.
http://www.wildfireg...ndpost&p=233049

missed that! I'm downloading it now and will try it out. So what work still needs to be done on the project to produce a stand alone app?

My strategy was to make the pyrogenesis part a sub-project that builds a framework. So that might make the Premake part easier. Since Premake only needs to build the pyrogenesis subproject. Then you can keep the stuff thats nasty inside the hand made project, stuff that won't change very much.


#56 historic_bruno

historic_bruno

  • WFG Programming Team

  • Primus Pilus
    (1,924 posts)

Posted 11 February 2012 - 10:41 PM

View Postafeder, on 11 February 2012 - 12:04 AM, said:

1) At least one of the bundled libraries (spidermonkey) need to be patched, if using the approach I've come up with so far. I don't know how this would fit into the plans. Probably the patch can be included in the SVN and then update-workspaces.sh can apply it if instructed to target Android...
This is no problem, there are already plans to separate the game's libraries by OS. In other words, we'll have \libraries\win32, \libraries\osx, \libraries\linux, etc. Developers will only checkout whichever directory they need. It wouldn't be too hard to add some Android-specific patches, including whatever eases 0ad development on Android, assuming we decide to support Android.

Quote

2) The filesystem on Android is different from any other platform, so all the paths need to be rethought. As far as I understand now, all apps can access the SD card (path: "/sdcard"). It might be a good idea to designate this as the place to put mods - then anyone can distribute mods through Android Market and have them install themselves in this shared space. Due to space requirements, things like game assets might also be stored on the SD card. Other things, like config and savegames, might be placed in the game app's "special data directory" - this is a directory which no other apps can access, residing in the device's internal flash memory, which is erased when the app is uninstalled.
Sounds good to me. Maybe start a new topic or Trac ticket for this, as you get closer to a working version of the game.
Ben Brian [ aka historic_bruno ]

Wildfire Games Programmer
Contact me: ben@wildfiregames.com

#57 historic_bruno

historic_bruno

  • WFG Programming Team

  • Primus Pilus
    (1,924 posts)

Posted 18 February 2012 - 10:53 PM

OK, my libraries build script is nearly done. The game libraries directory will change to look like this: (edit: updated per Philip's corrections)
libraries
    \-- source (source, headers, and build scripts shared between different OSes)
        \-- cxxtest
        \-- enet
        \-- fcollada
        \-- nvtt
        \-- spidermonkey
        \-- valgrind
    \-- osx
        \-- build-osx-libs.sh
    \-- win32
        \-- bundled static libs and headers for Windows builds, with wxWidgets included
"source" is used to avoid duplicating the same files on different platforms, since our NVTT and FCollada are highly modified from the official releases. Its contents are shared by Unix-likes, OS X, and Windows.

"osx" will contain only a build script, used to download all dependency source tarballs and build them in one step. 64-bit Windows and 32-bit OS X are not supported, yet.

The way of making this clear, I believe, is to have an empty "libraries" directory in SVN, with a readme that explains the further process. You will need to do a separate checkout of the libraries for your OS of choice (separate from the rest of the source code). This also has the benefit of reducing clutter and the checkout size, since Linux users won't be downloading prebuilt Windows libs. Of course BuildInstructions will be modified to reflect this change.

Every new OS X developer will run "build-osx-libs.sh", make some tea/coffee, wait maybe 10-30 minutes until the script finishes, then have all the dependencies necessary to build the game, without needing Mac Ports, though Xcode may still a dependency to get the Unix build tools (there's probably other ways to get them if someone is so inclined, but we're not supporting that yet). Currently it only builds static libs, I see this posing the least difficulties in creating a portable version of the game.

For simplicity, this will be the only build method for OS X, unless we want to complicate things with one or more of the following:
  • Support Mac Ports / other package managers - this would require a lot of conditional handling in Premake because search paths are different and pkg-config would be needed
  • Offer prebuilt libs in libraries\osx as we do on Windows - would save some time in the build, especially on slower computers, but again requires lots of special handling in Premake and hardcoding the results of sdl-config and wx-config (similarly increases maintenance for us)
  • Build dynamic/shared libraries instead - not difficult to do in the build script, but it does require the extra install_name_tool steps. Benefit would be people could swap different library versions in a bundle without having to rebuild the whole game, in case of incompatibilities. Is this is a reasonable concern? I don't know :)
And finally here is what the script builds:
  • SDL 1.2.15
  • Boost 1.48.0
  • wxWidgets 2.9.3
  • libjpg 8c
  • libpng 1.5.8 (the reason for this is even though it's included on Snow Leopard, its version 1.2 is ancient and the game won't build with it due to missing some API features. Seems like less effort to just build it ourselves, unless someone is familiar with libpng development)
  • libogg 1.3.0
  • libvorbis 1.3.2
  • libxml2 2.7.8 (this is also bundled with SL, but I need xml2-config from a consistent version, I think, so that's why I'm building it. Other suggestions welcome)
  • ENet 1.3.3
  • Spidermonkey 1.8.5
  • NVTT (custom)
  • FCollada (custom)

All these libraries are built as 64-bit against SDK 10.6 with minimum required OS X version 10.6, since we're not supporting 10.5 yet (if ever). They are make install'ed into the local directory, so no root permissions are needed, but again this means the absolute path gets built into the *-config utilities. Even moving your build directories will require a full rebuild.
Ben Brian [ aka historic_bruno ]

Wildfire Games Programmer
Contact me: ben@wildfiregames.com

#58 Sonarpulse

Sonarpulse

  • Community Members
    PipPip

  • Sesquiplicarius
    (161 posts)

Posted 18 February 2012 - 11:48 PM

Nice work! Is this is based around premake or something else? I think your current method is good, but I would rename some folders so as other platforms, package managers, etc are support, stuff still makes sense. (I put those other features in the hierarchy too, not saying you should add them now)
Prerequisites
\-- Source
\-- Include
	\-- just library headers (of course)
\-- Dynamic Lib
    \-- android arm (windows 8)
	\-- osx x86_64
	\-- osx x86_32
	\-- iOS arm (windows 8)
	\-- windows x86_64
	\-- windows x86_32
	\-- windows arm (windows 8)
\-- Static Lib
    \-- android arm (windows 8)
	\-- osx x86_64
	\-- osx x86_32
	\-- iOS arm (windows 8)
	\-- windows x86_64
	\-- windows x86_32
    	\-- current contents of libraries
	\-- windows arm (windows 8)

Edited by Sonarpulse, 18 February 2012 - 11:50 PM.

Posted ImagePosted ImagePosted Image

#59 historic_bruno

historic_bruno

  • WFG Programming Team

  • Primus Pilus
    (1,924 posts)

Posted 19 February 2012 - 01:34 AM

View PostSonarpulse, on 18 February 2012 - 11:48 PM, said:

Nice work! Is this is based around premake or something else? I think your current method is good, but I would rename some folders so as other platforms, package managers, etc are support, stuff still makes sense. (I put those other features in the hierarchy too, not saying you should add them now)
Well the library build script has nothing to do with Premake, but of course the results are used by Premake when building the game, so I did have to modify it a bit (mostly extern_libs4.lua). Now I just need to make sure I can copy a build from Lion to Snow Leopard, if it works I'll be happy, attach the script and hopefully we can get the ball rolling on this :) I don't know if the change will be made for Alpha 9 as that's less than a month away.

I think the libraries could possibly be organized further as you say, but that's not really my goal. For instance, there's a lot of Windows dlls in binaries\system that have no use except on Windows, it would be nice to do something about that. Some day :)
Ben Brian [ aka historic_bruno ]

Wildfire Games Programmer
Contact me: ben@wildfiregames.com

#60 General Octavian

General Octavian

  • Community Members
    Pip

  • Discens
    (67 posts)

Posted 19 February 2012 - 05:59 AM

Will this build allow for atlas and sound available on the mac version?




0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users