Light Style© by Fisana

Jump to content


Photo

Build environment and deployment on the Mac


  • Please log in to reply
325 replies to this topic

#21 historic_bruno

historic_bruno

    Primus Pilus

  • WFG Programming Team
  • 2,491 posts

Posted 15 January 2012 - 11:43 PM

I like the idea of that Wiki page, I updated it with a table for Windows and clarified some things. Might I suggest we combine all three OS paths tables into one? They should have the same number of entries and that way it would be easier to compare at a glance.
  • 0
Ben Brian [ aka historic_bruno ]

Wildfire Games Programmer
Contact me: ben [at] wildfiregames [dot] com

#22 historic_bruno

historic_bruno

    Primus Pilus

  • WFG Programming Team
  • 2,491 posts

Posted 16 January 2012 - 01:20 AM

perhaps all other data could go in "~/Documents/0ad" - roughly equivalent to My Documents on Windows?

Some opposing viewpoints:

http://vgable.com/blog/2010/06/02/nshomedirectory-is-a-bad-thing/
"Code that uses NSHomeDirectory() is probably doing The Wrong Thing. Itís not appropriate to clutter up the userís home directory ó internal application-data should be stored in the Application Support directory (or a temporary file if itís transient). So I canít think of a good reason to get the path to the userís home directory. Every use of NSHomeDirectory() Iíve seen is spamming the home directory, or getting a subdirectory in a brittle way."

http://cocoawithlove...on-support.html
"On the Mac, the correct location to store persistent user-related files for your application is in a directory with the same name as your application in the Application Support directory for the current user."


They suggest using NSSearchPathForDirectoriesInDomains to find the user's Application Support directory (instead of something like getting the $HOME environment variable, or trying to construct the path manually), which is part of the Cocoa API, so we'd probably need to write a C wrapper in order to use it.

Apple also has some clarifications about when to use ~/Library/:

Application Support directory: The Application Support directory is where your app stores any type of file that supports the app but is not required for the app to run, such as document templates or configuration files [...] To get the path to this directory use the NSApplicationSupportDirectory search path key with the NSUserDomainMask domain.

Caches directory: The Caches directory is where you store cache files and other temporary data that your app can re-create as needed. This directory is located inside the Library directory.


~/Documents is not even listed as a possibility.
  • 0
Ben Brian [ aka historic_bruno ]

Wildfire Games Programmer
Contact me: ben [at] wildfiregames [dot] com

#23 Echelon9

Echelon9

    Discens

  • Donator
  • 87 posts

Posted 16 January 2012 - 12:38 PM

Yes using ~/Documents isn't right.~/Library/Application Support/0AD is the way to go
  • 0

#24 Yves

Yves

    Primus Pilus

  • WFG Programming Team
  • 1,028 posts

Posted 16 January 2012 - 07:46 PM

I think you're right, I applied the suggested changes made earlier in this thread and didn't investigate further if those are correct.

The Caches directory is where you store cache files and other temporary data that your app can re-create as needed. This directory is located inside the Library directory.

Cache changed from "~/Library/Application Support/0ad/cache" to "~/Library/Caches/0ad"

The Application Support directory is where your app stores any type of file that supports the app but is not required for the app to run, such as document templates or configuration files.

Changed Userconfig from "~/Documents/0ad/config" to "~/Library/Application Support/0ad/config"

About Application Support directory:

The files should be app-specific but should never store user data.


About "Documents"

Important The files in the userís Documents and Desktop directories should reflect only the documents that the user created and works with directly. Similarly, the media directories should contain only the userís media files. Those directories must never be used to store data files that your app creates and manages automatically. If you need a place to store automatically generated files, use the Library directory, which is designated specifically for that purpose.

Screenshots, Savegames and User Mod definitely go into that category in my opinion and we should keep them as they are now in our table.

I'm not quite sure about the Mods and Logs.
My suggestions are as follows...
Mods: I agree on "~/Documents/0ad/mods". The user will probably want to download a mod somewhere and copy it to a directory, so we could say he "works with it directly". This directory should also be visible (Library is not).

Logs: The user doesn't create logs manually, but he should be able to find them and therefore they should not be in a hidden directory.


Another thing:
Userconfig root doesn't make sense at all in my opinion because we already have Userconfig and I can't see any relation of Logs to configuration.
I've marked this line for removal from the table.


Do you agree on the suggested changes?



The quotes are from:
The Mac Application Environment
File System Programming
  • 0

#25 historic_bruno

historic_bruno

    Primus Pilus

  • WFG Programming Team
  • 2,491 posts

Posted 16 January 2012 - 09:13 PM

Cache changed from "~/Library/Application Support/0ad/cache" to "~/Library/Caches/0ad"
Changed Userconfig from "~/Documents/0ad/config" to "~/Library/Application Support/0ad/config"

Agreed.

About "Documents"

Screenshots, Savegames and User Mod definitely go into that category in my opinion and we should keep them as they are now in our table.

The thinking behind "user mod" is that it will always be mounted in the VFS, unless they're an SVN user, and everything that the game tries to write in public/ now, will go in the user mod directory instead. So in that sense, it's not user managed, because the user doesn't have a choice to save things there (until we have a mod editor), but they could certainly copy files in and out of that directory. The only time we give the user a file dialog to save/load files is with Atlas-related tools, but this is wrong because they don't directly access the underlying file system, going through the VFS instead.

The fact Application Support is hidden on Lion is troublesome, that might be the best argument for storing mods elsewhere :(

Saved games are definitely managed by the game, as the user doesn't get to choose the names or where they are located. We'll be adding a "delete" button to the load game menu, so at that point they will be fully managed by the game.

It looks like Steam uses Documents at least for screenshots:

https://support.steampowered.com/kb_article.php?ref=4545-SDXV-0334
Locate the screenshot image which was created after closing the game (screenshots will be saved to the game folder under your username folder in the SteamApps directory).
Example: /Documents/Steam Content/[account name]/team fortress 2/tf/screenshots


But if you want to move Steam game data to a new computer, you only move files in Application Support:

http://osxdaily.com/2011/06/18/move-steam-games-save-files-to-a-new-hard-drive/
1. Navigate to your existing Steam library, located at: ~/Library/Application Support/Steam/
2. Copy the entire Steam folder and itís contents to that exact same location on the new hard drive (~/Library/Application Support/)


Another argument for using Application Support for mods, it provides an easy single location to uninstall game data or move to a new computer.
  • 0
Ben Brian [ aka historic_bruno ]

Wildfire Games Programmer
Contact me: ben [at] wildfiregames [dot] com

#26 Yves

Yves

    Primus Pilus

  • WFG Programming Team
  • 1,028 posts

Posted 16 January 2012 - 09:36 PM

Saved games are definitely managed by the game, as the user doesn't get to choose the names or where they are located. We'll be adding a "delete" button to the load game menu, so at that point they will be fully managed by the game.

The user doesn't choose where to put the files, but he at least triggers creating/deleting of savegames.
Also the savegames aren't tied to a specific system and someone could want to copy them from one computer to another, share them with friends or whatever.
I would tend to say they are more user-managed than game-managed.

About the mods I'm not sure because I don't know enough details about how they will work and how they will be distributed and used.
Probably Application Support is better.
  • 0

#27 Echelon9

Echelon9

    Discens

  • Donator
  • 87 posts

Posted 16 January 2012 - 10:37 PM

Discussion is progressing well.

In relation to mods and total conversions, I'd like to suggest that they should go in /Applications/0ad/ and then be dealt within the VFS system (which I'm not fully across just yet) - whether that means in practice a zip file in the top level folder, a sub-folder or someother compressed archive.

To my mind where a mod changes -- or even potentially totally converts gameplay elements in a total conversion -- the adoption of those changes should be quite clear and deliberate, almost forming a new game. This is why it makes sense to have them within the /Applications/0ad/ folder.
  • 0

#28 historic_bruno

historic_bruno

    Primus Pilus

  • WFG Programming Team
  • 2,491 posts

Posted 16 January 2012 - 10:46 PM

The VFS (virtual file system) means you can mount real OS paths as relative to a virtual root in the VFS, then almost every file the game uses goes through VFS (the virtual path gets translated to a real path by I/O functions). You can mount multiple real paths to the same VFS directory, with different priorities. Basically it's a way of abstracting how different OSes and file systems handle paths, and allows us to easily support mods that overwrite parts or all of the game data. Atlas-based tools also use VFS, they pass messages from the UI to the game engine, which handles the I/O. Currently it's a bit messed up because although VFS reads follow the priorities system, writes do not, making the concept of a user mod useless until that's resolved.

IMO, total conversions should consist of a new app bundle, because it's likely the engine itself will be changed. In the case of other mods, I don't think we should place anything in or with the app bundles after distributing them.
  • 0
Ben Brian [ aka historic_bruno ]

Wildfire Games Programmer
Contact me: ben [at] wildfiregames [dot] com

#29 Yves

Yves

    Primus Pilus

  • WFG Programming Team
  • 1,028 posts

Posted 17 January 2012 - 07:36 PM

I think we have enough overview and can start doing some work. We can still change any of those paths later if we need to.
I've added some description to each task on the wiki-page and made a note who is assigned to that task.
Feel free to take an unassigned task.
I've only taken those tasks I already started working on, but If nobody wants to do the others, I will take care of them. :)

@historic_bruno
I've assigned you one of the tasks because I think it covers one of the ticket's you are assigned to.
Is that OK?


edit: wrong link.
  • 0

#30 Juicyfruit

Juicyfruit

    Tiro

  • Community Members
  • 8 posts

Posted 17 January 2012 - 08:10 PM

I would also like to contribute some libraries or look at the build script for them.

Maybe we can collaborate on github or gitorious or something as a sort of staging area before it goes into the main tree.
For the short time we could make a script that just does a fix-up on the resulting binary by moving files into it and calling install_name_tool.
I think we should focus on making one bundle that works on snow leopard and lion i386 and x86_64. We should links some things static ( thinking about boost specifically ). But most of all we should work to get it working really well soon then use that experience to fix the building process properly.
  • 0

#31 Yves

Yves

    Primus Pilus

  • WFG Programming Team
  • 1,028 posts

Posted 17 January 2012 - 08:38 PM

Maybe we can collaborate on github or gitorious or something as a sort of staging area before it goes into the main tree.

Not sure if we need that. I think we won't change many existing files.

For the short time we could make a script that just does a fix-up on the resulting binary by moving files into it and calling install_name_tool.

That's quite similar to what I'm doing with my script except that I'm not using the bundle the current build-process creates but create a new one instead.
What else could we do in the future? The installnames are predefined by our repository structure because after building it should work as it is for testing and development.


I think we should focus on making one bundle that works on snow leopard and lion i386 and x86_64.

Currently we tend to only support x86_64 on the Mac. Most of the clients support that and it makes it a lot easier in the first place.

We should links some things static ( thinking about boost specifically ). But most of all we should work to get it working really well soon then use that experience to fix the building process properly.

Why should we link it static? Changing the installname is easy.
  • 0

#32 Juicyfruit

Juicyfruit

    Tiro

  • Community Members
  • 8 posts

Posted 17 January 2012 - 08:59 PM

Why should we link it static? Changing the installname is easy.


It seems common practice for boost on os x bundles I could find that use boost. Also when we dont use much functionality of a library and only one executable is going to use it anyway we might as well link it static to get a smaller total size faster load time etc.

Not sure if we need that. I think we won't change many existing files.

For getting together a set of well build libraries to populate the libraries directory to build agains. I used some extra flags to make nvtt not build support for cude/cg/jpeg/png/openexr for example. Its not like the game uses much more from there then the texture encoding things. It would be nice to hash out and share the libraries we build. Also due to the api changes between 10.6 and 10.7 we need to check that the SDL patches or snapshot we end up using does indeed work nicely on both platforms. I think the lion api it uses also works on snow but not sure, the comment in the code states the it uses the newer functionality, but it uses the old one for 10.6 even though the new way is available. Its the little things that make the resulting binary work well for a lot of people. Also I would like confirmation if openal-soft works as well for other people as it does for me. I dont have access to snow leopard at the moment but would love the feedback.

One of the reasons I am trying to make cmake work is that I dont like premake or know much about it. However for building gnu makefiles it seems very well suited. But then we need a script to add some OS X fancy things to the bundle it produces. Or am I wrong. For that we need an icon, a plist a direcotry to store libraries / frameworks. (Do we store dylib's in libraries and frameworks in frameworks ? ).
  • 0

#33 Yves

Yves

    Primus Pilus

  • WFG Programming Team
  • 1,028 posts

Posted 17 January 2012 - 09:36 PM

About the repository, It's probably useful if we want to share the data more often and collaborate more closely.
I've never made a repository on github/gitorious, so if you could create it, I'd support it and upload my work in progress there too. :)

It seems common practice for boost on os x bundles I could find that use boost. Also when we dont use much functionality of a library and only one executable is going to use it anyway we might as well link it static to get a smaller total size faster load time etc.

I will have to think a bit more about it because I don't really have important pro or contra arguments ;).

But then we need a script to add some OS X fancy things to the bundle it produces. Or am I wrong.

Yes, that's true.
I guess a few lines of code can explain best what I meant (see attachment).
It's a very early work in progress, so don't expect too much. ;)

Attached Files


  • 0

#34 Echelon9

Echelon9

    Discens

  • Donator
  • 87 posts

Posted 18 January 2012 - 05:11 AM

Keep in mind that Xcode4 can integrate the processes of creating properly formed bundles, as a post-compilation step, including copying files/folders into the app bundle.
  • 0

#35 Sonarpulse

Sonarpulse

    Sesquiplicarius

  • Community Members
  • PipPip
  • 166 posts

Posted 20 January 2012 - 05:54 AM

Seeing that github is brought up:
https://github.com/0ad
this is official, right?
  • 0
Posted ImagePosted ImagePosted Image

#36 Jeru

Jeru

    Factotum

  • Web Development Team
  • 4,990 posts

Posted 20 January 2012 - 08:31 AM

@Sonarpulse: Yes! :)
  • 0

Aviv Sharon [ aka Jeru ]

Wildfire Games 0 A.D. PR & Social Media Contributor
Contact me:
E-mail & Google Talk: aviv dot sharon at gmail dot com
MSN: lc_jerusalem at hotmail dot com
Facebook, Twitter, LinkedIn


#37 Yves

Yves

    Primus Pilus

  • WFG Programming Team
  • 1,028 posts

Posted 22 January 2012 - 10:49 AM

Task nbr 6 completed in changeset 10946. I've closed ticket #947 because it's about the broken app bundle.
I've now also taken Task nbr 1 because that's a requirement for even a first basic app bundle.

Btw. I was looking at some codeblocks-problems on OSX but gave it up. Premake is really a mess on those "less supported" platforms and I hope CMake is much better.
  • 0

#38 feneur

feneur

    Cartographer of imaginary worlds

  • 0 A.D. Project Leader
  • 7,985 posts

Posted 22 January 2012 - 01:29 PM

Task nbr 6 completed in changeset 10946. I've closed ticket #947 because it's about the broken app bundle.
I've now also taken Task nbr 1 because that's a requirement for even a first basic app bundle.

Btw. I was looking at some codeblocks-problems on OSX but gave it up. Premake is really a mess on those "less supported" platforms and I hope CMake is much better.

Does those tasks have "proper" Trac tasks?
  • 0

Erik Johansson [ aka feneur ]

Wildfire Games
Contact me: feneur@wildfiregames.com



Support Wildfire Games!


#39 Yves

Yves

    Primus Pilus

  • WFG Programming Team
  • 1,028 posts

Posted 23 January 2012 - 06:17 PM

Does those tasks have "proper" Trac tasks?

Some of them have more or less ;).
Different people created trac-tickets about the same thing or similar things. You see in the article that I tried matching some of the existing tickets to tasks but some tasks are still without ticket. I will create new tickets for those tasks without tickets if that's how it should be done. :)
  • 0

#40 feneur

feneur

    Cartographer of imaginary worlds

  • 0 A.D. Project Leader
  • 7,985 posts

Posted 23 January 2012 - 09:09 PM

Some of them have more or less ;).
Different people created trac-tickets about the same thing or similar things. You see in the article that I tried matching some of the existing tickets to tasks but some tasks are still without ticket. I will create new tickets for those tasks without tickets if that's how it should be done. :)

The main thing is that things are being done, the exact way doesn't matter that much. Since we do use Trac in general it's probably best to use it in this case as well, to make it easier to keep track on what's being worked on and not, and as you say there are some pre-existing already etc, so it's probably the least confusing in any case :)
  • 0

Erik Johansson [ aka feneur ]

Wildfire Games
Contact me: feneur@wildfiregames.com



Support Wildfire Games!