16 replies to this topic

[Kazu Kun]

So I have been looking around, and I don't really see anywhere where we would be able to update the game?
Are you guys going to just release a patch for the game, and then a separate copy of the full game?
Or how are you going to go about it?

[fcxSanya]

Kazu Kun, on 10 February 2012 - 09:42 AM, said:

So I have been looking around, and I don't really see anywhere where we would be able to update the game?
Are you guys going to just release a patch for the game, and then a separate copy of the full game?
Or how are you going to go about it?

Currently the game is in alpha stage, so we just release installers/packages of full version of 0 A.D. for each new alpha every couple of months. Those, who want to follow development more closely can use the SVN version and update it whenever they want. In the beta stage it probably will be the same and AFAIK for open-source games this model usually used after alpha/beta stage too.
Patch is something more related to commercial games world, with it's deadline rush before fixed release date, which is set be publishers without much taking into account the state of the project; then feature complete, but buggy release, complaining users/reviewers and a post release bug-fixing rush with 1.0.0.0.X patches.
In open source games there is usually no one feature complete release, but rather incremental releases with either bugfixes and new features with relative long testing phase. And release will be rather delayed than released with known bugs or untested functionality. And all bug fixes and new features even if they are done after few days after release are delayed until the next one.
We have a plan what features we want to have in 1.0 version and what features we can delay until later, so when all needed features will be ready we will switch from alpha stage to beta and work on fixing bugs/balancing etc. until the game will be ready to release, then we will release 1.0 version and theoretically there should not be any small patches right after this. We are not sure what will be next: 1.1 or 1.5 or 2.0 (i.e. how large will be changes in subsequent releases).
As far as I understood ricotz does some 'update' packages in 0ad.dev PPA (Ubuntu packages). And there was some discussion on IRC about something like build-in update system, but we don't have any steady plans about it.

[Kazu Kun]

Ah, Okay

Thank you very much, that pretty much answered everything about my question in every perspective

[Sonarpulse]

Built in update systems are cumbersome and usually platform specific. But something I that would be nice is to use jigdo or zsync to allow users to quickly update their installers. In addition, once either of those programs make a new "base", the actually updating can be cancelled, and the incomplete download can be used with BitTorrent. That combination is a very efficient way to roll out an update for alphas or releases, and requires no upstream changes.

[Ykkrosh]

It's not possible to incrementally update the installer .exe files (or equivalents on other platforms) - they're compressed so their content becomes completely different whenever a single byte is changed. Any kind of patching system would need to be slightly more integrated with the post-installation game, so it can use the old installed public.zip as the base for the new version.

[Sonarpulse]

Ykkrosh, on 12 February 2012 - 11:35 AM, said:

It's not possible to incrementally update the installer .exe files (or equivalents on other platforms) - they're compressed so their content becomes completely different whenever a single byte is changed. Any kind of patching system would need to be slightly more integrated with the post-installation game, so it can use the old installed public.zip as the base for the new version.

see: http://zsync.moria.org.uk/

Quote

zsync has special handling for gzipped files, which enables update transfers of files which are distributed in compressed form.

Don't winzip and gzip use the same underlying compression algorithm? I'd hope that zsync's logic would work on that algorithm alone and not be hung up on the differences between *.tar and winzip's built in archive method

[Potter]

@Sonarpulse
zsync (from what is said in their website) is basically just a file copy/sync program.
So, like Ykkrosh said, it can be used to integrate with an already installed game to sync any updated files from a central server.

There is a difference between an installer and an installed game. An installer contains all files to be installed (along with any metadata) in a compressed form. An installed game contained all the files in an uncompressed form. So, updating an installer does not really make sense and make require lots of unnecessary technical workarounds. That's why all patching systems target updating the installed game, as it is much much simpler.

And, gzip is one of the compression algorithms widely used. So, it can be used to compress the patched files, but the percentage of compression varies depending upon the type of file compressed.

[Ykkrosh]

Sonarpulse, on 12 February 2012 - 09:46 PM, said:

Don't winzip and gzip use the same underlying compression algorithm? I'd hope that zsync's logic would work on that algorithm alone and not be hung up on the differences between *.tar and winzip's built in archive method

See http://zsync.moria.o...er/ch03s02.html:

Quote

More complex formats, like zip files, would introduce too much complexity into the client, as it is unclear what one would do with the uncompressed stream corresponding to such a file: it needs the metadata from the compressed archive to be useful.

The Windows installer uses the LZMA compression algorithm (plus whatever headers NSIS adds), which is even less likely to work. The installed game's public.zip is uncompressed so something like normal rsync/zsync would work for that.

[Sonarpulse]

Hmm, good link. I also read http://zsync.moria.org.uk/server and https://lists.ubuntu...uly/028529.html

Can windows installers be made uncompressed, and public.zip compressed (i guess as a gzip) instead?

[Ykkrosh]

The current approach (public.zip with no internal compression, then everything packed together and compressed with LZMA by the installer) minimises the initial download size - any other combination will make it larger. Most people will probably only play the game once, so I think it makes sense to optimise for initial download size, rather than doing a tradeoff of worse initial size for better incremental update ability. (And trying to reuse off-the-shelf tools probably wouldn't be any easier than developing a custom updater that incrementally updates the installed public.zip files, which would allow the best initial download size plus efficient updates.)

[Sonarpulse]

how much data is outside public.zip?

Yeah, as soon as I read that headers cause such problems, even with known compression algorithms, I was worried. Hopefully zsync can be upgraded someday. It could be extraordinarily usefully combined with a package manager.

[Ykkrosh]

Everything is in public.zip, except for .exe/.dll/etc files on Windows, and except for binaries/data/tools/ and binaries/data/config/.

[Sonarpulse]

wouldn't that mean that compressing public.zip and uncompressing the installer would hardly change the installer size? (besides the fact that LZMA is better than DEFLATE) (public zip could then be uncompressed after being downloaded so runtime speeds were not affected).

Also, are there any large compressed files within public.zip?

[Ykkrosh]

Sonarpulse, on 25 February 2012 - 07:21 PM, said:

(besides the fact that LZMA is better than DEFLATE)

That's a critical fact and makes a big difference . (Also, zip compresses each file individually, while the installer compresses everything as a single chunk so it can exploit redundancy between files too.)

Quote

Also, are there any large compressed files within public.zip?

I think the main ones are lots of Ogg Vorbis files (which are hard to compress further), and lots of DDS textures (which are compressed but are easy to compress further).

[Sonarpulse]

Can the installer be made on linux? I think I might want to test uncompressed installer + compressed public.gz (Whoops, that's what i meant to say) with zsync.

[Ykkrosh]

The Windows installer can be - source/tools/dist/build.sh creates it (plus the Unix tarballs etc), using NSIS via Wine. (NSIS can be run natively on Linux instead, but it needs to be compiled in 32-bit mode so it's a pain on 64-bit Gentoo and I stuck with Wine instead.)

[Sonarpulse]

Awesome! hopefully i can get to that soon, and maybe edit the script to work with wine and native unix too (I am on 32 bit). It should come in handy if/when a 64-bit version of the game is made.