About Gallery SF.net Project Bug tracker Downloads Forums Manual Wiki DevBlog News

Archive for June, 2008

Extensions Renaming

Thursday, June 12th, 2008

Recently, there were a lot of questions on why we are renaming extensions for image files and textures. The changes were started after the 0.5.0 release and can be found in SVN only at this point in time. They will be in released versions from 0.5.1 on.

In 0.5.0 we had png, jpg, and bmp and the extensions did not necessarily represent the codec used in the file. Having files with .bmp extensions that in fact were .jpg, or .png, was completely messy. Also, with 0.5.0 we have switched all in-game graphics (images and textures) to use one of the S3TC DDS compression algorithms (DXT1, DXT3, or DXT5. See info on compression in http://en.wikipedia.org/wiki/S3_Texture_Compression). This means that the exiting extensions became misleading and obsolete.

The S3TC (DDS) compression is currently the most widely used and has the best tradeoff between memory/bandwidth consumption and graphics quality for most of the cases as compared to other compression algorithms like Dithering, 3Dc, or FXT1 (see comparison in http://www.digit-life.com/articles/reviews3tcfxt1/) and is being supported by all GPU manufacturers and the two most prominent graphics APIs, DirectX and OpenGL. This still does not mean that it will remain that way in the next 2-5 years (for example see the specialized normal map compression algorithm in http://ieeexplore.ieee.org/iel5/4089190/4089191/04089271.pdf).

The above issues (compression and naming mess) are the two main reasons that have lead us to the conclusion that renaming graphics extensions is of advantage giving us at the same time clarity about the contents of the image or texture and preparing the ground for any future codec changes that might come on the way. With this changes we are bringing our support for texture compression up to state-of-the-art.

The renaming is well underway with only small portions of data still open up for conversion. The new extensions come in 2 flavors:

  • .texture contains, besides the base texture, mipmap levels and is used everywhere where objects are drawn at different distances, i.e. in space. The GPU will automatically switch the mipmap level according to the object’s distance from the viewer. There is no reason to draw 1024×1024 pixels when a ship is 15km away and only visible as one or two pixels on the screen. Textures can come with transparency (e.g. sun flares) or without (ship diffuse textures).
  • .image files are graphics without the mipmaps and are used for all graphics where the distance to camera does not change. Space backgrounds, cockpit gauges and HUD images are the places where we apply the image type in space. Mipmap-less images are also used for splash screens, cargo/weapons/upgrades, base and planet backgrounds, to name some of the used places. They equally can come in transparent or opaque flavors.

There is no association whatsoever with specific directories where the files reside. For example, under the animations folder we will find subdirectories with .image types (for comm animations or splash screens) and other subdirs with .texture types for example for explosions or blinking lights.

The idea was that .image and .texture aren’t misleading, there are no existing file formats with that extension. And, since it’s format-agnostic as well, it will also make things easier to maintain in the future since a change in format (say, make them png) needs no change to the resources referencing them (like sprite files, animation files, meshes, or system definition files). And this is the point that makes the rename really necessary, since especially mods have asked for support of higher-quality textures by replacing the dds textures with lossless png textures, or high-quality jpegs. If we said “let’s rename them to .dds”, then the mod would have to have a dds file that was a png. So the only real choice is the format-agnostic choice. Since, over time, the desired codec used for images has changed and will be changing with the available technology, we decided that a codec agnostic extension was the best course of action to stop the constant confusion that people have been experiencing when the filename’s extension doesn’t match the real codec used.

The renaming might be annoying for those who have to re-download all graphics again. Though we were very careful to use the svn move command for renaming, it appears that svn doesn’t want to behave correctly in all the cases and, instead of moving also the files in your working copy, makes a completely new download. But this one renaming and re-download is to stop any need to change the names ever again whenever we change the underlying data type and to stop user confusion over conflicting info. It should have been done long ago, but has always been put aside until today.

The recent folder structure was also changed on they was and is now as follows:

  • data holds the dds compressed images and textures
  • masters holds the source and project files used to create the art and master images with the correct codec-dependent extensions.
  • hqtextures holds the non-dds compressed high quality png or jpeg files (though with the same extension as in data).

This means that the data folder isn’t intended for artists at all. It’s only VS UTCS game data. You have the masters repository where things have proper extensions and are artist-friendly, but data needs not be so. This means that you shouldn’t be editing data images but always edit/generate master images and then export to data. hqtextures are optional if you want to see in-game high quality graphics. It will not be compressed on loading into the GPU, so beware, the memory consumption will increase. You have been warned.

With the renaming comes the question: how can I view the images and textures without resorting to a complete masters repository download? It is possible, however your viewer will be required to understand the dds format. For GIMP (Win/Posix) and IrfanView (Win) there are plug-ins available that can read dds files (see http://vegastrike.sourceforge.net/wiki/Links:Graphic_Applications#2D_Graphic_Converters for a list). Several viewers come with native dds support (e.g. KDE supports DDS hence also GwenView, KView, and Konqueror). If in trouble, try renaming the graphics to .dds or .png and see if your application can read it.

Actually, all software should rely on file magic mime typing to determine the codec type of the file (where such a distinction can be made) and not extensions. Extensions are for humans. In theory, no application worth using should give any problem about .image or .texture file loading if it has the appropriate decoder installed. Unfortunately, the reality can be different and a bit more annoying. In Windows, there are several ways of associating extensions with specific applications (e.g. Open With../Always use this program…), which should ease the desktop integration of the new extensions. In Linux you can do a similar operation (Open With…, select the application e.g. GwenView or KView, and check “Remember application association…”) and will be able to view the images directly from Konqueror. While there might be other ways under Linux which I haven’t yet explored, Mac remains black magic to me.

Hopefully this clarifies things up a bit.