dabroz.scythe.pl

Today I run my game and was very surprised about mysterious window:

Console opened at 02:58:46 09-Dec-11
        GL_CLAMP_VERTEX_COLOR_ARB is clamped.
        GL_CLAMP_FRAGMENT_COLOR_ARB is clamped.
        There are 0 constants bound by this program.
        Texture 0 uses an 32 bit floating point format.
        Texture 0 is bound to texture target GL_TEXTURE_2D.

If you are wondering what is it, it seems that recent NVidia drivers added this feature if you enable debug flag in your OpenGL 3+ context. Also, debug output quality seems to be improving:

OpenGL message 131218: Program/shader state performance warning: Fragment Shader is going to be recompiled because the shader key based on GL state mismatches. [source: API] [type: performance] [severity: medium]

It's nowhere near DirectX debugging tools, but hey, that's still better than 4-color LED.

Notice: I've created Dab blog. Most post will be mirrored here.

Few weeks passed since first release of Dab. I’ve been working on the codebase, refactoring a lot of code and adding static compilation (i.e. creating executable files). And finally I can introduce to you first standalone game written in Dab.

JIT Shooter is a project that I was using to create first usable version of Dab. And now you can download its .exe binary that doesn’t need Dab IDE.

Download JIT Shooter Standalone

However there are few issues:

• you need OpenGL 3.2+ (to save additional functions calls shaders use explicit attrib locations)
• you need MS C runtime 7.1 (you can get it here, game uses few functions like fopen/fread)
• game uses STB image to load textures, those are compiled as DLL

But still, there is massive work ahead. I’ve removed few hacks just to add different hacks. However, next version (0.3) will feature proper handling of function pointers and few other modifications that will allow to reduce number of hard-coded tricks. Version 0.2 should happen in next few days.

BTW: I’m very happy with executable size. 39KB for playable native game is really not much. And with better loader (now all CRT/OpenGL calls are resolved in runtime very naively) it can be even smaller.

Other news: I’ve set up this blog as I’m not very happy with existing Dokuwiki site. I’ll probably experiment with wordpress wiki plugins instead.

I’ve also created Dab Google+ profile. Feel free to like+1!

Few months ago I've written a rant about OpenGL error handling. The bad news is that its debug capabilities haven't improved much since. The good news is that now I know the history behind this completly insane interface.

In late '70, there was a computer called TRS-80 (from Wikipedia):

Level I Basic was based on Li-Chen Wang's free Tiny BASIC, additional functions added by Radio Shack.[8] It achieved a measure of noteworthiness due in large part to its outstanding manual,[5] written by David Lien, which presented lessons on programming with text and humorous graphics, making the subjects very easy to understand. It had only two string variables (A$ and B$), 26 numeric variables (A - Z) and one array, A(). Code for functions like SIN(), COS() and TAN() was not included in ROM but printed at the end of the book. The only error messages were: "WHAT?" for syntax errors, "HOW?" for arithmetic errors such as division by zero, and "SORRY" for out of memory errors.

Rings a bell?

After few years on Warsztat (a Polish gamedev site) I’ve noticed an interesting phenomenon. Every now and then there are Compos (programming competitions) organized in two different flavours. Some compos are single-run events that last only few hours, others are long-term (several days/weeks). And as an extra catch, the former are usually restricted to basic APIs (SDL, OpenGL etc) while the latter are free-for-all (all sorts of engines, UDK/Unity allowed).

Now, results are somewhat shocking. Much more people participate in short compos than the long ones. But the best part is that quality of games created is just the same, no matter if the games was made in 2 hours or 4 weeks. Why?

1. Well, creating game in 4 weeks doesn’t usually mean neither 672 nor 224 working hours. In extreme cases 4 week compo is just a 2-hours compo, with 2 hours located at the very end of 4 weeks.
2. A lot of game value is an idea. Fact: you won’t come up with better idea in 4 weeks than in 10 minutes.
3. Development process for 2-hours game is very dense. And most of the time is spent on improving core features (because there are no others).
4. On the other hand, in long-term projects people start to focus on insignificant features. The moment you start improving abstract communication between task pools, adding GUI widgets so that you can make a built-in MP3 player or making splash screen data-oriented, your project is screwed to hell.

And this is probably the most important lesson to remember. If you need to do something very quick, code will be horrific, but also short, simple and fixable. With no time constraints, code will get new levels of complication, features and bugs. Time spent on maintaining won’t justify the result.

In 4 weeks, you can make several iterations of speed-coding, each time improving core features of the game. If you start with future proofness in mind, writing code and fixing bugs will consume most of the time. Sure, in 4 weeks you can make a ton of assets/levels, but how good are they is core gameplay is not good enough?

Finally, one solid C(++) tip: when adding new features, start with the smallest of available guns:

1. Global function — if you need to display score, don’t hesitate to add void DisplayScore() somewhere. If your game is single-player, store score as a global variable. See? You have just saved 10 minutes of writing getters, setters and designing communication between modules. If your game is multi-player then you will need to store and display scores per-player. But there is no reason to be able to display arbitrary score of arbitrary player if your game is not yet multiplayer. Believe me, you’ll have bigger problems than displaying score with that.
2. If your functions share common code or require helper functions, group them somewhere, possibly in separate file. Remember about static functions and variables — contrary to “OO” static, file static is about visibility. But it’s cool, because you can have a file for all font-related operations and store internal data in static global variable. Helper functions can be static, and public ones go to one shared header (if you write simple code, compilation time is never your enemy).
3. Promote functions to classes only when it’s relevant and useful. Remember, classes means objects, objects mean relations and relations means complexity. Is your gameplay so cool that you have time for code complexity?
4. Any design patterns or other exotic stuff is the last resort if none of above is good enough. Personally? Never got there.

(from #altdevblogaday)

I would like to present a very cool technique, widely used in C libraries, yet almost completely forgotten in C++.

We have a String class. It is a reference-counted, immutable string. Typical data structures in C++ would look like:

struct string
{
  string_data * data;
};
 

struct string_data
{
  int refcount;
  int length;
  char * data;

};

So, string is merely a reference to actual string_data. Creating new string objects (string foo = "hello world") looks like:

1. string_constructor(const char *)
2. allocate string_data
3. string_data_constructor(const char *)
4. allocate char[]
5. copy data, set all other members

In memory, it would actually look like:

So, we have one stack allocation, two dynamic allocations and our data is in three different places in memory. But in C++ terms it seems impossible to improve it. However, we can go C way and change string_data definition:

struct string_data
{
  int refcount;
  int length;

  char data[1];
};

Array of one character? I must be crazy, right?

Most C++ courses are not very precise about pointers and arrays. Especially, what is the difference of following instructions?

const char * string1 = "hello world";

const char string2[] = "hello, C++!";

And what is the difference of those?

int foo(const char *) { printf("*\n"); }

int foo(const char[]) { printf("[]\n"); }

Unfortunately, in C++ world where really various things can be static, answers are different. In first case, string1 and string2 are something else. In second case, there is no difference — actually, that code won’t compile due to non-unique overload.

Maybe little investigation? Let’s print some data:

printf("%p %p %s\n", string1, &string1, string1);
printf("%p %p %s\n", string2, &string2, string2);

What we do is printing expression as a pointer, address of this expression as a second pointer and null-terminated string pointed by expression. Results:

0x08048650 0xbfddbfa4 hello world
0xbfddbf98 0xbfddbf98 hello, C++!

Now you see magic behind it. 0xbf…… are stack addresses while 0×08048650 comes from somewhere else (likely from static, read-only data section). So, while pointer types hold address inside, arrays points to themselves. And we can abuse it!

“Typical” C++ allocation of string_data would look like this:

string_data * foo = new string_data;

It would allocate sizeof(string_data) bytes (9 + padding, probably 12). But in fact, we can allocate as much as we want! For example:

string_data * q = (string_data*)malloc(2 * sizeof(int) + 12);

new (q) string_data;

In this case — 2 ints and 12 bytes of string data. After successful allocation we need to manually invoke constructor, using placement new. Similarly, regular delete q won’t work — following code will:

q->~string_data();
free(q);

Of course if data type is simple (built-in datatypes, relaxed POD, whatever) there is no need to call constructor and destructor (although those calls probably would be simply optimized away). And malloc/free can be replaced with custom allocation routines.

And after such allocation, we can use data like it was not a char[1] array but rather a char[as much as we have allocated] array! How does memory look like with this data structure?

And there are numerous benefits of such approach:

• one dynamic allocation less
• data is less scattered in memory
• we saved some memory (how much? size of the pointer + size of allocation metadata)

In the next part, I’ll show how we can save even more allocations/memory.

(from #altdevblogaday)

Recently I'm creating a base library for all my gamedev adventures. It contains both very low-level features (allocator, string, vector, hashmap, errors etc) and reasonably middle-level features (like HTTP requests). Grabbing system information is rather high-level but very important, especially for OpenGL developer. It's very helpful if you can inform user that he just needs to update drivers.

Typical output:

[23:12:26] CPU : 8 x 2806 (2806) MHz (Intel(R) Core(TM) i7 CPU         930  @ 2.80GHz, FPU MMX SSE SSE2 SSE3 SSSE3 EST SSE4.1 SSE4.2 POPCNT HTT)
[23:12:26] RAM : 7956MB/12278MB
[23:12:26] GPU : NVIDIA GeForce GTX 460 [8.17.12.6658, 1-7-2011] :: NVIDIA Corporation GeForce GTX 460/PCI/SSE2 using OpenGL 4.1.0
[23:12:26] OS  : Windows 7 (6.1 Service Pack 1) 64 bit

Many folks complain on Windows API, it's backwards compatiblity and overall misery. But while working on sysinfo program, I've actually learned how powerful it is. Few examples:

Memory

On Windows there is GlobalMemoryStatusEx function that returns accurate results, in 64-bit format even for 32-bit applications.

On Linux there is /proc/meminfo but its results are very hard to interpret and are different than results of free command. In the end, I've used free -mo | head -n 2 | tail -n 1. Pure magic (and pure hope that free output format won't change).

Mac OSX amused me. As you may or may not know, I'm an iOS developer. I thought that Mac will be quite similar to its iOS counterpart, maybe with API like [[NSSystem sharedSystem] freeMemory]. Hell no! To obtain memory statistics, you need to access Mach (Mac OSX kernel) layer, like this:

mach_port_t host_port;
mach_msg_type_number_t host_size = sizeof(vm_statistics_data_t) / sizeof(integer_t);
vm_size_t pagesize;
vm_statistics_data_t vm_stat;

host_port = mach_host_self();
host_page_size(host_port, &pagesize);        
   
if (host_statistics(host_port, HOST_VM_INFO, (host_info_t)&vm_stat, &host_size) != KERN_SUCCESS) return;

natural_t mem_used = (vm_stat.active_count + vm_stat.inactive_count + vm_stat.wire_count) * pagesize;
mem_free = vm_stat.free_count * pagesize;
mem_total = mem_used + mem_free;
    
mem_free /= 1024 * 1024;
mem_total /= 1024 * 1024;

GPU

On all platforms I query OpenGL for vendor, renderer and both (OGL and GLSL) version strings. But this doesn't include driver information. And each vendor/platform has his own vision of those strings.

So, on Windows I use SetupAPI to obtain precise GPU information. With SetupDiEnumDeviceInfo I search for FILE_DEVICE_VIDEO devices. This is useful, because it allows to check multi-GPU configurations or to bypass additional layers of (OpenGL) emulation (like for example screen capturing applications). And with SetupAPI there is standarized driver information available.

But on Linux it's not so easy. I have no Linux-specific programming knowledge (not that I've used SetupAPI before). I didn't even search for kernel API for this. I've just used lspci command and searched for VGA compatible controller. For NVIDIA cards I've found that I can obtain driver info from /proc/driver/nvidia/version. For ATI or other vendors -- I have no idea. Fortunately, ATI seems to include driver information in OpenGL version string (like 3.3.10237 Compatibility Profile Context) so this should not be a problem. But if you know better solution, please let me know.

On Mac OSX I didn't even try. In fact you can't install or update GPU drivers on Mac, because those are not provided by NVIDIA or other IHV, but by Great Apple itself -- and that also means that OpenGL stopped at 2.1 version. Hate you, Apple. There is a magic number in OpenGL version string: 2.1 NVIDIA-1.6.26 but I have no idea how could I compare this to Windows/Linux driver version.

Conclusions

There is other data as well, but as complex as examples above. For example to obtain OS data there is GetVersionEx and GetNativeSystemInfo in WinAPI (Native flavour returns 64-bit information for 32-bit apps on 64-bit Windows). On Linux/OSX I just use uname -a (Mac OSX have no variations, and on Linux it would be hard because each distribution has its own vision of versioning). CPU was fortunately mostly cross-platform, because I focus on __cpuid data (BTW -- do you know how to obtain HTT and/or number of cores from __cpuid?).

So my verdict is: Windows rocks, other platforms are pure chaos. At least in grabbing system information. But really, WinAPI may be awkward or "too backward compatible" but at least is simple and powerful.

BTW, if you want to test sysinfo program on your PC, you can grab Windows, Linux or Mac OSX version and share results in comments. This is not the most recent version but results would be helpful.

I've found an intersting post on Twitter: You are NOT a Software Engineer. While I don't agree in general, it can bring us to a question: is programming an art?

Let's start with defining what is art. For me, art is a process of creating something where, at any time, you have full control of the output. For example, designing bridges is not an art, because you can't just remove a pillar here and add an arch there.

In more IT terms, subjectively:

• Doing 2D in Photoshop is an art. Even with multiple layers involved, you can always add another one on top and paint over to achieve pleasing effect.
• Sculpting in Z-brush is an art. At any moment you can add additional tentacle to your model, even since most of the time you shouldn't.
• Modelling in 3DS-max is an engineering. You need to maintain good topology, corrects edge loops and proper smooth groups. You can't easily modify finished model.

Of course I'm not telling that 3D artists are not artists. :) Art can (and usually does) involve engineering, and vice versa. It's just subtle yet important matter of balance. And we can observe that due to reduced technical complexity of doing ,,art'', it's more accessible to traditional artists. I mean, give talented painter a tablet and Photoshop and he will succeed. And Z-brush sculpting is just adding another dimension to it. On the other hand, there is no ,,off-line'' version of polygonal modelling.

So how can we treat programming with such definitions?

Unfortunately, programming is not an art. Most of the time you can't change features without effort. It's very apparent in gamedev where cost of code changes (esp. last-minute code changes) is much bigger than design/art.

I would love programming to be an art. But we need much better languages and tools to achieve it.