Media Molecule PMoly

lately I’ve been experimenting with a trick to save space on 64 bit architectures: rather than using pointers (which take 8 bytes of memory! 8! bytes! omg!), I’m using 32 bit indices into a single, gigantic 32 gig block I allocate at startup. as in:

u64 *bigblock=malloc(32*1024*1024*1024); // at startup. or choose a number smaller than 32 :)

struct foo { int x, y, whatever, whatever; };

// later…

u32 myfoo=MyAllocatorForBigBlock(sizeof(foo)); // using a custom allocator of your choice…

((foo*)(bigblock+myfoo))->x=100;

((foo*)(bigblock+myfoo))->y=200;

with some macro goodness, or in C++, a template class that looks a bit like a smart pointer but is in fact not smart, those casts I made explicit above go away it can be made to look really neat:

P<foo> myfoo=P<foo>::Alloc(); // P<foo> is really a 32 bit index just like before.

myfoo->x=100;myfoo->y=200; // the overloaded -> operator does the bigalloc+index thing for us

The indices in my case assume an 8 byte stride, ie all allocations happen on 8 byte boundaries (that’s why bigblock is a u64*), so you can address 4*8=32 gigs with 32 bit indices. For an in-memory database (my use case, as it happens), that’s plenty. I don’t want it swapping anyway, and 32 gig machines are reasonable these days. The compiler seems to do the right thing and generate quite efficient code (eg it keeps bigblock in a register (ecx), puts indices in say edx and does things like mov [ecx+edx*8+4],200)

another cute side benefit is that all the pointers in your system, now that they’re indices from a base address, are sort of ‘position independent’… you can mmap() or fwrite() the entire bigblock to and from disk, and next time you boot, it doesn’t matter if the base address moves, it all just works. no need for pointer fixups or anything like that.  win!

kinda makes 64 bit linux/windows feel a bit more like a nice embedded system with a predictable memory map and nice small 4 byte pointers. lovely!

Posted by: Alex

Categories: Code
Tags: 32 bit 64 bit indicies

Untitled Document Syndrome

2010January 28th

this post is pretty on the money - but it goes further. how many times have you been forced to name a project before you can start? too many. littlebigplanet, the multi-million-selling-product, still has the project name ‘ps3test1’ - yes, it really can trace its roots back to the first code we ever wrote on ps3. at the time I named that, I had *no* idea that it would grow into the final thing. oh, and the executable filename? pc.exe (windows) and pc.self (ps3). confusing!

on the other hand, when I start projects and choose a cool sounding name, oh I dunno, like “RoboEngine”, you can pretty much guaruntee that it will fail. it seems that the less well considered the filename, the more successful the content. at least for me…

Posted by: Alex

Categories: Code
Tags: littlebigplanet

At siggraph 2009, Natasha Tatarchuk invited several games industry graphics types to give a course on ‘advances in real-time rendernig in 3d graphics and games’. Follow the link for lots of lovely stuff, including a set of slides about graphics techniques used in LBP.

Posted by: Alex

Categories: Code
Tags: littlebigplanet rendering siggraph

c trick

2010January 27th

I love indexing constant strings, I’m sad like that.

for example,

char c=”0123456789abcdef”[b&15];

converts the low nybble of byte b into its hex character.

oldie, but goldie.

Posted by: Alex

Categories: Code
Tags: c tips

Quote of the moment

2010January 26th

“goto considered harmful? rubbish - it’s fine”

— Anton Kirczenow

Posted by: Alex

Categories: Code

Luscious luscious fluids

2010January 26th

MMM so tasty!

http://people.cs.ubc.ca/~rbridson/

Posted by: Anton

Categories: Code

warning signs

2010January 25th

when I’m coding, sometimes I get warning signs that I haven’t understood the problem fully. Normally, they take the form of gross looking code - ‘-1’ or ‘+1’ hanging round in loop limits, even worse, seemingly arbitrary constants, large amounts of unavoidable pointer dereferencing,…

nearlly always this is a sign that either I’m doing it wrong, or I got the data structure wrong. It is ALWAYS better to stop and rethink.

as I’ve got more experienced coding, I’ve continually lowered my tolerance to these warning signs, and it always helps. even if I find myself doing something fairly innocuous, like a nested loop, or if I find an unexpected number of edge cases I need to ‘code around’, I now stop and rethink. I don’t always get the right answer… but I’ve never regretted that mental check.

tl;dr version: develop your spidey coder sense for anything gross, and listen to it. it’s like the human form of -Wall, and it’s always worth it.

Posted by: Alex

Categories: Code
Tags: blind faith in the beauty of the universe tips

For those of you who use a wide-screen monitor, managing windows can be a bit annoying. oh for a tiling window manager for windows! ah well, this is nearly as good: it maps Ctrl+Alt+numeric keypad such that the current window snaps to the relevant half or quarter of the screen. makes it very easy to quickly organise your windows into tiles, without having to touch the mouse (as you do with windows 7 new features).

Posted by: Alex

Categories: Code

image

Posted by: Alex

Categories: Code
Tags: the matrix

Highcharts

2010January 3rd

Ccool interactive JavaScript charts: http://www.highcharts.com/

image

Posted by: Paul

Categories: Code
Tags: javascript