Learning by coding, or NIH syndrome

28.02.2010 12:04 in programming, rant

Using high-level stuff is fun. As long as it works.

Consider this C# code:

var s = new WebClient().DownloadString("http://foobar.biz").Replace("<br/>", "bar");
Console.WriteLine(s);

Cool, isn't it? Now imagine that you have to do the same thing in C, UNIX-like way (sockets etc). Yeah, seems like a real pain. And this is why high-level coding is fun.

But now assume something gone wrong. If you're "lucky" you'll get a nice exception. If you're not, you'll get some mess not related to expected result. What can you do next? Google "DownloadString mess" and read first three Stackoverflow topics? If this works, high-level programming is not as great as before, but still quite fun. But let's say that Google knows nothing. You can read API documentation. Oh yes, a lot of docs before you find something useful. And what if you find nothing?

The worst category of such bugs are "everything theoretically works, but I see nothing/crap". For example imagine you have high-level 3D engine and use such line of code:

engine.Render("car");

If it doesn't work as you expected, number of possibilities is +infinite. What can it be? Buffers? Shaders? Rendertarget settings? Where have you done it wrong? In loading stage? But there is no loading stage, engine loads everything from XML files itself, blah blah blah. And that is of course if you actually have knowledge about buffers and shaders and stuff. If you do high-level only, chances are you won't.

What I am going to say is that the only way to learn every tiniest details of some stuff is to code it COMPLETELY using most low-level you can afford. You can of course know no deep details of some stuff and still be happy human, worker and husband. But you won't know that tiny bit that makes it work. And if it fails, you'll be left alone on the ice floe.

Examples. You can know a lot about containers and algorithms, their Big O and characteristics. But will you ever be 100% sure what they exactly do to your objects? Another. You can happily use some UI library for years but when it comes to coding your own controls, their input handling, don't you feel a little nervous? If your control is on the scrollbox, in what exact order will they get mouse inputs? And why?

It is actually very easy to know if someone coded something or just used/read about it. One minute of talking and you know that he doesn't know all drawbacks, problems and dark sides.

But the sad fact is that things are getting now more and more complex. In 80' you actually could code just everything. And now? To be honest, to know every single details, you should start with x86 (and GPU, and HDD and...) emulator (VM), then do some OS programming, then code your C compiler (if you expect to reach 90+ age you can try C++), then most low-level libraries, and so on.

Yes, you could spend your whole life doing this. But... you would be surprised by NOTHING. :)

Comments:

  1. Paweł Rutkowski

    Paweł Rutkowski:

    Well said (or rather written) :)

    However, I wouldn't say you have to do it all - you can do it a little (much?) quicker with someone's help - you would probably get simillar knowledge(well, it always better to learn from mistakes you made) and it would take you not as much time as you said.

    That was at least the idea for schools and universities, as I see it :) It doesn't work in real life as good as I would wish for, but if you find a person who knows a lot about topic you are interested in - taking knowledge from him is the quickest way to learn. And you still can get the full knowledge of the topic - but, as you said, it takes time :)

    28.02.2010 15:59:20

  2. anonymous

    anonymous:

    Believe it's Vernor Vinge's science fiction novels that get to this at a macro level: civilizations far in the future whose technology is at such a high-level that when the lower-level components of that technology begin to erode, there's no one able to repair it (no one has the knowledge to do so both safetely at a low level and systemic level). The civilizations inevitably collapse. Kinda an apocalyptic look at large-scale complex software design :)

    The "sad fact" you refer to is true of any complex system. Consider the medical field. The general practitioner has a patient that reports having a headache. Can the doctor be expected to know everything that could be potentially happening on a cellular level? Is it a life-threatening brain tumor or is it patient not realizing it's simple dehydration? The result is a field with extreme levels of specialization where the general practitioner relies on referrals to the persons with a limited, but very focused areas of knowledge in order to best handle the problems.

    03.03.2010 21:26:24

  3. Reg

    Reg:

    Generally I agree with you, although I don't like this example with web error. For me abstraction is the main principle in all programming and a good API encapsulates both functionality AND error handling. If there are some crypic errors on the low level it's not your fault because you code in high level but it's a flaw in the low level layer.

    Consider any system API for file access. How often do you have problems with opening a file bacause of some low level stuff like cylinders, clusters, IDE or SATA standard, DMA etc.? Most of us don't even deal with that and there's nothing wrong about it.

    On the other hand, consider "friendly" Linux distributions where you have all these GUI tools to configure anything. They work as long as someting goes wrong and then you have to go down to these details of small console applications and text configuration files. For me it's an example of a failed abstraction.

    07.03.2010 17:15:49

  4. Kristine

    Kristine:

    I'm a bit late to the party, but I just discovered your blog and am going through its archive. I think that opensourced high-level stuff would provide a reasonable compromise: When it works, you simply use it, and if it doesn't work, you can dive into the source and learn why.

    Kristine

    18.02.2011 21:58:18

Leave comment: