Some big thanks to Tom Novelli!

Does someone remember this old bug involving broken debug output streams? Well, some nice person just gave me the key to solve this problem.

Today, Tom Novelli posted the following comment on the blog post linked above:

I ran into similar problems using C++ stream modifiers for colorized debugging messages. I wanted to save the previous color(s) on a stack:


cerr << color.push(C_RED) << "Fatal Error" << color.pop() << newline;

Trouble is, most compilers use right-to-left evaluation order, so pop() executes before push(). There is a solution but it’s not nearly as straightforward as putting push() and pop() in separate statements. C++ streams are a failure, I’m afraid. Like Amenel, I went back to using printf(). Then I went back to plain C, using the UThash macro library instead of STL maps/vectors/lists.

I really don’t think C++ is a good choice for OS implementation, especially on your first attempt. You’ll get distracted by all kinds of language issues. Some of the newer languages are designed better than C++, but those are mostly dynamic/interpreted languages with complex runtime environments, not suitable as system languages.

At first, I was going to say that we probably have different problems, because my modifier implementation only depends on the order in which the “<<” are evaluated, which is standard in the C++ spec.

To explain this better, let’s examine the example which I gave in that article, dbgout << txtcolor(TXT_LIGHTRED) << “RED” << txtcolor(TXT_LIGHTGRAY) << “GRAY”;

When I wrote the DebugOutput class which the dbgout stream is based on, one of my goals was to make it possible to seamlessly display several independant debug streams of text on the screen at once. Kind of like ncurses’ windows, but with a cleaner syntax. For this to work, modifiers had to solely modify the state of the stream which they are fed into.

To implement this behavior, I had created a set of objects that can be fed into a DebugOutput stream in order to modify its properties (text color, target area on the screen…), and made my modifier functions return such objects.

When one runs txtcolor(), as an example, it creates a DebugAttributeChanger object, which noticeably includes information on the text color change. This DebugAttributeChanger object can then be fed into a DebugOutput object like dbgout using the DebugOutput& operator<<(const DebugAttributeChanger& manipulator); method, and it is this method which is going to modify the state of the DebugOutput object based on the contents of the DebugAttributeChanger one.

So, what would happen if the C++ compiler evaluated txtcolor(TXT_LIGHTGRAY) before txtcolor(TXT_LIGHTRED)? Nothing special. The evaluated results would just sit there, waiting to be fed into the dbgout stream (in an order that’s guaranteed by the C++ standard) before they have some effect.

And then I thought.

And thought again.

And I realized my mistake.

The problem was that my modifier functions didn’t return a result on the stack, the usual way. Instead, they returned a reference to a static buffer, where the result was stored.

A single buffer. That was overwritten each time the function was run.

I probably did this out of fear that the stack could be overflowed with modifier objects in extreme use cases. However, that was not a reasonable fear. And it made my code sensitive to the order in which things are evaluated, which could well cause problems like the one I encountered, and is a pretty bad coding practice anyway (since, as we all know, compiler from the future will evaluate things in parallel, breaking all code which works this way).

I thus removed the superfluous “&” here and there in function prototypes, put my test code back in the main function, and compiled a floppy image.

That worked flawlessly.

So thanks, Tom, for helping me notice how stupid I am. And by the way, I do think that C++, or at least a subset of it, has some potential as a low-level language. Makes code easier to read for me, which is a pretty desirable characteristic in something as crashy and convoluted as an OS kernel.

7 thoughts on “Some big thanks to Tom Novelli!

  1. Tom Novelli April 5, 2011 / 8:00 pm

    Cool, glad I could help, in a roundabout way! A static buffer and a superfluous “&”, yeah, typical C++ gotchas. (Maybe it’s the language that’s stupid, not you :)

    Wouldn’t it be funny if all this OS and compiler work leads to… a simple OO color-text formatter ?!

    BTW, keep up the nice work. I just found your blog the other day and I’m maybe 25% through it. And whatever you do, don’t quit physics… it’s good stuff, even the old Newtonian stuff (have you seen Box2D, Chipmunk, etc?) and of course there’s plenty of opportunity for more programming fun in physics work. Computer Science has some good parts, but as a degree it’s pretty useless if it’s the usual Java programming, data structures, and some math.

  2. Hadrien April 5, 2011 / 10:10 pm

    C++ has its weird sides, granted, but in that case I do think it’s my fault :) In an attempt to write perfect code, I was re-reading some book about good coding practices, and followed the “use pointers/references for anything big” advice with a bit too much enthusiasm, putting pointers and references on just about everything that is larger than an uint64_t.

    Wouldn’t it be funny if all this OS and compiler work leads to… a simple OO color-text formatter ?!

    Yup, we need a clean and intuitive C++ interface to Ncurses ! This lib is full of awesomeness, it’s just too bad that all this awesomeness is hidden behind such a poor interface, with usability aberrations like putting y coordinates first…

    I don’t think I’d find it funny at first, though. I really put a lot of thinking and care in this project, and it matters a lot to me. Failure would be heartbreaking…

    And whatever you do, don’t quit physics… it’s good stuff, even the old Newtonian stuff (have you seen Box2D, Chipmunk, etc?) and of course there’s plenty of opportunity for more programming fun in physics work. Computer Science has some good parts, but as a degree it’s pretty useless if it’s the usual Java programming, data structures, and some math.

    Yeah, physics rocks, and you pretty much sum up why I’ve chosen it over CS at university. Felt like the stuff which I like in CS wouldn’t get me a job, and the stuff which would get me a job would be stuff which I don’t like so much (networking, web development, Java…)

  3. Hadrien April 9, 2011 / 8:44 am

    Possible, but in my case I can’t use the full C++ library at the kernel level for chicken and egg reasons (it depends on kernel-level calls which are not yet implemented)…

  4. Tom Novelli April 12, 2011 / 4:44 am

    Yup, we need a clean and intuitive C++ interface to Ncurses ! This lib is full of awesomeness, it’s just too bad that all this awesomeness is hidden behind such a poor interface, with usability aberrations like putting y coordinates first…

    Oh, you’re using ncurses! I’ve been using raw ANSI/VT100 escape codes for colorization the last few years, thinking ncurses was overkill. I would definitely use it for something like a text editor though.

    I don’t think I’d find it funny at first, though. I really put a lot of thinking and care in this project, and it matters a lot to me. Failure would be heartbreaking…

    Well, nowadays I think of osdev as a lifelong hobby, no success or failure, just a good thing to know if you’re into geeky computer/electronic things. Maybe the popular OSes will always be bloated crap, but at least we’ll have alternatives.

  5. Hadrien April 12, 2011 / 9:18 am

    Well, in the case of this project, I have some goals in mind, so I can use them to define success and failure :)

    I wànt to experimentally prove that my vision of a better OS is realistic by implementing it in practice in the form of a tech demo. If I manage to release a desktop OS that achieves at least a significant part of the goals, then it won’t matter if it’s not usable on a daily basis because it lacks sound or networking. The goal is to showcase some good tech. If people from the Haiku, Linux, or even Windows worlds find this interesting and try to copy it in there own products without even mentioning me, I’d love it. If enough people love this demo to want a “true” desktop OS to emerge from it, then it could become a lifelong hobby. But at the moment, I just aim at a nice demo.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s