Switching to Stackless: Why this, Why now?

Wanted to give a small status report. Things are advancing a bit each day...painfully, but semi-clearly.

  • The good news of having Ren-C entering this with such a wide range of wild abilities is that the design is very informed. So far, no previous fun-and-cool features look like they're going to get the axe because of the imposition of new architecture. It seems to be a strictly-enabling change. Even though it certainly throws some of the old ideas about how to write a debugger out the window...those ideas were rickety. And I see ways in which it will really empower new methods. I just have to see the rest.

  • Nothing is firmed up enough to where it's productive to start doing formal timing. However...no doubt some of the stackless operations are going to come with extra cost. But when I look at the kinds of convolutions you'd have to do to achieve these goals without them as interpreter features...it reminds me not to be "penny-wise and pound-foolish". Powerful operations that help you avoid writing huge and broken usermode surrogates that are MUCH slower can win in the big picture... so sacrificing a couple of micro-optimizations is :ok: see also the old SPECIALIZE performance comparison notes

By doing things like tackling PARSE and TRAP+FAIL early on, I'm not avoiding hard questions and just going around converting all the easy natives to stackless first. I'm pacing them out (e.g. I just did LOOP and UNTIL today, which took only minutes each.) Doing a couple at a time helps me let each instance of returning to it be a "new" experience where I might have new insights.

The three biggest unknowns right now are:

  • Debugging - It's very much on my mind to create an experience that's not just like the hacked up demo I did...but real. I want it to be better, with real automation and reflection. It's been a trickier thing than I first imagined to design a pleasing/extensible debugger in a non-threaded environment running in the same language it is debugging (!).

  • Path Dispatch - which was always weird and now has to be redone. And not that I anticipate getting something as bad as history's method would be hard (it seems to be another one of Rebol's semantics-free-zones). Hence what worries me about it that if I'm going to bother doing a full rewrite...I'd kind of like to make some headway on pinning down just what the heck is it. How is it generic, or safe, or how does it apply to custom types. I may have to kick that can down the road again and just focus on making it stackless.

  • Performance - I've been applying intuition to it but it's going to need some looking at with instrumentation and timings. We've spoken about performance regression testing...and it seems like the web automation demo via headless Firefox might be not just the most relevant target, but the most easy to get comparison figures for.

I started this around 18-Apr so it's been a bit over a month. It's a little hard to estimate how fast progress will be, especially in these three areas. But pure stackless processing is probably not more than a couple of months away; with less clear progress on debugging or guarantees about how fast it will be.

In any case, it looks like I might be on the cusp of another relocation shortly. :palm_tree: :oncoming_automobile: :palm_tree: So we'll see if that increases or decreases productivity.

4 Likes