Parallel FOR Example

Usage Show & Tell
1

One goal of Ren-C since the start has been to make sure people can write their own looping constructs that behave like the built-in loops. (That's why definitional return was so important, which paved the way for many other improvements to come.)

One loop construct requested by @gchiu was the ability to walk through blocks in parallel. It's in the tests, but I thought I'd share it here:

for-parallel: lambda [
    return: [any-value?]
    vars [block!]
    blk1 [<opt> any-list?]
    blk2 [<opt> any-list?]
    body [block!]
][
    while [(not empty? opt blk1) or (not empty? opt blk2)] [
        (vars): pack [(try first opt blk1) (first opt blk2)]

        attempt body else [  ; if pure NULL it was a BREAK
            return null
        ]

        ; They either did a CONTINUE the REPEAT caught, or the body reached
        ; the end.  ELIDE the increment, so body evaluation is WHILE's result.
        ;
        elide blk1: next opt blk1
        elide blk2: next opt blk2
    ]
]

You get this behavior:

>> collect [
       assert [
           20 = for-parallel [x y] [a b] [1 2] [
               keep reduce [x y]
               y * 10
           ]
       ]
   ]
== [[a 1] [b 2]]

There's a lot of nice little flourishes here. BREAK and CONTINUE work, thanks to the loop result protocol. Assigning the variables is handled elegantly by multi-return, where a SET-GROUP! retriggers as a SET-BLOCK!. ELIDE is put to good use to avoid a temporary variable for the loop product.

"Isn't it nice... when things just... work?"

2 Likes
2

I noticed that in the reduced case of receiving two voids, this will return what WHILE returns if it never runs its body. That is VOID.

However, that is contentious with the GHOST-in-NULL-out policy. I wrote some ponderings on what the right answer should be, and I lean towards saying that loops should obey GHOST-in-NULL-out.

If that were to be heeded here, it would need something like:

    ...
    blk1 [<opt> any-list?]
    blk2 [<opt> any-list?]
    body [block!]
][
    all [not blk1, not blk2] then [return null] 
    while [(not empty? opt blk1) or (not empty? opt blk2)] [
        ...

(There's probably a cleverer way to write it--there usually is--but that's straightforward enough.)

But I'm a little on the fence on if loops should do VOID-in-VOID-out vs. VOID-in-NULL-out, so I'm just going to start paying closer attention to which seems more useful.

(The answer might depend on the kind of loop. For instance, MAP-EACH being "expected" to produce a value should contrast it with NULL if it couldn't. Whereas FOR-EACH and FOR-PARALLEL and such might be more like EVAL and be expected to tunnel whatever evaluative result it gets.)

3

In making the change to where "strict mode is always enforced", a problem was noticed with FOR-PARALLEL:

The caller is passing in a VARS block like [x y]. And what was happening here was that due to attachment binding, X and Y were just being pushed out as declarations into whatever the binding of that block was.

That is not what was intended. What was intended was that X and Y be new definitions for the body of the loop. (Good catch, strict mode! :star2: )

Now the tricky part: what to do about it. The block you get was already bound at the callsite (wasn't quoted)... though despite that, plain words inside of it represent instructions to make new bindings for the body. So we need to override those bindings if we're going to use it as a SET-BLOCK! to do the assignments.

Hence vars needs to be "overbound" with new definitions.

A crude attempt might leverage the mechanics of "decide what to do about SET-BLOCK!" already built into WRAP:

blk: inside vars compose [(unbind vars):]  ; extract binding to tip
blk: wrap blk  ; e.g. gen bindings for [[x y]:]
vars: inside blk blk.1  ; put binding onto unbound block [x y]:

This lets WRAP take care of the "overbinding" because we moved the full binding of the vars block to the tip, and that's what WRAP does.

But better would be if WRAP just offered this as a fundamental operation. It could be the behavior applied when you pass it a SET-BLOCK!, but I don't like cueing behavior on type like that for such primitives.

So it could be wrap:set-block, or as you're passing a block just wrap:set to mean "wrap with set-block semantics".

vars: wrap:set vars

But wait, what about the body? :face_with_diagonal_mouth:

The body needs to be bound to the context created for the variables. And the variables need to be created only once if they're calculated from GROUP!s!

Okay, this is starting to get complicated. We need to COMPOSE the block to make sure any GROUP!s are evaluated once--not each time per loop iteration. And we need an operation that gives us back the context, and binds the variable block.

It seems WRAP needs a secondary multi-return, to give us back the context it made so we can use it on the body block.

[vars context]: wrap:set compose vars
body: overbind context body

Note that it would be "sketchy" if we tried to extract the context from the block result of WRAP. That block is bound not just to the context made for the variables, but other things (like the context to know how to evaluate embedded groups). We aren't interested in carrying over the full binding on vars from the callsite that had [x (some-func [blah y])] because SOME-FUNC and BLAH may be visible in the vars block but should not be made visible to the body.

Crazy stuff...but, this is what you have to do if you want things to actually work.