Refinement Arguments at Head of Args List, Not Tail

This question of refinement ordering is another one of those "really fundamental things that can't be skipped over in a minimum viable product, even a demo". :man_facepalming:

So it's been a real source of stress.

As is often the case, sitting back and erasing preconceptions, and asking questions from first principles, may be the way to solve it...

Why Do Refinements Exist?

e.g. why would you make APPEND with a :DUP and :PART option instead of writing separate APPEND and APPEND-DUP and APPEND-DUP-PART functions?

In many languages, you would write APPEND-DUP-PART as the "maximally configurable" core function with a bunch of arguments. Then you'd make specializations like APPEND-DUP call the core function with a default for the PART, and probably APPEND in turn by calling APPEND-DUP with a default DUP of 1, etc.

Advantages that come to mind of the refinement approach over this are:

  1. You don't have to remember the order of the optional parameters, you just have to remember their names, and you use them in any order (so long as you make your usage order match).

  2. One entry point for documentation, and that entry point is the "natural" name of the function where you have discoverability of what the options are.

  3. No need to write out the awkward cascade of specializations.

So at least three good things. But we've seen one rather key weakness:

The design chosen as most palatable for the default usage of the function name without optional parameterization dictates the parameter order, in a way that the refinements cannot override.

When you have individual functions like APPEND-DUP and APPEND-DUP-PART as their own entry points, those aren't beholden to the order that is most natural for APPEND. They can pick what makes the most sense for that "configuration".

And as we've noticed, optional or extra "configuring" parameters frequently make sense to appear earlier rather than later in the invocation. Though perhaps not always.

The historical refinement approach anchors the mandatory parameters in place, and configurations can only be added at the end.

APPLY as the Escape Hatch

It makes a big difference to have modern APPLY and its shorthand of //. This lets you pick whatever order of parameters makes the most sense to you.

I name COMPOSE as a good example because it shows that the "configuration" information is something that has a big impact on understanding the meaning of the operation as a whole:

compose:pattern [
    some bunch of {{code that}} <spans>
    #multiple {{lines}}
    [and could go on for pages]
] '{{}} 

So redoing that with APPLY, you can push the pattern up front:

compose // [
    pattern: '{{}}
    [
        some bunch of {{code that}} <spans>
        #multiple {{lines}}
        [and could go on for pages]
    ]
]

It's not terrible... it does resolve the tension between the desires of the default meaning of the word COMPOSE, and the need to see the configuring parameter before seeing the data operated on.

Using this shorthand you're paying for: // [ ] and a level of indentation, with some need to understand what // means. As opposed to:

apply compose/ [
    pattern: '{{}}
    [
        some bunch of {{code that}} <spans>
        #multiple {{lines}}
        [and could go on for pages]
    ]
]

But, still there's something unsatisfying about saying this is what you have to do.

You Could Fallback To... Making More Functions

The Non-Refinement world says that if the parameter-order needs of your "configured version" are in contention with your "default", make another function:

compose-pattern '{{}} [
    some bunch of {{code that}} <spans>
    #multiple {{lines}}
    [and could go on for pages]
]

And you can try some other naming ideas (I thought of COMPOSE2, to mean, "arity-2 compose", and it's briefer):

compose2 '{{}} [
    some bunch of {{code that}} <spans>
    #multiple {{lines}}
    [and could go on for pages]
]

You've lost your discoverability (HELP COMPOSE won't tell you about COMPOSE-PATTERN or COMPOSE2, the way HELP COMPOSE could find COMPOSE:PATTERN).

But when compared with COMPOSE:PATTERN in a refinements-take-their-args-first world, would COMPOSE2 still win out in usage?

compose:pattern '{{}} [
    some bunch of {{code that}} <spans>
    #multiple {{lines}}
    [and could go on for pages]
]

COMPOSE2 might still be something some people would choose (I doubt everyone). But at least you could just say compose2: compose:pattern/ and get the right behavior.

Positional Refinements: Best Of Both Worlds?

One idea might be that the position of a refinement in the parameter list actually makes a difference. This is something Ren-C can do since "refinements are their own arguments":

 compose: native [
     :pattern [...]
     template [...]
     :deep [...]
 ]

The idea that "refinement order matters" would mean that your callsite should probably be forced in line with that ordering. But is that so bad, considering you're forced in line with non-optional parameters already?

Let's say APPEND put :DUP before the main arguments, but :PART after the series parameter that it is constraining... (I don't know if this is desirable, let's just say it did)

>> append:dup:part 3 [a b c] [d e f] 2
== [a b c [d e] [d e] [d e]]

Now say you gave the order wrong:

>> append:part:dup 3 [a b c] [d e f] 2
** PANIC: Incorrect optional parameter order (should be :DUP:PART)

This then turns the tables, and says that if you want an arbitrary order of parameters that doesn't match the spec you should use APPLY.

Bridges The Non-Refinement World To The Refinement One

This effectively means that you are automatically getting the permutations of functions written for you:

APPEND
APPEND-DUP
APPEND-DUP-PART
APPEND-PART

But only those where the parameters follow the order in the definition.

It still lets you add the refinements after the fact, and you shouldn't get bitten by any callsites that mix and match refinements that haven't been used together before.

From an implementation point of view, this is a vast improvement. Right now, there's bend-over-backwards mechanics to figure out how to align an ordering of refinement usage that doesn't match the frame definition. That code is non-trivial, and removing it simplifies things.

Honestly I Think, I've Just Solved It

:bullseye:

Looks like a win to me.

Q: Should non-argument-bearing refinements be subject to interface order as well? :face_with_diagonal_mouth:

That's tough, but I kind of feel like the answer is "maybe so", if we want the mechanics to be simplest. Though it could probably still be simple as I can think of ways to handle parameterless refinements without requiring multiple passes. I guess I'd have to see how oppressive it winds up being to make it part of the order rule.

1 Like