The concept behind NULL is that it's a somewhat-ornery state, that most functions don't take by default. So it helps catch mistakes.
This makes me feel it's a bit haphazard to accept it at callsites:
>> count: null ; remember to assign this later...
== ~null~ ; anti
...
>> append:dup [a b c] [d e] count
== [a b c [d e]]
You got the same behavior as append [a b c] [d e], e.g. the refinement was "revoked" by being null.
But you've lost that protection idea. It would be safer if you had to write:
>> append:dup [a b c] [d e] count
** Error: APPEND expects INTEGER! for its DUP argument, not ~null~
>> append:dup [a b c] [d e] opt count
== [a b c [d e]]
And with <opt> as a parameter option, you have a smooth curve for converting a parameter between being a refinement and not a refinement. You get it as null internally to the function either way, making it pleasant to handle while preventing accidents on the interface:
foo: func [x [<opt> integer!]] [
if null? x [print "null"] else [print ["integer:" x]]
]
>> foo 10
integer: x
>> foo opt null
null
>> foo null
** Error: FOO doesn't accept NULL for its X argment
So now if you change X to be a refinement, you have the same behavior.
If we live in a world of answer #2 being how to forcibly say a refinement is not in use, what about refinements that don't take arguments? Their values are either ~null~ or ~okay~ antiforms. Should you have to put an OPT on a logic?
>> f: make frame! transcode/
>> f.source: "[a b c]"
>> f.step: opt transcode-mode = 'step
That doesn't seem like it should be necessary. You'd have to do it in things like apply too:
It may be that refinements without arguments are either [~null~ or ~okay~] antiforms, and don't behave the same as refinements with arguments, that expect you to forcibly opt out of them.
In which case, should argument-less refinements treat OPT'd as null also?
Hard to say, but I think the upshot here is that argument-less refinements are just different beasts, and do not have a clear parallel or migration path to <opt> arguments.
I updated this thread a bit due to "Lift The Universe", which makes it possible to store voids in FRAME! using ^META fields. That helps the FRAME! protocol to speak in "as-is" values, such that you don't get a field forced into a lifted representation just because sometimes it needs to take unstable antiforms... you can deal with those cases more narrowly.
But one question about something like an <opt> parameter would be: should an ENCLOSE or ADAPT be seeing that parameter as a null, or as a void?
APPEND is a good example, because it has an <opt> on its value parameter.
>> append [a b c] () ; append needs to run...doesn't return null
== [a b c]
The VOID becoming a NULL is a convenience for the internals of APPEND, so that it receives a "stable" antiform of null instead of an unstable void.
To think about how this might work compositionally, let's take a real example... like writing the KEEP of COLLECT in terms of APPEND.
Here's sort-of how it used to be written, when it assumed that it got to see VOID...updated for lift the universe:
keeper: specialize ( ; SPECIALIZE to remove series argument
enclose append/ lambda [ ; gets :LINE, :DUP
f [frame!]
<with> out
][
if void? f.^value [ ; imagine it sees VOID, not NULL
return null ; doesn't "count" as collected
]
f.series: out: default [make block! 16] ; won't return null now
eval f
f.value ; KEEP returns the input value as output
]
)[
series: <replaced>
]
This raises several questions, beyond just the question of voids and nulls for <opt>.
KEEP twists APPEND such that it returns the value you give it... should it be getting decayed inputs, or should it have access to the undecayed forms? e.g. could it say:
eval f
^f.value
If it got the inputs decayed and received PACK! for instance, that would allow:
But APPEND doesn't handle PACK!... its value argument is non-^META. Does that mean the decays have happened before the ENCLOSE has been reached? Does it mean other typechecking has been done before the ENCLOSE?
So far, the assumption has been that if you ENCLOSE a function, you're subject to its interface...the type checking is done before your enclosing function runs. So you'd have to adjust the interface before doing an enclose if you wanted to widen it to accept more types or change the parameterization. However, that means any parameters you change have to be typechecked again.
This runs up against ideas I've had about typechecking and decay being something that happens "inside" the function dispatcher. If APPEND hasn't been invoked yet, then something has to get involved and process the frame so that the ENCLOSE gets the FRAME! as APPEND would see it.
But you face questions there of whether conventions like <opt-out> fit in with that, too. Does an ENCLOSE of a function that has an opt out parameter get a chance to not opt out... e.g. does the opting out happen on the evaluation of the frame, or does it happen in the frame processing?
Practically speaking, it seems to me most ADAPT and ENCLOSE calls want type checking to have been run. At least, most do today.
What bearing does this have on building a FRAME! manually? You still have type checking run on it before execution. But type checking has to run on frames that you tweak during an ENCLOSE or ADAPT as well.
This implies that the <opt> transformations that turn voids into nulls are not part of type checking, because you don't want them to run every time you EVAL a frame. They're part of the parameter convention, so like quoting.
I tinkered with the code a bit to do this adjustment: that <opt> means the type-checking phase accepts NULL, but not VOID... and during callsite-oriented operations, both the <opt> parameters and argument-taking refinements will convert VOID to NULL.
In such a model, trying to build frames and set ^f.refine: () won't work, you use NULL. So in essence, the need to convert from VOID=>NULL is a responsibility of "frame builders" to decide if that's the "service level" they want to provide.
As I put it:
It's frustrating to have such a dichotomy. Because the whole point of "lift the universe" was to produce a single "currency" of FRAME!, so that phases didn't have to worry about it.
But this is saying "parameter convention" and "type checking" are intrinsically distinct. Which is obvious if you buy into the Rebol concept of "fexprs are good", because when you have a "quoted parameter" the only meaning of type checking is of the literal entity at the callsite, not the evaluation product.
Hence you always have a tension: are you the kind of thing that is concerned with "parameter fulfillment" or are you the kind of thing that does the "work once the parameters have been fulfilled".
Will Confess, I'm Saddened By The Tension
My wish with "Lift the Universe" was that it would create a nice, universal currency for FRAME!, so one didn't worry about "some moment" where frames transitioned from unlifted to lifted representations or anything like that.
I guess that if I'd thought about it a bit more--I'd have realized that "parameter conventions" (e.g. permitting literal arguments at all) raises a question of that "moment". Does an ENCLOSE or an ADAPT or other compositional mechanic need to consider whether a parameter is literal? If not, then taking the "literalness" into account is another "moment".
I don't think this is an indictment of ^META-representations being able to store unstable antiforms. I think that tool has cut down on the number of places where these "moments" exist, and shouldn't be blamed for this problem.
It's just that while it cut those moments down... it didn't wipe them out completely... and drew attention to the ones the medium has accepted.
Allowing such a thing as a "parameter convention" is a weird design tool, meaning that there is such a thing as "parameter fulfillment"... distinct from type checking.
The problem becomes how to articulate the layering
I think realizing that the needs of an <opt> parameter and a refinement-with-args are similar, unifies them in parameter convention.
It's fuzzy right now but I think it's sifting out to say what it means to operate at the FRAME!-level vs. at the "parameter" level is shaping up, a little bit.
It's not like design dichotomies like this are inherently illegitimate (think Kernel mode and user mode, via a bit the CPU recognizes). Sometimes you draw a line. It just wasn't obvious that refinements and optional parameters used the same bit, but... that's making more sense now, so the question is how to make the code that does it make sense.
Maybe this argument tampering is a mistake... and refinements should just be void
But we make dealing with voids more natural by saying that referring to things with leading colons manifests voids as nulls.
foo: func [arg :refine] [
if :refine [
print ["You'd have to put leading colons to logic-test refinements"]
...
]
...
]
How bad is this? You only have to put the colons on when you haven't tested the thing for voidness previously.
And you wouldn't have to opt a refinement to get a void out of it, you could just ^META it.
Annoying? Cool?
There's mechanical advantage to removing these moments where the dispatch mechanics have to massage or change the values in the frame.
If we could just leave the voids as voids, it would resolve this problem (which is more serious a problem than it might sound.)
And there wouldn't be <opt>, just <void>. You wouldn't turn voids into nulls.
It would also mean refinements could take LOGIC arguments... Because it would be ghost vs. null vs. okay. ghost if no refinement was provided. You could tell the difference.
We don't want to make things ugly for the sake of mechanical consistency...
...BUT.... some levels of coherence are overwhelming and shouldn't be fought against.
Another piece of the puzzle has clicked into place with the return of "HOLES"... what had once been antiform PARAMETER! in function frames, to indicate a parameter should be gathered.
Antiforms were frustratingly "not-quite-out-of-band-enough"; they were values that had to be considered by a function like INTEGER?. And so if you did a MAKE FRAME! for the INTEGER? function, there was a question of semantics of what it meant to have an antiform PARAMETER! in the value slot:
does the antiform parameter mean "this function is unspecialized?" (e.g. INTEGER? still needs an argument to test)
or does it mean "this function is specialized with an antiform parameter" (e.g. INTEGER? should return null, because that's not an integer)
But with bedrock PARAMETER! (a "dual state"), we get a hybrid solution. Something that cares (like function dispatch machinery) can find out that it's actually a hole--hence an unspecialized parameter. Things that don't care (like DEFAULT) can experience the value as if it's a GHOST!.
Hence MAKE FRAME! by default looks like a bunch of ghosts, to a casual observer.
This makes it feel all the more natural to say that an unused refinement was just never filled in.
(You might think there could even be a feature hiding in this: if we left the unused refinement cells as holes, then they'd know the types of the values they would have taken--while still reacting as GHOST! to most observers. But I don't think that's a great idea, because you couldn't easily keep that guarantee if someone wrote frame.refinement: if 1 = 2 [<value>] and overwrote the slot with void...that would throw away the information, and I don't think we'd want to be in the business of putting it back. So canonizing the refinements to "actual ghosts" seems the better plan.)
The 0-Argument Refinement Finesse
The unused 0-Arg refinement value has always been a pain point. Not just in Ren-C but in historical Rebol.
Rebol2 said refinements were either true, or none:
the universality of an untouched function parameter being in the "unperturbed" state ("all unused refinements are the same")
the greater usefulness of a 0-arg refinement being a LOGIC!
Ren-C offers a true "out"... because :REFINE will turn ghosts into NULL. This means the frame cell can be a GHOST! and yet you can experience it as a logic (or as a ghost, if you do ^REFINE). So long as the used state is an ~OKAY~ antiform, this works!
Wrinkle: COND vs. OPT
Generally speaking, COND (ghost/empty-pack) is used to "opt out", while OPT (none/empty-slice) is used to "opt in".
But when you use COND on a refinement, you're not opting out of the function call as a whole. You're just not asking for the refinement. The function still runs.
Hence a refinement has parity with a normal argument that takes GHOST!, and receives ghost when a ghost is passed.
This isn't a mechanical problem... but it's weird:
>> append [a b c] if 1 = 2 ['d]
== \~null~\ ; antiform
>> append:dup [a b c] 'd if 1 = 2 [10]
== [a b c d]
The only inkling I have of what it might imply is that the OPT/COND distinction is too subtle and weird, and maybe in the era of "hot potatoes" what COND makes should be a ~(veto)~... and these functions returning null are thus veto-reactive.
This makes more sense, even in the function spec. If you see <veto> in a function argument, that means if that argument is a ~(veto)~ then the function will gracefully return null.
(Note: you can't assume this systemically for any argument, e.g. VETO? has to accept a veto and return true. And you wouldn't want (integer? veto) to return false, it needs to error. I'm not sure if there's some general rule to when veto-ing is okay; you don't want something that is making a decision that could be false meaningfully to be vetoable, so anything that can return a LOGIC! naturally can't be veto'd...)
Wrinkle: 0-Arg-Refinement NULLs
If a refinement takes 0-arguments, it's awfully tempting to set that with a logic.
>> f: make frame! append/
>> f.series: [a b c]
>> f.value: [d e]
>> f.line: integer? f.value
Instinctively, I feel like the typechecking machinery has to be willing to canonize a null passed to a 0-arg refinement into a ghost.
The only reason I'd say it shouldn't do that would be if it can't. I've tried to talk about these "moments" of frame transition, and that's the only real worry... is when there isn't a "moment" to do it... or if the multiple moments have competing needs.
What's good about the canonization to the ghost is that it suggests all the moments have the same endpoint. It's not like some phases want an endpoint of null, and some phases want an endpoint of ghost... they all want ghost. So it isn't canonization that's the problem, it's different phases wanting different canonization.
I went ahead and pushed this through to see what I thought (sometimes it's easier to just do things than it is to think about them abstractly).
I will admit to not loving it
After having sought to end the GET-WORD! "pox" for deactivating what might-be actions, this is bringing back what looks like a pox on optional arguments.
One of the big justifications for this is that I was talking about removing the rule that you can't ask things like INTEGER? on a NULL. My rationale was that if you aren't using NULL as a "variable-not-set" state, then you can say that if you write integer? var you can be confident that the test is actually testing a variable that fetched.
So if refinements are not set, then :var turns that unsetness into NULL. And this ties into the narrative such that saying integer? :var shows you what you're doing: you are intentionally passing something that may be null into a type test.
This sounds fine in theory. But in practice, it's rather ugly, and I wonder if it's more dissatisfying than living with the potential for mistakes.
It seems the premise that "people should be using ghost for all unset states and not null" is just falling flat with me. It's falling so flat that I think the answer to "homogenize the frame" is actually make the unspecialized frame slots all NULL instead of make the unspecialized slots all GHOST.
Those are actually PARAMETER! "holes" that are just pretending to be null. But whatever they pretend to be--NULL or GHOST or TRASH--is a lie, and could be an actual in-band parameter value. Is it really so crucial that if you write f.series you get an error because you didn't assign it? Being able to say if f.series [...] and test if you have assigned it has benefits as well.
The Homogeneity Is The Non-Negotiable Change
Looking at the cleanups in the code, there's no question that the mechanics have to purge this problematic situation:
The empty state has to be the same answer for refinements as for everything else.
But having stared it in the eye hard enough, I think that answer is NULL.
My biases of trying to push unrefined things ever-further to the representational edge had been looking for the answer in unstable antiforms. But the answer had to go further--to bedrock states that evaluation could not produce. And once that bridge was crossed, choosing a maximally unfriendly state no longer has relevance.
So What About NULL-safe Typechecks, Then?
Now that VETO handling is built into every function, it seems to me that asking integer? cond var is a reasonable enough solution. VETO solves the problem of "the lie".
What About NULL-Safety For Refinements?
...y'know, the question that kicked this all off?
If NULL is the currency in the frame, we know this wouldn't be a problem now:
>> count: null
>> f: make frame! append/
>> f.series: [a b c]
>> f.value: [d e]
>> f.dup: count
>> eval f
If that isn't a problem, then should the rule for callsites be different... where you have to use void in order to get it to build the frame with a null?
Another question: should the typechecking machinery be willing to canonize ghost frame variables passed to refinements as nulls (the reverse of the null-canonized-to-ghost question).
I'm not 100% sure on this right now, but the thing I am sure of is that I'm panning the unused-refinements-as-ghost idea.
And I'm bringing back the type-checking strictness regarding nulls, where you have to use COND. The good news about that is that even if this were relaxed in the future and INTEGER? VAR were allowed to work on null variables, the COND would still work because you'll be able to veto integer regardless.
