Solving the Pox of the (^)... LIFT the Universe?

I've pushed this through... and by and large I'd say it's a really positive direction.

Of course, it's a GIGANTIC model change. So there are some issues which have to be faced.

Impact on Generators

Previously, the idea with a generator was that it could produce any value that wasn't an unstable ERROR! antiform. The only legal ERROR! to YIELD was one with id: 'done, and this would be interpreted as the end of the enumeration. YIELD-ing any other ERROR! would result in it being promoted to a PANIC.

I said this was better than using NULL, because it meant anything that could be stored in a variable could be returned... your generator could give back sequences like NULL, 10, NULL, 20 etc. If NULL wasn't in-band you could always TRY the result of your generator and get NULL for the only legitimate ERROR! it could return... being done. Otherwise you could test with done?

My hope was that generators would be able to power things like FOR-EACH, by providing a feed of values back. But with lift-the-universe, what's legal to store in a variable has broadened...when you're using ^META-representation.

If you have something like:

for-each [^val] generator-for-values-of-object obj [ 
    ...
]

(Note this may go back to being legal as for-each ^val soon, pending deep thought.)

So the OBJECT! can contain ERROR! values, stored via ^META assignment.

If we had to change all generators that fed back values to use a lifted protocol (like TWEAK does), that would make those generators less generally useful.

Note that conceptually, this problem isn't new. It just puts the problem a bit more in your face. A generator couldn't speak in terms of ERROR! values before--as I mentioned--so something gets twisted up here. The twist has just encroached upon something more fundamental, e.g. enumerating things that can be valid values in an OBJECT!. So it's not as easily dismissed as before with "well, if you want to return an error from your generator do it lifted".

There may be a workaround if PACK!s aren't taken literally. This was already something I was considering, e.g. I was thinking that the reason you could write:

for-each 'key obj [...]

for-each [key val] obj [...]

Was that the generator behind the scenes powering the OBJ fields would return a PACK! with the key and value in it. If it didn't return a PACK!, then what would happen when you said for-each 'key obj [...] would be that you'd get the key on one iteration, that key's value on the next, then the next key, then the next value... it would be the PACK! coming back from the generator that signaled "hey, I'm actually two values that should be part of one iteration"

So if that's already true--that PACK! isn't literal--then returning an actual pack would have to be done with a PACK-inside-a-PACK. And by the same token, an ERROR! could be handled by being inside a pack as well.

What's nice about this is that it pushes the "weirdness" off a bit. Generators that want to be compatible with FOR-EACH and friends don't have to speak fully in terms of a lifted protocol. They just have to return their PACK! and ERROR! wrapped inside a PACK!... that's the bargain. (Perhaps GHOST! would have to be inside a PACK! as well just to say "all unstable antiforms must be wrapped in a PACK! to be used with FOR-EACH/MAP-EACH? I don't know if there's a reason, but if the other two have to be then maybe I can find a good reason why it should apply to ghosts too.)

I think this seems like an acceptable tradeoff. It doesn't mean you have to use PACK! this way in your own off-the-cuff generators that aren't intended to be used with FOR-EACH. But you always had to deal with the ERROR! exception, and this just throws in another exception that solves the exception.

Sidenote: Representing "True Unset"

So this actually does have another wrinkle, which is that since object values can now hold states "beneath" even unstable antiforms... how to represent the state that leaks through to casual user consciousness: true unset?

>> f. make frame! negate/

>> f.number
** Panic: number is truly unset

>> f.^number
** Panic: look man, it's not TRASH!, it's TRULY UNSET, ^META won't help you

>> unset? $f.number  ; <-- special functions must be used!
== \~okay~\  ; antiform

So now, let's go back to our generator-powering-for-each:

for-each [key ^val] f [
   if unset? $val [print [key "is unset!"]]
]

Notice there's no problem with what state we want ^val to hold (though there is some question over whether you should have to say ^val to get the unsetness, or if plain val would allow it).

The problem is: how would a generator be able to return this state? :frowning:

Curiously, if you were building a PACK! by hand, you could make something that did it. The unlifted band has not been used in PACK historically... every element had to be a quasiform or quoted. But if you build a pack manually, you can actually put sub-band values in it... as long as they're not antiforms and can go in a BLOCK!:

>> anti ['number _]
== \~['number _]~\  ; antiform

I'm choosing _ here as an arbitrary out-of-band thing that could represent the truly unset state. But the key is that whatever it is, it can't be a quasiform or quoted. TWEAK should probably use the same state here (I've been using the ~ antiform so far, but that would be disconnected with this purpose--it's easy to change.)

How many sub-band values should be legal? I don't know, but if we made an exception for just this one then it seems to address this particular problem.

Does SET-BLOCK have to support it, just because it exists to paper over a representational hole in enumerating OBJECT! values via generators? I don't know if it does or not.

Does there have to be some helper for making it easier to make these?

>> unsettable-pack [1 + 2 <special-unset-signal> _]
== \~['3 _ '_]~\  ; antiform

Probably not. It's by design that PACK! is pretty easy to make.

>> anti reduce [lift 1 + 2 _ lift _]
== \~['3 _ '_]~\  ; antiform

Any in-the-box construct that obfuscates that probably isn't helping. If you find yourself generating a lot of these in some pattern you can write your own helper.

Side Note Addendum: Generator Binding Requests

Since I can think of solutions to the lift-the-universe problems...and see them as epicycles of an already existing problem... that's not such a big deal for the generator-powered FOR-EACH.

A bit more of a problem is the question of how to communicate $var to ask for binding to be imparted on a per-variable basis.

Maybe since the protocol is already special, passing a generator :BINDINGS might ask for an inflation where it returns 2x as many values via PACK!... the value and then the binding separately, and FOR-EACH/etc. would then merge the bindings onto the values if they were applicable to the corresponding variable.

It's an ugly idea, but it's the first idea I've had that might work. I just mention it because if we resorted to passing in the variables to the generator so it could see the decoration, that would also provide a channel for pure-unsetting the variable. But that doesn't jibe with how I'm thinking about this working.

1 Like