Iterating "NEW-LINE" Markers (and BLANK! Lines)

Development Iterators
5

I vaguely recollect PARSE/ALL when you had to explicitly ask to see whitespace in strings. The filtering was found to be problematic. Though maybe there's a more solid foundation for building opt-in filtering logic these days to where it could be the right answer.

...especially because I'm increasingly thinking fully reified NEW-LINE'd BLANK!s is probably the only sensible choice.

Let's consider a world where the following is not legal, due to newline significance and ELSE not being able to get its left hand side across a newline:

Then let's say someone comes up with a special meaning for ELSE with "nothing" on its left. (as there's meanings for RETURN with "nothing" on its right).

If you copy a block cell-by-cell (without special handling) you lose the newline. So you can change the semantics via what looks like a normal copy.

I think this points to the need for a BLANK! in cases like the above.

At minimum, we'd need to guarantee that if you copy the above cell-by-cell without tending to formatting what you wind up with is:

if condition [
   bunch of code here
], else [
   bunch of code here
]

This would imply that a BLANK! with a new-line marker on it, followed by content on the same line doesn't render anything:

if condition [
   bunch of code here
]  ; <-- there's a blank! at the end of this line...
else [  ; <-- ...or maybe it's at the start of this line
   bunch of code here
]

But if we're doing that, why not just preserve the new-line flag by making it not a formatting flag at all, but just the payload of a BLANK!.

That gets us back to this situation if you don't want newlines:

 block: [ \
     a b c \
     d e f \
     g h i \
 ]

That sucks, so the alternative could be a notation for saying "don't scan with blanks"

>> block: \[
       a b c
       d e f
       g h i
   ]\
== [a b c d e f g h i]

Maybe inside the bracket would make more sense when used with quasiforms, given that outer backslashes are used to represent antiforms at the moment...though that might not be a great idea anymore. (maybe vertical bars?)

>> splice: ~[\
       a b c
       d e f
       g h i
   \]~
== |~[a b c d e f g h i]~|  ; antiform (splice!)

Or Accept The BLANK!s, Solve It With Iterators

In the "solve it with iterators" worldview, you would write what you wanted as source:

 block: [ ; <-- blank! here
     a b c  ; <-- blank! here
     d e f  ; <-- blank! here
     g h i  ; <-- blank! here
 ]

That block would have 4 more cells in it than we are used to.

But at least it would be honest.

And as @IngoHohmann says, maybe it's just your pick to say whether "I want to experience this block without the blanks" when you use an iterator.

We might be able to get away from the uncomfortable terminology of NEW-LINE by asking the inverse of blanks... "are you comma-like?" If you're not comma-like, you're new-line like.

Definitely Different, Possibly Inevitable?

The hiddenness of the new-line flag makes it weird and hard to handle, when I think the reality is:

  • people care about it (and always have)
  • the evaluator semantics are irreversibly starting to care about it

It costs slightly more to have cells for it (4 platform pointers a newline, vs being free.) And it also means that the evaluator pays a little more to process it... although that's mitigated somewhat by the fact that you can just test the same byte as you test for anything else vs. having to bit-mask and look for a flag.

I'm not certain about this yet, but reifying newlines as BLANK! seems like it could be the way forward.

1 Like