Parsing Giant Streams Without Consuming Tons of Memory: How?

[hostilefork]

@bradrn

I suggest having a look at attoparsec.

Thanks! I've seen it, there's some complexity:

Parsing Huge Simulated Streams In Attoparsec | AE1020: Lazy Notebook

To get some forward motion and be able to start working on performance, I think I've made peace with the idea that it's better to build UPARSE coherently on top of the basic natives like FIND etc. and vet them on plain old series... making sure those primitives are rock solid.

As I've previously mentioned:

@hostilefork

I will point out that UPARSE builds on basics in ways that e.g. Red PARSE does not. Consider:

red>> find "abcd" ""
== none

red>> parse "abcd" ["" "abcd"]
== true

Is there a "" at the beginning of "abcd" or is there not? They have different answers because they don't run through a common code path.

UPARSE actually builds on FIND, and for this reason has a consistent answer:

>> find "abcd" ""
; first in pack of length 2
== "abcd"

>> parse "abcd" ["" "abcd"]
== "abcd"