I don't consider myself beholden to IEEE 754, any more than I consider myself beholden to ISO 8601, or ISO 3103. Rebol is a rebellion, after all. 
Review my list of benefits above (UPDATED).
Remember: this is coming from a place where Rebol2 and R3-Alpha made the decision to simplify programmers' lives by guarding them from the nasties altogether. You are the sort of person clearly who would prefer to go all-in (though your biases may change with time...)
But this is a middle ground. I'm sure there'd be someone out there would be mad when their spew of IEEE 754 literals in a text file from another language--with crazy states--didn't load without tweaking. Yet someone else out there is going to be glad they didn't just get streaming wild numbers into their program, that math blithely tolerated and caused baffling results. I'm pretty sure most Rebol programs are going to be in the latter category--which is what drove the original decision to prohibit the states.
"I don't know what the secret to success is. But the secret to failure is trying to please everybody."
Strongly agree. This is why only the antiforms would be legal.
They could still BINARY! convert as their IEEE representation (or textually convert):
>> sqrt -1
== ~NaN~ ; anti
>> to binary! sqrt -1
== #{FFF8000000000000}
Though I've never cared for the implicit choice of encoding on that. It's why I like ENBIN and DEBIN.
So I'd rather that be something like:
>> encode @IEEE-754 (sqrt -1)
== #{FFF8000000000000}
(Note that I am a stickler for not conflating quasiforms with their antiform state, e.g. I would not allow encode @IEEE-754 '~NaN~. I consider being committed to not doing such conflations to be a critical aspect of isotopic design. Though I can't control what people do in their own functions, and maybe someone has a good reason to do it...but I'm doubtful.)
Automatically converting the IEEE 754 notation to the ~NaN~ quasiform in loading blocks is clearly bad mojo. It's not only injecting an unexpected type for what you might have thought was a FLOAT!, the quasiform is not semantically an actual NaN until you unmeta it. So I'd say that 1.#NaN is simply an error as far as the scanner is concerned (which is the case historically and today). But the error could guide you to the existence of ~NaN~ if you wished to go that route... and if the literal wasn't actually the by-product of some error or unexpected condition (which I imagine 99% of the time it would be).