Jill Ramonsky
2003-10-20 14:37:50 UTC
Are the LS and PS characters actually used in real plain-text documents?
I ask because plain text documents are created by text editors. The text
editor I happen to use is TextPad (there are hundreds of others, and
everyone has their favorite). It can save in UTF-8, and so on. But it
always saves documents with CRLF separating the lines. (It's a Windows
system).
Going a little deeper, applications like this are often written in C or
C++. These languages have the convention that "\n" in a string literal
means "new line". Strictly speaking, BY DEFINITION (from the C and C++
specs), "\n" is supposed to mean LF, and nothing else, but programs
compiled on Windows will reinterpret "\n" in a string literal to mean
either LF only (when in memory) or CRLF (when encoded to or from a file
or stream opened in text mode). Yes, it's a kludge, but it obviously
works quite well. I suspect (but I don't know for sure) that the Mac
will interpret "\n" as CR only.
It would seem impossible (or at least, a violation of the C/C++ specs)
to reinterpret "\n" as LS in C/C++ ... but then again, that
specification has already been violated, so maybe the precedent is there
and that no longer matters.
Nonetheless, it would seem, at least /slightly/ sensible to me that text
files encoded as UTF-8 should be using LS instead of CRLF. But this
appears to be difficult to achieve. There is no C/C++ escape sequence
which is defined to mean LS (unless you're prepared to write
"\xE2\x80\xA2" instead of "\n" all over the place), and what "\n"
generates is platform-dependent.
We can't change C or C++, of course, but would it make sense for other
computer languages, in particular future computer languages, either to
redefine "\n" to mean LS (if the encoding is capable of representing
it)* or to introduce a new escape sequence, ("\l"?) to mean LS? (Of
course, if we introduced "\l" for LS, we could also introduce "\p" for PS).
Thoughts anyone?
Jill
*FOOTNOTE - this is actually quite difficult to achieve if you're
storing stuff internally as bytes. Windows knows whether or not to
convert LF->CRLF and vice versa by means of a parameter passed to
fopen(), but this parameter can only distinguish between "text" and
"binary", not between "latin-1 text" and "utf-8 text". Things get easier
if you stor chars internally as Unicode chars of course.
------------------------ Yahoo! Groups Sponsor ---------------------~-->
KnowledgeStorm has over 22,000 B2B technology solutions. The most comprehensive IT buyers' information available. Research, compare, decide. E-Commerce | Application Dev | Accounting-Finance | Healthcare | Project Mgt | Sales-Marketing | More
http://us.click.yahoo.com/IMai8D/UYQGAA/cIoLAA/8FfwlB/TM
---------------------------------------------------------------------~->
To Unsubscribe, send a blank message to: unicode-***@yahooGroups.com
This mailing list is just an archive. The instructions to join the true Unicode List are on http://www.unicode.org/unicode/consortium/distlist.html
Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
I ask because plain text documents are created by text editors. The text
editor I happen to use is TextPad (there are hundreds of others, and
everyone has their favorite). It can save in UTF-8, and so on. But it
always saves documents with CRLF separating the lines. (It's a Windows
system).
Going a little deeper, applications like this are often written in C or
C++. These languages have the convention that "\n" in a string literal
means "new line". Strictly speaking, BY DEFINITION (from the C and C++
specs), "\n" is supposed to mean LF, and nothing else, but programs
compiled on Windows will reinterpret "\n" in a string literal to mean
either LF only (when in memory) or CRLF (when encoded to or from a file
or stream opened in text mode). Yes, it's a kludge, but it obviously
works quite well. I suspect (but I don't know for sure) that the Mac
will interpret "\n" as CR only.
It would seem impossible (or at least, a violation of the C/C++ specs)
to reinterpret "\n" as LS in C/C++ ... but then again, that
specification has already been violated, so maybe the precedent is there
and that no longer matters.
Nonetheless, it would seem, at least /slightly/ sensible to me that text
files encoded as UTF-8 should be using LS instead of CRLF. But this
appears to be difficult to achieve. There is no C/C++ escape sequence
which is defined to mean LS (unless you're prepared to write
"\xE2\x80\xA2" instead of "\n" all over the place), and what "\n"
generates is platform-dependent.
We can't change C or C++, of course, but would it make sense for other
computer languages, in particular future computer languages, either to
redefine "\n" to mean LS (if the encoding is capable of representing
it)* or to introduce a new escape sequence, ("\l"?) to mean LS? (Of
course, if we introduced "\l" for LS, we could also introduce "\p" for PS).
Thoughts anyone?
Jill
*FOOTNOTE - this is actually quite difficult to achieve if you're
storing stuff internally as bytes. Windows knows whether or not to
convert LF->CRLF and vice versa by means of a parameter passed to
fopen(), but this parameter can only distinguish between "text" and
"binary", not between "latin-1 text" and "utf-8 text". Things get easier
if you stor chars internally as Unicode chars of course.
------------------------ Yahoo! Groups Sponsor ---------------------~-->
KnowledgeStorm has over 22,000 B2B technology solutions. The most comprehensive IT buyers' information available. Research, compare, decide. E-Commerce | Application Dev | Accounting-Finance | Healthcare | Project Mgt | Sales-Marketing | More
http://us.click.yahoo.com/IMai8D/UYQGAA/cIoLAA/8FfwlB/TM
---------------------------------------------------------------------~->
To Unsubscribe, send a blank message to: unicode-***@yahooGroups.com
This mailing list is just an archive. The instructions to join the true Unicode List are on http://www.unicode.org/unicode/consortium/distlist.html
Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/