Hexadecimal again (was RE: Clones)

[J***@aculab.com]

Thanks, but not good enough.

What guarantee do I have that other Unicode characters will not be added in
the future which have the property "Hex_Digit"?

How do I write an algorithm which will convert Unicode hex characters to
hexadecimal which is guaranteed to work for all future versions of Unicode?
(The algorithm is allowed to read the machine-readable charts, which may
obviously be replaced with updated versions as they arise).

Jill


-----Original Message-----
From: John Cowan [mailto:***@reutershealth.com]
Sent: Monday, August 18, 2003 4:39 PM
To: ***@Aculab.com
Cc: ***@unicode.org
Subject: Re: Clones (was RE: Hexadecimal)

Secondly, I believe that the code charts SHOULD provide machine-readable
information about the hexadecimal values of the letters "A" to "F".

0030;0
0031;1
0032;2
0033;3
0034;4
0035;5
0036;6
0037;7
0038;8
0039;9
0041;10
0042;11
0043;12
0044;13
0045;14
0046;15
0061;10
0062;11
0063;12
0064;13
0065;14
0066;15
FF10;0
FF11;1
FF12;2
FF13;3
FF14;4
FF15;5
FF16;6
FF17;7
FF18;8
FF19;9
FF21;10
FF22;11
FF23;12
FF24;13
FF25;14
FF26;15
FF41;10
FF42;11
FF43;12
FF44;13
FF45;14
FF46;15

Thuryago.


------------------------ Yahoo! Groups Sponsor ---------------------~-->
KnowledgeStorm has over 22,000 B2B technology solutions. The most comprehensive IT buyers' information available. Research, compare, decide. E-Commerce | Application Dev | Accounting-Finance | Healthcare | Project Mgt | Sales-Marketing | More
http://us.click.yahoo.com/IMai8D/UYQGAA/cIoLAA/8FfwlB/TM
---------------------------------------------------------------------~->

To Unsubscribe, send a blank message to: unicode-***@yahooGroups.com

This mailing list is just an archive. The instructions to join the true Unicode List are on http://www.unicode.org/unicode/consortium/distlist.html


Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/

[Rick McGowan]

Jill Ramonsky asked...

Post by J***@aculab.com
What guarantee do I have that other Unicode characters will not be
added in the future which have the property "Hex_Digit"?

You don't have a guarantee of much in the future, except as indicated in
the Unicode stability policies.

Realistically, however, you're probably not going to see the addition of
more hex digits, because all sixteen of them are already represented in the
standard.

Post by J***@aculab.com
How do I write an algorithm which will convert Unicode hex characters
to hexadecimal which is guaranteed to work for all future versions
of Unicode?

Use 0-9, A-F, a-f.

Rick



------------------------ Yahoo! Groups Sponsor ---------------------~-->
KnowledgeStorm has over 22,000 B2B technology solutions. The most comprehensive IT buyers' information available. Research, compare, decide. E-Commerce | Application Dev | Accounting-Finance | Healthcare | Project Mgt | Sales-Marketing | More
http://us.click.yahoo.com/IMai8D/UYQGAA/cIoLAA/8FfwlB/TM
---------------------------------------------------------------------~->

To Unsubscribe, send a blank message to: unicode-***@yahooGroups.com

This mailing list is just an archive. The instructions to join the true Unicode List are on http://www.unicode.org/unicode/consortium/distlist.html


Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/

[J***@aculab.com]

-----Original Message-----
From: Rick McGowan [mailto:***@unicode.org]
Sent: Tuesday, August 19, 2003 3:43 PM
To: ***@unicode.org
Subject: Re: Hexadecimal again (was RE: Clones)


My apologies if I have offended you (though I'm not quite sure how I might
have done). Let me reassure you that we're all "on the same side" here, as
it were. I know I'm a newcomer to this list, but I'm a programmer, and an
implementor. I want to promote Unicode, and I'm helping to make Unicode
work.

Look at it from my point of view. I'm English. I don't speak Chinese,
Japanase, Bengali, Indian, or ANYTHING that requires me to use a non-English
keyboard. I have _absolutely no idea_ how the Japanese write hex. For all I
know, maybe the Japanese use Japanese symbols for hex? For all I know, maybe
the Indians use Indian symbols for hex? I don't know. And it's precisely
BECAUSE I don't know that it would be arrogant of me in the extreme to
assume that all around the world, in every nation, in every language, and
for every alphabet, everyone, without exception, uses Latin letters for hex
digits. Why should I make an assumption so arrogant? I know for a fact that
people don't all use the Latin digits 0 to 9, so what on Earth gives me the
confidence that they will all use A to F? For all I know, the absence of
Bengali hex digits from the current Unicode standard _COULD_ be an omission
which will be corrected in the next edition. I wish to write libraries of
code that will work for everyone, and which will continue to work for
everyone long into the future.

Post by Rick McGowan

Post by J***@aculab.com
How do I write an algorithm which will convert Unicode hex characters
to hexadecimal which is guaranteed to work for all future versions
of Unicode?

Use 0-9, A-F, a-f.

With all due respect, that algorithm doesn't even meet the CURRENT Unicode
Standard.

I am very, very grateful to John Cowan, who emailed me to say that he is
going to propose adding to the standard the chart he posted. With that, this
quest effectively comes to an end, and I appreciate that.

You may think I'm some sort of moaning nitpicker, but these are real world,
non-trivial problems. The need for internationalisation demands that we drop
our assumption that Western standards will always be used by everybody. An
algorithm must rely on the tables alone. It cannot rely on anything else,
particularly not on an assumption of the universality of Western standards.

Thanks for answering, but please try to understand why I ask.

Jill


------------------------ Yahoo! Groups Sponsor ---------------------~-->
KnowledgeStorm has over 22,000 B2B technology solutions. The most comprehensive IT buyers' information available. Research, compare, decide. E-Commerce | Application Dev | Accounting-Finance | Healthcare | Project Mgt | Sales-Marketing | More
http://us.click.yahoo.com/IMai8D/UYQGAA/cIoLAA/8FfwlB/TM
---------------------------------------------------------------------~->

To Unsubscribe, send a blank message to: unicode-***@yahooGroups.com

This mailing list is just an archive. The instructions to join the true Unicode List are on http://www.unicode.org/unicode/consortium/distlist.html


Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/

[Tex Texin]

Post by J***@aculab.com
Thanks, but not good enough.
What guarantee do I have that other Unicode characters will not be added in
the future which have the property "Hex_Digit"?

One solution is to join the consortium and be able to vote against such a
thing happening!

If it is a concern you can still implement your algorithm to allow the hex
digits to be separately or externally specifiable, perhaps using John's chart.
(With perhaps a slight attendant security risk... ;-) )

Post by J***@aculab.com
From a practical standpoint, I think it is more likely that the base will

change rather than the hex characters.
After all, digits have been constant for a long time, but the base has
changed. Initially it was binary, then it was octal, and now hex arithmetic is
common. It seems more likely to me that we might switch to another base (32?
64?) as platforms expand, before we started adding redundant characters to hex
arithmetic. Somewhere, someday, some wristwatch-sized, space-deprived display
device manufacturer will be complaining that he doesn't have enough room on
his device to show the hex codes for the combining sequence of unicode
characters missing in his font, and so instead of hex, he wants to use base64
characters, but only if the characters are defined in the standard....


(Guess I am showing my age to be recalling flipping binary switches...) ;-)

All your base are belong to us!
tex


------------------------ Yahoo! Groups Sponsor ---------------------~-->
KnowledgeStorm has over 22,000 B2B technology solutions. The most comprehensive IT buyers' information available. Research, compare, decide. E-Commerce | Application Dev | Accounting-Finance | Healthcare | Project Mgt | Sales-Marketing | More
http://us.click.yahoo.com/IMai8D/UYQGAA/cIoLAA/8FfwlB/TM
---------------------------------------------------------------------~->

To Unsubscribe, send a blank message to: unicode-***@yahooGroups.com

This mailing list is just an archive. The instructions to join the true Unicode List are on http://www.unicode.org/unicode/consortium/distlist.html


Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/

[Jon Hanna]

Post by J***@aculab.com
From a practical standpoint, I think it is more likely that the base will
change rather than the hex characters.
After all, digits have been constant for a long time, but the base has
changed. Initially it was binary, then it was octal, and now hex arithmetic is
common.

No, first it was binary, then it was binary and now its binary. Different
human-readable formats have been (and continue to be) used to represent
this.

It seems more likely to me that we might switch to

Post by J***@aculab.com
another base (32?
64?) as platforms expand, before we started adding redundant
characters to hex
arithmetic.

What human-readability advantages (the only reason we use hex) would base 32
or base 64 representations have over hex? They aren't matched by a nice
number of bits for most systems; the reason for using hex rather than octal
is that 2 hex digits can exactly represent the range of a octet (the most
common size of bytes these days) and by extension of any word composed of an
integral number of octets. The next base to have that quality is base 256,
which would require us to ransack a few different alphabets and then maybe
create a few symbols in order for us to represent it.



------------------------ Yahoo! Groups Sponsor ---------------------~-->
KnowledgeStorm has over 22,000 B2B technology solutions. The most comprehensive IT buyers' information available. Research, compare, decide. E-Commerce | Application Dev | Accounting-Finance | Healthcare | Project Mgt | Sales-Marketing | More
http://us.click.yahoo.com/IMai8D/UYQGAA/cIoLAA/8FfwlB/TM
---------------------------------------------------------------------~->

To Unsubscribe, send a blank message to: unicode-***@yahooGroups.com

This mailing list is just an archive. The instructions to join the true Unicode List are on http://www.unicode.org/unicode/consortium/distlist.html


Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/

[Peter Kirk]

... The next base to have that quality is base 256,
which would require us to ransack a few different alphabets and then maybe
create a few symbols in order for us to represent it.

No, we could just use Ethiopic. Plenty of characters there. We could
even put some logic in the system e.g. by use the vowel parts of the
glyphs to indicate the lower three bits. I'm sure most people would
learn quickly. And if we used Ethiopic letters to define Unicode symbols
it might stop some people complaining that Unicode isn't African enough.
;-)
--
Peter Kirk
***@qaya.org (personal)
***@qaya.org (work)
http://www.qaya.org/




------------------------ Yahoo! Groups Sponsor ---------------------~-->
KnowledgeStorm has over 22,000 B2B technology solutions. The most comprehensive IT buyers' information available. Research, compare, decide. E-Commerce | Application Dev | Accounting-Finance | Healthcare | Project Mgt | Sales-Marketing | More
http://us.click.yahoo.com/IMai8D/UYQGAA/cIoLAA/8FfwlB/TM
---------------------------------------------------------------------~->

To Unsubscribe, send a blank message to: unicode-***@yahooGroups.com

This mailing list is just an archive. The instructions to join the true Unicode List are on http://www.unicode.org/unicode/consortium/distlist.html


Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/

[Tex Texin]

Post by Jon Hanna

Post by J***@aculab.com
From a practical standpoint, I think it is more likely that the base will
change rather than the hex characters.
After all, digits have been constant for a long time, but the base has
changed. Initially it was binary, then it was octal, and now hex
arithmetic is
common.

No, first it was binary, then it was binary and now its binary. Different
human-readable formats have been (and continue to be) used to represent
this.
It seems more likely to me that we might switch to

Post by J***@aculab.com
another base (32?
64?) as platforms expand, before we started adding redundant
characters to hex
arithmetic.

What human-readability advantages (the only reason we use hex) would base 32
or base 64 representations have over hex? They aren't matched by a nice
number of bits for most systems;

Only density. You are right 256 would be a more convenient base.
Fortunately with Unicode ransacking alphabets is easy!

Jon I was mostly being tongue in cheek and contrasting that relative to
needing new hex digits, a base change was more likely. However, I wasn't
saying that a base change is likely.
tex

the reason for using hex rather than octal

Post by Jon Hanna
is that 2 hex digits can exactly represent the range of a octet (the most
common size of bytes these days) and by extension of any word composed of an
integral number of octets. The next base to have that quality is base 256,
which would require us to ransack a few different alphabets and then maybe
create a few symbols in order for us to represent it.

--
-------------------------------------------------------------
Tex Texin cell: [phone] mailto:***@XenCraft.com
Xen Master http://www.i18nGuy.com

XenCraft http://www.XenCraft.com
Making e-Business Work Around the World
-------------------------------------------------------------


------------------------ Yahoo! Groups Sponsor ---------------------~-->
KnowledgeStorm has over 22,000 B2B technology solutions. The most comprehensive IT buyers' information available. Research, compare, decide. E-Commerce | Application Dev | Accounting-Finance | Healthcare | Project Mgt | Sales-Marketing | More
http://us.click.yahoo.com/IMai8D/UYQGAA/cIoLAA/8FfwlB/TM
---------------------------------------------------------------------~->

To Unsubscribe, send a blank message to: unicode-***@yahooGroups.com

This mailing list is just an archive. The instructions to join the true Unicode List are on http://www.unicode.org/unicode/consortium/distlist.html


Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/

[Jon Hanna]

Post by Tex Texin
Jon I was mostly being tongue in cheek and contrasting that relative to
needing new hex digits, a base change was more likely. However, I wasn't
saying that a base change is likely.

And I was being tongue in cheek (and ignorant of Ethiopian script) in
suggesting the use of base 256. However we both had serious points and my
serious point disagrees with yours.

Hexadecimal is used for good reasons; and while I'm not convinced about
Jill's point I'm not convinced otherwise either. What do hackers with non
Latin-based languages use for hex anyway?



------------------------ Yahoo! Groups Sponsor ---------------------~-->
KnowledgeStorm has over 22,000 B2B technology solutions. The most comprehensive IT buyers' information available. Research, compare, decide. E-Commerce | Application Dev | Accounting-Finance | Healthcare | Project Mgt | Sales-Marketing | More
http://us.click.yahoo.com/IMai8D/UYQGAA/cIoLAA/8FfwlB/TM
---------------------------------------------------------------------~->

To Unsubscribe, send a blank message to: unicode-***@yahooGroups.com

This mailing list is just an archive. The instructions to join the true Unicode List are on http://www.unicode.org/unicode/consortium/distlist.html


Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/

[Rick McGowan]

Post by Jon Hanna
What do hackers with non
Latin-based languages use for hex anyway?

They use 0-9, A-F, and a-f.

Hex is used mostly by programmers, mostly for computing, and mostly in
programming languages that have the digits and Latin letters built-in, and
that's what compilers expect to see. Hex doesn't have an independent
existence out in non-computing culture for, e.g., signs in the market place
or monetary values.

Rick


------------------------ Yahoo! Groups Sponsor ---------------------~-->
KnowledgeStorm has over 22,000 B2B technology solutions. The most comprehensive IT buyers' information available. Research, compare, decide. E-Commerce | Application Dev | Accounting-Finance | Healthcare | Project Mgt | Sales-Marketing | More
http://us.click.yahoo.com/IMai8D/UYQGAA/cIoLAA/8FfwlB/TM
---------------------------------------------------------------------~->

To Unsubscribe, send a blank message to: unicode-***@yahooGroups.com

This mailing list is just an archive. The instructions to join the true Unicode List are on http://www.unicode.org/unicode/consortium/distlist.html


Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/

[Ben Dougall]

Post by Rick McGowan

Post by Jon Hanna
What do hackers with non
Latin-based languages use for hex anyway?

They use 0-9, A-F, and a-f.

which'll be whatever characters happen to be used to represent those
sections of the character set on their machines: 0x30 - 0x39, 0x41 -
0x46 and 0x61 - 0x66.

Post by Rick McGowan
Hex is used mostly by programmers, mostly for computing, and mostly in
programming languages that have the digits and Latin letters built-in, and
that's what compilers expect to see. Hex doesn't have an independent
existence out in non-computing culture for, e.g., signs in the market place
or monetary values.
Rick

------------------------ Yahoo! Groups Sponsor ---------------------~-->
KnowledgeStorm has over 22,000 B2B technology solutions. The most comprehensive IT buyers' information available. Research, compare, decide. E-Commerce | Application Dev | Accounting-Finance | Healthcare | Project Mgt | Sales-Marketing | More
http://us.click.yahoo.com/IMai8D/UYQGAA/cIoLAA/8FfwlB/TM
---------------------------------------------------------------------~->

To Unsubscribe, send a blank message to: unicode-***@yahooGroups.com

This mailing list is just an archive. The instructions to join the true Unicode List are on http://www.unicode.org/unicode/consortium/distlist.html


Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/

[Rick McGowan]

Curtis Clark,

Caviar, 10kg, €FEED

Heh, heh... Don't you mean:

Caviar, Akg, €FEED

;-)

Rick


------------------------ Yahoo! Groups Sponsor ---------------------~-->
KnowledgeStorm has over 22,000 B2B technology solutions. The most comprehensive IT buyers' information available. Research, compare, decide. E-Commerce | Application Dev | Accounting-Finance | Healthcare | Project Mgt | Sales-Marketing | More
http://us.click.yahoo.com/IMai8D/UYQGAA/cIoLAA/8FfwlB/TM
---------------------------------------------------------------------~->

To Unsubscribe, send a blank message to: unicode-***@yahooGroups.com

This mailing list is just an archive. The instructions to join the true Unicode List are on http://www.unicode.org/unicode/consortium/distlist.html


Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/

[Philippe Verdy]

Post by Rick McGowan
Curtis Clark,

Caviar, 10kg, €FEED

Caviar, Akg, €FEED

And why not this menu:
€BEEFFACE VINAIGRETTE
WINE OF BOURGOGNE €A0C (3/4L)



------------------------ Yahoo! Groups Sponsor ---------------------~-->
KnowledgeStorm has over 22,000 B2B technology solutions. The most comprehensive IT buyers' information available. Research, compare, decide. E-Commerce | Application Dev | Accounting-Finance | Healthcare | Project Mgt | Sales-Marketing | More
http://us.click.yahoo.com/IMai8D/UYQGAA/cIoLAA/8FfwlB/TM
---------------------------------------------------------------------~->

To Unsubscribe, send a blank message to: unicode-***@yahooGroups.com

This mailing list is just an archive. The instructions to join the true Unicode List are on http://www.unicode.org/unicode/consortium/distlist.html


Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/

[Curtis Clark]

Post by Rick McGowan
Hex doesn't have an independent
existence out in non-computing culture for, e.g., signs in the market place
or monetary values.

Caviar, 10kg, €FEED
--
Curtis Clark http://www.csupomona.edu/~jcclark/
Mockingbird Font Works http://www.mockfont.com/




------------------------ Yahoo! Groups Sponsor ---------------------~-->
KnowledgeStorm has over 22,000 B2B technology solutions. The most comprehensive IT buyers' information available. Research, compare, decide. E-Commerce | Application Dev | Accounting-Finance | Healthcare | Project Mgt | Sales-Marketing | More
http://us.click.yahoo.com/IMai8D/UYQGAA/cIoLAA/8FfwlB/TM
---------------------------------------------------------------------~->

To Unsubscribe, send a blank message to: unicode-***@yahooGroups.com

This mailing list is just an archive. The instructions to join the true Unicode List are on http://www.unicode.org/unicode/consortium/distlist.html


Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/

[Jim Allan]

Post by Ben Dougall
which'll be whatever characters happen to be used to represent those
sections of the character set on their machines: 0x30 - 0x39, 0x41 -
0x46 and 0x61 - 0x66.

Not in EBCDIC (and other older character sets) they aren't. There are a
lot of mainframe systems still using EBCDIC encodings.

Jim Allan



------------------------ Yahoo! Groups Sponsor ---------------------~-->
KnowledgeStorm has over 22,000 B2B technology solutions. The most comprehensive IT buyers' information available. Research, compare, decide. E-Commerce | Application Dev | Accounting-Finance | Healthcare | Project Mgt | Sales-Marketing | More
http://us.click.yahoo.com/IMai8D/UYQGAA/cIoLAA/8FfwlB/TM
---------------------------------------------------------------------~->

To Unsubscribe, send a blank message to: unicode-***@yahooGroups.com

This mailing list is just an archive. The instructions to join the true Unicode List are on http://www.unicode.org/unicode/consortium/distlist.html


Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/

[Philippe Verdy]

Philippe.
Les messages non sollicités (spams) ne sont pas tolérés.
Tout abus sera signalé automatiquement à vos fournisseurs de service.
----- Original Message -----
From: "Jim Allan" <***@smrtytrek.com>
To: <***@unicode.org>
Sent: Thursday, August 21, 2003 2:54 AM
Subject: Re: Hexadecimal never again

Post by Jim Allan

Post by Ben Dougall
which'll be whatever characters happen to be used to represent those
sections of the character set on their machines: 0x30 - 0x39, 0x41 -
0x46 and 0x61 - 0x66.

Not in EBCDIC (and other older character sets) they aren't. There are a
lot of mainframe systems still using EBCDIC encodings.

And probably some remaining devices using 5-bit or 6-bit encodings...
Unicode does not specify encodings out of the UTF-* series.

I do think that there may also exist some EBCDIC-based transform for
Unicode similar to UTF-8, except that the UTF-8 bytes are remapped to
their basic EBCDIC codes (those that do not depend on locale variants,
and correspond to ASCII bytes and a few EBCDIC "C1" codes), using the
holes to remap the missing byte values needed to represent the full
range of UTF-8 encoding byte values 0x00 to 0xFB.



------------------------ Yahoo! Groups Sponsor ---------------------~-->
KnowledgeStorm has over 22,000 B2B technology solutions. The most comprehensive IT buyers' information available. Research, compare, decide. E-Commerce | Application Dev | Accounting-Finance | Healthcare | Project Mgt | Sales-Marketing | More
http://us.click.yahoo.com/IMai8D/UYQGAA/cIoLAA/8FfwlB/TM
---------------------------------------------------------------------~->

To Unsubscribe, send a blank message to: unicode-***@yahooGroups.com

This mailing list is just an archive. The instructions to join the true Unicode List are on http://www.unicode.org/unicode/consortium/distlist.html


Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/

[Doug Ewell]

Post by Philippe Verdy
And probably some remaining devices using 5-bit or 6-bit encodings...
Unicode does not specify encodings out of the UTF-* series.

SCSU:
http://www.unicode.org/reports/tr6/

BOCU-1:
(just a Technical Note, may not count as a Unicode "specification")
http://www.unicode.org/notes/tn6/

Post by Philippe Verdy
I do think that there may also exist some EBCDIC-based transform for
Unicode similar to UTF-8, except that the UTF-8 bytes are remapped to
their basic EBCDIC codes (those that do not depend on locale variants,
and correspond to ASCII bytes and a few EBCDIC "C1" codes), using the
holes to remap the missing byte values needed to represent the full
range of UTF-8 encoding byte values 0x00 to 0xFB.

UTF-EBCDIC:
http://www.unicode.org/reports/tr16/


-Doug Ewell
Fullerton, California
http://users.adelphia.net/~dewell/



------------------------ Yahoo! Groups Sponsor ---------------------~-->
KnowledgeStorm has over 22,000 B2B technology solutions. The most comprehensive IT buyers' information available. Research, compare, decide. E-Commerce | Application Dev | Accounting-Finance | Healthcare | Project Mgt | Sales-Marketing | More
http://us.click.yahoo.com/IMai8D/UYQGAA/cIoLAA/8FfwlB/TM
---------------------------------------------------------------------~->

To Unsubscribe, send a blank message to: unicode-***@yahooGroups.com

This mailing list is just an archive. The instructions to join the true Unicode List are on http://www.unicode.org/unicode/consortium/distlist.html


Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/