Recommend
21 
 Thumb up
 Hide
31 Posts
1 , 2  Next »   | 

BoardGameGeek» Forums » BoardGameGeek Related » BGG News

Subject: GeekMail is now UTF-8 rss

Your Tags: Add tags
Popular Tags: [View All]
Scott Alden
United States
Dallas
Texas
flag msg tools
admin
badge
Aldie's Full of Love!
Avatar
mb
I'm converting systems over to use UTF-8 encoding, and the latest update is GeekMail. Please let me know if you see any problems.
1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
...sure...
Netherlands
Rijen
Noord Brabant
flag msg tools
badge
Avatar
mbmbmbmbmb
All things normal.
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Got two game tables and a microphone
United States
New York
flag msg tools
badge
Avatar
mbmbmbmbmb
Aldie wrote:
I'm converting systems over to use UTF-8 encoding, and the latest update is GeekMail. Please let me know if you see any problems.


EVERYTHING'S GOING TO HELL IN A HAnDBASKET!!! Otherwise, everything is fine...
1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Kristian Madsen
Sweden
Bandhagen
n/a
flag msg tools
Avatar
mbmbmbmbmb
Just use a wicker basket, presto, fixed.

/kgm
2 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Katie Hill
United States
Ada
Michigan
flag msg tools
Avatar
mbmbmbmbmb
Aldie,

Hey! I just noticed you had a UF microbadge...

GO GATORS!!!!!
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Lajos
Japan
Hachiouji
Tokyo
flag msg tools
designer
badge
Avatar
mbmbmb
I did send myself a test message with the game title ZÈRTZ and some Japanese. This is what I got:
Quote:
Subject: Re:test - ZÈRTZ - 日本語のテスト.
test - ZÈRTZ - 日本語のテスト.

Below is what I got in a similar test, earlier today, before posting http://www.boardgamegeek.com/thread/313668
Quote:
Subject: test - ZÈRTZ - ???????
test - ZÈRTZ - ???????
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Gwen
Belgium
oost-vlaanderen
flag msg tools
badge
Avatar
mb
This is the result I get in geekmail for French (ç, é, è, à )and German (ö, ä, ü)letters :

Subject: test
çççççç éééé èèèè à à à à à Ööööö äääää üüüüü
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Scott Alden
United States
Dallas
Texas
flag msg tools
admin
badge
Aldie's Full of Love!
Avatar
mb
Hmm that's odd. I just sent Lajos and gwen messages with some characters. Maybe I missed something, but it's working for me.

I'll keep plugging.
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Scott Alden
United States
Dallas
Texas
flag msg tools
admin
badge
Aldie's Full of Love!
Avatar
mb
Ok I think I found the problem - try again now.
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Lajos
Japan
Hachiouji
Tokyo
flag msg tools
designer
badge
Avatar
mbmbmb
Yes, working fine now.



- edit -

But now Z�RTZ looks weird in the Recently viewed column...
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Tor Gjerde
Norway
Trondheim
Unspecified
flag msg tools
designer
http://landnam.old.no
badge
http://old.no
Avatar
mbmbmbmbmb
Old mail with non-ascii characters still works fine, but the GeekMail page is still encoded as iso-8859-1, unlike the front page which is utf-8. I have put a link to my own game 'Landnám' in my Quick Bar, and this appears correctly on some pages (those with utf-8 encoding) and broken on others (those with iso-8859-1).

...while I was typing the text below, the encoding for GeekMail was changed, and the content is still correct . GeekLists and game images are still iso-8859-1, and hence shows 'Landnám' as 'Landnám' in my Quick Bar.

I assume Aldie has a plan for how to complete the transition, so the following piece of advice is to everybody else who has to battle this particular hydra. In my experience, the safest transition mechanism is to convert all non-ascii content to entities before changing encoding, even though this significantly bloats non-English text. If this breaks code that assumes one byte per character, then the same problem would occur when converting to utf-8. Note that if this method is used, then all export to non-html formats (such as email copies of geekmail) needs to be converted from entities to an appropriate encoding.

In those ugly cases where content is a mixture of utf-8 and iso-8859-1 from the outset, conversion should be done separately on each unit of text that can be assumed to be of a single encoding. For each, try to convert from utf-8, and if that fails, try to convert from iso-8859-1 (or rather windows codepage 1252, as this is a superset of iso-8859-1). If that fails too, the content is either not text, or is in an encoding that would need to be known for the system to work to begin with. The reason this approach works, is that the chance of a non-utf-8 encoded string containing non-ascii characters to actually be valid utf-8 is vanishingly small, even in huge text collections.
2 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Hammock Backpacker
United States
Columbus
Ohio
flag msg tools
Hot Coffee...Mmmmmm
badge
Go Take A HIke!
Avatar
mbmbmbmbmb
The ä is wonky in my 'name' on the left.

When editing contact info it looks like this: Mär' kwŭnd

 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Ben Kirman
United Kingdom
Lincoln
Lincolnshire
flag msg tools
badge
Avatar
mbmbmbmbmb
Aldie wrote:
I'm converting systems over to use UTF-8 encoding, and the latest update is GeekMail. Please let me know if you see any problems.


Cheers for taking on a challenge with this. I know from bitter and painful experience how hard it is to switch an existing site to UTF-8 but it is the right thing to do
3 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Mad Scientist Philip von Doomula
United States
Orono
ME
flag msg tools
"I'm a leaf on the wind. Watch how I soar."
badge
I got in everyone's hostile little face. Yes, these are wooden cubes from boardgaming. Yes, I'm comfortable with that. I am enlightened.
Avatar
mbmbmbmbmb
Nice job Aldie.
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Joe Casadonte
United States
Media
Pennsylvania
flag msg tools
designer
badge
The compass always points to Terrapin.
Avatar
mbmbmbmbmb
Lajos wrote:
But now Z�RTZ looks weird in the Recently viewed column...


Aldie, can you make sure that diacritic characters are encoded properly? Firefox is very strict (annoyingly so, I wish I could "fix" it) about this, and will show a "?" in a black triangle instead of the proper character if it's not encoded properly. Most of the characters look OK on this page, but the above quoted ZÈRTZ is not displaying properly. Perhaps something in the post editing isn't UTF-8? I'm curious how my accented E will show up; I'm using QuickQuote, if that matters.
1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Mike Jones
United States
Gainesville
Florida
flag msg tools
Yeah it's here! Really it's right here.
Avatar
mbmbmbmbmb
I read UTF-8 and my mind processed WTF, eh?
4 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Giacomo Mangiarano
Italy
Castellana Grotte
(BA)
flag msg tools
Use the force
badge
Everything is proceeding as I have foreseen
Avatar
mbmbmbmbmb
Image's tags on mouse over are diplaying in a wrong way: Fu#ball Ligretto, Caf# International...
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
United States
flag msg tools
badge
Avatar
Aldie wrote:
I'm converting systems over to use UTF-8 encoding.


Poll
UTF-8 encoding, Mr. Aldie?
Aldie, don't wear the Ring !
Cast it into the fire !
You have my sword !
I can't carry UTF-8, but I can carry you, Mr. Aldie !
      54 answers
Poll created by Sexy Amy

12 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
J
United States
Lexington
Kentucky
flag msg tools
admin
Avatar
mbmbmbmbmb
this isn't a problem because of the UTF-8 change as I noticed it before, but if you send a geekmail and use the red shortcut tags for formatting and then hit preview, the text between the tags (and the brackets themselves) disappear (although it does show in the preview box)
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Diane Close
United States
Twin Cities
Minnesota
flag msg tools
badge
Avatar
mbmbmbmbmb
Aldie wrote:
I'm converting systems over to use UTF-8 encoding, and the latest update is GeekMail. Please let me know if you see any problems.


There are still problems with code misinterpretation errors in the forums, and here's one I got in geekmail just a few minutes ago. The original subject line was:

Package Problems?

This is what I got back as the subject line for the response:

Re: [BGG] Package =?UTF-8?B?cHJvYmxlbXM/?=
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Dave Dubin
United States
Champaign
Illinois
flag msg tools
badge
Avatar
mbmbmbmbmb
Aldie wrote:
I'm converting systems over to use UTF-8 encoding, and the latest update is GeekMail. Please let me know if you see any problems.




אַז מע דערצײלט אַ מעשׂה אַ פּױער, לאַכט ער דרײַ מאָל


Looks OK! Thanks, Aldie.

 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Michael Leuchtenburg
United States
Cambridge
Massachusetts
flag msg tools
badge
Avatar
mbmbmbmbmb
Awesome! I look forward to seeing the rest of things converted too. That'll make using the BGG API a lot easier.
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Anthony
Canada
Vancouver
BC
flag msg tools
Avatar
mbmbmbmbmb
WTFIUTF-8? I'm guessing it's not a virus.
1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Barry Goldstein
United States
Culver City
California
flag msg tools
badge
Avatar
mbmbmbmbmb
I think it has something to do with a Cthulhu cult Aldie joined.
It works ok, but costs several sanity points to cast.


Tsk tsk Aldie...The Great Old Ones are displeased.
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Paul - the
Sweden
Lund
flag msg tools
My name is Ozymandias, King of Kings; Look on my Works, ye Mighty, and despair!
badge
You spin me right round, baby - Right round like a record, baby - Right round round round - You spin me right round, baby…
Avatar
mbmbmbmbmb
Great stuff! thumbsup

Now I can finally use Scandinavian characters without the mails getting all funky.
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
1 , 2  Next »   | 
Front Page | Welcome | Contact | Privacy Policy | Terms of Service | Advertise | Support BGG | Feeds RSS
Geekdo, BoardGameGeek, the Geekdo logo, and the BoardGameGeek logo are trademarks of BoardGameGeek, LLC.