Recommend
18 
 Thumb up
 Hide
37 Posts
1 , 2  Next »   | 

Arimaa» Forums » News

Subject: April 18: David Wu's program (Sharp) wins the Arimaa challenge rss

Your Tags: Add tags
Popular Tags: [View All]
Phil Dreizen
United States
Kew Gardens
New York
flag msg tools
Avatar
mbmbmbmbmb
This past weekend a bot named "Sharp" won the Arimaa challenge!

From Wikipedia:

Arimaa Challenge

There is a $10,000 prize, available annually until 2020, for the first computer program to win the human-versus-computer Arimaa challenge. As part of the conditions of the prize, the computer program must run on standard, off-the-shelf hardware. As of 2014 the prize has not been claimed. However, the prize is expected to be claimed in 2015, because the current bot challenger (Sharp) has won 2 games in each series as of April 18, 2015.

My understanding is that the final condition of winning the prize, publishing a paper about the program that won, hasn't been done yet. Once that's been done all conditions will have been met.

Surprisingly, there isn't much on the net about this yet. Even arimaa.com barely has anything posted about it.


UPDATE (4/30/15) -- 2015 Arimaa Challenge Match results invalid:
The challenge has been invalidated due to side betting:
[geekurl=http://arimaa.com/arimaa/forum/cgi/YaBB.cgi?board=events;action=display;num=1430331089]Challenge Result declared invalid[/geekurl]

Omar Syed:
"I don't think any one of the people involved had bad intentions, but their actions have caused the challenge match to be influenced in ways that were not intended.... As such the result of the 2015 Arimaa Challenge Match are declared to be invalid and the challenge match will continue in 2016."

UPDATE (4/2/15)

Turns out the challenge is considered *valid* after all. It's a confusing situation.

Omar:
"Guys, I'm really sorry if I caused too much stress for you by creating this suspense and drama. When I found out about the bets that were placed I was quite disturbed and disappointed and I did not know it had happened in a previous Challenge match until I learned about it from this thread. However, I knew it was too late as per the challenge rules to do anything about it. But I thought it would be a good opportunity to add some suspense and drama to the challenge match being won. I hold no hard feeling from what anyone has said in this thread and I'm sorry if I hurt anyone else feelings by what I said...."


10 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Yours Truly,
United States
Raleigh
North Carolina
flag msg tools
badge
There must have been a moment at the beginning, where we could have said no. Somehow we missed it. Well, we'll know better next time.
Avatar
mb
kupad wrote:
This past weekend a bot named "Sharp" won the Arimaa challenge!

From Wikipedia:

Arimaa Challenge

There is a $10,000 prize, available annually until 2020, for the first computer program to win the human-versus-computer Arimaa challenge. As part of the conditions of the prize, the computer program must run on standard, off-the-shelf hardware. As of 2014 the prize has not been claimed. However, the prize is expected to be claimed in 2015, because the current bot challenger (Sharp) has won 2 games in each series as of April 18, 2015.

My understanding is that the final condition of winning the prize, publishing a paper about the program that won, hasn't been done yet. Once that's been done all conditions will have been met.

Surprisingly, there isn't much on the net about this yet. Even arimaa.com barely has anything posted about it.




Wow, that's HUGE, considering the origin of Arimaa was coming up with a game that would be easier for humans than for computers.

Yeah it looks like Arimaa's website has pretty much a headline about it and that's it. It's been only two days, but still you'd think there would be an article or something. Maybe they're writing one.

I google-newsed "Arimaa challenge" and even "Arimaa"... nothing. I would've thought the AI scene would be at least bit interested too.

Well congrats to David Wu, he gets $10K I guess!
2 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Fritz Juhnke
United States
Unspecified
Unspecified
flag msg tools
mbmbmbmbmb
David Wu gets $12K, in fact, but first he has to write a paper describing how sharp works and submit it for publication. The Arimaa Community it still reeling from sharp's victory; nobody expected it, not even David.
5 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Quinn Swanger
United States
Holly Springs
North Carolina
flag msg tools
badge
Avatar
mbmbmbmbmb
It looks more than a convincing performance too, I might add. Sharp clinched the challenge after only two rounds -- 6 wins to 0 losses! Two wins each against three different top players. Yikes!
6 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Russ Williams
Poland
Wrocław
Dolny Śląsk
flag msg tools
designer
badge
Avatar
mbmbmbmbmb
JohnnyDollar wrote:
Surprisingly, there isn't much on the net about this yet. Even arimaa.com barely has anything posted about it.
Yeah, I was trying to find info about it there and failed. :/
The front page has a link "Apr 18 David Wu's program (Sharp) wins the Arimaa challenge" but it goes to a generic announcement http://arimaa.com/arimaa/challenge/2015/ about what the conditions are, not news about Sharp's win (that page doesn't even mention Wu or Sharp).

I tried looking in the forums via the main page's link to Forum and utterly failed to login to the forum. The link goes to http://arimaa.com/arimaa/gameroom/forumLogin.cgi but the page says "The Arimaa Gameroom" and I had to login with my game account, not my forum account. Eventually I gave up in confusion. I'd not visited arimaa.com in a while, and this reminded me again how oddly confusing I find it... Is there some publicly visible forum discussion about it there, or do you have to login to view the forums?
1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Yours Truly,
United States
Raleigh
North Carolina
flag msg tools
badge
There must have been a moment at the beginning, where we could have said no. Somehow we missed it. Well, we'll know better next time.
Avatar
mb
For me part of the "mystique" of the game was that it had been designed specifically to be hard for computers to "solve" or at least to beat a human... so now that that's gone, the game is "just another game"...

Maybe the Arimaa website folk are terribly depressed by this and that's why there's virtually nothing posted about this impressive achievement.
2 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Russ Williams
Poland
Wrocław
Dolny Śląsk
flag msg tools
designer
badge
Avatar
mbmbmbmbmb
JohnnyDollar wrote:
For me part of the "mystique" of the game was that it had been designed specifically to be hard for computers to "solve" or at least to beat a human... so now that that's gone, the game is "just another game"...
I think the Arimaa challenge was merely a good PR hook to get attention/publicity for the game, but it's still the same good interesting game in any case. I liked it before, I still like it now (and I'm still a crappy player and I am sad I can't get it to the table these days...)
5 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Germany
flag msg tools
mbmb
About the Forum login:
I agree that it is kind of confusing and it has been for a while, the problem is that Omar doesn't have so much time anymore to fix issues.

You can log into the gameroom and then use the link in the gameroom on the top-right of the screen to get into the forum, though. I recommend clearing the cache or something similar. And never again click the link to the Forum from the main page, that is somehow really bugged.

About the Arimaa fanatics being distraught/depressed about the news and not posting because of this:
Omar is the only person running the website, so any announcement on the main page will have to come from him. Maybe he wants to wait until all 9 games have been played, the last 3 games will happen in the next 4 days.
After that, there is an award ceremony on May 2nd.(Exact Time not announced yet)
We had record attendance in the Arimaa chatroom during the 6th game of the Arimaa challenge, with 50 people in the chatroom at the peak moment.
There is discussion going on in the Forum about sharp and the future of the World Computer Championship and so on.

I didn't want to post anything here because I didn't want to get in Omar's way, I feel like he should have made an announcement of some sort, but again, maybe he will wait until all games are played.

I don't think anyone of the people who like playing Arimaa will stop playing Arimaa, just because a computer programm managed to beat the best humans. This does not change my enjoyment of the game in any way, and why should it?
One thing we might have to worry about is that we won't have such an active bot-developing community anymore and not as many new people being interested in Arimaa because the challenge is won.

Sharp's win is very impressive overall, Sharp was not in last year's challenge and the challenger bot was beaten 7-2. So this came as a surprise to many of us.

If you want to know more about the techniques/heuristics the developer used, you can come into the Arimaa chatroom at any time and ask the developer (if he is online) or attend the award ceremony, where the developer will be on Teamspeak (you don't need Teamspeak to listen, though) and will happily answer questions.
I think the developer will also be at one of the next three challenge games, but I don't know which one.
5 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Russ Williams
Poland
Wrocław
Dolny Śląsk
flag msg tools
designer
badge
Avatar
mbmbmbmbmb
Supersamu wrote:
About the Forum login:
I agree that it is kind of confusing and it has been for a while, the problem is that Omar doesn't have so much time anymore to fix issues.

You can log into the gameroom and then use the link in the gameroom on the top-right of the screen to get into the forum, though. I recommend clearing the cache or something similar. And never again click the link to the Forum from the main page, that is somehow really bugged.
I cleared my cache, logged into gameroom, then used the top right "Forum" link, and it still took me again to the same-looking login screen with text "The Arimaa Gameroom" but URL mentioning "forumLogin":
http://arimaa.com/arimaa/gameroom/forumLogin.cgi

Sigh. My diverse difficulties using the strange arimaa.com website (and that it simply stopped pairing me in automatically paired email games a long time ago with no explanation why and I couldn't figure out how to get that working again) is why I quit playing online. A good game in need of less buggy/flaky online support. :/
1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Yours Truly,
United States
Raleigh
North Carolina
flag msg tools
badge
There must have been a moment at the beginning, where we could have said no. Somehow we missed it. Well, we'll know better next time.
Avatar
mb
I noticed today at least on Google's front page results for "Arimaa," half-way down there's a news story link, but clicking through it turns out to be just a very short post from an "anonymous reader":

http://games.slashdot.org/story/15/04/19/2332209/computer-be...
2 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Tim Koppang
United States
Westmont
Illinois
flag msg tools
designer
publisher
"It's a magical world, Hobbes, ol' buddy..."
badge
"For the listener, who listens in the snow, and, nothing himself, beholds nothing that is not there and the nothing that is." -- Wallace Stevens
Avatar
mbmbmbmbmb
From the linked Slashdot thread, this sums up my questions nicely:

Zontar_Thing_From_Ve wrote:
Many years ago, it was assumed that in order for a computer to beat humans at chess that major advances would have to happen in artificial intelligence. In the end IBM just simply brute forced an approach by basically allowing the computer the equivalent of an open book test against a poor human who could only go by memory. Maybe some improvements got made in searching in order to beat humans, but I think that's about it. I know that arimaa was developed in the hope that it would spur new AI advances in order for a computer to beat humans. I have no idea - did Wu just simply brute force his way to victory? Is arimaa so little played by humans that the best players are far weaker at it than humans are at chess and thus it just wasn't all that hard to eventually write a program that could defeat the best human players? Did Wu actually advance the art any with his winning program?
2 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Russ Williams
Poland
Wrocław
Dolny Śląsk
flag msg tools
designer
badge
Avatar
mbmbmbmbmb
PS: if anyone else has trouble reaching the Arimaa.com forum, you can get there directly via the link http://arimaa.com/arimaa/forum/cgi/YaBB.cgi (which doesn't appear in the html source of the pages I visited). Thanks to Supersamu for this assistance!
1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Fritz Juhnke
United States
Unspecified
Unspecified
flag msg tools
mbmbmbmbmb
Sharp is an alpha-beta searcher, so in that sense not a radical departure from the basic method used to conquer chess. Also the evaluation function was primarily hand-coded as opposed to machine-learned from game databases or self play, at least as I understand it, so it isn't the type of AI some people were hoping for. Sharp represents incremental progress in AI, not a big breakthrough or a strikingly original method.

However, insofar as sharp is a "brute force" thinker, it is a very subtle kind of brute force. Arimaa's branching factor of 15,000 was overcome in part by designating only about 2% of moves as important ones to search, and looking at the other 98% only very superficially. Past efforts to filter moves like this have left exploitable "blind spots", so Wu had to do something clever to make sure critical moves weren't left out of the important set. I am curious how he did that.

Furthermore, the search was aggressively extended when goal was imminent. Quiescence search extensions are more difficult than in chess, where the number of moves that capture, check, or get out of check is small. The difficulty in Arimaa is that there can be a plethora of "get out of check" defensive moves that cause any quiescence search extension to explode, so Wu had to do something clever here too to have the extra search be targeted enough to be useful. I am also curious to read how he solved this problem.

The more the search tree becomes imbalanced, with some moves being discarded after cursory examination and other being pursued to great depth, the less one can dismiss the search engine as simply using brute force. Knowing which moves to think about is part of what distinguishes human intelligence from computer intelligence, and on that point sharp has blurred the lines a bit.
14 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Bob Jonas
United States
Alaska
flag msg tools
designer
badge
Avatar
mbmbmbmbmb
Sad news. Does this mean Arimaa will die? I hope not.
1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Russ Williams
Poland
Wrocław
Dolny Śląsk
flag msg tools
designer
badge
Avatar
mbmbmbmbmb
dashgalaxy86 wrote:
Sad news. Does this mean Arimaa will die? I hope not.
Chess didn't die despite programs as strong as people, so it doesn't seem like Arimaa should die.
1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Yours Truly,
United States
Raleigh
North Carolina
flag msg tools
badge
There must have been a moment at the beginning, where we could have said no. Somehow we missed it. Well, we'll know better next time.
Avatar
mb
russ wrote:
dashgalaxy86 wrote:
Sad news. Does this mean Arimaa will die? I hope not.
Chess didn't die despite programs as strong as people, so it doesn't seem like Arimaa should die.

Yeah, but Chess didn't originate as a "computer nemesis," with design inspirations specifically based on what would be hard for an AI to figure out.

That said, I've played it and it's a great game, and a unique game, so I don't think it will die. But, definitely a question I'm sure are on some people's minds.
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
United States
Washington
flag msg tools
very manly muppet
badge
muppet of a man
Avatar
mbmbmbmbmb
I've found that it's hard to get people to care about new abstract strategy games. The fact that arimaa was designed (up until now, successfully) to be resistant to brute-force AI was a cool hook/gimmick that I used in my sales pitch to get people interested in the game. It's also why I originally became interested in the game. Now that the challenge has fallen, I can't use that in my sales pitch anymore. It's very unfortunate, because arimaa is a fantastic game in its own right, but that's not immediately obvious until you play a few games, and now it will be harder to convince people to play those first few games.

I'm very disappointed that humans lost the challenge this year, but fortunately it won't be a blowout. Jean Daligault won his third game against Sharp. I watched that game, and midgame the weakness of arimaa AI became readily apparent. The game reached a position where, apparently, Sharp didn't see any immediate way to improve its position, and Sharp made several completely wasteful moves just shuffling pieces in the back row around while Jean slowly picked off Sharp's weak pieces. It's a real shame the humans lost to AI in this state. It didn't have to be this way.
6 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Avri
United States
Brooklyn
New York
flag msg tools
designer
badge
Avatar
mbmbmbmbmb
While "the challenge" is officially (about to be) over, I would hope that the annual AI contest followed by a Human versus AI challenge series will continue.

Good to see chessandgo (Daligault) take one from Sharp. Perhaps humans can rebound and reclaim the ascendency in 2016?
4 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Tim Koppang
United States
Westmont
Illinois
flag msg tools
designer
publisher
"It's a magical world, Hobbes, ol' buddy..."
badge
"For the listener, who listens in the snow, and, nothing himself, beholds nothing that is not there and the nothing that is." -- Wallace Stevens
Avatar
mbmbmbmbmb
nycavri wrote:
Good to see chessandgo (Daligault) take one from Sharp. Perhaps humans can rebound and reclaim the ascendency in 2016?
Agreed. I wonder if this is a matter of the human players learning from and about the Sharp program. In other words, knowing what they know now about Sharp, would the humans still lose? Would they still lose in a year after practicing? Etc.
2 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Yours Truly,
United States
Raleigh
North Carolina
flag msg tools
badge
There must have been a moment at the beginning, where we could have said no. Somehow we missed it. Well, we'll know better next time.
Avatar
mb
Arimaa Lives

cool
2 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
United States
Washington
flag msg tools
very manly muppet
badge
muppet of a man
Avatar
mbmbmbmbmb
browni3141 also won the third game against Sharp, so the final score is 7-2 for Sharp. One more win from either browni3141 or chessandgo, and the humans would have successfully defended the challenge.
2 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Robin Zigmond
United Kingdom
Durham
flag msg tools
mbmbmbmbmb
I for one don't care about the Arimaa challenge (except that I agree that Omar should have made more out of the publicity that Sharp's win in the Challenge had the potential to create) - I play Arimaa (actually I don't any more, because I don't have time for real-time games and for some reason the Autopostal stopped working ages ago, I also didn't play in the Postal Mixer last year because there is now some sort of charge for entering) because it's simply a great game. That's how I think it should be sold to people - a game which can be played with a chess set which otherwise has nothing in common with chess, which has straightforward rules (not quite as simple as Go, but way simpler than Chess) but staggering depth, one where it's easy to learn strategic concepts which can shape a whole game, but where the board position is constantly in flux despite it not being uncommon to go 30 moves by each side with few or no captures, and with some of the most hair-raising tactical endgames in any abstract.

Of course, none of this will interest anyone with no interest in abstract strategy games, which is most of BGG - yet Arimaa has a scandalously low profile here even relative to most other modern abstracts. (Probably because, unlike the likes of Hive and the project Gipf games, it is a deep game which tends to take at least an hour to play, so has little appeal as a family game or as a quick game for 2. Most "lifestyle" abstract gamers prefer to stick to Chess or Go.)
2 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Yours Truly,
United States
Raleigh
North Carolina
flag msg tools
badge
There must have been a moment at the beginning, where we could have said no. Somehow we missed it. Well, we'll know better next time.
Avatar
mb
robinz wrote:
- yet Arimaa has a scandalously low profile here even relative to most other modern abstracts.

Although BGG is how I discovered Arimaa, thankfully before it went OOP, and so now I'm a proud owner of one of the Z-Man edition copies. So at least it has some kind of presence here...
2 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Germany
flag msg tools
mbmb
I feel obligated to post this here, but I won't comment on it until I have collected my thoughts:

[geekurl=http://arimaa.com/arimaa/forum/cgi/YaBB.cgi?board=events;action=display;num=1430331089]Challenge Result declared invalid[/geekurl]

Feel free to cross-post this in the other threads discussing Arimaa atm, I didn't want to spam it in every single one of them.
5 
 Thumb up
0.05
 tip
 Hide
  • [+] Dice rolls
Robin Zigmond
United Kingdom
Durham
flag msg tools
mbmbmbmbmb
The following has just been posted my Omar on the above-linked thread (I was going to post a new link myself, but there seem to be issues, so this is easier):

omar wrote:
Guys, I'm really sorry if I caused too much stress for you by creating this suspense and drama. When I found out about the bets that were placed I was quite disturbed and disappointed and I did not know it had happened in a previous Challenge match until I learned about it from this thread. However, I knew it was too late as per the challenge rules to do anything about it. But I thought it would be a good opportunity to add some suspense and drama to the challenge match being won. I hold no hard feeling from what anyone has said in this thread and I'm sorry if I hurt anyone else feelings by what I said. The only person I let in on this was Janzert and only after it seemed that he might have been effected by this thread.

All is well in Arimaaland !!!


Seems that the whole thing was a (imo rather bad) joke on Omar's part. I didn't want to post this on the Arimaa forum for fear of offending Omar (I realise he may read this too, but at least it's not "his" site), but the whole thing seems to have served no purpose other than leave a rather bad taste in people's mouths.

On a more positive note, I only just realised I hadn't offered my congratulations to David for his hugely impressive achievement blush
2 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
1 , 2  Next »   |