Average points scored in a game of Scrabble.

What is a good score in a game of Scrabble? Nigel Richards averages 462.54 in international tournaments. He has gone past 400 points 84.41% of the time. Craig Beavers averages 438.59 and has gone past 400 points 73.34% of the time. Mind you these are international tournaments where they would be coming up against strong opponents most of the time. Their averages would go up considerably if national level and state level tournaments were also taken into consideration. Anyway, this is not an exhaustive study on average game score, it's merely a study on the use of average game scores to evaluate the fairness of the draw.

At the Age Group Scrabble Championship 2016, weaker players struggled to pass 300, the top rated players notched up 500+ and even 600+ scores with ease. Thatwas due to a huge disparity between the top players who are some of the best young Scrabblers in the world and others who compete only at the domestic level. This disparity makes it possible to use the average number of points scored by each player as an indicator for the challenges faced by that player.

Position Name Wins Margin Mean Points
13 Hasindu Arumapperuma 4 1409 506
1 Aabid Ismail 6 829 475
37 Thirandi De Silva 3 660 461
14 Shakeel Ahamed Ajim 4 754 459
3 Hayati Rassool 5 738 450
38 Janidu Karunarathne 3 619 443
15 Vignesh Pirapaharan 4 494 434
16 Hemaka Gunasekera 4 492 432
20 Janul De Silva 4 404 429
2 Migara Jayasinghe 5 746 428

Table 1 Average (mean) points per game and marging sorted by average points

Janul won the prize for highest game score by racking up 646 in the first round. Despite two losses defending champion Hasindu averaged 506 per game, 31 points higher than the new champion's average. Hasindu had three scores between 500 - 600 and once scored 625. Hasindu’s lowest score was 367. That means Hasindu’s lowest score is higher that 91% of scores in the entire tournament.

As with many other comparisons Aabid is right up there with the best. It shows that despite it’s faults King Of The Hill (KOTH) is capable of selecting a champion. However only three other prize winners make it to this list.

37th placed Thirandi averaged 461 per game which is the third highest despite losing three games and she has a healthy margin of 660 points too (sixth highest among all players). Janidu with 443 was 38th. Compare this with an average of 259 for 36th placed Jithmi Linara.

 

Position Name Wins Margin Mean Points
1 Aabid Ismail 6 829 475
2 Migara Jayasinghe 5 746 428
3 Hayati Rassool 5 738 450
4 Hansi Weerasooriya 5 630 425
5 Radinka Dissanayake 5 574 421
6 Rehan Wijesekera 5 481 427
7 Danusha Muhandiramge 5 421 406
8 Yenuli Hewage 5 417 379
9 Navodya Jayaratna 5 400 383
10 Akshai Yogalingam 5 269 326

Table 2 Average scores for the prize winners

The first 7 prize winners have very respectable average number of points scored but for the last three places the spread and the average per game is much below par.

Position Name Wins Margin Mean Points
2 Migara Jayasinghe 5 746 428
1 Aabid Ismail 6 829 475
20 Janul De Silva 4 404 429
13 Hasindu Arumapperuma 4 1409 506
38 Janidu Karunarathne 3 619 443
21 Thavalakshman Yoganathan 4 390 381
3 Hayati Rassool 5 738 450
5 Radinka Dissanayake 5 574 421
37 Thirandi De Silva 3 660 461
15 Vignesh Pirapaharan 4 494

434

 

Table 3 Average scores for the rated players.

Though six of the rated players have not managed to get on the prize list, they have done very well in terms of average points score. The lowest amongst them is 381 by Thavalakshman. He would be disapointed with that. But it is still a respectable score.

So in conclusion it can be seen that King Of The Hill is grossly unfair if you use Average Points Score as a measure of performance. Earlier we looked at Buchholz score and Average Opponent Rank to find that KOTH has been grossly unfair.

Do I rest my case? Not yet. We will next look at Aristomancy to judge the effectiveness of the pairing system

July 11, 2016, 5:21 a.m. » Tagged: Scrabble , KOTH , Pairing

Buchholz score as a measure of fairness of the draw.

Average Ratings of Opponents, the Buchholz and Sonneborn-Berger System are some of the methods recommended by FIDE for breaking ties in Chess tournaments. While ties are the order of the day in chess tournaments, in the Scrabble world it is virtually unheard of. But what works for tie breaking in chess works well for evaluating Scrabble pairings. We will take a closer look at Buchholz in this post particularly since it's one recommended by John Chew, the creator of Tournament Shell for exactly this purpose.

The sum of opponent scores or Buchholz is “..the idea is that the same score is more valuable if achieved against players with better performances in a given tournament. Looks like an ideal tie-breaking method and has been used since the Swiss system was invented.” (Swissperfect, 2008) 

In this context the score means the number of games won by each player. To avoid confusion with the number of points scored on the board, the term Bucholz score will be used when the discussion is about the total number of games won by a player's’ opponents. And this discussion is about evaluating the fairness of King Of The Hill pairings in Scrabble. This is fourth part in a series. 

For example Hayati played against Erandi (3 wins), Thirandi ( 3 wins), Danusha ( 5 wins), Janidu (3 wins), Lalinka ( 3 wins), Navodya (5 wins) so her Buchholz score is 22. This is what is looks like for all the prize winners.

 

Position name Buchholz score
1 Aabid Ismail 25
2 Migara Jayasinghe 19
3 Hayati Rassool 22
4 Hansi Weerasooriya 26
5 Radinka Dissanayake 24
6 Rehan Wijesekera 19
7 Danusha Muhandiramge 23
8 Yenuli Hewage 21
9 Navodya Jayaratna 22
10 Akshai Yogalingam 17

Table 2.4 Sum of opponents scores for each of the prize winners

Most of the prize winners have a Buchholz score greater than 20. For Akshai, it's only 17 but that's reasonably since he is placed only tenth overall isn't it? Let's see what the next table, it lists the players with the best Buchholz scores

 

Position Name Buchholz score
4 Hansi Weerasooriya 26
1 Aabid Ismail 25
5 Radinka Dissanayake 24
25 Qays Sangani 24
41 Lalinka De Silva 24
18 Nikesh Moorthy 24
20 Janul De Silva 23
16 Hemaka Gunasekera 23
7 Danusha Muhandiramge 23
75 Therunee Karunaratne 23
30 R Nikesh 23
23 Harindu Wijayanayake 23

Table 2.5 Players with the highest Buchholz scores.

There are 12 players with a Buchholz score of at least 23 and one of them Therunee ended up in 75th place. For comparison purposes the player at the 76th position had a score of 14. Among the prize winners only four players had a score of 24 or more. Let's also take a look at the table for top rated players.

 

Rank name Buchholz score
1 Migara Jayasinghe 19
2 Aabid Ismail 25
3 Janul De Silva 23
4 Hasindu Arumapperuma 20
5 Janidu Karunarathne 21
6 Thavalakshman Yoganathan 21
7 Hayati Rassool 22
8 Radinka Dissanayake 24
9 Thirandi De Silva 22
10 Vignesh Pirapaharan 17

Migara and Rehan suffered losses in the third and fourth rounds so their Buchholz scores are only 19 but Hasindu who suffered a loss in the second round and another loss in the third round still has a Buchholz score of 20 as shown in the table above.

Except for Vignesh most of the other rated players have Buchholz scores that compare favourably with the prize winners. Except that many of these players didn't actually win any prizes!

Though Janul lost the second game he obviously didn’t have an easy time of it from that point onwards, as can be seen by the fact that his opponents racked up 23 wins among themselves. In other words, Janul kept running into tough opponents who were all lurking in the second half of the field! Janul's Buchholz score is higher than that of five of the prize winners.

Thus in conclusion, the pairing system (King Of The Hill) has not treated all players equally as shown by the huge disparity in Buchholz scores. But I am not going to rest my case! There is a lot more evidence coming up!

July 5, 2016, 5:17 a.m. » Tagged: chess , scrabble , buchholz , KOTH

Fairness of KOTH pairing.

This is part three, in a series of posts on King Of The Hill pairings. The analysis is carried out using the results of the Age Group Scrabble Championship 2015.. The tables on this post can be recreated by web scraping the contents of the above link.

The fairness of the draw can be calculated based on three different criteria as described on the official website of tsh, the most widely used pairing software in Scrabble and the one used for this tournament

Average Opponent Rank

The rank of a player is the position he occupied at the end of the tournament. Therefore Average Opponent Rank is the sum of the rank of each opponent faced by a given player divided by the number of rounds.

As an example Qays played against Lochana, Zahra, Rizvi, Ashvin, Duilini and Radinka their ranks were 34, 42, 11, 58, 24 and 5 which add to 174 and averages to 29. Players with lower Average Opponent Rank (AOR) have faced stiffer competition.

The table below outlines the average opponent rank sorted by AOR in ascending order. Lower AOR means the player's opponents secured better placings.

Position Name Average Opponent Rank Number of prize winning opponents
1 Aabid Ismail 16.67 1
4 Hansi Weerasooriya 22.67 3
41 Lalinka De Silva 24.50 3
18 Nikesh Moorthy 27.17 2
75 Therunee Karunaratne 27.67 2
20 Janul De Silva 28.00 2
5 Radinka Dissanayake 28.17 2
25 Qays Sangani 29.00 1
7 Danusha Muhandiramge 31.50 2
37 Thirandi De Silva 32.17 1

Table 1: Best Average Opponent Ranks

Only four of the prize winners figure in this list but the champion has the best AOR of all players. 75th placed Therunee and 41st placed Lalinka have better AORs than eight of the prize winners. Lalinka and Hansi were the only players to meet three prize winners. Lalinka managed to win two of those games (vs Migara and Yenuli) yet her own position is 41st.

Compare the above with the next table which examines the AOR figures for top rated players.

Position Rating Name AOR Prize Winning opponents
1 1134 Aabid Ismail 16.67 1
20 1054 Janul De Silva 28.00 2
5 897 Radinka Dissanayake 28.17 2
37 847 Thirandi De Silva 32.17 1
3 924 Hayati Rassool 32.50 2
38 964 Janidu Karunarathne 35.33 2
21 938 Thavalakshman Yoganathan 37.00 2
2 1148 Migara Jayasinghe 42.83 0
13 1053 Hasindu Arumapperuma 44.83 0
15 840 Vignesh Pirapaharan 49.17 0

Though top rated players are the most likely contenders for the title as well as the remaining nine prizes, and the stated objective of KOTH is to pair contenders, three of the top rated players haven’t been paired against prize winners at all. These three happen to be two former under 18 champions and a two time under 13 champion!!

Thus the Average Opponent Rank shows that there is considerable disparity in the manner in which rated players were paired and KOTH has failed to fulfil its objectives. Based on Average Opponent Rank, King of the Hill has been anything but fair

(Note: Average opponent rank table for Prize winners is presented in section 2.3)

July 4, 2016, 7:15 a.m. » Tagged: KOTH , Pairing , Scrabble

Analysis of Pairing Systems : Definition of Terms.

This the continuation of a series of articles that analyses the King of The Hill pairing system. Before we get to the analysis proper, we need to get some definitions out of the way.

KOTH

The King Of The Hill Pairing system, which is a strict 1 plays 2, 3 plays 4 .. (2n-1) plays 2n pairing system.

Swiss

A reference to the Swiss Pairing system which is almost always the format that is used for tournaments where the number of players exceeds the number of rounds. It finds widespread usage in scrabble and some other sports too

Prize winner

It is customary in Scrabble tournaments to award prizes and/or certificates to players who finish with in the top 10 at the end of the tournament. These persons will be called prize winners

Rated Player

Age Group Scrabble Championship is the only tournament conducted by the SLSL that specifically targets beginners. As a result it attracts a disproportionate number of novices. There is a huge gap between these players and rated players. For the purpose of this discussion a rated player is a person whose name appears on the SLSL ratings list. Further the players with the 10 highest ratings will be called the top rated players.

Contender

A person who is either a prize winner or a top rated player

Strong Player

Players who have scored an average of 365 points per game will be defined as a strong player. Only the top quartile had an average greater than 365 .

June 30, 2016, 4:44 a.m. » Tagged: KOTH , Scrabble , Pairing

King Of The Hill Pairing System - An Analysis.

The King Of The Hill pairing system is one where the “Top ranked plays second, third plays fourth, etc. Usually used for the last round(s) of the tournament” (NSA 2008, p53) “KOTH pairings are often used in the final round(s) of a tournament to ensure that contenders face each other” (Chew, John 2015)

This is the start of a series of posts on the King Of The Hill Pairing system based on the results of the results of the Sri Lanka Age Group Scrabble Championship 2015. Before you run of thinking Sri Lanka, Bah! Consider that this was a tournament in which three of the top 10 finishers at the World Youth Scrabble Championship took part. In fact Sri Lanka is a country that has produced many a strong performer at international level

KOTH was used for pairing 5 out of 6 rounds at the All Island Age Group Scrabble Championships in 2015 and 2014. In previous years when KOTH was only used it was only in the last round of the tournament. The format is also used in 6 out of 18 rounds at the Scrabbler of the Year tournaments held four times a year.

“It is estimated that as many as 75-80% of all tournaments use the KOTH for the last one or two rounds, including many R-R events. But, almost none use KOTH in the early rounds.” (NSA 2008, p53)

 

References:

Chew, John. 2015. 'Tsh: Commands'. Poslarchive.Com. http://www.poslarchive.com/tsh/doc/commands.html#pairing.

>2008. New York: National Scrabble Association.

June 28, 2016, 5:41 a.m. » Tagged: scrabble , pairing , KOTH

A dilapidated Scrabble Power House

What does the 2003 UK Scrabble Champion and the 2005 Australian Scrabble Champion have in common?  They are both of Sri Lankan origin (Harshan and Naveen).  For good measure Naveen Fernando won the Australian title again in 2010.  Before that he had won the Gulf Championship. Ancient history you say? Then consider that Sri Lanka was the only country to have three players in the top 10 at the 2015 World Youth Scrabble Championship. One of them won the Under 13 title and Sri Lanka also won the team title. 

When the incomparable Bobby Fischer emerged as the World Chess Champion, it was likened to grass growing through the cracks on a busy sidewalk. Then Sri Lankan Scrabblers shining on the world stage is the equivalent of a tree sprouting on the middle or a busy elevated highway. Missaka Warusawithana played at the World Championship as a 15 year old. Mind you this is the senior event, not World Youth Championship. Missaka was 15th that year. A few years later Naveen bettered it moved it from 15 to 13, 11, 8 and 3 (though he wasn't representing Sri Lanka at the time). There it has languished for a long time.

Sri Lanka Scrabble team for WYSC 2016

Photo by Manjula de Silva

You would think that a tiny Indian ocean country where English is spoken as the first language by less than 50,000 persons might not be able to achieve these sort of performances but they have. Sri Lankans have done even better at WYSC consistently having one or two players in the top 10 and having three in 2015. So why hasn't Sri Lanka had a top 10 finisher in the WSC since 2005? When you read the next few posts you will find out!

If Sri Lankan eventually wins the World Scrabble Championship, it would be inspite of the Sri Lanka Scrabble League rather than because of it

foot note: This article isn't exactly tech geek but it's a prelude to a series of posts that will be geek (even though not exactly tech geek)

June 26, 2016, 1:40 p.m. » Tagged: scrabble

Sending Google Cloud Notifications with python and XMPP

Have you seen the sample code in Android Docs? for sending GCMs with python an XMPP? Sure you have. You probably know that import xmpp here refers to the xmpppy library.

xmpppy hasn't been updated for at least a decade. Heck there's even an ICQ number for the author on the project home page. And yes, the project is hosted on Source Forge!! Never mind all that. the real problem is that xmpppy doesn't play very well with modern versions of openssl and god knows what other incompatibilities there are.

  File "push/ccs.py", line 319, in 
    ts.setup_xmpp()
  File "/..../push/base.py", line 140, in setup_xmpp
    self.xmpp_client.connect(server=(settings.CLOUD_MESSENGER['SERVER'],self.params['PORT']), secure=1, use_srv=False)
  File "/usr/local/virtualenv-django18/lib/python2.7/site-packages/xmpp/client.py", line 200, in connect
    if not CommonClient.connect(self,server,proxy,secure,use_srv) or secure<>None and not secure: return self.connected
  File "/usr/local/virtualenv-django18/lib/python2.7/site-packages/xmpp/client.py", line 178, in connect
    transports.TLS().PlugIn(self,now=1)
  File "/usr/local/virtualenv-django18/lib/python2.7/site-packages/xmpp/transports.py", line 281, in PlugIn
    if now: return self._startSSL()
  File "/usr/local/virtualenv-django18/lib/python2.7/site-packages/xmpp/transports.py", line 316, in _startSSL
    tcpsock._sslIssuer = tcpsock._sslObj.issuer()
AttributeError: '_ssl._SSLSocket' object has no attribute 'issuer

So you might shop around for another python xmpp library, there are quite a few. But the majority of these libraries seem to be forks of xmppy itself but these forks have merely copied the files and just left them there. Twisted and SleekXMPP seem to be the only libraries and are being maintained. So let's see if we can modify google's sample to work with either of these two and in the long run that really is what you should do. But for now there is one, just one fork (https://github.com/ArchipelProject/xmpppy/) that's only two years behind. You will have to install it manually though.

 

source /usr/local/myvirtualenv/bin/activate
pip uninstall xmpppy
wget https://github.com/ArchipelProject/xmpppy/archive/master.zip
unzip master.zip
cd xmpppy-master
python setup.py build
python setup.py install

Hopefully the next breaking change to openssl is a long way away.

May 2, 2016, 1:31 p.m. » Tagged: python , android , xmpp , xmpppy , gcm , ccs

Updrading Postgresql 9.3 to 9.5 with Postgis.

Upgrading Postgresql minor versions is quite straght forward. You can do so with the pg_upgrade tool that's part of the binaries. But there is always a but, this is true only when Postgis extensions are not being used.

The first error in the upgrade process had nothing to do with postgis though. It was due to the wrong encoding. 

encodings for database "postgres" do not match:  old "UTF8", new "SQL_ASCII"

This was a problem that could be easily overcome by dropping the database and recreating it with the proper encoding.

/usr/pgsql-9.5/bin/initdb -E 'UTF-8' 9.5

But you will run into trouble once again with libraries

Checking for presence of required libraries                 fatal

Your installation references loadable libraries that are missing from the new installation.  You can add these libraries to the new installation, or remove the functions using them from the old installation.  A list of problem libraries is in the file:
    loadable_libraries.txt

Failure, exiting

The contents of the loadable_libraries.txt file is as follows:

Could not load library "$libdir/rtpostgis-2.1"
ERROR:  could not access file "$libdir/rtpostgis-2.1": No such file or directory

 

Could not load library "$libdir/repmgr_funcs"
ERROR:  could not access file "$libdir/repmgr_funcs": No such file or directory

Could not load library "$libdir/tablefunc"
ERROR:  could not access file "$libdir/tablefunc": No such file or directory

The requirement of tablefunc.so can be fullfilled by installing  postgresql95-contrib.x86_64 while repmgr_funcs.so is obviously from  repmgr95.x86_64

The postgis requirement isn't so easy.  What comes with postgresl-9.5 on centos is postgis-2.2 whereas 9.3 had 2.1. You have two choices; You can manually install 2.1 on the new server or you can do a hard upgrade. The first option is a little bit more work (you will invariably have to install the -devel versions of various libraries in order to be able to compile postgis from source) but it will minimize downtime. Hard upgrade involves dumping the entire database manually and importing it back.

So I decided to take the manual compile path only to find that it cannot be crossed.  When tying out the compile Postgis option it was discovered that the --prefix parameter to configure was broken. So the only choice for me is to do a hard upgrade. So it's a hard upgrade, and the hard upgrade is very hard because there are all kinds of inexplicable errors showing up. Here are a few examples:

ERROR:  invalid byte sequence for encoding "UTF8": 0xe0 0xaf 0x2e
ERROR:  value too long for type character varying(128)
ERROR:  insert or update on table "announcements_announcement" violates foreign key constraint "announcements_announcement_creator_id_fkey"
DETAIL:  Key (creator_id)=(1) is not present in table "auth_user". 

The data that doesn't fit into 128 chars now obviously fitted in the old version, this and the encoding errors indicate that some form of charset conversion is happening. More pressing is the foreign key errors. These users definitely exist on the roadlk database.

Update May 01: Finally solved this by not using postgis_restore and importing the database direct with pg_restore!

April 29, 2016, 6:21 a.m. » Tagged: postgis , postgresql , pg_dump