Motherboard or logic board or whatever

Yesterday, my father’s mac refused to power up. He had shut it down normally the previous day. I never shut down mine for fear that it wouldn’t start up again and in his case it didn’t. So he tried all the usually remedies like resetting the SMC etc etc but when none of them worked, I was asked to take a look. I immediately suspected a broken cable like happened with mine. After taking the damned thing apart (there are about 20 screws to undo), I found the cable was ok. Anyway to be safe, I tested with another cable only to find that the computer doesn’t start up.

I bet you are itching to ask if I tried a different battery or a different power supply. You bet I did. So the only logical conclusion now is that the mother board (Illogically named the Logic board by Apple is toasted). And guess what? this is the second time that the mother board on this computer has got toasted! The first time around it was still under warranty but that is no longer the case.

There are plenty of logic boards for sale on Ebay. But they are of no use whatsoever.  That’s because Apple being as stupid as they are, have thought fit to solder the CPU onto the mother board. Yes you heard that right, in Apple notebooks you cannot upgrade the CPU unless you are an electronics engineer. Even then most electronics engineers may not have the equipment needed to replace the CPU or the motherboard at home.

The best thing to do with this macbook seems to be to cannibalize it for parts, any of the macbooks that belong to other family members or close friends are sure to develop some fault before long.

Pingdom test while cloud front is enabled.

Static content hosted on S3

Without and content distribution

All the images can be clicked on to be zoomed. As you can see, content distribution really does have a positive impact when the site’s visitors are from the United States.

It is debatable whether using a CDN will speed up your site. In fact, I just discovered that using Cloudfront actually appears to be slower than just using S3 or indeed delivering static content from an Elastic Compute Cloud server.

Page load times all content on an EC2 server

Static content from Amazon Cloud Front

http://raditha.com/blog/images/aws/s3hosting1.png

The reason for this apparent slowdown is that CDNs are generally geared to send you route you to the closest geographic location. The closest geographic location may not always be the closest network path. Even they are the same, it’s still possible that there is less bandwidth in the fiber optic cables for that location, as a result if lots of surfers are connecting to that location, you might find things a little bit slow.

Users from Sri Lanka are directed to the CloudFront servers in Singapore. Even though Singapore is much closer to me than the United States, most traffic from Sri Lanka ends up heading Westwards towards US than eastwards towards Singapore. The same applies for most parts of South Asia including India.  Most ISPs in different parts of the world are better equipped to route traffic towards the US than towards other countries for the simple reason that historically a vast majority of web servers were located in the US. In fact the business model of small web hosting companies is to rent out racks full of servers from large us hosting companies and to sublet them to their customers. Which means, whenever you connect to the website of the office across the street from yours, you are actually accessing a server in the United States.

Page size with the @anywhere javascript included

Page load statistics without the @anywhere code

Sounds bad huh? well maybe, but not even half as bad as Facebook’s like button. It adds about 70Kb to the page size and doesn’t do anything other than display the like button. In the case of @anywhere, it’s got a load of features it’s just that we are not making use of them.

Now they are blaming us for their own faults!

Those who were patient enough to stare at the semi frozen browser window were rewarded with the above dialog. That’s right nothing wrong at their end, we have a firewall issue. Except that in my firewall all outgoing ports are open. Which is pretty much the same for quite a few others who have commented on it.

The truth is that this is not a firewall issue. It’s not a scalability issue. S3 can and will scale very well, It’s just that someone has written buggy JavaScript and doesn’t want to admit it. In the meantime many users will jump to the wrong conclusion and get off the cloud.

EncFS Password:

This can be easily fixed by typing:

chmod +x /usr/bin/fusermount

encfs is pretty useful because you don’t need to start of by creating a large single file to hold all your other files. Simply create two folders on your hard drive, one to contain the encrypted files and the other as the mount point where the decrypted files show up  (but only if you provide the right password).  Today, I tried to setup such a system on a Centos 5.4 box (well it wasn’t really a centos 5.4 box it was virtualize, but never mind) and ran into an error straight away:

Oh by the way, encfs is not found on the Centos Base repos. You need to get it from EPEL. There were exactly four results in Google for this error and none of them seemed likely to be the solution. Pretty annoying, because I had in fact set this up before on a 32 bit CentOS machine before (this time, it’s a 64 bit machine). Rather than compiling from sources, which I am loath to doing, I thought to have a look at some of the other options available.

1) PhoneBook

Phonebook also appears to support steganography content and sounds similiar to truecrypt but it’s a bit of overkill for me and it hasn’t been maintained since 2004.

2) CryptoFS

Sadly the site seems to have been taken offline.

3) MinforFs

‘MinorFs is a userspace filesystem (fuse) for Linux providing private storage to pseudo persistent processes. This allows programs that are run by a user to keep some data safe from all potential malware that runs with all this users privileges.’ – once again it appears to be overkill

4) MetFS

‘MetFS is a filesystem software in userspace that is encrypted, dynamic sized (increase when new data added, decrease when data removed), single file when it’s unmounted’ – oh well, if the file grows and shrinks dynamically I can live with that. The only catch is that it doesn’t have an RPM. It’s released under an open source license but the download page says, it may only be used for educational purposes. Explain that one to me.

Sadly MetFS wouldn’t compile either:

metfs.c: In function ‘metfs_extract_file’:

metfs.c:802: error: ‘TAR’ undeclared (first use in this function)

metfs.c:802: error: ‘t’ undeclared (first use in this function)

metfs.c:820: warning: implicit declaration of function ‘tar_extract_all’

metfs.c: In function ‘main’:

metfs.c:910: warning: implicit declaration of function ‘fuse_main’

make: *** [all] Error 1

This is the old UI saved here for posterity.

The old UI, saved here for posterity.

Oh and by the way, this is totally untested on Windows. I will be absolutely delighted to hear that this doesn’t work on IE 6, 7 or even 8. If it’s indeed broken, I have absolutely no plan of fixing it.

For example the Amazon SimpleDB extension resides in a folder named ~/.mozilla/firefox/2f1ex7k8.default/extensions/sdbizo@bizo.com/ and the relevent section of the install.rdf file looks

     <em:targetApplication>
        <Description>
          <em:id>{ec8030f7-c20a-464f-9b0e-13a3a9e97384}</em:id>
          <em:minVersion>1.5</em:minVersion>
          <em:maxVersion>3.5.*</em:maxVersion>
        </Description>
     </em:targetApplication>

Grid created using the jQuery tablesorter plugin

All in all, it took me a full day to figure this out. I know it’s more cool to say I did it in half and hour but the truth is that I couldn’t. Well I could have done so if I had used jQuery and one of it’s plugins. It probably wouldn’t have taken me even 5 minutes in fact because I’m quire familiar with jQuery as well as it’s TableSorter plugin. Much of the time taken by this task was spent in reading the documentation and the rest, hitting the reload button on firefox and chrome.

A YUI DataTable.

The day spent was worth it because the UI of the YUI DataTable looks pretty good and there are many more configuration options than available with jQuery plugins. But it’s not going to beat ExtJS. While I was laboring with the YUI datatable, I tweeted about it and Fahim Farook suggested that I look at ExtJS instead and I did, en passant. After a half hour diversion, I had a working  data grid in Exts (there you are I am a cool geek). It looked much better than what even YUI produced. But there is a catch, the ExtJS library is more than 2MB in size.

ExtJS Data Grid

A 2MB library for a web page that usually has less than 50kb of content would rule out ExtJS for most situations. However, you can ‘trim the fat’ and create a customized version of ExtJS. It would contain the bare minimum code needed to support the set of function that you are trying to make use of. However,the customized library I downloaded was still over 200Kb in size and both Firefox  and Chrome immediately started complaining about undefined objects. An indication that the library file is incomplete.

So to cut a long story short the next time i need a data grid, what would I use? It depends. If the software being designed is an Air app, I would definitely insist on ExtJS. If download time and or development time really mattered it would be jQuery, else it would be YUI.

related: jQuery TableSorter and XSL Transformation

All this started when I decided to write a small script to analyze the content of a database using python rather than my usual first choice PHP. But I haven’t really coded anything at all with Python since I learned it nearly two years ago. So I thought, why not use Python, well the time I spent researching the cause of this problem could have been utilized to write the darned thing in PHP, Perl or Java or heck even C.

The last time, I did anything with Python, I ran the scripts either from the command line or using mod_python. I can’t remember the editor or IDE I was using at the time but I pretty sure it wasn’t Netbeans. The problem pops up only when you try to run the script from inside Netbeans itself. The very same script can be executed from the shell using the python interpreter (see, what did I tell you about mysqldb being installed?)

Searching for Import Error: No module named MySQLdb netbeans came up with a blank. Digging a bit deeper, I realized that Netbeans using the jython intepreter rather than the default python 2.6 interpreter that’s available on Fedora. That in turn enabled me to dig up a post on Stackoverflow which explained that MySQLdb cannot be used with jython. You need to use something called zxJDBC instead. Damned if I am going to bother with that. Instead I just changed the python platform and solved it easily.

To change the platform, Right Click on the projects root node (the project name in the ‘projects tab’), when the dialog pops up select the ‘python’ item from the tree on the left and then change the platform.

Changing the Python Platform (click to enlarge)

If by some chance, the installed python interpreter does not show up here, click the manage button and on the new dialog, choose the ‘auto-detect’ button. If today isn’t your lucky day, netbeans will refuse to find your default python interpreter. But that’s ok, just run to the command line do a `which python` and then use the path to the interpreter as reported by which to add a ‘New’ platform.

Adding a new python platform (click to enlarge)

Hopefully you might find my older post ImportError: No module named dom.minidom also to be usefull

Notes: With amazon ec2, data transfer in is free till 30th June 2010.

Amazon and go grid provide  detailed specifications about the CPU power that you get.

‘One EC2 Compute Unit (ECU) provides the equivalent CPU capacity of a 1.0-1.2 GHz 2007 Opteron or 2007 Xeon processor.’ (Amazon)

‘GoGrid is built upon a Grid of servers (nodes), managed by a hardware virtualization layer (Xen-based hypervisor) which allows several guest operating systems (virtual servers) to be executed on the same computer hardware at the same time. Our nodes are custom built multi-processor, Intel-based computers. In order for GoGrid to guarantee a minimum CPU to a virtual server, we will always maintain a ratio of 1 Xeon core (equivalent to a P4 2.0 chip) to 4 GB of RAM across our entire grid.’ (Go Grid)

Rackspace is less clear about their CPU speeds, the speed in the chart above is derived from the information given in their FAQ. Opsource have not mentioned the speed of their CPU at all. The prices are calculated assuming $0.04/CPU + 0.025/GB RAM + 0.003 for 10GB storage

As you can see, it’s pretty hard to compare these prices. It’s much more difficult than comparing apples and oranges. Just to make the waters a little murkier, all these vendors have pre-paid packages which allow you to obtain significant discounts by agreeing to purchase a certain number of CPU hours per month. If you want to run windows, you need to pay extra.

One fact that does stand out: Rackspace has the lowest charges but their CPU is only 1/4 the power of the Amazon version. So if you were to multiply the rackspace price by 4, you would get a figure of USD 0.06 per hour for 1.6GHz/1.0GHz/40GB . Amazons price for a 1.6GHz/1.6GB/160GB is 0.085 you decide whether the extra 2 cents for 600MB of memory is worth the extra 2.5 cents.

We will revisit this topic later to compare the pre paid plans made available by the different vendors and also take a look at cloud storage.

References:

Cloud Servers Pricing by Rackspace Cloud Computing & Hosting [Online]. Available from : http://www.rackspacecloud.com/cloud_hosting_products/servers/pricing [Accessed : 17, April, 2010].

Amazon.com, 2010. Amazon Elastic Compute Cloud (Amazon EC2) [Online]. Available from : http://aws.amazon.com/ec2/#pricing [Accessed : 17, April, 2010].

GoGrid Cloud Hosting : Frequently Asked Questions [Online]. Available from : http://www.gogrid.com/support/faqs.php [Accessed : 17, April, 2010]

OpSource Cloud – Pricing information [Online]. Available from : http://www.opsourcecloud.net/getstarted/pricing/ [Accessed : 17, April, 2010].

References were generated with deadlinedue, a free service by Rad Inks that’s hosted on an amazon EC2 instance.

PHP Warning: SimpleDB::batchPutAttributes(): ServiceUnavailable: Service AmazonSimpleDB is currently unavailable. Please try again later
in /var/www/****/SimpleDB/sdb.php on line 1021
Items FAILED
ErrorCode: ServiceUnavailable

However at about the time that twitter went offline, we noticed almost every other insert producing an error. When that happens we have coded our script to sleep a few seconds and retry the insert. The second attempt was completed successfully almost everytime. I went to tweet about it to find that twitter has gone offline.

AFAIK, twitter has not publicly stated what their database platform is. They might do so at chirp so for the moment I am going to hazard a guess that they are using SimpleDB to store some of their data. All their static content is served out of Amazon S3 so this is not a wild guess (I hope). I am now going to leap to the conclusion that the two errors are related and make a fool of myself.

• Annyoing memory leaks. Of course they will say ‘there are no memory leaks’, but if you leave FF running for a day or two, you find that it’s using around 2GB of memory. The longer you keep if running and the more tabs that you have open the worse it gets. Not only does this slow down firefox itself, but it slows down the whole computer as well.
• High CPU Usage I don’t know what causes it but there are times when FF drinks up 80% or even more of available CPU capacity to do nothing. There is no reason for a browser to need more than 5% of a Core 2 duo processor’s capacity to process a web page and that too will be a spike that disappears after the page has been rendered. But what really happens is that processor usage never goes down. As with the high memory usage, this slows down the whole computer and also increases power consumption while reducing the lifetime of the processor.
• GTK-Gnash I hate to knock open source software but gtk-gnash is just awful

Firefox might still come out of retirement if I cannot figure out how to set a master password for saved passwords. Surely google doesn’t expect me to store my passwords where anyone can see them? I also need to find replacement for all the firefox extension that i’ve been using.

I plugged in the power cable and tried again with the same result. Next I let the battery charge some more and tried again. The last attempt was with a borrowed battery but the machine still refused to wake up. Time to take off the screws, there are many screws, including a few loose ones belonging to Jobs. I removed the top panel, unplugged the ribbon cable connecting the top panel to the mother board, put it back in and tried again. Guess what? nothing happens. Then I unplugged the machine again and shorted the pins for the power button connector with the tip of a ball point pen. Viola the machine started up! I shut it down again by connecting the mouse (there is no other way), connected the cable and tried again. Nothing happens. Thought to remove the cable and spray some contact cleaner, but the cable came apart.

I looked around for a replacement cable but it’s not available. The local apple agents would happily sell me the top panel with the cable included and I would happily by from them if they had the darned thing in stock, which they don’t. Now I am waiting for them to get one down, let’s see how it goes.

Linux gives you two options for creating file systems; at the kernel level or in the user space.  e2compr is a kernel patch that supports compressing an ext2 file system while the user space tools are based on fuse. Using kernel space drivers could be messy if the file system chosen like e2compr is always playing catchup with the kernel version. The only compressed file systems supported in the mainstream kernel are jffs2 and squashfs.  The former is for flash drives and the latter is ready only. Though JFS is available for linux, it’s compression mode is not.  While both KDE’s Dolphin and Gnome’s Nautilus support mounting a gzipped tar archive of a zip file as a node in the file system in read/write mode, they are not suitable for web apps. So fuse seems  the way to go.

There are eight file systems listed on the Fuse compressed file system page. Four are read only, two are abandoned, two more are heading down that route. One is in it’s early development stage. The remainder was FuseCompress.

This one is real easy to install just do ‘yum install fusecompress’ setup is even easier.

fusecompress  /mnt/compressed   /mnt/uncompressed

Copying over a set of 30,000 files (a mix of text and binary data) with a total weight of 85MB took 36 seconds. The same operation when repeated, completed in 20 seconds because of file system caching.  When the files were copied without compression the operation completed in 1 second, so much for my assumption that a compressed system would be faster.

The sphinx index isn’t really a mySQL database, so you cannot join it to your other tables. That’s where the Sphinx storage engine comes into the picture. It allows you to create a table in your existing databases to represent the Sphinx index. That would allow you to actually use the sphinx data in a join or sub query. You might ask why goto all this trouble? Why not just use full text? Full text search has it’s limitations, most notable being how slow it is when you have large table. A full text search on a table with 8 million rows (which is about as much as we have on Pitupasa.com at the moment) will take 10-15 seconds to complete. Sphinx on the other hand returns the results in a fraction of a second.

At first glance, it seems that enabling the sphinx storage engine is kidstuff. Just download the sphinx and mysql source tarballs, copy a set of files from sphinx into the mysql source tree, do a BUILD/autorun.sh followed by ./configure –with-plugins=sphinx  (see full instructions at http://www.sphinxsearch.com/docs/current.html#sphinxse-mysql50 ). Except that it doesn’t work.

configure error: unknown plugin: sphinx

Yes, I know what I am doing. No there weren’t any typos. I am not the first one to run into it. The others who ran into this problem don’t seem to have found the solution either. Then I ran into another set of instructions at http://www.sphinxsearch.com/wiki/doku.php?id=sphinx_sphinxse_on_rhel there the instruction is to use ‘-with-plugin’ but shouldn’t that be ‘–with-plugins’ ? since I have already tried ‘–with-plugins’ I tried ‘–with-plugin’ (note the s). I tried ‘–with-plugin’ and the compiler didn’t complain about the missing sphinx plugin as above. But still the sources in the sphinx folder were not getting built. By the way, ‘configure –help’ shows the correct parameter to be –with-plugins. But then none of these things will result in the sphinx storage engine being built.

The docs refer to a prepatched mysql source tarball, unfortunatley it appears to have been taken off line. There is a patch for mysql 5.0.x but I don’t want to use that because I want to be able to partition the table and indexes (a feature that is not available in mysql 5.0.x). Oh well looks like it’s time to look at mnogosearch.

It is not possible to assign complex types to nodes in /var/www/deadline/ISBN.php on line 480, referer: http://deadlinedue.com/index.php?lookf
or=http%3A%2F%2Fraditha.com&find=find
*** glibc detected *** /usr/sbin/httpd: double free or corruption (fasttop): 0×81cdfc78 ***

PHP is pretty old now, it’s not something that you expect to segfault, so initially I thought the culprit would be APC – Advanced PHP Cache. These PHP accelerators or caches are known to crash every once in a while. That can be easily fixed by removing the accelerator. In this case we could afford to do so since we had upgraded the server very recently and optimized the code which resulted in a speed boost and reduced memory usage. I was barking up the wrong tree,  disabling APC didn’t do any good. So guess it’s time to upgrade PHP?

Now you might ask, shouldn’t we have used the same version of PHP in our development and production servers to ensure that this sort of thing didn’t happen? Right you are but who expects PHP of all things to crash like this. It’s really rediculous that deadlinedue is hosted in the cloud. So we could easily have made a snapshot of it and started another server in less than 5 minute. We could have tested with that guinea pig and then gone live, but no, I was too cocky.

This is Centos 5.4 and there are no RPMs available for PHP 5.3.1, So it was time to compile from scratch. Which usually means you need to run ./configure about half a dozen times, fixing each of the missing dependencies it reports until it completes without error. Fortunately there is plenty of bandwidth on Amazon EC2 and the servers are blazing fast. Most deps can be installed in less time than it takes to type out the yum install command.

After all this the problem still wasn’t solved, a lot of modules we needed like JSON , DOM and heck even Mysqli were not getting added!

PHP Warning:  PHP Startup: apc: Unable to initialize module\nModule compiled with module API=20050922\nPHP    compiled with module API=20090626\nThese optio
ns need to match\n in Unknown on line 0
PHP Warning:  PHP Startup: dbase: Unable to initialize module\nModule compiled with module API=20050922\nPHP    compiled with module API=20090626\nThese opt
ions need to match\n in Unknown on line 0
PHP Warning:  PHP Startup: dom: Unable to initialize module\nModule compiled with module API=20050922\nPHP    compiled with module API=20090626\nThese optio
ns need to match\n in Unknown on line 0
PHP Warning:  PHP Startup: json: Unable to initialize module\nModule compiled with module API=20050922\nPHP    compiled with module API=20090626\nThese opti
ons need to match\n in Unknown on line 0

Given enough time, I can track down their causes and fix these errors, but time was exactly what I was not having on my hands. So tried looking around to find out if there are any RPMs available from third party repositories that would update PHP to version 5.3.1. Fortunately there was.  The repo is called Web Tactic. Update using the RPMs took just seconds and good news, no more segmentation faults. The bad news is that APC is no longer available.  I tried  I tried ‘yum install php-pecl-apc’  without success. Then tried ‘pecl install apc’ and got the following error:

/tmp/pear/temp/APC/php_apc.c: In function ‘zif_apc_compile_file’:
/tmp/pear/temp/APC/php_apc.c:881: warning: unused variable ‘eg_class_table’
/tmp/pear/temp/APC/php_apc.c:881: warning: unused variable ‘eg_function_table’
/tmp/pear/temp/APC/php_apc.c: At top level:
/tmp/pear/temp/APC/php_apc.c:959: error: duplicate ’static’
make: *** [php_apc.lo] Error 1
ERROR: `make’ failed

Oh well, can live without APC for the moment.

One of the data items returned by the get list id method includes the number of members in your list. The get list members method returns 20 members at a time and you need to call the it multiple times using different cursor locations to retrieve the complete membership. When you count them and compare against the number returned by the get list id method, you will find that they do not match. Your own account is not part of the dataset that is returned.

By the time you are reading this post, the list might have changed. So I have saved a copy of the XML here. On the other hand when you access the list through twitter.com, you can see that I am included in my own list. In fact my screen name (e4c5) appears twice!

The work around I suppose is to not rely on the members count returned by the get list id method.

Unfortunately I didn’t meet with a lot of success the Fedora 11 AMI refused to boot but trouble started even before, first with yum reported numerous depsolving errors. I tried to sort them out by using the –skip-broken flag to no avail.

Total size: 252 M
Is this ok [y/N]: y
Downloading Packages:
Running rpm_check_debug
ERROR with rpm_check_debug vs depsolve:
libcrypto.so.7 is needed by (installed) httpd-tools-2.2.14-1.fc10.i386
libssl.so.7 is needed by (installed) httpd-tools-2.2.14-1.fc10.i386
Complete!
(1, [u'Please report this error in http://yum.baseurl.org/report'])

I worked around this by uninstalling the complaining RPM (yum remove httpd-tools-2.2.14-1.fc10.i386) and putting it back after the upgrade had completed. Nevertheless the machine reused to boot up. I tried to do this as a two step process, stopping at F10, booting up that image and then upgrading to F11. Unfortunately that didn’t quite work out either. The terminal displayed by the Amazon Management Console remains blank without anything ever showing up on it. Then I tried changing the RAM Disk and Kernel,  tried  using aki-20c12649 as the kernel and ari-21c12648 as the ramdisk with the same result.

In the end I gave up on this temporarily and used a third party Centos 5.4 image (uploaded by RightScale) for the task at hand. Going to come back to this problem later. Want to try installing a new image on the loopback device.

So what doesn’t work? The Adobe AIR debugger. It gets stuck at ‘Launching New_Configuration’ and this dialog pops up without the air app starting up.

Sounds like a socket error but this is running on localhost and all my ports are open to the local machine. As usual when I run into a problem there is nothing on google and there is nothing on the log file. At this stage I would gladly switch to Netbeans. I use for almost everything other than air. Unfortunatley I never figured out how to use the netbeans debugger with AIR. Couldn’t find any howto either.

Next I came across a reference to a third party debugger that you can use with any AIR app, regardless of the IDE that you use. It looks like a great project. Unfortunately the developers forget to create some documentation. There isn’t even a getting started guide.

I am on holiday so I am doing this on my Mac the only options available are to go back to version 1.5 or to wait till I get back home next week.

You think I have a large list of java apps running? look closely, it’s just two! One is Netbeans. Netbeans is a Sun project, so you have to expect it to behave weirdly but look at the second one. It’s Gnash; the open source replacement for the Adobe Flash player. That line is just massive. Looks like Gnash is adding all the HTTP variables into the environment. No hang on a second, Firefox is sending all the HTTP variables to Gnash through the command line. What the hell? is this how browser plugins are supposed to work? Surely this is not secure?

If you URL decode the out put of ps it’s double WTF.

Even a copyright notice is passed as a parameter!

{“profile_image_url”:”http://a1.twimg.com/profile_images/311054570/SecuritySystemPlaza.comLogo_normal.jpg”,”created_at”:”Sun, 29 Nov 2009 12:39:51 +0000″,”from_user”:”SecurityPlaza”,”to_user_id”:null,”text”:”GE SECURITY – USA TX6010011 ONLY
$78.66 AT WWW.SECURITYSYSTEMPLAZA.COM”,”id”:6168853273,”from_user_id”:28463195,”geo”:null,”iso_language_code”:”en”,”source”:”<a href="http://apiwiki.
twitter.com/" rel="nofollow">API</a>”}

Here the from_user field is given as SecurityPlaza and the from_user_id field shows a value of 28463195, if you look at http://twitter.com/users/28463195.xml you will see that the user id of 28463195 corresponds to @ballercoolkid on the other hand http://twitter.com/users/show.xml?screen_name=SecurityPlaza  gives a user id of 56297110 to <a href=”http://twitter.com/SecurityPlaza”>@SecurityPlaza</a>. So which one of these users sent out the tweet? it was @SecurityPlaza

Here is the second example:

{“profile_image_url”:”http://a1.twimg.com/profile_images/364463510/logo_normal.jpg”,”created_at”:”Sun, 29 Nov 2009 09:32:56 +0000″,”from_user”:”hindichannels”,”to_user_id”:null,”text”:”Sacked opposition party member to spearhead campaign against Genera
l Fonseka: Nov 29, Colombo: A member of the main… http://bit.ly/8OBKwS”,”id”:6166287234,”from_user_id”:32271894,”geo”:null,”iso_language_code”:”en”,”source
“:”&lt;a href=&quot;http://twitterfeed.com&quot; rel=&quot;nofollow&quot;&gt;twitterfeed&lt;/a&gt;”}

Here the user hindichannels has been associated with the userId of 32271894 where as that ID really belongs to @mrsdraz the correct id for @hindichannels is 61683443 interestingly the results are easily reproducable. Two different searches produced the same errornous output (‘fonseka‘ and ‘colombo‘)

I tried these searches between 6:00-7:00pm on Nov 29th by the time that you try these web API calls on your own, the search results would have changed dramatically. So here is a cached copy for you. By the way, the search results may have contained other inaccuracies but these are the only ones that I noticed thanks to some safety checks we have in the Pitupasa code base.

The XML search results does not have this problem due to the simple fact that the user ids are not included in it!

Users don’t need any special effort for Oauth but developers do. Writing an application that uses OAuth instead of passwords is supposed to take just a wee bit more effort, (but all seasoned programmers know what exactly that means).

One of the issues we ran into was caused by Twitter returning an ‘Incorrect signature’ error for List Delete member api calls. The signature was being generated using the TwitterOauthLib. The signature was correct. It worked fine for all other methods excepts for DELETE calls. If you passed in the same parameters to a different Oauth library, you would get the exact same signature so are we seeing a bug in the twitter API?  You might also suspect that this is because the underlying HTTP client library didn’t support the DELETE HTTP method. In this case it’s curl and the DELETE method is supported. Nevertheless I tried using an HTTP POST while sending the _method=DELETE parameter as suggested by the Twitter API documentation with out any luck.

On further investigation I found that all the parameters were being posted for both DELETE and POST requests. RFC 2616 doesn’t specify how you should pass parameters (request body or query string) for DELETE requests. I decided to change the TwitterOauth code to include the parameters as query strings (get the patch). Sure enough twitter accepted the request without the incorrect signature error.  The modified code now looks like this:

  function http($url, $method, $postfields = NULL) {

    $ci = curl_init();

    /* Curl settings */

    curl_setopt($ci, CURLOPT_CONNECTTIMEOUT, $this->connecttimeout);

    curl_setopt($ci, CURLOPT_TIMEOUT, $this->timeout);

    curl_setopt($ci, CURLOPT_RETURNTRANSFER, TRUE);

    curl_setopt($ci, CURLOPT_HTTPHEADER, array('Expect:'));

    curl_setopt($ci, CURLOPT_SSL_VERIFYPEER, $this->ssl_verifypeer);

    switch ($method) {

      case 'POST':

        curl_setopt($ci, CURLOPT_URL, $url);

        curl_setopt($ci, CURLOPT_POST, TRUE);

        if (isset($postfields)) {

          curl_setopt($ci, CURLOPT_POSTFIELDS, $postfields);

        }

        break;

      case 'DELETE':

        curl_setopt($ci, CURLOPT_CUSTOMREQUEST, 'DELETE');

        if (isset($postfields)) {

            curl_setopt($ci, CURLOPT_URL, "{$url}?{$postfields}");

        }

        else

        {

            curl_setopt($ci, CURLOPT_URL, $url);

        }

        break;

      default:

            curl_setopt($ci, CURLOPT_URL, $url);

            break;

    }

    $response = curl_exec($ci);

    $this->http_code = curl_getinfo($ci, CURLINFO_HTTP_CODE);

    $this->last_api_call = $url;

    curl_close ($ci);

    //if ($this->format === 'json' && $this->decode_json) {

    //  return json_decode($response);

    //}

    return $response;

  }

Also had to do a modification to the oAuthRequest method, which was automatically appending the twitterId to the url it’s not required (and in fact leads to an incorrect link for list methods). That means if you have been using an unpatched TwitterOauth library and have been passing in the twitterId as a parameter, you will have to change that.

Oh and btw, follow me on twitter

service fuse does not support chkconfig
error: %preun(fuse-2.7.4-3.fc11.x86_64) scriptlet failed, exit status 1

Trying to manually remove the fuse package with yum gets you nowhere fast:

Running Transaction
service fuse does not support chkconfig
Error in PREUN scriptlet in rpm package fuse
fuse-2.7.4-3.fc11.x86_64 was supposed to be removed but is not!

Using `rpm -e` produces the exact same error as package-cleanup (which is because package-cleanup was merely passing through the error message it received from rpm). Fortunately the –noscripts paramter to rpm can be used to force the uninstall.

rpm -e –noscripts fuse-2.7.4-3.fc11.x86_64 fuse-2.8.1-1.fc11.x86_64

After the rpm command was executed in this manner, package-cleanup also would run to completion with out any errors. But it has a funny idea about what completion is; there are still 433  .fc11 rpms lying about. Most of these packages are from third party repos such as RPM fusion. A yum update ought to fix it and indeed the applying 684MB of updates resulted in all but 68 of the fedora 11 rpms disappearing from the system. Some of these are old kernels/kernels headers which can be removed with `package-cleanup –oldkernels` that still leaves around 60 rpms that you might have to manually uninstall and re install. You might also right the following simple command to remove them all en mass:

yum remove `rpm -qa | grep fc11`

The only catch is that it might remove some other important packages but putting those back in might be easier than removing 60 rpms one at a time. BTW why are these rpms tagged as FC11 or FC12, with FC standing for Fedora Core when the OS itself is no longer called Fedora Core? That name hasn’t been used for ages. Now it’s just Fedora