A more meaningful way to reschedule Zyzzyva cardboxes

Playing competitive Scrabble but not using Zyzzyva cardboxes? Well you should. Using Zyzzyva cardboxes but you have an unmanagable number of quizzes coming up each day? Read on.

We are not going to talk about program's built in rescheduling feature. It's usefull if you miss your quizzes for a few days  but it's not very usefull if the number of questions grows to unmanagble levels after you have just added a large number of words to your vocabularly. As always, prevention is better than the cure and there is way to prevent or minimize it from happening; When you learn a set of new words, make sure that you do a standard quiz with them before adding them to the cardbox. That way the questions are spaced out instead of being bunched up. Even then, if you keep adding a few hundred words each day for about week such as when preparing for a tournament, you will end up with a cardbox that looks something like this.

Zyzzyva

Our strategy will be to spread these out over several days. Half the questions scheduled for the next 12 days will be pushed back by 12 days. Thus the number of questions coming up over each of the next 12 days will be halved and the number after the 12th day will increase. Twelve is a number I chose arbitarily. You can use any number but make sure it's not too large a value or the cardbox the question belongs to will need to be changed as well. How exactly do we do the change? by editing the data file. The quiz data is held in two files named ~/Zyzzyva/quiz/data/CSW12/Annagrams.db and ~/Zyzzyva/quiz/data/CSW12/Hooks.db note that ~ denotes your home folder.

Close any active instances of zyzzyva and double click on the data file. If you are nervous backup the data file before you try this out. When you click you will see something like the following pop up on your desktop.

sqlite

If you don't see something like that, it means you do not have sqlite installed on your computer. No fear, you can install sqlite quite easily on your computer. But if that feels a bit daunting you can add the sqlite exention to chrome or firefox and edit the data file in your browser. Once you are ready, you just need to copy paste the following text into the sqlite console and click run.

update questions SET next_scheduled = next_scheduled + 86400*12
    where next_scheduled is not null  and next_scheduled < strftime('%s','now') + 86400*12 
    AND next_scheduled % 2 =0

Now close sqlite3 and reopen Zyzzyva you will find that the quizzes have been rescheduled in a more meaningful manner than is possible with the built in feature.

 

June 3, 2015, 5:59 a.m. » Tagged: zyzzyva , scrabble , cardbox , sqlite

Vim and python debug

Vebugger

This is the first VIM based debug connection that I managed to setup. Vebugger will stop at break points and you can use it's commands to inspect variables. But it's somewhat tedious because it doesn't seem have an interactive shell like pdb does.  Vebugger is a frontend that can be used with many different languages. In case of  python what this actually does is connect to pdb, so I am left wondering if it would be better off to just use pdb by itself.

Having said that this project looks like it's been actively maintained and hopefully in a few months time we might see some more functionality and it will become more usefull..

Vim-debug

I tried to install it by adding Plugin 'jaredly/vim-debug' without any luck. Well the installation did happen by the :Dbg command wan't enabled. Then I uninstalled it with :PluginClean and tried a manual install with better results. In fact I could quite easily start the debugger after restarting vim just by  :Dbg . and hey presto all sorts of splits appeared with stacks, variables etc. Impressive. And just to think this software hasn't been updated for 2 years!

Now for the acid test, can it debug Django? unfortunately not. :Dbg manage.py runserver 0.0.0.0:8000 yields

Traceback (most recent call last):
  File "", line 1, in 
  File "/usr/local/virtual-django-15/lib/python2.7/site-packages/vim_debug/commands.py", line 19, in debugger_cmd
    return start(*shlex.split(plain))
TypeError: start() takes at most 1 argument (3 given) 

But I am hopefull that I will be able to make something out this for example you can just create another script than invokes manage.py with the set of arguments that you need. Clunky but usable. The code is in python and looks really good if you ignore the fact that it conflicts with many keymaps including Command-T. This conflict  can apparently be fixed by adding let g:vim_debug_disable_mappings = 1 to .vimrc . However in my case it didn't happen and I had no choice than to uninstall it.

Vdebug

It can be setup with Vundle but you need to download pydbgp from Komodo afterwards start the debug server by hitting <F5> in vim and run your script from the terminal python -S ~/bin/pydbgp -d localhost:9000 hello.py for me it didn't quite work out in the first attempt.

Traceback (most recent call last):
  File "/home/raditha/bin/pydbgp", line 71, in 
    import getopt
ImportError: No module named getopt

Turns out this is caused by the -S option for python which says don't imply 'import site' on initialization whatever that means! I am guessing it has something to do with site-packages in a virtualenv but at any rate running the script without the -S option worked! So let's leave it at that

Now the acid test. Does it work with Django? Typed python ~/bin/pydbgp -d localhost:9000 manage.py runserver and was quite pleased to see it stop at the first line inside manage.py and then hit again to goto the browser and load a url. At that stage gvim stopped responding completely. However the following message could be seen in one of the debug windows Debugger engine says it is in interactive mode,"+\ "which is not supported: closing connection"

I tracked it down to runner.py in vdebug but didn't have the time to dig any deeper. If nothing else works out I will come back here

DBGPavim

Like Vdebug, has the Komodo remote debug client listed as a dependency. Unfortunately I couldn't elicit any response from it by pressesing or any other keys. (The fact that the plugin has been activated could be seen b typing :map)

vimpdb

Nextup was vimpdb. According to the documentation this shouldn't be installed as a plugin but using pip. Then you need to add import vimpdb; vimpdb.set_trace() to your code and start the script. This is similar to adding import pdb; pdb.set_trace() for use with pdb. The difference here is that a new instance of vim will be launched instead of pdb when you run the script. It doesn't seem to be able to reuse an already started vim session. The basic script could be debugged wtihout any trouble and it didn't freeze or thrown an error message when django manage.py runserver eas exectued! Now the bad news, it didn't stop at a break point when a url was loaded in the browser.

vim-addon-python-pdb

And finally vim-addon-python-pdb but not much luck with that either.

Error detected while processing function python_pdb#Setup..python_pdb#RubyBuffer:
line   11:
E117: Unknown function: async_porcelaine#LogToBuffer
Error detected while processing function python_pdb#Setup..python_pdb#UpdateBreakPoints..python_pdb#BreakPointsBuffer:
line    2:
E117: Unknown function: buf_utils#GotoBuf
E15: Invalid expression: buf_utils#GotoBuf(buf_name, {'create_cmd':'sp'} )
line    3:
E121: Undefined variable: cmd
E15: Invalid expression: cmd == 'e'
Error detected while processing function python_pdb#Setup..python_pdb#UpdateBreakPoints:
line   37:
E117: Unknown function: vim_addon_signs#Push
line   43:
E716: Key not present in Dictionary: write

May 19, 2015, 12:30 a.m. » Tagged: vim , python , django , debug

Which PostGIS function to use?

Which PostGIS function or operator do you use if you wanted to find the overlap between different lines? There are several candidates which immidiately come to mind;  ~, && , ST_Overlaps and and ST_Intersects if you are in a hurry or TLDR; use ST_Overlap. The two operators They all do job but in different ways, takes different amounts of time and return slightly different results. 

Using only operators             time  4.3237  rows  1208
Using ~ operator and ST_Overlap  time  5.5201  rows  233
Using ST_Dwithin                 time  5.5551  rows  2500
Using ST_Intersects              time  5.3128 rows  746  

ST_DWithin seems to match everything to everything else. Using ~ and &&. Usin operatings seem to be a shade quicker than the other options. But the operators only deal with bounding boxes not the lines themselves so your milage may vary depending on your context. This test was carried out with a nested for loop with the other loop having 250 objects and the inner loop having 25. The data came from the road.lk carpool tables. Picking the right sample is very important in this test since different samples could produce different results so the sample was actually changed multiple times.  The results followed the same pattern except for this one instance where all the routes were disjoint! 

Using only operators      time  0.0174  rows  0
Using operators + overlap time  0.0176  rows  0
Using ST_Dwithin          time  0.0177  rows  0
Using ST_Intersects       time  0.0177  rows  0   

I should have used number formatting but I didn't so sue me. Getting back to the first data set, the number of matches vary by a factor of eleven so we ought to check the quality of the matches. That can be done by using the overlapping distances.

Using only operators             time 7.1954  rows  1208  total overlap  0.199858778468
Using ~ operators and ST_Overlap time 6.9210  rows  233   total overlap  0.199858778468
Using ST_Dwithin                 time 10.7650 rows  2500  total overlap  0.199858778468
Using ST_Intersects              time 7.1929  rows  746   total overlap  0.199858778468 

 

Surprise, surprise all the overlapping distances are identical. So clearly the a.line && b.line greatly over estimates the number of overlaps and so does ST_Dwithin. Counter intuitively ST_Overlaps and ST_Intersects both seem to have the edge over the operators.

Before winding down, I wanted to do one final test. As things stand the queries look like' INNER JOIN pool_route b ON a.line ~ b.line OR b.line ~ a.line or a.line && b.line ' and ' INNER JOIN pool_route b ON a.line ~ b.line OR b.line ~ a.line or ST_Overlaps(a.line , b.line) ' will there be a difference if the contains operator (~) is removed?

 

Using only operators             time  7.4457  rows  1208  total overlap  0.199858778468
Using ~ operators and ST_Overlap time  6.0198  rows  64    total overlap  0.199858778468
Using ST_Dwithin                 time  10.7326 rows  2500  total overlap  0.199858778468
Using ST_Intersects              time  7.3100  rows  746   total overlap  0.199858778468     

 

Just ignore the timing the difference is too small to be taken seriously but look at how the number of matches has dropped dramatically from 266 to 64 for the ST_Overlap query. Yet the final overlapping distance hasn't changed. At this point I double checked and triple checked to make sure that my code doesn't have any bugs and it looks like there isn't. The only thing to do is to try with a much bigger dataset

 

Using only operators time  317.762  rows  83075  total overlap  116.168481483
Using ST_Overlap     time  223.555  rows  13884  total overlap  116.167709497
Using ST_Dwithin     time  416.819  rows  160000  total overlap  116.168481483
Using ST_Intersects  time  276.907  rows  46804  total overlap  116.168481483

This is with a much larger dataset, 400 objects in the outer loop, 400 objects in the inner loop. If you want to have look at the code that was used here you go:

import os, sys

if __name__ == '__main__':  #pragma nocover
    # Setup environ
    sys.path.append(os.getcwd())

    os.environ.setdefault("DJANGO_SETTINGS_MODULE", "main.settings_dev")
    
from django.db import connection
from pool.models import Route

import time

cursor = connection.cursor()

routes = Route.objects.order_by('id')[0:100]
routes2 = Route.objects.order_by('-id')[0:250]

t0 = time.time()
sum = 0
distance = 0

for r1 in routes:
    for r2 in routes2 :
        q = cursor.execute('SELECT count(*) , sum(ST_Length(ST_intersection(a.line,b.line))) FROM pool_route a '

                ' INNER JOIN pool_route b ON a.line && b.line '
                ' WHERE a.id = %s AND b.id = %s', [r1.id, r2.id])
        row = cursor.fetchone()
        sum += row[0]
        if row[1]:
            distance += row[1]
t1 = time.time() - t0
t0 = time.time()


print 'Using only operators time ' , t1 , ' rows ' , sum ,' total overlap ', distance
sum = 0
distance = 0

for r1 in routes:
    for r2 in routes2 :
        q = cursor.execute('SELECT count(*) , sum(ST_Length(ST_intersection(a.line,b.line))) FROM pool_route a '
                ' INNER JOIN pool_route b ON ST_Overlaps(a.line , b.line) '
                ' WHERE a.id = %s AND b.id = %s', [r1.id, r2.id])
        row = cursor.fetchone()
        sum += row[0]
        if row[1]:       
           distance += row[1]

t1 = time.time() - t0
t0 = time.time()
print 'Using ~ operators and ST_Overlap time ' , t1 , ' rows ' , sum ,' total overlap ', distance

sum = 0
distance = 0

for r1 in routes:
    for r2 in routes2 :
        q = cursor.execute('SELECT count(*) , sum(ST_Length(ST_intersection(a.line,b.line))) FROM pool_route a '
                ' INNER JOIN pool_route b ON ST_Dwithin(a.line, b.line, 50) '
                ' WHERE a.id = %s AND b.id = %s', [r1.id, r2.id])
        row = cursor.fetchone()
        sum += row[0] 
        if row[1]:
            distance += row[1]

t1 = time.time() - t0
t0 = time.time()
print 'Using ST_Dwithin time ' , t1 , ' rows ' , sum, ' total overlap ', distance
sum = 0
distance = 0


for r1 in routes:
    for r2 in routes2 :
        q = cursor.execute('SELECT count(*) , sum(ST_Length(ST_intersection(a.line,b.line)))  FROM pool_route a '
                ' INNER JOIN pool_route b ON ST_Intersects(a.line, b.line) '
                ' WHERE a.id = %s AND b.id = %s', [r1.id, r2.id])
        row = cursor.fetchone()
        sum += row[0] 
        if row[1]:
            distance += row[1]
t1 = time.time() - t0

print 'Using ST_Intersects time ' , t1 , ' rows ' , sum, ' total overlap ', distance

Update: a bug in the code was fixed and the blog post formatted (by hand)

May 16, 2015, 4:11 a.m. » Tagged: postgis , python , road.lk

Configure the vim plugins and other stuff.

The story so far is that I have moved from pydev to vim, and then dumped the initial vim setup which used python mode in favour of setting everything up from scratch. Then struggled a bit with autocomplete for django and now it's time to tweak some of the plugins.

minibufexpl

Right-o now it's time to configure the plugins that were installed the other day. Let's start with minibufexpl they haven't made any key mappings so if you want to use it you need to keep typing in long commands or you need to create a map in your .vimrc file. Before doing that you can type :nmap to see what the current mappings are (you dont' want a conflict do you?) I chose <leader>\b for buffer explorer :MBEToggle and I am not bothered about open and close mappings. The next thing is to change the positioning from a horizontal split to a verticle and make it 20 columns wide, and also open on the left.

let g:miniBufExplVSplit = 20   " column width in chars
let g:miniBufExplBRSplit = 0
let g:miniBufExplStatusLineText = 'Buffers'

It also changes the minibufexpl's status bar to just buffers. Wonder if the status line can be removed completely.

Command-T

While minibufexpl might be cool it might actually be redundant because command-t has both a file and buffer select/search/autocomplete feature. In fact with Command-t installed you probably don't need NERDTree either. In my case the debugger took over both default mappings for the buffer and files and they had to be added back in with

nnoremap  t :CommandT
nnoremap  b :CommandTBuffer

Tasklist.

This one didn't survie. The plugin is five years old and can be replaced with a simple vim command (which is nothing more than a comment to a stackoverflow answer)

noremap <leader>todo :noautocmd vimgrep /TODO/j **/*.py<CR>:cw<CR>

Snipmate

Here is another one that didn't survive. I've never really liked using snippets. You are more likely to spend time thinking up usefull snippets and putting them into a file rather than actual coding. And this is a lot of extra stuff to remember.

Oh, BTW snipmate is often linked in blogs as msanders/snipmate.vim; that's indeed the original. However there is a more upto form at garbas/vim-snipmate this one doesn't have the snippets though, but you would probably want to look up snippets specific for your programming language than the rather generic (and outdated) snippets you find in the original repo.

NERD Commenter

It's one of the things I picked up from coming home to vim. NERD Commenter allows you to add remove comments very easily. If you never had to comment out a large chunk of code temporarily your brain is probably wired differently from everyone else's

 

Sparkup

If you write HTML (and with everyone claiming to be a full stack developer who doesn't?) you would love sparkup you write what looks like CSS and the plugin will convert that to a set of tags. Pretty nifty.

 

The keyboard mappings

use Ctrl-s to save files when in insert mode

imap <C-S> <ESC>:w<CR>i

add Ctrl-c, v, x support and also make vim use the system clipboard

" add 'standard' copy paste stuff 
vmap <C-c> "+y1
vmap <C-x> "+c
vmap <C-v> c<ESC>"+pg
imap <C-v> <C-r><C-o>+

" use the system clipboard so that lines yanked in vim can be pasted elsewhere
set clipboard=unnamed

Rope

Before ending this post, Let's get back to rope for a bit. The first time you open a project you will need to do :RopeGenerateAutoimportCache to set things up

May 13, 2015, 5:15 a.m. » Tagged: vim , plugins

Vim Autocomplete for Django

This is the continuation of a journey that began by switching from eclipse/pydev to vim for python/django development via python mode. Looking to setup auto completion for python and django. Let's jump straight in

Rope

Originally rope was installed as part of python-mode and I quickly tired of it and switched to jedi it was only after installation that I discovered it to be incompatible with python-mode. That was what prompted my decision to give up on python mode (it has far too many blackboxes) and install everything manually. I managed to get auto complete working beautifully with rope or so I thought.

autocomplete django code with vim and rope

and this is the configuration that was used

let ropevim_vim_completion = 1
let ropevim_extended_complete = 1
let ropevim_enable_autoimport = 1
let g:ropevim_autoimport_modules = ["os.*","traceback","django.*"]
imap <C-space> <C-R>=RopeCodeAssistInsertMode()<CR>

The trouble with this setup is that auto completion would add the package name after the class name!. For example if you were to select HttpResponse from the dialog, what actually gets added is HttpResponse : django.http.HttpResponse. Using the let ropevim_vim_completion = 1 also means taking vim auto import out of the picture. The meaning of Auto import here is to automatically add the import statement to the top of the code without moving the cursor. This is one of the most usefull features of pydev

If you set let ropevim_vim_completion = 0 auto complete still works but you lose the popup dialog. Instead you get a line near the bottom of the screen with the choice. You need to cycle though the options with the tab key.

Jedi

The real problem with jedi is that you need to know that full class name including it's package for auto complete to work. For example just typing Http<Ctrl Space> will not give you HttpResponse or any of the related packages from django.http. For me this is a real problem, I have never bothered to remember the full package names for classes in any of the languages I have ever used. There isn't a need for it. That's what IDEs are for!

Conclusion.

So it seems that both Jedi and Rope have their short comings. Where Jedi is strong rope is lacking and vice verce. Hay hang on a second. Can't we use both? The answer my dear madam is that you can! This is the relevent section of the .vimrc

" Use both ropevim and jedi-vim for auto completion.Python-mode 
" let ropevim_vim_completion = 1
let ropevim_extended_complete = 1
let ropevim_enable_autoimport = 1 
let g:ropevim_autoimport_modules = ["os.*","traceback","django.*"] 
imap <s-space> <C-R>=RopeCodeAssistInsertMode()<CR>

let g:jedi#completions_command = "<C-Space>"
let g:jedi#auto_close_doc = 1

Now when you are not sure of the package name you hit Shift-Space to do an rope auto import + auto complete action. You don't get a pop up but have to tab through the options. When the class has already been imported or you know the full name, hit Ctrl Space and get jedi to do the dirty work for you.

May 11, 2015, 12:12 p.m. » Tagged: vim , python , django

Good bye python-mode

I recently switched from pydev to vim for all python work. But after struggling with a nasty bug in my code for nearly two hours I was beginning to have second thoughts and started eclipse/pydev, ran it through the debugger. After that It took just 3 minutes to fix the bug including eclipse start up time. Ok, maybe it's because I haven't had any luck setting up a python debugger for vim. Maybe I should have just used pdb. Your devtools are support to make things easier not harder.

Debugging isn't the only issue, django auto complete still doesn't work with either jedi or rope. I've tried about a dozen remedies listed on various places including stackoverflow with no luck at all. It's tempting to make the swith back to pydev a permanent one,  but I am going to give it one more shot. The crux of the problem maybe with python-mode. Python-mode has a lot of other plugins under the hood and they are all sealed into small black boxes. It's just like driving a modern car. You are never really sure who is the boss.

Now I am going to uninstall python-mode, follow the outdated guide at sontek and have another go at it. Hopefully at the end of the day, I might have a working vim setup and a modernized version of the sontek guide! This is just an as it happened commentary. After I settle in I will create a howto which will hopefully be helpfull to someone else.

A couple of changes to the plugins. Instead of pyflakes I am installing syntastic and not going to install ack either. Both of these items might return to the menu  if I find that vim's search features don't meet expectations. The same goes for gundo, which seems like a nice plugin, but can it replace git? make-green and pytest are to other plugins to miss out. The sontek blog gives a link to the NERD Tree plugin is given as vim-scripts/The-NERD-tree while there is a better maintained version at 'scrooloose/nerdtree' by the same guy who bring you syntastic.

That's not the only plugins where the upto date version different from the location given in that blog post; others included ropevime (correct loctaion: python-rope/ropevim) and minibufexpl (correct as fholgado/minibufexpl.vim). All these plugins were installed using Vundle which probably wasn't around at the time the post was written. Vundle adding and removing plugins becomes completely painless. 

 

Now to start up. Predictably there is an error:

Error detected while processing function LoadRope:
line    4:
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ImportError: No module named ropevim
Error detected while processing /home/raditha/.vim/bundle/TaskList.vim/plugin/tasklist.vim:
line  369:
E227: mapping already exists for \t
Press ENTER or type command to continue

pip install rope ropevim takes care of the first one while for the second you will need to create a separate mapping according to your taste in keystrokes! Now the key to the whole puzzle: 

" Rope AutoComplete
let ropevim_vim_completion = 1
let ropevim_extended_complete = 1
let g:ropevim_autoimport_modules = ["os.*","traceback","django.*", "xml.etree"]
imap <c-space> <C-R>=RopeCodeAssistInsertMode()<CR>

This is something I discovered stackoverflow, with it autocomplete suddenly starts working pretty much like it does in pydev. So does auto import. Start typing and press Ctrl-Space to activate autocomplete, once the autocomplete is done do ::RopeAutoImport to have the import statement automatically added towards the top of the file. This is rather clunky and the default keymapping C-c r a c is worse! but you are free to map it to any key of your choosing and I am selecting Ctrl-F1

And that brings an abrubt end to the proceedings but I will be back soon.

The photo is a public domain pic by openclips

May 9, 2015, 12:15 p.m. » Tagged: vim , python , pydev

Postgresql streaming replication

If you read the postgresql streaming replication documents it sounds really really complicated, but it isn't. There is a clever little tool called repmgr that can set up multiple slaves for you quite easily.  repmgr can also take care of promoting a slave to a master automatically if needed but you still have to tell the database clients that the server has changed.

Getting started

You need two servers with postgresql installed. For this discussion we will use 9.3 but you can use 9.4 as well. If you have an RPM based system such as Centos 7 it goes like this:

 yum -y install http://yum.postgresql.org/9.3/redhat/rhel-7-x86_64/pgdg-redhat93-9.3-1.noarch.rpm
yum -y groupinstall "PostgreSQL Database Server 9.3 PGDG" 
yum -y install repmgr93 

 

You will need a pair of SSH keys preferably without a password because repmgr uses rsync to copy a snapshot of the master server onto the slave. The key should belong to the postgres user which means it should be in /var/lib/pgsql/.ssh/ folder (a folder that you will most probably have to create). The id_rsa.pub should be in the authorized_keys file on the master. Don't forget to check the permissions and ownerships of this files SSH is very particular about it (chmod 700 .ssh and chmod 600 .ssh/*

Setting up the master

Yum placed a file named repmgr.conf in /etc/repmgr/9.3/repmgr/ for me. I decided the hell with it and created a file like this:

cluster=roadlk
node=1
node_name=192.168.1.1
conninfo='host=192.168.1.1 user=repmgr dbname=repmgr'
pg_bindir=/usr/pgsql-9.3/bin/

Note that in some places in the repmgr documentation the username is given as repmgr_usr in and other places it's repmr similarly the database name is given as repmgr_db or repmgr it doesn't matter what you choose for the username or db name but make sure that it's the same everywhere

Now register the master with repmgr repmgr -f /etc/repmgr/9.3/repmgr.conf --verbose master register a minor detail is that you will need to run this command as postgres user and repmgr will probably need to added to the path.

The slave

If you were to do a manual replication this is the tricky part. But thanks to repmgr it's a one liner (run as postgres).

repmgr -d repmgr -U repmgr --force --verbose standby clone 192.168.1.1

If you didn't do an initdb you don't need the --force option. Now here is the gotcha, the above command copies over the postgresql.conf and the pg_hba.conf files from the master. That means you will need to edit postgresq.conf and change the listen directive. Without it the server will try to listen on the wrong IP address.

Next steps are straightforward, create the repmgr.conf file

cluster=roadlk
node=2
node_name=node2
conninfo='host=192.168.1.2 user=repmgr dbname=repmgr'
pg_bindir=pg_bindir=/usr/pgsql-9.3/bin/

Start the server (systemctl start postgresql-9.3-service in Centos 7 /etc/init.d/postgresql start in older versions) and finally register the slave (run this command as postgres).

repmgr -f /path/to/repmgr/node2/repmgr.conf --verbose standby register

Now you can go and put the kettle on or whatever else it is you like to wear. The rest of this post is about what lead to my writing about postgresql replication.

For nearly two years road.lk operated with just log shipping WAL archives. We took a pg_basebackup about once a month (alright alright whenever I remembered to).  All the log files were rsynced to another machine but not applied. A warm standby it certainly wasn't. The logs just sat there in a folder waiting to be applied if needed. But we did test it once in a while though and the last time process took hours and hours, so after the carpool was launched I decided that streaming replication is the way to go. 

On Monday road.lk was offline for nearly an hour. It was due to the primary database server crashing, no data was lost but the automatic failover hadn't kicked in automatically. To make the failover automatic, you need to change the /etc/hosts file on the server and send a HUP signle to gunicorn but the tiny bash script that I wrote for this had a bug. It's just my luck that the outage occurred the very moment I stepped out and I am the only one with the keys to the server.

May 8, 2015, 5:48 a.m. » Tagged: postgresql , streaming , replication , repmgr , roadlk

Taking on side on the vim vs emacs debate

it started with a casual comment on twitter. Then at Gaveen's suggestion I decided to finally take a side and had to decide which side! After using a plethora of IDEs all these years I had very little knowledge vim and what I knew of emacs could be written on the back of a stamp. The little bit I knew about vim was thanks to the all the times when I need to edit something on a remote server. Even then for heavy work I prefer to edit locally and rsync.

These days most of my code is python with a bit of Java on Android but surely VIM isn't suitable for the former? As for the latter python-mode seems to be highly recommended so I decided to install it using vundle. This is what the plugins section of my .vimrc looks like

Plugin 'gmarik/vundle'
Plugin 'tpope/vim-fugitive'
Plugin 'scrooloose/nerdtree'
Plugin 'klen/python-mode'
Plugin 'majutsushi/tagbar'
Plugin 'davidhalter/jedi-vim'
Plugin 'joonty/vdebug'

As you can see I have decided not to rely on rope but to use jedi-vim instead, even then I had to make a slight change because autocomplete would automatically select the first item before I had time to select anything. For example when I typed the name of a Django model followed by the '.' jedi would append __str__ at the blink of an eye. That behaviour can be changed with the following settings in .vimrc

let g:jedi#popup_select_first = 0
let g:jedi#popup_on_dot = 0

It's only been a few days but I've already found Nerdtree to be indispensable while tagbar I haven't really made much use of yet. The biggest issue that I need to sort out though is debugging. So far I have tried Vundle without much success. Well, i can in fact get to stop at breakpoints but at that stage the vim tab with the code stops responding.

May 4, 2015, 6:30 a.m. » Tagged: python , vim , emacs