Loading object properties from an external file

Recently I came up with the idea that I will hold the running properties in the configuration file. The idea was to use the Python syntax but keeps it simple as possible.
The second thing is that the config file can be anywhere and I do not want force people to make any Python modules (with those strange __init__.py files). They will simply forget to create.
On the other hand I want to keep those attributes in the class object because then I can use that object in other part of the program. First idea was to simply use execfile()
Here is how I overcome the problem, I create a BaseSettings class with the constructor that takes an path to the settings file:

class BaseSettings(object):
  def __init__(self, settings_file):
    """Constructor of the settings class.

      Args:
        settings_file: The full path to the settings file.
    """

    _required_sections = set([
        'ANGLES', 
        'DIHEDRALS', 
        'PAIRS', 
        'ATOM_PAIRS', 
        'BOND_CONFIG',
        'TOTAL_NUMBER_OF_CROSSLINKS'
        ])
    settings_variables = {}
    copy_globals = globals().copy()
    try:
      execfile(settings_file, copy_globals, settings_variables)
    except SyntaxError as ex:
      print 'There is an error in your config file %s in line %s' % (
          ex.filename, ex.lineno)
      print '%d: %s' % (ex.lineno, ex.text)
      print ex.msg
      sys.exit(2)

    # Checks if some attributes are missing in the settings file.
    missing_sections = _required_sections - set(settings_variables)

    if missing_sections:
      Exception(
          'Invalid settings file, those sections are missing: %s' % (
              ','.join(missing_sections))
    # Loads the variables as it they would be and attributes of the class.
    self.__dict__.update(settings_variables)

and then I use execfile() with the copied globals() and the empty dict settings_variables that works as the container for the locals(). I had to copy globals dict because otherwise I had those attributes in the globals.
Afterwards the variables from the settings file can be found in the settings_variables dictionary. So the only things was to update the __dict__ of the class with those variables.

Composite unique constraint

Composite unique constraint are very common in real database entities. In Django models, those constraints are expressed by the unique_together property of the Meta class.
Lets say that we have an Django model class:
class ModelA(models.Model):
  property1 = models.CharField(max_length=255, null=True)
  property2 = models.IntegerField()

  class Meta(object):
    unique_together = ('property1', 'property2')

It looks fine, now lets say that we want to save something in this model:
m3 = ModelA(property1=None, property2=1)
m3.save()
m4 = ModelA(property1=None, property2=1)
m4.save()

What we could expect is that the second save will fail because of the composite unique constraint. What actually will happen is that there will be two rows in db with property1 set to NULL and property2 set to 1. This is because the SQL standard define that NULL is not a value but the state so in that case the NULL value is not treat as unique accros the rows.  
To solve the issue we have to take care about checking the uniqueness on the Django model level. The save method of the models has to be overwrite. Below is the example how to define abstract class which can be used instead of Django models.Model class.

class Model(models.Model):
  class Meta(object):
    abstract = True

  def _check_uniqueness(self):
    nullable_field_names = set([x.name for x in self._meta._fields() if x.null])

    for unique_together in self._meta.unique_together:
      if set(unique_together).intersection(nullable_field_names):
        values = [(x, getattr(self, x)) for x in unique_together]
        queryset = {}
        for field, value in values:
          if value is None:
            queryset['%s__isnull' % field] = True
          else:
            queryset[field] = value
        exists_query = self.__class__.objects.filter(**queryset)
        if self.pk is not None:
          exists_query = exists_query.exclude(id=self.id)
        if exists_query.exists():
          return unique_together
    return None

    def save(self, *args, **kwargs):
      unique_together = self._check_uniqueness()
      if unique_together is not None:
        raise db.IntegrityError('columns %s are not unique' % ', '.join(unique_together))
      super(Model, self).save(*args, **kwargs)

How to get the current list of indexes and columns?

So how to get the list of all columns for all tables with additional information about indexes.
This little query will return the three columns: table_name, column_name and index_name (if there is no index then it will be null).

select 
    ic.table_name, ic.column_name, iss.index_name
from
    information_schema.columns ic
        inner join
    information_schema.tables it ON it.table_name = ic.table_name
        and it.table_type = 'BASE TABLE'
        left join
    information_schema.statistics iss ON iss.column_name = ic.column_name
group by ic.table_name , ic.column_name
order by ic.table_name;

Unique id across multiple tables on MySQL

The idea is to have one global unique id for every entries in MySQL tables. So instead of using auto_increment property for primary key I would like to have some kind of global generator. Here is the first draft of the solution.

1. First step is to create a table in which the ids will be stored:

CREATE TABLE `uuid` (
  `id` INT(11) NOT NULL AUTO_INCREMENT,
  `stub` TINYINT(11) NOT NULL,
  PRIMARY KEY (`id`)
) ENGINE=InnoDB;

Next we have to define triggers. Lets assume that we have some tables t2 and t3. We have to create triggers on each of them. The trigger is simple as a rock:

DELIMITER $$
CREATE TRIGGER `t3_uuid` BEFORE INSERT
  ON `t3`
  FOR EACH ROW BEGIN
    REPLACE INTO `uuid` (`stub`) VALUES (1);
    SET NEW.id = LAST_INSERT_ID();
  END$$
DELIMITER ;

We create the trigger t3_uuid on the table t3 which will be run before each insert and will set `id` from the LAST_INSERT_ID() command.

Dictionary with regular expression

import re

class RegexDict(dict):
  def __init__(self, d):
    dict.__init__(self, d)

  def __getitem__(self, key):
    try:
      return super(RegexDict, self).__getitem__(key)
    except KeyError as ex:
      for kkey, value in self.iteritems():
        if re.match(kkey, str(key)):
          return value
      raise ex

usage:

z = RegexDict({
  'a': 100,
  '\d+': 300,
  1: 3
  })
z['re#\d+'] = 100

print z['a']
print z[2]
print z['a']
print z[1]
print z['re#200']

results:

100
300
3
100

Convert jmol file to png

Simple note, from time to time I am looking for command which convert my xyz file into png. Here is the command:

C="connect delete; rotate y 15; rotate x 20; spacefill 80; background white; zoom 110;"
for f in *.jmol; do echo "${f}"; echo "$C write $f.png; exit" | jmol -nL "${f}" -s - ; done 

First line set up config, in this case we disable bonds, rotate in y-plane an about 15 degree and in x-plane 20 degree, set background to white, zoom molecule and set size of atoms to 80%;

Remote PDB and Django

Recently I have had to debug some error in Django system. In fact the problem was somewhere inside views and the debug page did not give any important information (as usually) and randomly show different place in .py files where exception was raised. I could not put standard pdb debugger, cause the problem  appeared only on nginx+uwsgi setup. I could not reproduce it on internal django webserver.
How to put debugger in uwsgi environment? Of course we do not have access to terminal in that case but we can use remote debugger. I have found very small and simply debugger rpdb:

http://tamentis.com/projects/rpdb/ 
Simply install it by pip and then put in the code:
 
import rpdb; rpdb.set_trace()
 
By default, rpdb listen on 4444 port so the next step is to connect to it by e.g. nc:
 
$ nc zuzia 4444

[teodor@zuzia production]$ nc zuzia 4444
--Call--
> /home/teodor/project/test/lib/python2.7/site-packages/rpdb/__init__.py(37)shutdown()
-> def shutdown(self):
(Pdb) w
  /home/teodor/project/test/lib/python2.7/site-packages/django/core/management/commands/runserver.py(107)inner_run()
-> run(self.addr, int(self.port), handler, ipv6=self.use_ipv6)
  /home/teodor/project/test/lib/python2.7/site-packages/django/core/servers/basehttp.py(696)run()
-> httpd.serve_forever()
  /usr/lib64/python2.7/SocketServer.py(227)serve_forever()
-> self._handle_request_noblock()
  /usr/lib64/python2.7/SocketServer.py(284)_handle_request_noblock()
-> self.process_request(request, client_address)
  /usr/lib64/python2.7/SocketServer.py(310)process_request()
-> self.finish_request(request, client_address)
  /usr/lib64/python2.7/SocketServer.py(323)finish_request()
-> self.RequestHandlerClass(request, client_address, self)
  /home/teodor/project/test/lib/python2.7/site-packages/django/core/servers/basehttp.py(570)__init__()
-> BaseHTTPRequestHandler.__init__(self, *args, **kwargs)
  /usr/lib64/python2.7/SocketServer.py(639)__init__()
-> self.handle()
  /home/teodor/project/test/lib/python2.7/site-packages/django/core/servers/basehttp.py(615)handle()
-> handler.run(self.server.get_app())
  /home/teodor/project/test/lib/python2.7/site-packages/django/core/servers/basehttp.py(283)run()
-> self.result = application(self.environ, self.start_response)
  /home/teodor/project/test/lib/python2.7/site-packages/django/contrib/staticfiles/handlers.py(68)__call__()
-> return self.application(environ, start_response)
  /home/teodor/project/test/lib/python2.7/site-packages/django/core/handlers/wsgi.py(273)__call__()
-> response = self.get_response(request)
  /home/teodor/project/test/lib/python2.7/site-packages/django/core/handlers/base.py(111)get_response()
-> response = callback(request, *callback_args, **callback_kwargs)
  /media/Crypted/teodor/projects/test/web/views.py(525)wrapper()
-> return panel_view(request, *args, **kw)
  /media/Crypted/teodor/projects/test/web/views.py(540)test()
-> import rpdb; rpdb.set_trace()
  /home/teodor/project/test/lib/python2.7/site-packages/rpdb/__init__.py(62)set_trace()
-> debugger.shutdown()
> /home/teodor/project/test/lib/python2.7/site-packages/rpdb/__init__.py(37)shutdown()
-> def shutdown(self):
(Pdb) 

Django template tag: display some content in specified time range

Django template tags are very powerful tool for extending this template framework, in fact built-in set of filters and tags is very large but from time to time it is the need to do something custom. 
Recently I have got a task that required to display some content in specified time range. I could not find built in solution so I have prepared template tag that behave like {% if %}{% else %} tag set.

Code is available here: https://github.com/MrTheodor/django-snippets

How to use it?

Usage:
   1. Define in settings dict TIME_BLOCK with names of block that has to be
   shown on certain time:
       TIME_BLOCK = {
           'block1': {
               'from_date': datetime.datetime(2012, 04, 28, 13, 48),
               'to_date': datetime.datetime(2012, 04, 30, 13, 48),
               'show': True
           }
       }
   
   2. In template use it in such way:
       
       {% showtime block1 %}
           content that will be displayed between 2012-04-28 13:48 and 2012-04-30
           13:48
       {% else %}
           content that will be displayed before and after that time
       {% endshowtime %}

  There is also the possibility to do it in the reverse way by using option 'show'
 

Enjoy!

Mendeley with Fedora 16


Few months ago I wrote about my problems with running this very nice software on my Fedora 14.

After while, I've upgrade my Fedora to 15, next to 16 and recently I've noticed that Mendeley again have not worked.
What I've got on console is some memory pointer error:

[teodor@zuzia lib]$ mendeley
Running /home/teodor/download/apps/mendeleydesktop-1.1.3-linux-x86_64/bin/..//lib/mendeleydesktop/libexec/mendeleydesktop.x86_64
loaded the Generic plugin
QSharedPointer: pointer 0x260f550 already has reference counting

Google said that problem is related to the Qt library, again.. this time you can simply run mendeleydesktop with option that switch app to use internal version of library. I've modified running script and set this option on by add after line 199 this statement:

use_system_qt = False

That's all for a while... Mendeley starts but it shows similary error like in previous post.
QSslSocket: cannot call unresolved function d2i_X509
QSslSocket: cannot call unresolved function d2i_X509
QSslSocket: cannot call unresolved function SSLv3_client_method
QSslSocket: cannot call unresolved function SSL_CTX_new
QSslSocket: cannot call unresolved function SSL_library_init
QSslSocket: cannot call unresolved function ERR_get_error
QSslSocket: cannot call unresolved function ERR_error_string
QSslSocket: cannot call unresolved function SSLv3_client_method
QSslSocket: cannot call unresolved function SSL_CTX_new

This time solution is rather different. You need to install package openssl-devel:

su -c 'yum install openssl-devel' 

gnome-shell-ext-icon-manager

Small news. Week ago I've published new version of this plugin. It should work on GNOME 3.2;
https://github.com/MrTheodor/gnome-shell-ext-icon-manager

Gnome 3.2 and two screens

GNOME 3.2 try to be considered as the most user-friendly environment around the Linux world. Maybe that's why most settings has to be set up by dconf-editor or via so very intuitive dialog box:
Guess what, I want to set primary display to Goldstar. And I can't do it by some select box, dialog box or other stuff. So the tip:
To set up primary display, you need to drag this black bar on display which you want to be your primary display.
Next point, of course it works, your topbar is moved on your primary desktop, but what was funny  Message bar (bottom bar) was still on laptop display. So next tip:
You need to move your displays on the same level to set up message bar on primary display

Why it cannot look like here, with small checkbox Primary display ? Or maybe it looks to much complicated for typical GNOME user?



Icons in Gnome shell

Another small extension that give you some control on your gnome-shell. 

 IconManager (maybe this name is not adequate) lets you to manage icons that will appear in your top panel. It can be useful when you want to move Tomboy or gnote icon and remove at the same time icons for bluetooth, a11y (accessibility), display, network manager.




Get repository:
git clone https://MrTheodor@github.com/MrTheodor/gnome-shell-ext-icon-manager.git

and follow the instruction in INSTALL file.


How to use?

1. run dconf-editor
Press Alt+F2, type dconf-editor

2. Find settings
org/gnome/shell/extensions/icon-manager

3. Edit key
* top-bar: put icons that you want to remove from top-bar or move from tray bar

4. To accept changes you need to restart gnome-shell
Alt+F2, type r

Repository: 
https://github.com/MrTheodor/gnome-shell-ext-icon-manager 

Skype and Gnome-Shell

I am fascinated by new Gnome 3. Not only let's say foreground is interesting but also what is behind. The whole concept of desktop, application oriented GUI, also gnome-shell seems to be very useful. Everything looks very easy.
One of nice feature is to set global presence of user. I think that this idea came up firstly with Ubuntu or previous version of GNOME. In fact I have never used it until now. When you switch your presence to Busy, whole system instantly will stop bothering you with some notification. 
Signal about presence  is emitted by SessionManager via DBus mechanism. This is another smart idea in GNOME. Of course programs need to watch for this signal. Empathy (or telepathy), Pidgin are prepared to observe presence signal. To be completely happy I have to force Skype to observe presence signal.
Cause extensions for Gnome-shell are written in JavaScript language (yes, it is the same JavaScript which is used by web developers) so writing new one is not so difficult. This short extension for gnome-shell, link presence signals from SessionManager with Skype.

How to install?

Simply download extension:
git clone https://github.com/MrTheodor/gnome-shell-ext-SkypeNotification.git

mkdir -p ~/.local/share/gnome-shell/extensions
cp -r gnome-shell-ext-SkypeNotification/SkypeNotification@krajniak.info ~/.local/share/gnome-shell/extensions
next restart gnome-shell by pressing Alt+F2 and typing r
Reposistory for plugin: https://github.com/MrTheodor/gnome-shell-ext-SkypeNotification

Mendeley with Fedora 14 on 64 bit


I was looking for tool which apart of storing citation reference, will also index my pdf files on hard disk and gives some interface to search for articles, organize them in some groups and  Mendeley do all of this tasks. It is a very useful academic reference tool for organize your scientific articles in one place. What is more, pdf reader has tool for making annotation in pdf and after that can export pdf with those additional information.

Mendeley has got also web interface arrange in the way of social application, so there is a list of your friends, groups, some kind of twitter like tool.  The user can also create own profile with some information. What is more interesting, is the synchronization between desktop application and web application so it can be use to share data with co-workers.


Problems with Mendeley on Fedora 14 64-bit

After switched to Fedora, I had problem with using Mendeley. First problem was with missing libssl.so.0

/home/teodor/download/apps/mendeleydesktop-0.9.9-rc1-linux-x86_64/bin/../lib/mendeleydesktop/libexec/mendeleydesktop.x86_64: error while loading shared libraries: libssl.so.0: cannot open shared object file: No such file or directory


simple solution is to link library from /usr/lib/ to lib/ in Mendeley dir.

ln -s /usr/lib64/libssl.so.10 lib/libssl.so.0


One remark, if your system is 32-bit, you need to link from /usr/lib/ instead of lib64
Second problem is with Qt library. Mendeley throw errors when it tries to connect to the server

QSslSocket: cannot call unresolved function SSLv3_client_method
QSslSocket: cannot call unresolved function SSL_CTX_new
QSslSocket: cannot call unresolved function SSL_library_init
QSslSocket: cannot call unresolved function ERR_get_error
QSslSocket: cannot call unresolved function ERR_error_string
QSslSocket: cannot call unresolved function SSLv3_client_method
QSslSocket: cannot call unresolved function SSL_CTX_new
QSslSocket: cannot call unresolved function SSL_library_init
QSslSocket: cannot call unresolved function ERR_get_error
QSslSocket: cannot call unresolved function ERR_error_string
QSslSocket: cannot call unresolved function SSLv3_client_method
QSslSocket: cannot call unresolved function SSL_CTX_new
QSslSocket: cannot call unresolved function SSL_library_init
QSslSocket: cannot call unresolved function ERR_get_error
QSslSocket: cannot call unresolved function ERR_error_string
QSslSocket: cannot call unresolved function SSLv3_client_method
QSslSocket: cannot call unresolved function SSL_CTX_new
QSslSocket: cannot call unresolved function SSL_library_init
QSslSocket: cannot call unresolved function ERR_get_error
QSslSocket: cannot call unresolved function ERR_error_string

Solution is very simple, you need to remove Qt files from lib/ in Mendeley directory. After that Mendeley will start to using Qt libraries from your system instead of those distributed with it.

rm lib/libQt*


Enjoy using Mendeley

game of life in javascript

Life is hard to understand, but here you can play in it ;)
Game of Life

I was wondering how to use this new html5 canvas element so I wrote this piece of code. Enjoy!


How to convert bin/cue to iso

I always forget how to convert cue cd image to iso. So in fact this note is for my memory.
Converting to iso format can be done by using bchunk. Ubuntu users have this program in repository so the procedure is very simple
  1. sudo apt-get install bchunk
  2. bchunk CDImage.bin CDImage.cue CDImage
That's all.

Suspend plugin for Rhythmbox

Suspend plugin for Rhythmbox music player. This plugin shutdown or suspend your computer after the last song from your playlist or queue. It is very useful when you listening to music at night and you don't remember to shutdown/suspend your computer.

Download

Last version is always available on Google Code:
Google Code downloads

Instalation

  • uncompress files from archive
  • put folder suspend-plugin in .gnome2/rhythmbox/plugins

Quick tip: Mysql ERROR 1005


ERROR 1005 (HY000) at line 296: Can't create table './debradb/comment.frm' (errno: 150)


check what is going on by passing this sql command to mysql command line:

SHOW ENGINE INNODB STATUS;

and next look for error ;). In most case errors are made by


  • incorrect data types and flags for primary key and foreign keys

  • lack of primary key, in innodb your relation have to contain primary key

  • all relation have to be the same type. In default installation of mysql set as default engine myisam


Quick tip: Display raw html

How to display raw html



Simple, if you generate scaffold, edit show.html.erb and remove h


<%=h @static.content %>

after that, @static.content would be display without html escape. It's very useful if you add
to you application some static pages managed from control panel and contains raw html.

Ruby on Rails: First impression


Next day learning Ruby on Rails. My first question was, where is documentation, where is entry point. Of course, there is lots of screencast (grr..), videocast, voicecast, but I was looking for documentation similarly to django. You know, page where there would be described all functions. I don't think about API doc, for first user is too hard to understand and to find entry point.



Scaffolds, something that is called killer function. Well it's nice when everything is generated, but why for holy shit I can't add new method to controller. For example I try to add method login and call controller using url /users/login. What I get? Of course error, login could not be find. For goodness what login, I try to call method, not to show object. Developers like an user shouldn't be taken by surprise.



Models with migration are very nice functionality, I saw it last day in Doctrine.



Conventions, conventions and again conventions. You have to write upper case letters, end table with s and other strange CONVENTIONS. I hate it. In django/python I have on, use tabs or spaces to indent and that's all.