django multidb support for admin site by subdomain

usecase: the client logs in to the admin site by going to clientname.sitename.com/client_admin.

Why? because it's nice to use the django admin site. Because we want the client to only access the client's database.

How?

first off, setup your DNS to forward *.sitename to your server. Than in your server config (I use nginx), send *.sitename to uwsgi.

(note: if you're not using uwsgi with django here, you'll have to use your imagination).

In your wsgi script (you know, the one that launches django for each incoming request?), put something like this:

class RequestWSGIHandler(django.core.handlers.wsgi.WSGIHandler):

def get_response(self,request):

domain = request.__dict__['META']['HTTP_HOST']

domain = domain.split('.')

if len(domain) > 3:

db_name = domain[0]

print "caught domain %s"%db_name

os.environ['SITE_NAME'] = db_name

return super(RequestWSGIHandler,self).get_response(request)

# Lastly, load handler.

# application = django.core.handlers.wsgi.WSGIHandler()

application = RequestWSGIHandler()

Yes, it's kind of quick and dirty. I've subclassed the django wsgi handler to intercept every request, grab the subdomain if there is one, and set it in the OS env. Then using the (happily outdated) python 2.6* super() syntax, I call the standard wsgi handler.

Then when you get to settings.py, you can grab that env variable if it's there and save it to your settings object. It's now available when you need it.

For instance!!!, you can subclass admin.ModelAdmin to support multiple databases (see here). In the linked example (you have to scroll halfway down, they basically overwrite the key class methods and add in "using=new database" to all db queries) the new database is static, but since you now have a dynamic database name in settings.whatever you called it, you can pick your extra database access name on the fly. hurrah.

A caveat:

(this is a bit scary...) in the django documentation, the good django devs say to always import settings from django.conf. I wonder why? Maybe to avoid the same multithread concurrency issues I very briefly glossed over in my last post. Who knows. Problem is, if you do that with the above example, some of your calls (though not all) won't pick up your dynamic subdomain-influenced database. Why the hell not? Beats me. I have "from django.conf import settings" everywhere in that project except that file, for which I use "import settings".

and as a further caveat, keep in mind that if you import the settings from django.conf in a file that's loaded before admin.py, or wherever you want the real dynamic settings, it won't work!

I didn't really go into it, but this is a 4.5 facepalm issue right here.

Dynamic Databases in Django

My current (paid) project has me managing a separate database for every client in Django. This has been a great challenge. Since 1.2, Django has had multidb support, so that's not hard - the hard part is all of the edge cases.

For instance, we want to be able to add clients. On the fly. We plan to have many - like more than 20. So we certainly don't want to have our database definition in settings, all written out like the Django tutorials. At the very least, a loop over db names.

def add_db_to_databases(DATABASES,name):

if name in DATABASES:

return DATABASES

DATABASES[name] = {

'HOST': 'localhost',

'PORT': '',

'NAME': name,

'USER': '',

'PASSWORD': '',

'ENGINE': '',

'OPTIONS': {

'autocommit': True,

}

}

return DATABASES

for name in pro_dbs.names:

DATABASES = add_db_to_databases(DATABASES,name)

What I did there is take the names from another python file, which contains a simple python list of names.

I needed to be able to add clients on the fly. This is the hard part; as of yet I have two stumbling blocks with only partial workarounds.

1. I'd love to have the database names in a database themselves. Soooo much better than reading the python file with the names into a list, appending the new name to the list, than writing back to the file. But how to load from a database in the django settings file itself? It's been engineered not to allowed that.
2. I'd love to be able to update settings without restarting the server. You can do certain things in that vein by messing with django.conf.settings, but it's unclear how well that'll hold up under multithreading.

All in all, not a facepalm worthy subject, but very interesting.

web scraping and microformats

This proved very helpful to me. It's a short, concise explanation of how to extract data from the structured markup of a website in python. It's a no-brainer, but sometimes you need someone to show you the no-brainer before it becomes one for you.

The bottom line is that html is a string. It's also a tree. You can parse it either way. Microformats and the semantic web aren't only useful to big spidering companies like the goog; they're useful to all of us.

Twitter is useless?

So I had a great idea for a JQuery plugin. What's better, I was going to write it too.

A single line initializes and adds the plugin to your page, which then allows people to tweet from said page, including a shortened link to said page, hash tags of your choice - but that's just the beginning. Then, the plugin retrieves all tweets with the same shortened link, organizes them using create_date and @whoever, and displays them on your page as hierarchical comments.

Basically, a totally painless and free commenting system hosted and authenticated by twitter, displayed on your page with one line. I like. Except for one problem - twitter search doesn't work for included URLs.

If anybody knows how to possibly retrieve all tweets about a given url (or the last 200), please let me know. It's a terrible wasted potential not to allow this.

For instance, here's a twitter account of mine from another project: . The last couple tweets have bitly links. You'd think a twitter search for the link would show up the tweet, but NO! nothing going. Even when you switch from their ridiculous "popular tweets" to "all tweets". What kind of tweet search misses on unique text from a tweet?

facepalms = only 2, but general annoyance is through the roof.

UPDATE:

I take it back - some urls hit, some don't. Even better. Possibly, broken urls don't hit? But I'm not sure.

holy sweet victory

Just figured out something that's been bugging me nonstop for weeks, which has no documentation online: the jquery plugin fancybox doesn't like to have its stuff overwritten, which is to say, you shouldn't define something like this:

var fancy_box_options = fancy_box_defaults;
fancy_box_options.type = 'iframe';
fancy_box_options.width = 696;
fancy_box_options.height = 499;
fancy_box_options.scrolling = 'no';
fancy_box_options["onStart"] = function(link){
var new_link = $.param.querystring($(link).attr("href"), "gmail=" + $("#gmail").val());
$(link).attr("href", new_link);
};

$("a.iframe").fancybox(fancy_box_options);
$("a.iframe_defaults").fancybox();

In one function, and then soon after define:

<% if current_user && @popup_help %>
var fancy_box_options = fancy_box_defaults;
fancy_box_options.type = 'iframe';
fancy_box_options.width = 696;
fancy_box_options.height = 499;
fancy_box_options.scrolling = 'no';

fancy_box_options.href = '<%= popup_help_path(:gmail => current_user.email) %>';
$.fancybox(fancy_box_options);

in another. You'll get a wholly unhelpful error about href being undefined.

All better now -
#facepalms only 4, because I already had a lot of trouble with this previously, I'm over it now, and the sweet release of figuring it out is worth so, so much.

ubuntu problems

so for the last while my wireless card has been acting up _sometimes_, on certain wireless networks but not others. I finally got a clue as to what might be going on: wireless N. It's an N card, but the driver/kernel stuff going on in ubuntu has a known problem with certain wireless cards.
Edited my modprobe.d/options.conf with the line:

options iwlagn 11n_disable=1 11n_disable50=1

and then (hopefully won't have to restart) ran:

sudo rmmod iwlagn
sudo modprobe iwlagn

And that seems to have improved things. So far.

#facepalms: like a lot. Until now. Hopefully.

rails fields_for, object sorting

So what I needed to do was select a subset of one model which happened to have a key into another, sort it by an attribute of the other model, and display a list of the results in a form.
here's the query to find, filter, join, and order:

@user_categories = Category.joins(:user_categories).where('user_categories.user_id' => current_user.id).order("name asc")

and here's the bit of the form:

<%= f.fields_for :user_categories, @user_categories do |user_category| %>

and then to access the objects:

<%= user_category.label :name, user_category.object.name %>

Things to notice:
the fields_for uses a parameter :user_categories which references the name of the model, followed by the object which is actually a list of objects of type Category, so I'm assuming this is to let rails know how to treat it. Also, note that to reference the actual objects, you have to use model_class.object.method. the model_class is actually some sort of form builder helper before the call to .object.

This only took an hour to figure out - #facepalms = 3.

My foray into Quantified Self research

So this is really cool - with the advent of the high speed internet in your pocket smartphone, everybody can do their own quantified self research with ease and convenience. Forget apps, it's even simpler than that. Just make a google docs form . You can choose anything about yourself that you want to track - for instance, say water intake. You can make a form that has inputs (type of your choosing - check box, radio button, text, etc) corresponding to what you want to track. Every time anybody fills out the form, a line will be added to a google docs spreadsheet with the info that they filled out.

The trick here is, give yourself and no one else the url for your form. Bookmark it on your smartphone. Fill it out all day long, whenever appropriate. I've been tracking all my bodily inputs and outputs, as well as mood, sleep habits, and a couple other things. I generate about 10 or 15 lines on the spreadsheet a day. The form takes about 15 seconds to fill out, which I do during downtime (like while I'm peeing...).

At the end of the month, I'll have more than 400 datapoints. Since it's in a spreadsheet already, no extra tabulation is necessary. As long as you design your form/spreadsheet combo with some insight, you'll be able to learn things about yourself that you never knew before.

Internet exasperations

this one is like a 9 on the facepalm scale. Inexplicably, certain wireless networks (especially dsl and 4g) seem to work, but gradually stop connecting very often, throwing up network errors (could not load variety) on my browser, but then connecting again (quickly, even) after a couple attempts.

I thought I might have just made it better by switching to google's dns servers, but I'm not actually sure it's helped yet.

Note, of course, that I'm using wicd on ubuntu 11 (not network-manager, which was actually catastrophically freezing my computer once in a while).

ideas?

--edit:

Wow, looking back, realized I never shared my solution: turns out certain dell wireless cards have a problem with wireless standard n, but not g. wish I had all the appropriate linkage and help at my fingertips, but if you find this post, that should get you started. I disabled N for my card, and everything is fine (albeit slower than need be). Who knows, mayhap it be fixed by now. I don't remember how I disabled it, and I probably don't need to think about this whole issue anymore.

Rails 3 Unobtrusive Javascript is Unintuitve

So what I wanted to do was link from a partial view to a controller method through a route which took a title parameter and from the controller method run a javascript file in order to append content to a div with a unique name defined by the title param.

In django it's pretty straightforward, but ruby on rails 3 had me scratching my head for a good long while.

Here's the solution, or at least the parts that were difficult:

the link:

<%= link_to "Comments", show_comments_path( :format => 'js', :comment_id=> session[:id] ), :remote => true %>

This whole :remote => true business tells rails 3 that this is an ajax call. The :format param does NOT get passed into the url, but DOES render the response as html, most of the time. Which is to say, rails will run the .js.erb file with the name matching your route.

Here's the route:

match 'show_comments/:comment_id', :to => 'deals#show_comments', :as => :show_comments

So first rails runs the show_comments method from the appropriate controller, which you can use to set state variables for the javascript. Then, whatever is in show_comments.js.erb will be run, and the current page will NOT be reloaded or redirected. No partial needed. No view needed. The controlling method will do all of this as the default return, so no return or render needed either, only the .js.erb file which has your prototype or jquery calls, or whathaveyou.

facepalms: 4 (steady, slow, only slightly infuriating progress)

more ubuntu fights

so I know I thought I fixed my virtualenv and virtualenvwrapper problems recently, but apt-get wasn't letting me install anything! This error:

Traceback (most recent call last):
File "", line 1, in
ImportError: No module named virtualenvwrapper.hook_loader
dpkg: error processing install-info (--configure):
subprocess installed post-installation script returned error exit status 1
Errors were encountered while processing:
install-info

So it seemed like it had something to do with install-info, whatever that is, but NO - I'd forgotten about some lines in my /etc/environment file that had to do with virtualenv, before I started using .bashrc. took em out, all better.
facepalms: 2

event data in jquery

this one stumped me for quite a bit. If you want to bind a click event to a div id in jquery to a preexisting function which takes arguments, you need:

$('#id').click({param: param}, function_to_run );

to bind it. Here, id is the id of the div or whatnot, param (the second one, the first one could be anything) is the variable you want to send, and the function_to_run is already written. No parens after the name, you're sending it as an object.

The tricky part is in the function itself:

function function_to_run(event) {
param = event.data.param;

get it? The click event sends the event itself as the one argument to function_to_run, and the parameters you give it are stored in data. just so you know.

around a 6 on the facepalm scale.

upgrading to ubuntu natty

annoyingness right off the bat. maybe a 7 on the facepalm scale.

1. the login screen allows you to choose between unity and classic, but only after you click the login name to get the password box out. before then, no indication of any possible choices on the bottom bar.

2. loaded up classic, but compiz settings were off.

3. my ATI graphics seem choppy (ouch).

4. virtualenvwrapper acting funky - specifically,

ImportError: No module named virtualenvwrapper.hook_loader

even though things generally seemed to work. Wow, I'm going to have to figure out how to inline code on blogspot here.

Anyways, the problem turned out to be that natty installed python 2.7.1 as default. Nobody warned me! I added the line

export VIRTUALENVWRAPPER_PYTHON=/usr/bin/python2.6

To send everybody back to python2.6. Problem solved.

Unity isn't super annoying or buggy yet, but I'll give a shoutout if it becomes so.

Welcome to creature feature

yes, it's Yet Another Programming Blog.

I'll also throw in some deep thoughts once in a while.