tricks for constantcontact oauth2

Just so you know, you have to specify a redirect url for both legs of OAuth2 integration with ConstantContact. And it has to be the same exact url. Even tho the second (POST) leg probably won't end up redirecting anywhere.

Not only that, it has to be the exact same as the oauth_callback setting for the api key you're using. Where do you set this? You sign in to the community portal at constant contact, click the "api keys" tab, and then click on the api key itself, which is a link which takes you to its management page. With no explanation on this, it took me about 20 minutes to figure out, so hopefully this helps somebody else.

Oh - and if you're still getting uri-mismatch errors during this process? remember that the callback has to be https, for real - you need an ssl cert.

facepalms: 4

django-celery with virtualenv

this page has some great tips for daemonizing celery for django. They even have a section for django under a virtualenv.

But, just remember that if you have any environment variables that you export before running django, that your django settings rely on to load correctly, you should probably export them in your daemon script. Unfortunately the django-celery documents don't know about these environment variables (I'm talking about you, export ENV='pro'), so it's up to you to stick em in there.

Furthermore, consider restarting the celery daemon whenever you deploy. I hear supervisord makes this easy; I'm going to do that next.

facepalms: 2

email attachments with django postmark

A quick fix - and there are some pull requests for this on bitbucket, so feel free to look for them, but you can do it yourself -

in postmark/backends.py, attachments are expected to be in dictionary form:

{'Name':XXXX,'Content':XXXX,'ContentType':XXXX}

however, the default django core email attachment method wants them as a tuple. Make your own patch if you want, it's pretty straightforward. Postmark's error messages are fairly useless.

facepalms: 3

a quick gripe

the worst thing about the official google reader app for android (besides the annoying new swipe-based UI) is that there's no way to see alt text on images (think XKCD).

google gdata oauth

there's a great big honkin awesome library for python called gdata - it lets you interface with a ton of their APIs, including spreadsheets and docs. We use it internally at SimpleRelevance to write all sorts of logging directly to shared spreadsheets - a big process will finish and have reams of data to share, but rather than parse out a flatfile every time, I wrote a wrapper that takes the data and spits it into a beautifully formatted google spreadsheet. From there, charts and such are easy.

We also now use it externally - clients can log in and authenticate through oauth with google, and then we can write their predictions to a spreadsheet in their own account.

You probably know about oauth - it's nice because the client never has to supply us with any login credentials - the whole thing is very secure. Unfortunately, it was a little painful to set up. Like, 1 hour of productive work and 2 hours of fighting with stupid. Why?

I'll tell you why.

1. There are a lot of outdated tutorials.
2. The gdata plugin, while awesome, has tons of legacy code and 2 completely different and mostly working ways of doing everything.
3. There are tutorials for the old path, tutorials for the new, and tutorials that mix the two.
4. This page has the most beautiful, well-written, cogent, perfect, comprehensive example of how to set up oauth with gdata. Unfortunately it gets confusing in a crucial bit at the end.

Although it's great, check it out. It has examples every step of the way in 4 different languages, for both gdata paths. That's 8 examples every step of the way (and you know that oauth2 is a 3 step process - that's around 24 pieces of code).

It's actually spot on all the way through to the end. The confusing part comes when you have to exchange your oauth token for the longterm access token - the thing that actually authenticates and lets you access stuff.

The tutorial has this line:

access_token = client.UpgradeToOAuthAccessToken()  # calls SetOAuthToken() for you

But I had trouble. Frankly, I messed up. But I couldn't get it to work until I did this:

client.UpgradeToOAuthAccessToken()
access_token = client.token_store.find_token(oauth_token.scopes[0])

It turns out that the first line actually does returns the access_token, so the google tutorial is correct. This was fixed some time in the last something or other. Used to be you had to use that second line to get it. Older tutorials don't reflect this change. I was getting tired by then. I missed the boat. Cue frustration.

Then, I my second point of confusion: the request token key and secret (leg 1) are the same as the oauth token key and secret (leg 2) but the access token key and secret are totally different and new. In retrospect, this makes perfect sense (from a security POV), but at the time I was baffled. Don't try to upgrade your oauth token and then save its key and secret to authenticate with. It won't work.

The google tutorial tells you to save it but leaves it to you to figure out how to use the access token object. Really, it's easy:

save access_token.key

save access_token.secret.

You can use the session, you can use a database, you can use a session stored in your database, whatever you want! I think the access token lasts for a while.

Anyway, oauth is hard, but I'm really getting the hang of it. Feel free to email me if you need help with this one.

facepalms: 4.5.

django multidb support for admin site by subdomain

usecase: the client logs in to the admin site by going to clientname.sitename.com/client_admin.

Why? because it's nice to use the django admin site. Because we want the client to only access the client's database.

How?

first off, setup your DNS to forward *.sitename to your server. Than in your server config (I use nginx), send *.sitename to uwsgi.

(note: if you're not using uwsgi with django here, you'll have to use your imagination).

In your wsgi script (you know, the one that launches django for each incoming request?), put something like this:

class RequestWSGIHandler(django.core.handlers.wsgi.WSGIHandler):

def get_response(self,request):

domain = request.__dict__['META']['HTTP_HOST']

domain = domain.split('.')

if len(domain) > 3:

db_name = domain[0]

print "caught domain %s"%db_name

os.environ['SITE_NAME'] = db_name

return super(RequestWSGIHandler,self).get_response(request)

# Lastly, load handler.

# application = django.core.handlers.wsgi.WSGIHandler()

application = RequestWSGIHandler()

Yes, it's kind of quick and dirty. I've subclassed the django wsgi handler to intercept every request, grab the subdomain if there is one, and set it in the OS env. Then using the (happily outdated) python 2.6* super() syntax, I call the standard wsgi handler.

Then when you get to settings.py, you can grab that env variable if it's there and save it to your settings object. It's now available when you need it.

For instance!!!, you can subclass admin.ModelAdmin to support multiple databases (see here). In the linked example (you have to scroll halfway down, they basically overwrite the key class methods and add in "using=new database" to all db queries) the new database is static, but since you now have a dynamic database name in settings.whatever you called it, you can pick your extra database access name on the fly. hurrah.

A caveat:

(this is a bit scary...) in the django documentation, the good django devs say to always import settings from django.conf. I wonder why? Maybe to avoid the same multithread concurrency issues I very briefly glossed over in my last post. Who knows. Problem is, if you do that with the above example, some of your calls (though not all) won't pick up your dynamic subdomain-influenced database. Why the hell not? Beats me. I have "from django.conf import settings" everywhere in that project except that file, for which I use "import settings".

and as a further caveat, keep in mind that if you import the settings from django.conf in a file that's loaded before admin.py, or wherever you want the real dynamic settings, it won't work!

I didn't really go into it, but this is a 4.5 facepalm issue right here.

Dynamic Databases in Django

My current (paid) project has me managing a separate database for every client in Django. This has been a great challenge. Since 1.2, Django has had multidb support, so that's not hard - the hard part is all of the edge cases.

For instance, we want to be able to add clients. On the fly. We plan to have many - like more than 20. So we certainly don't want to have our database definition in settings, all written out like the Django tutorials. At the very least, a loop over db names.

def add_db_to_databases(DATABASES,name):

if name in DATABASES:

return DATABASES

DATABASES[name] = {

'HOST': 'localhost',

'PORT': '',

'NAME': name,

'USER': '',

'PASSWORD': '',

'ENGINE': '',

'OPTIONS': {

'autocommit': True,

}

}

return DATABASES

for name in pro_dbs.names:

DATABASES = add_db_to_databases(DATABASES,name)

What I did there is take the names from another python file, which contains a simple python list of names.

I needed to be able to add clients on the fly. This is the hard part; as of yet I have two stumbling blocks with only partial workarounds.

1. I'd love to have the database names in a database themselves. Soooo much better than reading the python file with the names into a list, appending the new name to the list, than writing back to the file. But how to load from a database in the django settings file itself? It's been engineered not to allowed that.
2. I'd love to be able to update settings without restarting the server. You can do certain things in that vein by messing with django.conf.settings, but it's unclear how well that'll hold up under multithreading.

All in all, not a facepalm worthy subject, but very interesting.