Porting Postgres to google cloud SQL

I know it's quite a peculiar problem - you have a postgres database, and suddenly you're migrating to Google App Engine and you want to use the new Google Cloud SQL (look it up, it's public and it's pretty cool).

Anyway, you need to port the old data. Google cloud sql is cool except for a big problem - sql dump imports have 0 error reporting - if it fails, it just turns red and tells you that an unknown error occurred. So here's how I made it work:

1. dump the postgres stuff selecting the tables you want with a couple extra options on:
2. pg_dump --column-inserts --data-only POSTGRES_DATABASE_NAME -t TABLE_NAME -t ANOTHER_TABLE_NAME -f NEW_FILE_NAME.sql [note: you need to have psql privileges already here].
3. delete the top lines of the dump file created in 2) until the first "insert" line.
4. load it into mysql locally, where you can catch any errors:
5. mysql -u USER -p DATABASE_NAME < NEW_FILE_NAME
6. dump it from the local mysql:
7. mysqldump -u USERNAME -p --add-drop-table MYSQL_DATABASE_NAME TABLE_NAME ANOTHER TABLE_NAME> FIXED_SQL_DUMP_FILE.sql
8. add as the first line in the new dumpfile: "use DATABASE_NAME;" (ignore the quotes, add the name of the database you want the data loaded into on google).
9. Now you can load this new file into a google cloud storage bucket using their web browser gui and from there import it into cloud sql.
10. pray, as you wait for the stupid thing with no error reporting to turn green.

facepalms: 7

app engine and django mail

lo and behold, local email (as in, sending email to console) in django works with the app engine development server, but not with the app engine development backend / taskqueue server. It gives you an attributeerror: Exception Value: 'module' object has no attribute 'getfqdn'

django content types and south

If you are missing content types in your from django.contrib.contenttypes.models.ContentType table, and you are using south on multiple databases, it's likely that you haven't synced AND MIGRATED all of your apps (even if you don't use them) on your master database.

facepalms: 1

tricks for constantcontact oauth2

Just so you know, you have to specify a redirect url for both legs of OAuth2 integration with ConstantContact. And it has to be the same exact url. Even tho the second (POST) leg probably won't end up redirecting anywhere.

Not only that, it has to be the exact same as the oauth_callback setting for the api key you're using. Where do you set this? You sign in to the community portal at constant contact, click the "api keys" tab, and then click on the api key itself, which is a link which takes you to its management page. With no explanation on this, it took me about 20 minutes to figure out, so hopefully this helps somebody else.

Oh - and if you're still getting uri-mismatch errors during this process? remember that the callback has to be https, for real - you need an ssl cert.

facepalms: 4

django-celery with virtualenv

this page has some great tips for daemonizing celery for django. They even have a section for django under a virtualenv.

But, just remember that if you have any environment variables that you export before running django, that your django settings rely on to load correctly, you should probably export them in your daemon script. Unfortunately the django-celery documents don't know about these environment variables (I'm talking about you, export ENV='pro'), so it's up to you to stick em in there.

Furthermore, consider restarting the celery daemon whenever you deploy. I hear supervisord makes this easy; I'm going to do that next.

facepalms: 2

email attachments with django postmark

A quick fix - and there are some pull requests for this on bitbucket, so feel free to look for them, but you can do it yourself -

in postmark/backends.py, attachments are expected to be in dictionary form:

{'Name':XXXX,'Content':XXXX,'ContentType':XXXX}

however, the default django core email attachment method wants them as a tuple. Make your own patch if you want, it's pretty straightforward. Postmark's error messages are fairly useless.

facepalms: 3

a quick gripe

the worst thing about the official google reader app for android (besides the annoying new swipe-based UI) is that there's no way to see alt text on images (think XKCD).