more python SUDS advice

Everybody knows the a SOAP API is a horrid thing to work with, even a well designed one. But everybody also knows that SOAP is going to be around for a while.
Recently, while working with the ExactTarget SOAP API in Python, I figured out another helpful trick with SUDS, which is the main python SOAP wrapper.

Let me preface this by saying that suds is not a great library; particularly, it was written by and for people from another era of web development and does not stand up to today's needs. Why doesn't somebody write a better one? because everyone who does modern web dev hates SOAP in the first place.

But anyway, one annoying feature of suds is that when you create an object from its object factory, it always fills in all properties with an empty string, even those that are optional and have sane defaults. So essentially, it breaks all objects right out of the gate, forcing you to go through one by one and choose those sane defaults explicitly.

However, there's another way - for any properties that are giving you trouble in this fashion, just delete them after creating the object:
del object.AnnoyingProperty

Suds will then not send the property, allowing the API server to choose the sane default.

facepalms: 6

How to do get a good security audit for your startup

Recently at SimpleRelevance we decided it was time for a security audit on our website and especially our dashboard. We have quite a bit of client information that we would never want to share with the world.

Security audits are kind of like STD testing - even when you feel 100% fine, it's better safe than sorry.
Unfortunately, also like STD testing, if you cheap out, you won't get tested for everything.

So our problem was - how do we not get ripped off, while still ensuring a comprehensive audit - we're talking more than just simple pentests here. It's so hard to verify that you've gotten a good security audit, since the company can just come back and say "we didn't find any issues - your system is bulletproof!", and you can't prove that negative. In the end we did a lot of research and interviews and chose based on the data at hand. But since then I've thought of a fun way that might work even better.


1) through research, find N>1 companies that you'd consider paying for research.
2) bargain each one down (you were going to do this anyway, right? We ended up paying about 65% of the original ask for the company we chose).
3) tell each one: "I'll make you a deal. I'll pay you full price on the test, if you agree to the stipulation that I'm making this same deal with another security company, and whoever finds fewer or less important security holes doesn't earn any money, and whoever finds the most gets the full price [plus 10% if you need to sweeten it]".
4) Some of the companies are going to say no. Decide which company is most excited by the idea of the deal - they are clearly the winner, so congrats - you found the best one! Tell them none of the others would take the deal, so unfortunately you can't do it, but you'll still pay them the discounted rate you'd already agreed upon for a full audit, and you applaud their style.

ipython and the django shell: strange scoping errors

this is a minor issue that has annoying repercussions. on most versions of django, if you use ipython and start it with ./manage.py shell, you cannot define global variables and then use them in local functions. it gets ugly quick. more info here:
https://github.com/ipython/ipython/issues/62

patch is here:
https://github.com/django/django/pull/512/files

and it definitely fixes the issue.

python httplib2 7.x and ssl

this is not a completely new issue, and the internet helped me find the solution in about 5 minutes, but it was not very clear or obvious so I thought I'd write it down succinctly for posterity:

Python's httplib2 package did not validate ssl certificates in versions < 7.0, and started to from 7.0 and up. Unfortunately, it ships with its own set of trusted ssl certificates which comprise only a medium-sized subset of all of your favorite sites' certs (for instance, wikipedia fails when pinged at https://en.wikipedia.org!). The error looks like this:

  File "/home/deploy/.virtualenvs/sandbox.koaladeal.com/local/lib/python2.7/site-packages/httplib2/__init__.py", line 1597, in request
    (response, content) = self._request(conn, authority, uri, request_uri, method, body, headers, redirections, cachekey)

  File "/home/deploy/.virtualenvs/sandbox.koaladeal.com/local/lib/python2.7/site-packages/httplib2/__init__.py", line 1345, in _request
    (response, content) = self._conn_request(conn, request_uri, method, body, headers)

  File "/home/deploy/.virtualenvs/sandbox.koaladeal.com/local/lib/python2.7/site-packages/httplib2/__init__.py", line 1281, in _conn_request
    conn.connect()

  File "/home/deploy/.virtualenvs/sandbox.koaladeal.com/local/lib/python2.7/site-packages/httplib2/__init__.py", line 1036, in connect
    raise SSLHandshakeError(e)

SSLHandshakeError: [Errno 1] _ssl.c:504: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed

 So the solution is to use ubuntu's system default cert file, which lives at /etc/ssl/certs/ca-certificates.crt. I ended up overwriting the one that shipped with httplib2, which fixes the problem globally, but fails if we ever reinstall httplib2. Hence this blog post for posterity. If anyone knows of another way to globally install a new certificates file for httplib2 without changing the package itself, I'm all ears. I might just onboard the package to our project (but then I have to remember never to pip install it...).

by the way, this page was most helpful.

facepalms: 3

how to not run your business

tell me - if you're going to pay for hosting for a corporate wordpress blog, what's the chance you're not going to use your corporate email account and what's the chance you're not going to host the blog at your corporate url? in other words, for what business would this error message not come up?

None. Facepalms: 5 (horrifically hilarious, luckily not a problem I have to deal with today.)

magento and the dreaded WebFault: Product not exists

Sometimes using SOAP with the Magento V2 API you may search for a product's image (by calling catalog_product_attribute_media.list with the sku as only argument) and receive this hilarious message:

"Product not exists." <-- you can't make this stuff up!

Hilarious but sad, because there's the image right there, laughing at you from the website, and yet your API doesn't have access.

luckily there's an easy fix. in short: append a space to the end of the sku. done. psha. magento, we love to hate you.

facepalms: 5

celery + djcelery problem with virtualenv and virtualenvwrapper

this is a tricky one - out of nowhere, production env boxes started failing at celery startup with:



ImportError: cannot import name current_app
when importing djcelery. Versions of celery were fine.
It turns out if you import celery and run
import celery.current_app you'll see the real problem, which is that the virtualenv binary is out of sync with the new python binary from a recent security update - specifically, os.urandom has been changed/removed.

if you have virtualenvwrapper, and you let $ENV=YOUR_ENVIRONEMNT_NAME
So the answer is:


deactivate (in case an env is running)
cd ~/$ENV_HOME (.virtualenvs, for me)
rm $ENV/bin/python
virtualenv $ENV


this will rebuild your python binary with the correct python post-security fix, without losing any other packages. happy hacking!