tag:blogger.com,1999:blog-65804432614718790692024-03-12T19:52:27.934-07:00Creature Featuremostly code problems + solutions, a few deep thoughtsUnknownnoreply@blogger.comBlogger48125tag:blogger.com,1999:blog-6580443261471879069.post-87028697541388487692020-03-17T09:17:00.000-07:002020-03-17T09:17:28.404-07:00dependencies and devdependenciesbeen a while since I never code any bugs anymore but I thought I'd mention:<br />
if you have rails webpacker upgrade issues with the webpack loader not finding random files when you build in staging, try moving pipeline and compilation and babel related dependencies out of devDependencies in your package.json and into the main dependencies block. worked for me!<br />
<br />
facepalms: 7 (only took 5 hours to figure it out...)Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-6580443261471879069.post-63564809143004508742019-07-13T06:05:00.000-07:002019-07-13T06:05:31.248-07:00compiling and then transpiling sveltejs: babeljs, webpack, and a safari 9 problemBeen 2 years since my last post here!<br />
just had a very fun deep dive into this annoying bug that was only affecting this one old browser.<br />
<br />
I wrote it up here:<br />
<a href="https://stackoverflow.com/questions/57019417/how-to-transpile-compiled-svelte-with-babel-for-safari-9/57019418#57019418">https://stackoverflow.com/questions/57019417/how-to-transpile-compiled-svelte-with-babel-for-safari-9/57019418#57019418</a><br />
<br />
Anyway if you haven't checked out <a href="https://svelte.dev/">https://svelte.dev</a> I highly encourage it - it's faster, easier, and cooler than reactjs.<br />
<br />
This bug took me 5 hours to find and fix, but it was a fun and interesting sleuthing session the entire time that has left me feeling proud of my abilities, so I'm gonna go with facepalms: only 2 on this one.Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-6580443261471879069.post-26700554621104387322017-04-25T11:25:00.000-07:002017-04-25T11:25:18.115-07:00mobile touch gotchaI once made an angular firebase demo site called <a href="http://fridgemagnets.upchicago.org/">Fridge Magnets With Friends</a>.<br />
You can move magnets around on a virtual fridge. This involves a lot of dragging elements.<br />
One day mobile drag stopped working, even tho I employed the clever <a href="http://touchpunch.furf.com/">jquery ui touch punch</a>.<br />
<br />
I spent hours switching the whole thing to use <a href="https://github.com/fatlinesofcode/ngDraggable">ngDraggable</a>, because their mobile demo did work in mobile. Altho using ngDraggable required me to take out jquery (actually it didn't, but some forums said that would help make it work in mobile) - so I spent a while redoing the angular app without jquery.<br />
Moment of truth: no luck.<br />
<br />
Finally I hijacked the browser with a little shim code:<br />
<blockquote class="tr_bq">
<blockquote class="tr_bq">
$document.on('touchstart', function(e){</blockquote>
<blockquote class="tr_bq">
console.log(e)</blockquote>
</blockquote>
});<br />
<br />
Because for the life of me I couldn't figure out why my elements weren't receiving the touchstart event in mobile.<br />
This way I could see what element the e event was pointing to.<br />
Turns out my footer CSS was very improperly done and covering the entire screen, so the event for touchstart, which has to have a stoppropogate() for other reasons in mobile only, never made it to the dragable elements.<br />
Took out the footer and both solutions started working; however, jquery-ui + touch-punch still worked better than ngDraggable, so in the end, 4 hours of work (work that was, admittedly, kind of fun) got reduced to commenting out a couple lines of html. At least I understand mobile touch events now.<br />
<br />
facepalms: 9.5Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-6580443261471879069.post-88016530219307771612017-04-19T13:13:00.003-07:002017-04-19T13:14:02.008-07:00CSS property of the day: pointer-eventsDo you have an absolutely positioned div that you want to show up on top of another element, but you want click and hover events (for instance) to pass through to the back element? Not such an uncommon use case!<br />
just call our old (new, for me) friend<br />
<blockquote class="tr_bq">
pointer-events: none;</blockquote>
And it'll pass those events right through.<br />
<br />
facepalms: 1 (either I'm getting better at CSS or I just got lucky. According to <a href="https://medium.com/@isaaclyman/8-css-gotchas-to-start-your-morning-off-right-c5daade0731d">this hilarious article</a>, I got lucky.)Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-6580443261471879069.post-39185832887577228512017-03-11T10:14:00.000-08:002017-03-11T10:14:12.057-08:00quick note on webpack imports and es6 class inheritance<div class="tr_bq">
I know it's crazy that I'm only now using modern javascript class inheritance for the first time, but I've been more of a backend programmer most of my career.</div>
<br />
I've got a new hobby project that I'm doing with modern webpack / es6 / vue.js, and I just learned something that I didn't find an obvious answer to in the googs.<br />
<br />
Basically, I'd been creating utility files with instantiated objects saved as var whatever or let whatever and then exported at the bottom:<br />
<br />
<blockquote class="tr_bq">
export default {<br /> obj: obj<br />}</blockquote>
<br />
And this was all well and good. But then when I wanted to define a class in the same file, the export default was overriding keyword "this". So the functions defined on the class referencing this were actually referencing my export default. I could use a fairly circular reference to get back to the instantiated class object, but really that whole route is silly.<br />
<br />
The way to make this work is to define (but not instantiate) each class in its own file. The class itself is your export default in that file:<br />
<br />
<blockquote>
export default YourClass extends Whatever {<br /> constructor () {<br /> this.stuff<br /> }<br /> yourFunction () {<br /> this.stuff + 1<br /> }<br />}</blockquote>
<br />
Then instantiate the class and use it in another file. Should have been obvious but well es6 takes some getting used to.Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-6580443261471879069.post-15807421810514473302015-12-02T19:27:00.001-08:002015-12-02T19:27:30.456-08:00on being a professional coderI recently finished Robert "Uncle Bob" Martin's <i><a href="http://www.amazon.com/The-Clean-Coder-Professional-Programmers/dp/0137081073">The Clean Coder</a> </i>and I'd like to use this space to share, very briefly, my takeaways from the book, as I head into a new team lead position.<br />
<br />
<br />
<ul>
<li>Accurate estimation of coding tasks is essential if you want to maintain trust between coders and managers</li>
<ul>
<li>don't be afraid to push estimation back due to scope creep, etc</li>
</ul>
<li>TDD is a way to write better code faster in the long run </li>
<ul>
<li>(I already knew this but it was good to hear it again [and again, and again, and again])</li>
</ul>
<li>Pair programming is also a way to write better code faster in the long run</li>
<ul>
<li>Never tried this in depth, but I'd really like to. Guess as a team lead that might even be up to me!</li>
</ul>
</ul>
<div>
anyway I'm inspired to be more professional in my coding life, so I guess he succeeded. Book's a quick read, too.</div>
Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-6580443261471879069.post-73686010226306998472015-09-20T19:33:00.001-07:002015-09-20T19:43:02.260-07:00ad blocking "controversy" aka foolishnesstl;dr: I<b>f your website can't make money without crappy ads, and nobody looks at your crappy ads because of advances in technology, then blaming that technology puts you squarely on the wrong side of history.</b><br />
<br />
The creator of <a href="http://www.marco.org/2015/09/18/just-doesnt-feel-good">peace</a>, one of the most popular iOS ad blocking apps, just pulled his app after a day and a half because it "didn't feel right" to him. While that's his choice, this strikes me as misguided.<br />
<br />
There's been a lot of<a href="http://www.wired.com/2015/09/content-blocking-apps/"> media coverage lately</a> about ad blocking, and complaints about a regurgitated and irrelevant tech news cycle aside, I have seen a quite a few well meaning people stating something like:<br />
<blockquote class="tr_bq">
ads make the internet go round. it's irresponsible to enable ad blocking software.</blockquote>
As a web developer and armchair philosopher, I'm on the record as being <i>for</i> ad blocking software, insofar as I think it only makes the internet a better place. Crucially, I'd remind the reader that under our current more-or-less capitalist system, no business models are guaranteed by government or (mostly) morality. And tho I think capitalism has some blind spots (see healthcare, maybe?), advertising on the internet ain't one of them.<br />
<br />
We would only<i> improve</i> the internet by striking back against giant, centralized ad serving systems. How anyone could honestly defend cross-site targeted ads is beyond me. Do they improve anyone's browsing experience? Do they honestly make all that much money for the sites that run them? For the businesses that pay for them?<br />
<br />
I humbly submit that any website that subsists on google (or similar, but right now it's mostly google)'s ad network, that could not possibly conceive of another way to monetize (of which I'll speak more below) <i>does not deserve</i> to make money. This is the good part of capitalism. This is how we improve technology and social organization as a species. This is the process that government interruption / manipulation frequently subverts (see: the DMCA, drug laws, FCC regulation, oil subsidies, etc).<br />
<br />
In fact, the DMCA is a useful parallel from recent history: Pirating music is very different from stealing a physical CD, all whiny record execs to the side. But a powerful industry complained that their business model had become obsolete, and instead of letting that business model die a natural death, our government guaranteed it with the DMCA's intellectual property laws. As a result, we are stuck with all sorts of <a href="http://hackaday.com/2015/08/08/corey-doctorow-rails-against-the-effect-of-drm-and-the-dmca/">negative externalities</a>. I'd write more about it but Cory Doctorow explains it way better.<br />
<br />
Without the DMCA, all the useless blood sucking middlemen (aka giant record labels) would have evolved or shrunken away, and maybe our entire culture would have been better off (fewer cookie cutter pop stars, anyone?). Smaller labels would have had no trouble monetizing, just like they do today anyway - with more interesting physical releases, more/better concerts, a resurgence in vintage-format media, pay-what-you-can downloads, free streaming to help spread music, and much more.<br />
<br />
Ad blocking is the same way. I wouldn't be surprised if Google lobbied congress to pass an anti-adblocking law. When you think about it, and for the reasons stated here, it'd be very similar to the DMCA. Google has enough at stake here to make spending hundreds of millions in lobbying dollars quite worthwhile for Google. And as we know, <a href="http://www.washingtonpost.com/news/the-fix/wp/2015/04/21/why-more-money-is-spent-on-lobbyists-than-on-congress/">congress is cheap</a>.<br />
<br />
Instead of rooting for the old ad-supported paradigm, let's brainstorm how a newer, sleeker web could monetize:<br />
<br />
<ul>
<li>more thoughtfully crafted ads, negotiated more directly with businesses, not served from ad farms, and hence not so easily blocked (aka <a href="https://northernlightsdirect.com/rise-of-native-advertising/">native ads</a>)</li>
<li>opt-in / crowdfunded memberships with perks (plenty of tipping services out there on the net)</li>
<li>merchandising</li>
<li>online or offline "rent party" type fundraisers, where website fans could meet and mingle</li>
<li>In general, new products that people are more interested in paying for</li>
</ul>
<div>
To reiterate and conclude: if your website can't make money without crappy ads, and nobody looks at your crappy ads because of advances in technology, then blaming that technology puts you squarely on the wrong side of history.<br />
<br />
A similar opinion I just found:<br />
https://snelling.io/on-ad-blocking</div>
Unknownnoreply@blogger.com4tag:blogger.com,1999:blog-6580443261471879069.post-56170043251130205272015-02-23T22:29:00.000-08:002015-02-23T22:29:12.420-08:00talk about facepalmslearning javascript: valuable<br />
<i>re</i>learning trigonometry: way valuable<br />
spending 2 hours trying to code Math.atan2 in plain javascript because you were too stubborn to spend 6 seconds searching for how to get that stupid angle at any quadrant of the unit circle: <b>priceless!</b><br />
<b><br /></b>
in related news, Math.atan2 is very useful (altho why its arguments are (deltaY, deltaX) and not vice versa eludes me), and <a href="http://pixijs.com/">pixi.js</a> is very fun.Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-6580443261471879069.post-87913660068649547802015-02-18T13:22:00.000-08:002015-02-18T13:22:06.321-08:00unexplainable 500 errors with flask and jquery ajax on google app engineI was trying to do something simple - get a preloaded partial from the server via ajax. my api call worked directly in the browser, worked when I ran it with $.get() from the console after the page was loaded, but was returning 500 during page load (as in, only failing when it was called as js running at after dom load). I could confirm that the correct flask view was running and succeeding using logging, but the 500 I was getting had NO responsetext, and nothing useful..<br />
<br />
After fighting and fruitless searching for 40 minutes, I disabled the flask debug toolbar. Voila! everything worked as expected.<br />
<br />
Moral of the story: F$@# the flask debug toolbar.<br />
<br />
facepalms: 8Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-6580443261471879069.post-73360921664097936012015-02-03T12:42:00.000-08:002015-02-03T16:33:17.922-08:00displayport / hdmi not recognized in ubuntu 14 when eighth inch audio jack is not plugged into sound system (lenovo t430s)yeah you read that title right. can you believe it? I fought with my computer for an hour early this morning, installed a super fresh kernel, played with intel drivers, but nothing wanted to make ubuntu 14 recognize the displayport. It was working the night before!<br />
<br />
finally gave up and went to put some music on. boom! screen turned on. what the hell? unplugged the eighth inch cable again. boom! screen turned off. also, the amplifier turned off and on each time I plugged or unplugged the cable. I'm still 99% baffled here but there's another clue - a buzz on the line that I can audibly hear thru the speakers when this particular sound system is connected, that also <i>changes</i> when I restart the computer.<br />
<br />
Note that when I plugged headphones into the eighth inch jack, the screen did <i>not</i> turn on (altho there may have been some signs of life briefly).<br />
<br />
So best guess (which is probably very wrong?) is that there's a loose connection somewhere in the laptop, and that the added power from my stereo thru the eighth inch connection (which should be flowing the other direction, but this <i>is</i> an input / output jack) is bridging the loose wire? hence the buzz?<br />
<br />
UPDATE 6:30pm same night:<br />
scratch the above theory - <a href="https://www.youtube.com/watch?v=gbIv7W7rhx4">new shit has come to light!</a><br />
when I moved my laptop to the living room and plugged the mini displayport into the projector, everything worked as always - without a sound system being plugged in. So then, being the good scientist that I am, I lugged one of my screens into the living room and plugged it into the displayport, and holy moly, it worked fine, no audio cord plugged in. So it's actually <i>proximity</i> to my crazy stereo system that's causing this issue. Unfortunately I don't know enough about sound systems and electronics to posit a complete theory, altho I bet this is enough for <i>someone</i> to be able to. My general idea now is that the magnetic field cast by the bass amplifier on the floor right next to the monitor is somehow involved.<br />
<br />
As before, if you have any ideas, I'd love to hear em.<br />
facepalms: 7.5 (would have been 10 but tempered by the sweet relief of seeing that screen turn on)Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-6580443261471879069.post-66117187368172302014-03-18T14:15:00.002-07:002014-03-18T14:15:45.869-07:00how to tell if an API works well before you write the integrationI connect with APIs for a living, or at least for part of a living. I do it all the time.<br />
Often the ops team will ask me - how long to do an integration with ESP X, or Analytics firm Y? short of actually doing the integration, it used to be hard to give a good answer to this question. Now, tho, I have the perfect heuristic - it's never failed me.<br />
<br />
Just take a look at the API docs. doesn't really matter what they say; if they're well formatted, easy to navigate, and well thought through, API integration will be a breeze. That simple.<br />
<br />
Something like this makes an easy heuristic:<br />
<br />
<ul>
<li>no API docs - refuse to do the job</li>
<li>API docs not available online but emailed to you in pdf form (or shared as gdoc): 40-50 hours</li>
<li>API docs in one monolithic html page: 30-40 hours (obviously all of this depends on depth of integration too)</li>
<li>docs have a formatted nav bar at left and some readable styling: 20-30 hours</li>
<li>docs have the above in addition to "web 3.0" style formatting, and maybe some helpful examples: 10-20 hours</li>
<li>docs have the above plus toggleable examples in different programming languages, maybe a link to wrapper libraries, are well written, and code blocks are formatted with special code block formatting: 1-10 hours.</li>
</ul>
<div>
you can use this, free of charge :)</div>
<div>
best part of it is that I taught the ops team how to do this, and now they know exactly what estimate I'll give before they even ask me. Pretty soon they won't even need me at all, what with services like <a href="http://zapier.com/">zapier</a>.</div>
Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-6580443261471879069.post-22049491860653971162014-02-04T09:27:00.000-08:002014-02-04T09:27:45.825-08:00for the recordthe python CSV package is not as useful as it could be. no unicode support, can't handle all sorts of useful things like switching out the line delimiter (unless I'm missing something), and frequently just falls over for no reason.<br />
<br />
I always try to use it, because modularity, and DRY, and yet I end up just writing up a quick csv parser myself for the job at hand like 60% of the time. Makes me think I should look for other CSV parsing packages, or just write my own.Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-6580443261471879069.post-66830252102954875082013-10-30T09:05:00.000-07:002013-10-30T09:05:09.640-07:00more python SUDS adviceEverybody knows the a SOAP API is a horrid thing to work with, even a well designed one. But everybody also knows that SOAP is going to be around for a while.<br />
Recently, while working with the ExactTarget SOAP API in Python, I figured out another helpful trick with SUDS, which is the main python SOAP wrapper.<br />
<br />
Let me preface this by saying that suds is not a great library; particularly, it was written by and for people from another era of web development and does not stand up to today's needs. Why doesn't somebody write a better one? because everyone who does modern web dev hates SOAP in the first place.<br />
<br />
But anyway, one annoying feature of suds is that when you create an object from its object factory, it always fills in all properties with an empty string, even those that are optional and have sane defaults. So essentially, it breaks all objects right out of the gate, forcing you to go through one by one and choose those sane defaults explicitly.<br />
<br />
However, there's another way - <b>for any properties that are giving you trouble in this fashion, just delete them after creating the object:</b><br />
del object.AnnoyingProperty<br />
<br />
Suds will then not send the property, allowing the API server to choose the sane default.<br />
<br />
facepalms: 6Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-6580443261471879069.post-57686392795602643772013-10-18T12:53:00.001-07:002013-10-18T12:53:38.635-07:00How to do get a good security audit for your startupRecently at SimpleRelevance we decided it was time for a security audit on our website and especially our dashboard. We have quite a bit of client information that we would never want to share with the world.<br />
<br />
Security audits are kind of like STD testing - even when you feel 100% fine, it's better safe than sorry.<br />
Unfortunately, also like STD testing, if you cheap out, you won't get tested for everything.<br />
<br />
So our problem was - how do we not get ripped off, while still ensuring a comprehensive audit - we're talking more than just simple pentests here. It's so hard to verify that you've gotten a good security audit, since the company can just come back and say "we didn't find any issues - your system is bulletproof!", and you can't prove that negative. In the end we did a lot of research and interviews and chose based on the data at hand. But since then I've thought of a fun way that might work even better.<br />
<br />
<br />
1) through research, find N>1 companies that you'd consider paying for research.<br />
2) bargain each one down (you were going to do this anyway, right? We ended up paying about 65% of the original ask for the company we chose).<br />
3) tell each one: "I'll make you a deal. I'll pay you full price on the test, if you agree to the stipulation that I'm making this same deal with another security company, and whoever finds fewer or less important security holes doesn't earn any money, and whoever finds the most gets the full price [plus 10% if you need to sweeten it]".<br />
4) Some of the companies are going to say no. Decide which company is most excited by the idea of the deal - they are clearly the winner, so congrats - you found the best one! Tell them none of the others would take the deal, so unfortunately you can't do it, but you'll still pay them the discounted rate you'd already agreed upon for a full audit, and you applaud their style.Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-6580443261471879069.post-86350904926728374572013-06-11T13:49:00.000-07:002013-06-11T13:49:05.815-07:00ipython and the django shell: strange scoping errorsthis is a minor issue that has annoying repercussions. on most versions of django, if you use ipython and start it with ./manage.py shell, you cannot define global variables and then use them in local functions. it gets ugly quick. more info here:<br />
<a href="https://github.com/ipython/ipython/issues/62">https://github.com/ipython/ipython/issues/62</a><br />
<br />
patch is here:<br />
<a href="https://github.com/django/django/pull/512/files">https://github.com/django/django/pull/512/files</a><br />
<br />
and it definitely fixes the issue.Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-6580443261471879069.post-73912499233643288062013-01-15T11:24:00.000-08:002013-01-15T11:28:51.962-08:00python httplib2 7.x and sslthis is not a completely new issue, and the internet helped me find the solution in about 5 minutes, but it was not very clear or obvious so I thought I'd write it down succinctly for posterity:<br />
<br />
Python's httplib2 package did not validate ssl certificates in versions < 7.0, and started to from 7.0 and up. Unfortunately, it ships with its own set of trusted ssl certificates which comprise only a medium-sized subset of all of your favorite sites' certs (for instance, wikipedia fails when pinged at https://en.wikipedia.org!). The error looks like this:<br />
<br />
<blockquote class="tr_bq">
<span style="background-color: white; font-family: arial, sans-serif; font-size: 13px;"> File "/home/deploy/.virtualenvs/</span><a href="http://sandbox.koaladeal.com/local/lib/python2.7/site-packages/httplib2/__init__.py" style="background-color: white; color: #1155cc; font-family: arial, sans-serif; font-size: 13px;" target="_blank">san<wbr></wbr>dbox.koaladeal.com/local/lib/<wbr></wbr>python2.7/site-packages/<wbr></wbr>httplib2/__init__.py</a><span style="background-color: white; font-family: arial, sans-serif; font-size: 13px;">", line 1597, in request</span><br />
<span style="background-color: white; font-family: arial, sans-serif; font-size: 13px;"> (response, content) = self._request(conn, authority, uri, request_uri, method, body, headers, redirections, cachekey)</span><br />
<br style="background-color: white; font-family: arial, sans-serif; font-size: 13px;" />
<span style="background-color: white; font-family: arial, sans-serif; font-size: 13px;"> File "/home/deploy/.virtualenvs/</span><a href="http://sandbox.koaladeal.com/local/lib/python2.7/site-packages/httplib2/__init__.py" style="background-color: white; color: #1155cc; font-family: arial, sans-serif; font-size: 13px;" target="_blank">san<wbr></wbr>dbox.koaladeal.com/local/lib/<wbr></wbr>python2.7/site-packages/<wbr></wbr>httplib2/__init__.py</a><span style="background-color: white; font-family: arial, sans-serif; font-size: 13px;">", line 1345, in _request</span><br />
<span style="background-color: white; font-family: arial, sans-serif; font-size: 13px;"> (response, content) = self._conn_request(conn, request_uri, method, body, headers)</span><br />
<br style="background-color: white; font-family: arial, sans-serif; font-size: 13px;" />
<span style="background-color: white; font-family: arial, sans-serif; font-size: 13px;"> File "/home/deploy/.virtualenvs/</span><a href="http://sandbox.koaladeal.com/local/lib/python2.7/site-packages/httplib2/__init__.py" style="background-color: white; color: #1155cc; font-family: arial, sans-serif; font-size: 13px;" target="_blank">san<wbr></wbr>dbox.koaladeal.com/local/lib/<wbr></wbr>python2.7/site-packages/<wbr></wbr>httplib2/__init__.py</a><span style="background-color: white; font-family: arial, sans-serif; font-size: 13px;">", line 1281, in _conn_request</span><br />
<span style="background-color: white; font-family: arial, sans-serif; font-size: 13px;"> conn.connect()</span><br />
<br style="background-color: white; font-family: arial, sans-serif; font-size: 13px;" />
<span style="background-color: white; font-family: arial, sans-serif; font-size: 13px;"> File "/home/deploy/.virtualenvs/</span><a href="http://sandbox.koaladeal.com/local/lib/python2.7/site-packages/httplib2/__init__.py" style="background-color: white; color: #1155cc; font-family: arial, sans-serif; font-size: 13px;" target="_blank">san<wbr></wbr>dbox.koaladeal.com/local/lib/<wbr></wbr>python2.7/site-packages/<wbr></wbr>httplib2/__init__.py</a><span style="background-color: white; font-family: arial, sans-serif; font-size: 13px;">", line 1036, in connect</span><br />
<span style="background-color: white; font-family: arial, sans-serif; font-size: 13px;"> raise SSLHandshakeError(e)</span><br />
<br style="background-color: white; font-family: arial, sans-serif; font-size: 13px;" />
<span style="background-color: white; font-family: arial, sans-serif; font-size: 13px;">SSLHandshakeError: [Errno 1] _ssl.c:504: error:14090086:SSL routines:SSL3_GET_SERVER_</span><wbr style="background-color: white; font-family: arial, sans-serif; font-size: 13px;"></wbr><span style="background-color: white; font-family: arial, sans-serif; font-size: 13px;">CERTIFICATE:certificate verify failed</span></blockquote>
<br />
So the solution is to use ubuntu's system default cert file, which lives at /etc/ssl/certs/ca-certificates.crt. I ended up overwriting the one that shipped with httplib2, which fixes the problem globally, but fails if we ever reinstall httplib2. Hence this blog post for posterity. If anyone knows of another way to globally install a new certificates file for httplib2 without changing the package itself, I'm all ears. I might just onboard the package to our project (but then I have to remember never to pip install it...).<br />
<br />
by the way, <a href="http://code.google.com/p/httplib2/issues/detail?id=202">this page</a> was most helpful.<br />
<br />
facepalms: 3Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-6580443261471879069.post-41143169448921328112012-11-20T13:39:00.002-08:002012-11-20T13:39:14.002-08:00how to not run your business<img alt="Inline image 1" src="https://mail.google.com/mail/ca/u/0/?ui=2&ik=6eef1dde5b&view=att&th=13b1fbffdfefd553&attid=0.1&disp=emb&realattid=ii_13b1fbf855ea6f47&zw&atsh=1" /><div>
<br /></div>
<div>
<br /></div>
<div>
tell me - if you're going to pay for hosting for a corporate wordpress blog, what's the chance you're <i>not </i>going to use your corporate email account and what's the chance you're <i>not</i> going to host the blog at your corporate url? in other words, for what business would this error message <i>not</i> come up?</div>
<div>
<br /></div>
<div>
None. Facepalms: 5 (horrifically hilarious, luckily not a problem I have to deal with today.)<br /><br /></div>
Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-6580443261471879069.post-28867401578113002052012-10-10T15:00:00.000-07:002013-01-15T11:24:55.800-08:00magento and the dreaded WebFault: Product not existsSometimes using SOAP with the Magento V2 API you may search for a product's image (by calling catalog_product_attribute_media.list with the sku as only argument) and receive this hilarious message:<br />
<br />
"Product not exists." <-- you can't make this stuff up!<br />
<br />
Hilarious but sad, because there's the image right there, laughing at you from the website, and yet your API doesn't have access.<br />
<br />
luckily there's an easy <a href="http://www.magentocommerce.com/boards/viewthread/196167/">fix</a>. in short: append a space to the end of the sku. done. psha. magento, we love to hate you.<br />
<br />
facepalms: 5<br />
<br />Unknownnoreply@blogger.com1tag:blogger.com,1999:blog-6580443261471879069.post-52771207910107707822012-10-08T08:20:00.000-07:002013-01-15T11:25:55.353-08:00celery + djcelery problem with virtualenv and virtualenvwrapper<br />
<br />
this is a tricky one - out of nowhere, production env boxes started failing at celery startup with:<br />
<br />
<br />
<br />
ImportError: cannot import name current_app<br />
when importing djcelery. Versions of celery were fine.<br />
It turns out if you import celery and run<br />
import celery.current_app you'll see the <a href="http://www.virtualenv.org/en/latest/news.html">real problem</a>, which is that the virtualenv binary is out of sync with the new python binary from a recent security update - specifically, os.urandom has been changed/removed.<br />
<br />
if you have virtualenvwrapper, and you let $ENV=YOUR_ENVIRONEMNT_NAME<br />
So the answer is:<br />
<br />
<br />
deactivate (in case an env is running)
<br />
cd ~/$ENV_HOME (.virtualenvs, for me)
<br />
rm $ENV/bin/python
<br />
virtualenv $ENV
<br />
<br />
<br />
this will rebuild your python binary with the correct python post-security fix, without losing any other packages. happy hacking!Unknownnoreply@blogger.com8tag:blogger.com,1999:blog-6580443261471879069.post-28566570485895532862012-07-08T09:55:00.002-07:002013-01-15T11:25:40.010-08:00redis vs rabbit with django celeryif you're planning on putting a django celery app into heavy production use, your message queue matters. At SimpleRelevance we used RabbitMQ for months, since it's probably the most famous of the available options.<br />
It took about 8 seconds to restart out celery workers, but that was fine. In general it was fairly reliable, and the way it routed tasks to different celery workers on different boxes often felt like magic.<br />
<br />
However, getting results of a task back using rabbit as a result backend was a different story - it often took <i>minutes</i>, even hanging the box it was on. And these weren't big results either.<br />
<br />
So for the record here, we switched to Redis. Not only is restarting about 3X faster, but as a results backend it also wins - no more hanging, and results come back as soon as they're ready. My sysops also tells me it was much easier to install and configure.<br />
boom.<br />
<br />
----<br />
update!<br />
actually it turns out redis starts to perform very badly when faced with a deep queue in our production environment. So the optimal setup for us turns out to be RabbitMQ for the queue handling, and Redis for the result backend.Unknownnoreply@blogger.com2tag:blogger.com,1999:blog-6580443261471879069.post-91970096157059303132012-07-04T07:47:00.000-07:002013-01-15T11:25:17.642-08:00"Error in service module" Ubuntu pam login failThis was a strange one. One of the newer computers at the lab where I moonlight, running Ubuntu 11.04 (yes, I probably should upgrade to the next LTS, I guess), suddenly stopped logging in. This was preceded by a pink screen and a bunch of errors about the harddrive (so they tell me; I wasn't there. oh, the lives of not IT folk - like living underwater).<br />
So now, whenever they logged in through the GUI, nothing happened - click the user, type the password, straight back to login screen.<br />
So I ctrl alt F1 to TTY1, and log in, and the only error I got was:<br />
"error in service module". Not helpful.<br />
Googling that was helpful in that it started to point the blame at PAM, or Pluggable Authentication Modules, which I now know way too much about.<br />
<br />
But anyway I ended up booting into recovery mode which got me past the login, and then overwriting the /etc/init/tty1.conf to skip login even when not in recovery mode following <a href="http://askubuntu.com/questions/8467/auto-login-to-console-as-root-no-xserver">this</a> advice, which allowed me to access the internet and external drives on the machine.<br />
<br />
Then I realized that ubuntu logs <i>everything</i>, and I checked /var/log/auth.log, which mentioned a bunch of missing files in /etc/pam.d/ and other fun places. All such fun directories were <b>empty</b>. very strange. So I copied over all of the files from another ubuntu machine into said directories, including a bunch of .so files, and what do you know, login worked. That simple.<br />
<br />
Moral of the story - read the logs when something goes wrong that you don't understand. Ubuntu writes a lot of logs, and there's a reason for that. I know I made it seem easy here but it probably took 2-3 hours to do all of the above. facepalms: 4Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-6580443261471879069.post-30788838443151057982012-05-31T13:20:00.001-07:002012-05-31T13:20:54.388-07:00another python suds tipIf you are having trouble creating nested XML for array objects like this:<br />
<blockquote class="tr_bq">
<configurations><br />
<configuration><br />
stuff<br />
</configuration><br />
</configurations></blockquote>
<br />
while using suds' Factory methods, you should try to create the struct from scratch, by passing whatever method is appropriate a list of dictionaries or whatever nested structure applies:<br />
<br />
<blockquote class="tr_bq">
[{'Configuration':obj},{'Configuration':obj}]</blockquote>
<br />
facepalms: a millionUnknownnoreply@blogger.com0tag:blogger.com,1999:blog-6580443261471879069.post-71065694930911203462012-05-31T12:07:00.001-07:002012-05-31T12:07:52.701-07:00suds empty tag issueI just posted a really nice Stack Overflow solution about this:<br />
<a href="http://stackoverflow.com/questions/9388180/suds-generates-empty-elements-how-to-remove-them">http://stackoverflow.com/questions/9388180/suds-generates-empty-elements-how-to-remove-them</a><br />
The general point is that although Suds is an awesome python library that lets you connect to SOAP clients (I mean, really? welcome to the 21st century, people) with <i>relative</i> ease, it has the bad habit of adding empty tags for optional properties of objects. This tends to confuse (poorly written) API endpoints.<br />
<br />
facepalms: 6Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-6580443261471879069.post-36928785977576673462012-03-30T13:23:00.002-07:002012-03-30T13:30:23.964-07:00django celery with remote worker nodesI set up rabbitmq and celery. Plenty of good tutorials online about how to do that. Then I wanted a bunch of different linode boxen all running the same django project, with the following setup:<div style="font-style: normal; "><br /></div><div style="font-style: normal; ">1 server running mysql and nothing else</div><div style="font-style: normal; ">1 server running nginx serving http requests and routing tasks to rabbitmq / celery</div><div style="font-style: normal; ">1 server running rabbitmq and celery and django</div><div style="font-style: normal; ">N boxes running django and celery</div><div style="font-style: normal; "><br /></div><div style="font-style: normal; ">Turns out, it's easy! </div><div><ul style="font-style: normal; "><li><span style="font-size: 100%; ">All of the above hook into the mysql server by setting the HOST and PORT settings in the django settings. </span></li><li><span style="font-size: 100%; ">Each slave celery box uses an environment variable to take care of any individual settings it might need, but in general each of them uses django-celery's BROKER_HOST and BROKER_PORT options to connect to the rabbitmq server.</span></li><li><span style="font-size: 100%; ">using fabric makes deploying code to all of them fairly simple</span></li></ul><div>Believe it or not, rabbitmq effortlessly figures out who's got a free worker between all of your boxes and <i>just does it</i>.</div></div>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-6580443261471879069.post-43771896008785211302012-02-22T13:22:00.006-08:002012-03-30T13:33:40.532-07:00app engine and appcfg import errors<h5 style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding-top: 0.4em; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; border-top-width: 0px; border-right-width: 0px; border-bottom-width: 0px; border-left-width: 0px; border-style: initial; border-color: initial; border-image: initial; font-size: 1em; font-weight: normal; "><span style="font-style: normal; margin-top: 0px; margin-right: 0.3em; margin-bottom: 0px; margin-left: 0.3em; "><br />If you get an error that looks like the below, be sure that you are importing the correct version of django</span><span style="font-size: 1em; ">(which is to say, that if you are using the builtin version, be sure you are not also importing another version of django from your sitepackages). Appcfg automatically follows all symlinks and packages up everything on your pythonpath, <i>and the import order in production is different (actually reversed!) from that in development.</i> </span></h5><div style="font-style: normal; "><span style="font-size: 1em; "><br /></span></div><div style="font-style: normal; "><span style="font-size: 1em; ">I solved this problem by deactivating my virtualenv whenever I deploy.</span></div><div style="font-style: normal; "><span style="font-size: 1em; ">facepalms: 8 (3 hours of nonsense for a 1-line fix!)</span></div><div style="font-style: normal; "><span style="font-size: 1em; "><br /></span></div><span>Traceback (most recent call last): File "/base/python27_runtime/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 193, in Handle result = handler(self._environ, self._StartResponse) File "/base/python27_runtime/python27_lib/versions/third_party/django-1.2/django/core/handlers/wsgi.py", line 232, in __call__ self.load_middleware() File "/base/python27_runtime/python27_lib/versions/third_party/django-1.2/django/core/handlers/base.py", line 40, in load_middleware mod = import_module(mw_module) File "/base/python27_runtime/python27_lib/versions/third_party/django-1.2/django/utils/importlib.py", line 35, in import_module __import__(name) File "/base/python27_runtime/python27_lib/versions/third_party/django-1.2/django/middleware/transaction.py", line 1, in <module> from django.db import transaction File "/base/python27_runtime/python27_lib/versions/third_party/django-1.2/django/db/__init__.py", line 77, in <module> connection = connections[DEFAULT_DB_ALIAS] File "/base/python27_runtime/python27_lib/versions/third_party/django-1.2/django/db/utils.py", line 92, in __getitem__ backend = load_backend(db['ENGINE']) File "/base/python27_runtime/python27_lib/versions/third_party/django-1.2/django/db/utils.py", line 50, in load_backend raise ImproperlyConfigured(error_msg) ImproperlyConfigured: 'google.appengine.ext.django.backends.rdbms' isn't an available database backend. Try using django.db.backends.XXX, where XXX is one of: 'dummy', 'mysql', 'oracle', 'postgresql', 'postgresql_psycopg2', 'sqlite3' Error was: cannot import name backends</module></module></span>Unknownnoreply@blogger.com0