compiling and then transpiling sveltejs: babeljs, webpack, and a safari 9 problem

Been 2 years since my last post here!
just had a very fun deep dive into this annoying bug that was only affecting this one old browser.

I wrote it up here:
https://stackoverflow.com/questions/57019417/how-to-transpile-compiled-svelte-with-babel-for-safari-9/57019418#57019418

Anyway if you haven't checked out https://svelte.dev I highly encourage it - it's faster, easier, and cooler than reactjs.

This bug took me 5 hours to find and fix, but it was a fun and interesting sleuthing session the entire time that has left me feeling proud of my abilities, so I'm gonna go with facepalms: only 2 on this one.

mobile touch gotcha

I once made an angular firebase demo site called Fridge Magnets With Friends.
You can move magnets around on a virtual fridge. This involves a lot of dragging elements.
One day mobile drag stopped working, even tho I employed the clever jquery ui touch punch.

I spent hours switching the whole thing to use ngDraggable, because their mobile demo did work in mobile. Altho using ngDraggable required me to take out jquery (actually it didn't, but some forums said that would help make it work in mobile) - so I spent a while redoing the angular app without jquery.
Moment of truth: no luck.

Finally I hijacked the browser with a little shim code:

        $document.on('touchstart', function(e){

            console.log(e)

                            });

Because for the life of me I couldn't figure out why my elements weren't receiving the touchstart event in mobile.
This way I could see what element the e event was pointing to.
Turns out my footer CSS was very improperly done and covering the entire screen, so the event for touchstart, which has to have a stoppropogate() for other reasons in mobile only, never made it to the dragable elements.
Took out the footer and both solutions started working; however, jquery-ui + touch-punch still worked better than ngDraggable, so in the end, 4 hours of work (work that was, admittedly, kind of fun) got reduced to commenting out a couple lines of html. At least I understand mobile touch events now.

facepalms: 9.5

talk about facepalms

learning javascript: valuable
relearning trigonometry: way valuable
spending 2 hours trying to code Math.atan2 in plain javascript because you were too stubborn to spend 6 seconds searching for how to get that stupid angle at any quadrant of the unit circle: priceless!

in related news, Math.atan2 is very useful (altho why its arguments are (deltaY, deltaX) and not vice versa eludes me), and pixi.js is very fun.

unexplainable 500 errors with flask and jquery ajax on google app engine

I was trying to do something simple - get a preloaded partial from the server via ajax. my api call worked directly in the browser, worked when I ran it with $.get() from the console after the page was loaded, but was returning 500 during page load (as in, only failing when it was called as js running at after dom load). I could confirm that the correct flask view was running and succeeding using logging, but the 500 I was getting had NO responsetext, and nothing useful..

After fighting and fruitless searching for 40 minutes, I disabled the flask debug toolbar. Voila! everything worked as expected.

Moral of the story: F$@# the flask debug toolbar.

facepalms: 8

displayport / hdmi not recognized in ubuntu 14 when eighth inch audio jack is not plugged into sound system (lenovo t430s)

yeah you read that title right. can you believe it? I fought with my computer for an hour early this morning, installed a super fresh kernel, played with intel drivers, but nothing wanted to make ubuntu 14 recognize the displayport. It was working the night before!

finally gave up and went to put some music on. boom! screen turned on. what the hell? unplugged the eighth inch cable again. boom! screen turned off. also, the amplifier turned off and on each time I plugged or unplugged the cable. I'm still 99% baffled here but there's another clue - a buzz on the line that I can audibly hear thru the speakers when this particular sound system is connected, that also changes when I restart the computer.

Note that when I plugged headphones into the eighth inch jack, the screen did not turn on (altho there may have been some signs of life briefly).

So best guess (which is probably very wrong?) is that there's a loose connection somewhere in the laptop, and that the added power from my stereo thru the eighth inch connection (which should be flowing the other direction, but this is an input / output jack) is bridging the loose wire? hence the buzz?

UPDATE 6:30pm same night:
scratch the above theory - new shit has come to light!
when I moved my laptop to the living room and plugged the mini displayport into the projector, everything worked as always - without a sound system being plugged in. So then, being the good scientist that I am, I lugged one of my screens into the living room and plugged it into the displayport, and holy moly, it worked fine, no audio cord plugged in. So it's actually proximity to my crazy stereo system that's causing this issue. Unfortunately I don't know enough about sound systems and electronics to posit a complete theory, altho I bet this is enough for someone to be able to. My general idea now is that the magnetic field cast by the bass amplifier on the floor right next to the monitor is somehow involved.

As before, if you have any ideas, I'd love to hear em.
facepalms: 7.5 (would have been 10 but tempered by the sweet relief of seeing that screen turn on)

more python SUDS advice

Everybody knows the a SOAP API is a horrid thing to work with, even a well designed one. But everybody also knows that SOAP is going to be around for a while.
Recently, while working with the ExactTarget SOAP API in Python, I figured out another helpful trick with SUDS, which is the main python SOAP wrapper.

Let me preface this by saying that suds is not a great library; particularly, it was written by and for people from another era of web development and does not stand up to today's needs. Why doesn't somebody write a better one? because everyone who does modern web dev hates SOAP in the first place.

But anyway, one annoying feature of suds is that when you create an object from its object factory, it always fills in all properties with an empty string, even those that are optional and have sane defaults. So essentially, it breaks all objects right out of the gate, forcing you to go through one by one and choose those sane defaults explicitly.

However, there's another way - for any properties that are giving you trouble in this fashion, just delete them after creating the object:
del object.AnnoyingProperty

Suds will then not send the property, allowing the API server to choose the sane default.

facepalms: 6

python httplib2 7.x and ssl

this is not a completely new issue, and the internet helped me find the solution in about 5 minutes, but it was not very clear or obvious so I thought I'd write it down succinctly for posterity:

Python's httplib2 package did not validate ssl certificates in versions < 7.0, and started to from 7.0 and up. Unfortunately, it ships with its own set of trusted ssl certificates which comprise only a medium-sized subset of all of your favorite sites' certs (for instance, wikipedia fails when pinged at https://en.wikipedia.org!). The error looks like this:

  File "/home/deploy/.virtualenvs/sandbox.koaladeal.com/local/lib/python2.7/site-packages/httplib2/__init__.py", line 1597, in request
    (response, content) = self._request(conn, authority, uri, request_uri, method, body, headers, redirections, cachekey)

  File "/home/deploy/.virtualenvs/sandbox.koaladeal.com/local/lib/python2.7/site-packages/httplib2/__init__.py", line 1345, in _request
    (response, content) = self._conn_request(conn, request_uri, method, body, headers)

  File "/home/deploy/.virtualenvs/sandbox.koaladeal.com/local/lib/python2.7/site-packages/httplib2/__init__.py", line 1281, in _conn_request
    conn.connect()

  File "/home/deploy/.virtualenvs/sandbox.koaladeal.com/local/lib/python2.7/site-packages/httplib2/__init__.py", line 1036, in connect
    raise SSLHandshakeError(e)

SSLHandshakeError: [Errno 1] _ssl.c:504: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed

 So the solution is to use ubuntu's system default cert file, which lives at /etc/ssl/certs/ca-certificates.crt. I ended up overwriting the one that shipped with httplib2, which fixes the problem globally, but fails if we ever reinstall httplib2. Hence this blog post for posterity. If anyone knows of another way to globally install a new certificates file for httplib2 without changing the package itself, I'm all ears. I might just onboard the package to our project (but then I have to remember never to pip install it...).

by the way, this page was most helpful.

facepalms: 3

magento and the dreaded WebFault: Product not exists

Sometimes using SOAP with the Magento V2 API you may search for a product's image (by calling catalog_product_attribute_media.list with the sku as only argument) and receive this hilarious message:

"Product not exists." <-- you can't make this stuff up!

Hilarious but sad, because there's the image right there, laughing at you from the website, and yet your API doesn't have access.

luckily there's an easy fix. in short: append a space to the end of the sku. done. psha. magento, we love to hate you.

facepalms: 5

"Error in service module" Ubuntu pam login fail

This was a strange one. One of the newer computers at the lab where I moonlight, running Ubuntu 11.04 (yes, I probably should upgrade to the next LTS, I guess), suddenly stopped logging in. This was preceded by a pink screen and a bunch of errors about the harddrive (so they tell me; I wasn't there. oh, the lives of not IT folk - like living underwater).
So now, whenever they logged in through the GUI, nothing happened - click the user, type the password, straight back to login screen.
So I ctrl alt F1 to TTY1, and log in, and the only error I got was:
"error in service module". Not helpful.
Googling that was helpful in that it started to point the blame at PAM, or Pluggable Authentication Modules, which I now know way too much about.

But anyway I ended up booting into recovery mode which got me past the login, and then overwriting the /etc/init/tty1.conf to skip login even when not in recovery mode following this advice, which allowed me to access the internet and external drives on the machine.

Then I realized that ubuntu logs everything, and I checked /var/log/auth.log, which mentioned a bunch of missing files in /etc/pam.d/ and other fun places. All such fun directories were empty. very strange. So I copied over all of the files from another ubuntu machine into said directories, including a bunch of .so files, and what do you know, login worked. That simple.

Moral of the story - read the logs when something goes wrong that you don't understand. Ubuntu writes a lot of logs, and there's a reason for that. I know I made it seem easy here but it probably took 2-3 hours to do all of the above. facepalms: 4

app engine and datastore keys

say you know the key name's string of a Model instance and you want to fetch it from the datastore. It goes like this:

Model.get(db.Key.from_path("Model",key_name))

why so complicated? why can't you just Model.get(key_name)? Couldn't tell you, but was getting inexplicable errors that way.

facepalms: 4

tricks for constantcontact oauth2

Just so you know, you have to specify a redirect url for both legs of OAuth2 integration with ConstantContact. And it has to be the same exact url. Even tho the second (POST) leg probably won't end up redirecting anywhere.

Not only that, it has to be the exact same as the oauth_callback setting for the api key you're using. Where do you set this? You sign in to the community portal at constant contact, click the "api keys" tab, and then click on the api key itself, which is a link which takes you to its management page. With no explanation on this, it took me about 20 minutes to figure out, so hopefully this helps somebody else.

Oh - and if you're still getting uri-mismatch errors during this process? remember that the callback has to be https, for real - you need an ssl cert.

facepalms: 4

django-celery with virtualenv

this page has some great tips for daemonizing celery for django. They even have a section for django under a virtualenv.

But, just remember that if you have any environment variables that you export before running django, that your django settings rely on to load correctly, you should probably export them in your daemon script. Unfortunately the django-celery documents don't know about these environment variables (I'm talking about you, export ENV='pro'), so it's up to you to stick em in there.

Furthermore, consider restarting the celery daemon whenever you deploy. I hear supervisord makes this easy; I'm going to do that next.

facepalms: 2

email attachments with django postmark

A quick fix - and there are some pull requests for this on bitbucket, so feel free to look for them, but you can do it yourself -

in postmark/backends.py, attachments are expected to be in dictionary form:

{'Name':XXXX,'Content':XXXX,'ContentType':XXXX}

however, the default django core email attachment method wants them as a tuple. Make your own patch if you want, it's pretty straightforward. Postmark's error messages are fairly useless.

facepalms: 3

a quick gripe

the worst thing about the official google reader app for android (besides the annoying new swipe-based UI) is that there's no way to see alt text on images (think XKCD).

google gdata oauth

there's a great big honkin awesome library for python called gdata - it lets you interface with a ton of their APIs, including spreadsheets and docs. We use it internally at SimpleRelevance to write all sorts of logging directly to shared spreadsheets - a big process will finish and have reams of data to share, but rather than parse out a flatfile every time, I wrote a wrapper that takes the data and spits it into a beautifully formatted google spreadsheet. From there, charts and such are easy.

We also now use it externally - clients can log in and authenticate through oauth with google, and then we can write their predictions to a spreadsheet in their own account.

You probably know about oauth - it's nice because the client never has to supply us with any login credentials - the whole thing is very secure. Unfortunately, it was a little painful to set up. Like, 1 hour of productive work and 2 hours of fighting with stupid. Why?

I'll tell you why.

1. There are a lot of outdated tutorials.
2. The gdata plugin, while awesome, has tons of legacy code and 2 completely different and mostly working ways of doing everything.
3. There are tutorials for the old path, tutorials for the new, and tutorials that mix the two.
4. This page has the most beautiful, well-written, cogent, perfect, comprehensive example of how to set up oauth with gdata. Unfortunately it gets confusing in a crucial bit at the end.

Although it's great, check it out. It has examples every step of the way in 4 different languages, for both gdata paths. That's 8 examples every step of the way (and you know that oauth2 is a 3 step process - that's around 24 pieces of code).

It's actually spot on all the way through to the end. The confusing part comes when you have to exchange your oauth token for the longterm access token - the thing that actually authenticates and lets you access stuff.

The tutorial has this line:

access_token = client.UpgradeToOAuthAccessToken()  # calls SetOAuthToken() for you

But I had trouble. Frankly, I messed up. But I couldn't get it to work until I did this:

client.UpgradeToOAuthAccessToken()
access_token = client.token_store.find_token(oauth_token.scopes[0])

It turns out that the first line actually does returns the access_token, so the google tutorial is correct. This was fixed some time in the last something or other. Used to be you had to use that second line to get it. Older tutorials don't reflect this change. I was getting tired by then. I missed the boat. Cue frustration.

Then, I my second point of confusion: the request token key and secret (leg 1) are the same as the oauth token key and secret (leg 2) but the access token key and secret are totally different and new. In retrospect, this makes perfect sense (from a security POV), but at the time I was baffled. Don't try to upgrade your oauth token and then save its key and secret to authenticate with. It won't work.

The google tutorial tells you to save it but leaves it to you to figure out how to use the access token object. Really, it's easy:

save access_token.key

save access_token.secret.

You can use the session, you can use a database, you can use a session stored in your database, whatever you want! I think the access token lasts for a while.

Anyway, oauth is hard, but I'm really getting the hang of it. Feel free to email me if you need help with this one.

facepalms: 4.5.

django multidb support for admin site by subdomain

usecase: the client logs in to the admin site by going to clientname.sitename.com/client_admin.

Why? because it's nice to use the django admin site. Because we want the client to only access the client's database.

How?

first off, setup your DNS to forward *.sitename to your server. Than in your server config (I use nginx), send *.sitename to uwsgi.

(note: if you're not using uwsgi with django here, you'll have to use your imagination).

In your wsgi script (you know, the one that launches django for each incoming request?), put something like this:

class RequestWSGIHandler(django.core.handlers.wsgi.WSGIHandler):

def get_response(self,request):

domain = request.__dict__['META']['HTTP_HOST']

domain = domain.split('.')

if len(domain) > 3:

db_name = domain[0]

print "caught domain %s"%db_name

os.environ['SITE_NAME'] = db_name

return super(RequestWSGIHandler,self).get_response(request)

# Lastly, load handler.

# application = django.core.handlers.wsgi.WSGIHandler()

application = RequestWSGIHandler()

Yes, it's kind of quick and dirty. I've subclassed the django wsgi handler to intercept every request, grab the subdomain if there is one, and set it in the OS env. Then using the (happily outdated) python 2.6* super() syntax, I call the standard wsgi handler.

Then when you get to settings.py, you can grab that env variable if it's there and save it to your settings object. It's now available when you need it.

For instance!!!, you can subclass admin.ModelAdmin to support multiple databases (see here). In the linked example (you have to scroll halfway down, they basically overwrite the key class methods and add in "using=new database" to all db queries) the new database is static, but since you now have a dynamic database name in settings.whatever you called it, you can pick your extra database access name on the fly. hurrah.

A caveat:

(this is a bit scary...) in the django documentation, the good django devs say to always import settings from django.conf. I wonder why? Maybe to avoid the same multithread concurrency issues I very briefly glossed over in my last post. Who knows. Problem is, if you do that with the above example, some of your calls (though not all) won't pick up your dynamic subdomain-influenced database. Why the hell not? Beats me. I have "from django.conf import settings" everywhere in that project except that file, for which I use "import settings".

and as a further caveat, keep in mind that if you import the settings from django.conf in a file that's loaded before admin.py, or wherever you want the real dynamic settings, it won't work!

I didn't really go into it, but this is a 4.5 facepalm issue right here.

holy sweet victory

Just figured out something that's been bugging me nonstop for weeks, which has no documentation online: the jquery plugin fancybox doesn't like to have its stuff overwritten, which is to say, you shouldn't define something like this:

var fancy_box_options = fancy_box_defaults;
fancy_box_options.type = 'iframe';
fancy_box_options.width = 696;
fancy_box_options.height = 499;
fancy_box_options.scrolling = 'no';
fancy_box_options["onStart"] = function(link){
var new_link = $.param.querystring($(link).attr("href"), "gmail=" + $("#gmail").val());
$(link).attr("href", new_link);
};

$("a.iframe").fancybox(fancy_box_options);
$("a.iframe_defaults").fancybox();

In one function, and then soon after define:

<% if current_user && @popup_help %>
var fancy_box_options = fancy_box_defaults;
fancy_box_options.type = 'iframe';
fancy_box_options.width = 696;
fancy_box_options.height = 499;
fancy_box_options.scrolling = 'no';

fancy_box_options.href = '<%= popup_help_path(:gmail => current_user.email) %>';
$.fancybox(fancy_box_options);

in another. You'll get a wholly unhelpful error about href being undefined.

All better now -
#facepalms only 4, because I already had a lot of trouble with this previously, I'm over it now, and the sweet release of figuring it out is worth so, so much.

ubuntu problems

so for the last while my wireless card has been acting up _sometimes_, on certain wireless networks but not others. I finally got a clue as to what might be going on: wireless N. It's an N card, but the driver/kernel stuff going on in ubuntu has a known problem with certain wireless cards.
Edited my modprobe.d/options.conf with the line:

options iwlagn 11n_disable=1 11n_disable50=1

and then (hopefully won't have to restart) ran:

sudo rmmod iwlagn
sudo modprobe iwlagn

And that seems to have improved things. So far.

#facepalms: like a lot. Until now. Hopefully.

rails fields_for, object sorting

So what I needed to do was select a subset of one model which happened to have a key into another, sort it by an attribute of the other model, and display a list of the results in a form.
here's the query to find, filter, join, and order:

@user_categories = Category.joins(:user_categories).where('user_categories.user_id' => current_user.id).order("name asc")

and here's the bit of the form:

<%= f.fields_for :user_categories, @user_categories do |user_category| %>

and then to access the objects:

<%= user_category.label :name, user_category.object.name %>

Things to notice:
the fields_for uses a parameter :user_categories which references the name of the model, followed by the object which is actually a list of objects of type Category, so I'm assuming this is to let rails know how to treat it. Also, note that to reference the actual objects, you have to use model_class.object.method. the model_class is actually some sort of form builder helper before the call to .object.

This only took an hour to figure out - #facepalms = 3.

Rails 3 Unobtrusive Javascript is Unintuitve

So what I wanted to do was link from a partial view to a controller method through a route which took a title parameter and from the controller method run a javascript file in order to append content to a div with a unique name defined by the title param.

In django it's pretty straightforward, but ruby on rails 3 had me scratching my head for a good long while.

Here's the solution, or at least the parts that were difficult:

the link:

<%= link_to "Comments", show_comments_path( :format => 'js', :comment_id=> session[:id] ), :remote => true %>

This whole :remote => true business tells rails 3 that this is an ajax call. The :format param does NOT get passed into the url, but DOES render the response as html, most of the time. Which is to say, rails will run the .js.erb file with the name matching your route.

Here's the route:

match 'show_comments/:comment_id', :to => 'deals#show_comments', :as => :show_comments

So first rails runs the show_comments method from the appropriate controller, which you can use to set state variables for the javascript. Then, whatever is in show_comments.js.erb will be run, and the current page will NOT be reloaded or redirected. No partial needed. No view needed. The controlling method will do all of this as the default return, so no return or render needed either, only the .js.erb file which has your prototype or jquery calls, or whathaveyou.

facepalms: 4 (steady, slow, only slightly infuriating progress)