Wednesday, May 13, 2015

How Shift is Using Depicted (dpxdt) and Why You Should Too

I've been meaning to write a post about how we are using Depicted (aka dpxdt) at Shift for awhile now. Since it just saved my arse from pushing a broken release to production I think now is the time for me to both pay homage and share how we're using it in case anyone else might find it useful.

But first .. let me share with you how Depicted saved me today! I'm working on a major refactor change for Shift's URLs, and I expect zero visual change as far as the user is concerned. So I'm intentionally not going to update the URLs in our Depicted config because I expect that everything should be properly redirected. So I pushed my changes to our production server, which can serve both my new version and our current version concurrently (thanks App Engine!). Then I initiated a dpxdt run like so (I'm leaving out some unnecessary args here):

dpxdt/diff_my_urls.py \
  --test_host "http://my-changes.server.com" \ 
  --ref_host "http://driveshift.com"

Among a few other gems, I got the following diffs:


In this case it's pretty obvious that I forgot to add a redirect for an URL. Whoops! Good thing our users never got this 404 page.



And here in this second diff it's obvious that on of our AJAX requests which loads data about our Car Enthusiasts returned an empty list. I was able to very quickly figure out my bug, and write the unit test for it.









In both of these cases I was able to easily amend my change before I'd already pushed to production. If I had pushed to production I would have had to roll it back and re-push. This is a huge developer time saver and of course provides anyone on the team with insight about how any given change will impact users.

Brett Slatkin, the author of Depicted, describes it as "the ultimate, automated end-to-end test." Depicted allows you to compare screenshots for a new release against screenshots from a "golden" release to catch any visual changes - ideally, before you make the new version live. If you're not using visual diff yet to test your pushes you should be.

As compared with other integration testing tools like Selenium or Casper, it's super easy to debug the results of a Depicted visual diff. Visual diff is also better representation of a bug in the User Experience of your site - it's precisely what your customers see, as opposed to testing state about the underlying DOM.

Adding a Depicted test is also WAY easier than writing another Casper or Selenium test - Depicted uses a YAML configuration file, which we at Shift generate via python script at the end of our CircleCI build. While YAML allows you to do "mix-ins" we wanted to optimize our config to be super easy to add an URL to and also wanted to bake in some special functionality. Ours base config looks like this:


build_configs:
  Desktop:
    viewportSize:
      width: 1280
      height: 1024
  Tablet:
    viewportSize:
      width: 768
      height: 1024
  Phone:
    viewportSize:
      width: 480
      height: 1024

# It can take a while for all the assets on our page to load.
resourceTimeoutMs: 60000

resourcesToIgnore:
  - mixpanel
  - adroll
  - adnxs
  - doubleclick
  - twitter.com/oct.js
  - perfectaudience
  - getsentry
  - googleadservices
  # Because phantomjs is a little peculiar about webfonts, let's not bother.
  - fonts
  - cloud.typography.com
  - vimeo

# Attempt to deal with inconsistent font rendering leading to false positives.
injectCss: |
  html > body,
  html > body * {
    font-family: Arial, sans-serif !important;
  }

urls:
  404:
    url: /404.html

  500:
    url: /500.html

  About:
    url: /about
    # Because we tend to get false negatives due to the fade-in
    # effect from the G Maps API, we'll just hide the map before 
    # the screenshot.
    injectCss: |
      #about-page .static-map {
        visibility: hidden
      }

  Buy:
    url: /buy

  Sell:
    url: /sell

  SF Cars:
    url: /cars/san-francisco

urls_staging:
  Car Tracker:
    url: /my-car/some_key

urls_production:
  Car Tracker:
    url: /my-car/some_other_key

You can tell from that config that it supports the idea that we generate different configs for staging and production, where our datastores contain different entities.

There are also a few scenarios you might notice in our config with regards to Google Maps and font rendering. Fonts are supposed to be better with phantomjs2 so we'll see if we can nuke the font overrides at some point.

We still need to set up some golden data on our servers to guarantee we can test screenshots with stable data, and we also want to make every change to our master branch do a rolling-forward-golden test on our staging server so that it might be more obvious how any change affects master, visually (whether it's a production push or not).

There's still lots to do!

Depicted is amazing. I will make the no-duh prediction here that someone will productize a visual diff service in the next year in the same way that Sentry and NewRelic have done for visualizing and alerting developers on code-level exceptions. Until then, we're really happy deploying our docker VM to Google App Engine and running our own Depicted server.

Depicted has changed how I write code and how I think about production. Production is the best place to do integration testing (ideally before any new version goes live =). There's an old saying about how users don't care about our code, they care about what they see and experience - Depicted reinforces this all the way through a change - I don't care about lines-of-code in a Changelist nearly as much as I care about visual outcome. How many times have you asked someone to send you a screenshot of their changelist or to show you a changelist running on their dev server before you can totally understand it? Depicted obviates the need for this and ultimately can make the code review process much more sane. Imagine if every code review automatically included the screenshot diffs produced by a change? We're going to get there at Shift soon, and I hope that this practice becomes commonplace.

P.S. Thanks to Mikey Lintz for noticing that dpxdt = Delta Pixels Delta Time







Tuesday, June 17, 2014

Some GOTCHAs I've noticed with React

  • Facebook's React has brings render diffing to us mortals, and it's something I'm very excited about. While I was working at Twist, we did a lot of profiling of render performance of the webview - and our general conclusions were that template rendering in JS wasn't a speed killer, but DOM manipulation was. So anything to prevent accidental or unnecessary parent-level changes should be pretty game-changing for performance.

    Since I'm learning it I'm coming across a few things that weren't immediately obvious to me, so I'm just going to share them here in case anyone else comes across these as well.

    • render needs to return a single node - aka wrap your elements in a div. When you don't you get a rather strange looking Parse Error: unexpected identifier. I asked a coworker before "fixing" this by wrapping my template code in a single div. Seems like something JSX could just do for you if it detects this.
    • Text strings are wrapped in spans. I have  and React adds a Some text. This just happens to break some code I was using to call preventDefault or not on form elements when turning "click" into "tap".
    • SublimeText's validation plugin hates JSX in file.js. So I'm just using file.jsx.
    • I've done something stupid in my JSX file before such that it parses but doesn't end up executing and I didn't get any sort of error. Oh yeah, it's when I forgot the @jsx React.DOM annotation above one of my objects. Maybe it's for the whole file and not per-module..

    All in all, I'll say that I'm really digging things about React. I'm doing an integration with Backbone, so I'm still using it for models and routing. It's a pretty natural substitute for Backbone's relatively weak View class. And I've never been happy with the mustache/hogan template integration really. The communication from React back and forth to the data layer is still something I'd like to see improve. I think that Backbone's storage/API hooks are a little clunky. I'd prefer something more like localStorage+Parse|Meteor for this bit.

Monday, October 15, 2012

Android browser + position:fixed + maps = pain

Here are two test cases that have made me cringe lately:

You'll need to try it on the Android browser.

This one is with Bing Maps:
http://urloritdidnthappen.appspot.com/render/254001

and this one with Google:
http://urloritdidnthappen.appspot.com/render/288001

See the big red rectangle with position:absolute hovering on the map?

Notice how with position: fixed in the header that thing disappears?

I suspect some sort of z-index tomfoolery going on, which probably doesn't bode super well for the performance of position: fixed on Android. We can only hope for the day when ChromeWebView becomes available to us HTML5 app developers.

Sunday, June 24, 2012

Velocity 2012!

Velocity Conference 2012

This year's Velocity conference schedule looks excellent, with several repeat speakers from years past, but also lots of new ones. I'll be moderating a panel on Wednesday at 5:20pm titled JavaScript Performance MythBusters, via JSPerf as well as a Browserscope lightning demo around 10:30am on Wednesday.

If you're there and want to say hi, shoot me an email at elsigh at gmail. =)

Here are the links to the slides:

Wednesday, November 9, 2011

Hide the URL Bar in Mobile Safari - Fast!

Damn you, 60 pixel URL bar.
As a web developer who's recently made the transition to working more on mobile, one of the APIs that I *really* wish mobile browser vendors would provide is one to hide the URL bar. At the moment I'm most concerned with our iOS users and reclaiming that 60 pixels at the top:



There are a lot of exhaustive and good posts about the hack to hide the navigation bar - using window.scrollTo(0, 0).

One thing that kept tripping me up in my own app was the fact that it's an absolutely positioned, full-screen web app - in other words, content did not naturally flow past the bottom of the viewport, so scrollTo wouldn't hide the bar unless I made the app the size of the screen minus the url bar and I didn't really want to set the body height in script after loading like some of the other examples I saw and which I found to be pretty brittle - client.innerHeight eat your heart out.

The other hassle I found was that event though I could get text on the screen and was deferring all scripts to the bottom of the page, there was no way to get the stupid bar to hide until the arbitrary body onload event finally hit.

Make BODY onload fire ASAP.
Ultimately my fix involved removing all the SCRIPT or IMG tags from the DOM initially, in other words anything that might prevent body onload from firing immediately. Now there's a single body onload listener which fires super fast and hides the navigation bar and *then* it uses LABjs to load/run any scripts. The following bit of CSS + SCRIPT ensures that the navigation bar goes away like lightning:

CSS:

/* Portrait mode */
@media screen and (max-width: 320px) {
  .container {
    min-height: 416px;  /* viewport 356px + 60px navigation bar */
  }
}
/* Landscape mode */
@media screen and (min-width: 321px) {
  .container {
    min-height: 268;  /* viewport 208px + 60px navigation bar */
  }
}
SCRIPT:
body.onload = function() {
  window.setTimeout(function() {
    window.scrollTo(0, 0);
  }, 0);
  window.setTimeout(function() {
    $LAB.setGlobalDefaults({
      'AlwaysPreserveOrder': true
    });
    $LAB.runQueue();
    {% if is_production %}
      $LAB.script('https://ssl.google-analytics.com/ga.js');
    {% endif %}
  }, 0);
}; 

My house, my rules, I win.
There are lots of good reasons to use a script loader if you have multiple scripts, but for my production app I have only one. Maybe the 5k hit of LABjs is unnecessary in production, but this seems an acceptable tradeoff at the moment - I really like the queueScript and queueWait API in LABjs.

Fortunately the loading spinner image in our CSS file doesn't hold up body onload, so the overall net effect is really lovely - URL bar gone, spinner spinning, code loading then running - ideal. This approach seems to get rid of the URL bar faster than any other app I've seen which does so (gmail, Financial Times, Twitter).

Monday, October 17, 2011

iOS5 in Private Browsing Mode

I love feature detection. Sometimes it fails you in a really unfortanate way.
Two examples of late


> window.localStorage
 ▶ StorageConstructor

> window.localStorage.setItem('foo', 'bar')
> Error: QUOTA_EXCEEDED_ERR: DOM Exception 22



Q: Guess what most libraries do to detect support?

A: if ('localStorage' in window)


Well, nothing a try catch can't fix.

/facepalm

Sunday, October 16, 2011

Hide the Mother Effing URL Bar

The should be an API for mobile browsers to do this. Can I get an amen?