MorgBorg Momentum

Just Keep Swimming

0 notes

Searching Encrypted Data

Damn thorough explanation of both the problem of searching encrypted data and a solution. Really helpful for understanding the complexities of the issue gems like Strongbox are attempting to solve.

Filed under encryption

0 notes

If we let ourselves, we shall always be waiting for some distraction or other to end before we can really get down to our work. The only people who achieve much are those who want knowledge so badly that they seek it while the conditions are still unfavourable. Favourable conditions never come.
C.S. Lewis

1 note

API Call Optimization Investigation: Find the Bottleneck!

Context

I wrote a Rails API that is consumed by an Angular front-end (Angular is integrated with the Rails application via angularjs-rails gem).

Problem

One API call retrieves all records a user is allowed to see. On one seeded user I am retrieving 64 records. 

200 OK in 2065ms (Views: 4.9ms | ActiveRecord: 98.3ms)

Far from ideal.

Fixing

First I checked out the implementation of the method associated with that particular API call. It was horrendous, looping through the data around four times and retrieving far more information than was necessary for the view. So I deleted it and rewrote it sensibly (test driven development, baby! making deleting code not terrifying since Some-Date-Long-Ago). I was confident this would solve the issue. But, while the time went down, not significantly.

I then installed bullet which found one case of eager loading being unnecessary. Still no major difference (no surprise).

At this point I was about to set timers all throughout my method and the methods it calls to figure out what the heck was happening, but something shiny stopped me. Something shiny called New Relic. I’ve never used them before, but we had a subscription so why not go the two birds, one stone route?

New Relic is the easiest thing to set up, ever. Within twenty minutes, I had my application and my server returning data which New Relic rendered beautifully. The first thing New Relic told me was that my version of Rails had a known security risk, so I should update it. Already it was being helpful!

The second thing New Relic told me was that one of my API calls was taking it’s sweet ass time to return (well aware, but happy to see it flagged it) but it couldn’t give me further information out of the box. However, it did let me know that I could create custom metrics within my code to give me more in-depth information on where things were going wonky. I played with that for about half an hour before I decided this was a rabbit hole of potential awesomeness but would take me a bit of time to fully understand (it was seeing the custom metric but not returning data) and was not my immediate concern. 

Benchmarking (aka timers everywhere) time. I seeded my local database with 64 records and then retrieved them via the API. While it wasn’t quite as bad as production, it still took a full second. To be specific, my benchmark was this:

0.980000   0.020000   1.000000 (  1.001986) 

There are submethods within the main methods, but there was no need to get that granular. In my loop through the data I was decrypting information which I had investigated five months ago when I first implemented encrypted data and decided was not going to be  large issue in terms of performance…or so I thought. My mental filing system of ‘things not to worry about’ is clearly off.

Commenting out that one line of the code brought me down to 300ms. It also delivered me drivel. So, that isn’t an option. And encryption is a must due to the nature of the application. So, the problem, at it’s core, remains. 

Now What

Currently my Angular application doesn’t render the page until it has data from my API…which means the entire app is waiting 1-2 seconds with white space just staring at you. Unimpressively.

There are a few options I see to solve this problem.

One, put in a loading swirly in the datatable and have the page load without waiting for the API to return. (bandaid!)

Two, follow an example I’ve seen in another API where the API returns the first X entries and then returns a value indicating the “page” of data you should request next (will be nil when you’re on the last page). Then my view can wait to load until the first API call has returned (with the first ten or so records) and by the time they actually get around to manipulating the table the rest of them will likely be accessible as well. (plan!)

Filed under rubberducking rails angular newrelic

0 notes

Unit Testing Best Practices in AngularJS

"Chuck Norris’ code tests itself, and it always passes, with 0ms execution time."

Filed under angularjs