November 21, 2019

Installing applications on a new Mac using Brewfiles

My first Mac — the 2013 MacBook Pro — is still alive. Putting aside the occasional service battery” warning, it’s actually going pretty strong. This machine has far exceeded my expectations in terms of durability and longevity.

Soon enough, though, I will purchase a new machine; either because this one dies, or because Apple finally releases a promising replacement (read: no keyboard issues).

Naturally, I had been looking for a way to automate what I consider the most dreaded part of migrations: reinstalling macOS applications, packages, and command line utilities.

I wrote about macOS migrations with Homebrew and Brewfiles on Open Folder. I estimate it shaves off 95% of the time (and effort) it takes to install applications on a new Mac.

Yes, it involves using the Terminal, but I’ve seen iOS shortcuts more complicated than that.

Share: Facebook · Twitter
June 16, 2018

Let them drag & drop

A common annoyance with Google Sheets is that drag & drop can potentially mess up a lot of things. This is because changing the location of cells by dragging them somewhere else also changes the reference to those cells in formulas.

A simple, easily solvable example: If cell D1 is supposed to hold the formula =SUM(B3:B4), and we then select and drag those two cells down, the formula in cell D1 will now say =SUM(B5:B6). Sometimes this is the desired result, other times it isn’t, and if it’s the latter case, it’s easy enough to fix by using INDIRECT: =SUM(INDIRECT(‘B3:B4’)).

But here’s a more complicated scenario: you are one of many collaborators on a protected sheet, of which you are only permitted to edit a certain range: B2:C20. In other words, this is the only unprotected range in the sheet.

Everything is cool. Unless you decide to drag the first cell in the range. Here’s a short video I’ve created to demonstrate this:

Notice how when you try to drag the two rows selected from row 3 back to row 2, Sheets throws an error. This is because you are now no longer allowed to edit row 2 — the new unprotected range is B3:C20. You are effectively locked out of a range you are supposed and were originally permitted by the owner to edit. Since you are only a collaborator, you can’t simply go to the protected ranges” menu and reconfigure them.

This is how all spreadsheets work, it isn’t specific to Sheets. And still, it seems like the wrong behavior from a user-experience standpoint: all cells that were manipulated in the example were within the unprotected range. It’s not like we dragged a protected cell into the range. I also believe that with protected sheets, only the owner should dictate which cells remain unprotected. But Sheets not only allows collaborators to lock themselves out of cells, it allows them to lock out other collaborators too.

So as the maintainer and owner of more than a dozen spreadsheets used in my company for mission-critical tasks, I set out to find a solution.

Telling tens people to use copy-paste instead of dragging wasn’t an option. You don’t count on humans to not do things they’re used to doing. Using INDIRECT as is possible with formulas was also out of question because Google doesn’t support it when dealing with ranges.

Since we can’t really prevent collaborators from locking themselves out of their own ranges, we need a way to verify, and if needed, change the sheet’s (un)protected ranges to what we intend them to be.

As in many cases, what isn’t possible in the interface can be achieved quite easily with JavaScript in the script editor:

// protect the sheet, add a description
var protection = sheet.protect().setDescription("Protection Validator");
// select the range we want to un-protect  
var unprotected = sheet.getRange('B2:C20');
// remove the protection from the range
protection.setUnprotectedRanges([unprotected]);

This code is set to run whenever a change is made to the spreadsheet. No matter how people misuse” it, protected ranges are always re-set to the desired result. They get to use drag and drop, and I get less phone calls.

Share: Facebook · Twitter
February 3, 2018

On apps that use fake progress bars

A thought-provoking discussion ensues on Hacker News regarding the practice of displaying fake progress bars in web services and apps. Opinions vary on whether these are indeed benevolent deceptions”, as the linked-article suggests, or not. Still haven’t made up my mind.

Share: Facebook · Twitter
December 10, 2017

Running a local script “from” Google Sheets

It’s been two years since I started learning Python. Incidentally, or not, it has also been two years since I last posted something on here. It’s been a fun ride and I’ve now got several projects going on: web-scrapers and APIs for personal use, automation scripts, and one nice but practically useless predictive model utilizing machine learning.

Challenging subjects like ML are fun to practice, but nothing beats automation in my world.

For a recent project, the need was to run some local script whenever a button is clicked in a Google Sheets spreadsheet. Each script automates a specific After Effects composition, based on data in its respective spreadsheet. So we have sheet-script pairs, one button per sheet, and many users interacting with different sheets.

Google makes it fairly easy to manipulate Drive files based on certain events. Apps Script runs on their own cloud platform and can do pretty much anything a human can with Google’s own services. Invoking a locally hosted script isn’t one of them though.

Generalizing the problem helped pave the way: what we needed was to notify the machine, in realtime, of an event taking place on the web. Once that’s done, executing the appropriate script is straightforward.

There are two ways to make a computer aware of remote events: we either have it pull them periodically from the sender, or we have the sender push each event to the machine. The pull approach is simpler, but wouldn’t have worked here: it’s possible to write a script that pings Google’s servers every second, but that’s also a surefire way to go over the APIs quota.

So we need the sender to push the notification, something Google’s platform doesn’t provide out of the box. But actually, the only thing a Google script can push out of the box is even better: a filesystem event. That is, if Google Drive is synced as a local folder, changes in this folder are normal file events as far as the operating system is concerned, and these are among the easiest to monitor. Now the question becomes how it is best to do that.1

For our use-case, I decided to create a timestamped JSON file each time there’s a button event. It sure would’ve been nice to do the tracking in one file, but with multiple users and sheets, such design would create a substantial risk of race-conditions.2

Here’s where I complicate things a bit: Since I already had Dropbox installed, I decided to save the JSON files there, and not in Google Drive. This changes the implementation, but not the underlying concept.

So, each time one of the users clicks a button, a Google script is invoked, creating a new file in the Dropbox subfolder json_jobs”:

### filename example: rhino-12-8-2017 10/32/30.json
{
"processed": false,
"script": "C:\\ae_folder\\ae_script_1.ps1",
"generated_at": "12-8-2017 10:34:02AM",
"delegator_spreadsheet": "spreadsheet_id",
}

Whenever a file like this is created, it’s grabbed and inspected by a Python script running in the background: this script then invokes the PowerShell script at the specified path, moves the resulting files to their final destination, and if all went good, changes the processed flag to True. This way retries won’t reprocess completed jobs.3

But that’s the end result. The very first step was to write the script that creates the JSON file and saves it to Dropbox. Here’s a slightly modified version of the main function, the one connected to the button-click event:

function main(spreadsheet_id) {
var json_dict = fillDict(spreadsheet_id)
var json_obj = convertDictToJSON(json_dict)
sendJSONToDropbox(json_obj)
};

Connected” is a little misleading here: the file holding this code isn’t actually bound to any one spreadsheet. Instead, all spreadsheets involved import and utilize it as a library, passing their IDs whenever they call it. From there, we pass spreadsheet_id to a function that constructs a dictionary with the appropriate values, convert the dictionary to a JSON object, and send it to sendJSONToDropbox():

function sendJSONToDropbox(json_object) {
var timestamp = timeStamp()
var parameters = {
"path": "/json_jobs/rhino" + "-" + timestamp + ".json",
"mode": "add", // do not overwrite
"autorename": true,
"mute": false // notify Dropbox client
};


var headers = {
"Content-Type": "application/octet-stream",
'Authorization': 'Bearer ' + 'dropbox_access_token',
"Dropbox-API-Arg": JSON.stringify(parameters)
};


var options = {
"method": "POST",
"headers": headers,
"payload": json_object
};

var API_url = "https://content.dropboxapi.com/2/files/upload";
var response = JSON.parse(UrlFetchApp.fetch(API_url, options).getContentText());
Logger.log(response)

};

And that’s one way to run a local script from Google Sheets.


  1. Yes, we do end up listening to something, but polling the filesystem is overwhelmingly cheaper than polling Google’s servers. ↩︎
  2. I haven’t found a way to directly append data to a Dropbox file using the API. ↩︎
  3. This is one of several reasons to call the scripts indirectly via Python. The main reason: I didn’t write these scripts myself, and have no intentions whatsoever to learn PowerShell. ↩︎
Share: Facebook · Twitter
April 19, 2017

London Police Spying on Journalists

From WSWS via Hacker News:

The existence of a secretive unit within London’s Metropolitan Police that uses hacking to illegally access the emails of hundreds of political campaigners and journalists has been revealed. At least two of the journalists work for the Guardian .

Green Party representative in the British House of Lords, Jenny Jones, exposed the unit’s existence in an opinion piece in the Guardian. The facts she revealed are based on a letter written to her by a whistleblower.

In May 2015 it was unveiled that the UK government quietly rewrote its Computer Misuse Act to exclude police and intelligence agencies. I wrote about it here. Today we see the bastard child.

Share: Facebook · Twitter
January 8, 2016

Paypal Freezes Startup’s Funds, Offers Loan

Hacker Paradise founder Casey Rosengren, on Medium:

On one hand, Paypal was saying that our account was too risky for them to release the $20,000 they’d taken as collateral. On the other hand, they were saying our account was credible enough to offer us a loan.

We chose to approach this in the spirit of Hanlon’s Razor: when in doubt, assume ignorance, not malice. So, we emailed a screenshot of the ad above to the startup evangelist, assuming it was some kind of clerical error.

Unfortunately, for customers outside the US there is no practical alternative to Paypal. I still use them for most of my recurring payments, with the occasional online purchase.

Share: Facebook · Twitter
September 15, 2015

Disrupt Yourself

Readers of this blog should already know I’m a big fan of Horace Dediu. This episode of his podcast was even better than what regular listeners like me have come to expect.

Share: Facebook · Twitter
July 23, 2015

Kiwi for Gmail: A Review

In Gmail in a Box, I mentioned having bought Kiwi for Gmail. This review will first cover the app itself, and an optional appendix at the end will detail my initial experiences as an early adopter.

Kiwi for Gmail Logo

Background

Zive Inc. launched their Kickstarter campaign for Gmail for Mac” on November 15, 2014, dubbing it the true desktop email client for Gmail”.1 The project was fully funded within six days, and by the time the pledge ended, Zive had raised $42,200: more than two times their initial goal. The app’s name was changed in May after Zive were approached — politely they say — by Google.

Kiwi for Gmail launched on June 23 and costs $10 on the Mac App Store.

Two Kiwis

A free version — Kiwi for Gmail Light” — can also be found on the App Store. Contrary to the full version, Kiwi Light lets you add only one account, and does not support shortcuts, important-only notifications, do-not-disturb mode, or Gmail plugins.

Setup

The setup process is very simple. A tutorial is presented on first launch, after which you’re taken to a standard login window, and then to your inbox. Two-step verification is supported, so there is no need to create an app-specific password for Kiwi.

Preferences

Once you set up your first account, up to five more can be added through the accounts” tab in the preferences pane. There, it’s also possible to change various account-specific settings: notifications, sounds, and colors.

Kiwi for Gmail Screenshot

There are three other tabs besides accounts” in the preferences pane:

General: where you can set Kiwi to launch on login, and supposedly make it your default client. Supposedly, because this box is greyed-out for Yosemite users. Zive says this is a bug” that affects all email clients, and that the only way to set one as the default is via Mail.app’s preferences. However, to merely access those preferences one first needs to have an account authenticated in Mail.app. Annoying.

Notifications: set global settings for notifications and sounds, and choose whether the unread mail badge is shown on the dock, menubar, or both.

Shortcuts: Kiwi supports several of them, but only the new email” shortcut appears here. Press ⌘ ⌥ ⌃ M anywhere and a new window opens with a list of your accounts. The list’s item order corresponds to the keyboard’s numbers, so typing 2” would select the second account. There isn’t a way to choose a default account to compose from.

Inside the app, ⌘ ⇧ and [ ] are used to switch between accounts. To cycle between active windows, you can press ⌘ `.

I was puzzled to see no shortcut for simply bringing up Kiwi’s window. Adding shortcuts is relatively easy in OS X, but every email client should ship with a default, configurable one.

Design & User Interface

Kiwi’s menubar item comes in the shape of an envelope (no kidding!) and displays the total count of unread items.2 A neat menu opens when you click on it, featuring individual counts, as well as buttons to start composing for each account.

Kiwi for Gmail Screenshot

A thin, colored strip is visible right below Kiwi’s title bar. Envelopes — each in its designated account color — are horizontally stacked from the right. Switching from one account to the other will update the address on and the color of the ribbon on the left. Other than these, there aren’t any noticeable aesthetic differences compared to the browser experience.

Kiwi for Gmail looks as nice as a web-view” app can. Zive can’t control Gmail’s own interface design, but they have done their part of the deal.

Features Overview

Multi-account support: Zive promised 100% fidelity and zero friction for Kiwi’s biggest selling feature. And the execution here is flawless. I’ve set up three different accounts in less than ten minutes, and switching between them is instantaneous. For those with more than one Gmail account, the benefits here can’t be overstated.

Drive support & attachment handling: Seems to work great but asks to you switch to the older look because your browser (which is actually the new window opened inside of Kiwi) is old and unsupported”. A killer feature for those who use Drive regularly.

Gestures: left is back” and right is next”, which is just about enough. These work inside the web-view and not for going back and forth between accounts.

Lab support: I have several lab features turned on and they all worked fine inside Kiwi.

Zen Switch: the ability to turn-off all notifications until the next day. Works as expected.

Filter by flags: I haven’t tested this one, but if works as advertised it should be pretty useful for those who do use flagging.

Gmail plugin support: coming soon” according to Zive’s product page.

Bottom Line

A month after its launch, Kiwi for Gmail does what it set out to very well. There are minor annoyances, but my Gmail experience has been less cumbersome and nicely streamlined after the Driver bug got fixed. (see below)

For me, Kiwi was well-worth its price.

Appendix: Labour Pains & How to Piss off Your Customers

I have a soft spot for indie developers, so I’ll try to proceed as gracefully as I can.

Let’s start with launch day: in one of the first batches sent to customers, thousands of email addresses were exposed because whoever sent it was careless enough to simply dump them in the to:” field. Zive were quick to respond, but the damage had already been done.3

Second, the Google Drive feature wouldn’t work during the first week. When I contacted Zive, I was told this bug was affecting some retina Macs, and that I should turn on low-resolution mode until a fix is released”. When was a fix coming? No ETA.”

I replied saying I would’ve gotten insulted. Would have”, because I’m sure Zive’s support rep wasn’t using a retina screen himself. Otherwise, this idea wouldn’t have even crossed his mind. It’s not inconvenient but usable” or a trade-off”, as he insisted thereafter, it’s impossible.

My gripe isn’t with the bug. Stuff happens, and again: if you’re a small team working under tight conditions, I can sympathize. Just don’t give me the impression I’m complaining about some minor issue, or insist I patch it with something that makes the app itself unusable. How about this instead: Sorry our app/this feature is nonfunctional, we’re working on a fix”.

The exchange that followed with Zive’s support, while polite and prompt, was quite frustrating. Several days after I tweeted about it, Zive’s founder Eric Shashoua got in touch and said he’d like to know more about how his team had handled this. More importantly, a new version rolled out on June 30, fixing the Google Drive bug.

All things considered, Kiwi is an excellent app if you want the full Gmail experience. I can’t help but feel it shipped one or two weeks too early though.


  1. There’s a technical explanation on how Kiwi is different than other single-site browser apps” in the FAQ section↩︎
  2. You can exclude accounts on individual basis. ↩︎
  3. No, the irony of this happening to an email client developer didn’t escape me. I don’t think it’s fair to judge the app itself by this, or to give Zive more hard time than they’ve already had for it. ↩︎
Share: Facebook · Twitter
July 20, 2015 July 7, 2015

A Pond, Not a River

Alright Internet, it’s been a helluva week. My post (ostensibly) about paid apps has been read by over 13,000 people and shared by dozens. I am full of gratitude and appreciation. There’s already an outline for a sequel that is geared more towards the developer side of the app market, but that will have to wait until my RSI-like problems begin to alleviate.1

A confession: there is one phrase in the aforementioned article that I thought meant something entirely different than what it turned out to mean. Here’s the relevant snippet from my piece:

The impending yearly car test, why do you need to remember when that is? The house insurance you need to renew, the phone call you promised to make for a friend, your mobile plan expiring soon, the meeting you just agreed to. You think you don’t spend time thinking about these little, mundane bits, but they are there, accumulating brain-cruft at the back of your head, not letting you achieve a mind like water”.

Mind like water” is a simile which David Allen mentions in his book about the Getting Things Done” framework. I bought it a few days ago as I’m increasingly intrigued by Allen’s brainchild, and have been considering Omnifocus, an app that’s based upon it.

When I referred to mind like water”, I was pretty sure water” was recruited to emphasize flow. It makes sense: an unobstructed mind flows powerfully, like a river. But apparently, the karate master that coined it saw a different image. I really like how Allen describes it in his book:

In karate, there is an image that’s used to define the position of perfect readiness, mind like water”: imagine throwing a pebble into a still pond, how does the water respond?

The answer is, totally appropriately to the force and mass of the input; then it returns to calm. It doesn’t overreact, or underreact.

So there you go, a complete opposite of the state I had in mind. And perhaps one that’s appealing even more.

Bonus: an excellent, actionable essay on designing metaphors.


  1. I’ve been employing Siri for my reminders, and to write” this post, I’m using Yosemite’s built in dictation system. It’s less awful than what I thought it would be, but that’s for another occasion to discuss. ↩︎
Share: Facebook · Twitter