Separating Spam from Ham with the shell
Here’s how attempting to export images off a Word Document led to a quest for data deduplication and classification using the shell.
The images I wanted to export were MS Word diagrams drawn in Word, rather than PNG files1. Because those doodle-shapes do not export to PNG well, I first copy-pasted them into Powerpoint to get the familiar “save as picture” context menu. But a couple of images were still deformed beyond recognition.…
Reproducible workspace deployment with Ansible and Vagrant
Like many developers, I keep track of my configuration files in version control, to make moving to new machines is simpler and share code with others. This process started out for my Emacs config files at first, but grew to consume other aspects: bash functions, git aliases… Since 2016, I have accumulated over 500 git commits.
Now the dotfiles are tracked, the next biggest pain is to install packages and folder structures and obscure commands required to set up some subsystems needs which config files.…
Using Awk to replace only some lines
I was trying to convert bank-issued CSV files that have a weird data layout, for automatic processing by the excellent ledger. This made me learn a new thing about awk. Follow me down the UNIXy goodness!
The data I was getting looks like this:
Transaction Date,Transaction Description,Debit Amount,Credit Amount,Balance 29/02/2018,Restaurant,14.5,,1234.2 12/03/2018,ATM withdrawal,50,,1184.2 23/04/2018,Deposited check,,100,1284.2 Note the last line having 100 in a different column: The problem is that, when receiving money, the transaction is logged in a separate entry (“Credit Amount”) than the (more common) case of debit, which goes to “Debit Amount” column.…
Git diff from empty commit
In order to troubleshoot a bug in the very early history of a project using git, I needed a way to show the git diff from “nothing” to the first commit. Since git keeps a linked graph of commits, what I wanted was a diff from the commit before the first one. Searching the onlines dug up this answer on StackOverflow, showing that there’s a specific commit hash we can use to diff from “nothing” to the first commit:…
Reverse-engineering webapp APIs with Firefox
I’m writing code to back up my user data off a website that lets me see all of my info (including querying by time, account etc) but doesn’t have export features (officially). I am certain there’s an API behind the site that I just have to make sense of.
Since the webapp is requesting data from the API when I click, we should be able to record the web traffic to explore the API.…
Git diff for prose
When writing long sentences in documentation repositories, git tends to show really unhelpful diffs. They are unreadable because long lines aren’t broken, which hides edits happening towards end of line. A colleague of mine asked me if git couldn’t be configured to make this sort of thing more obvious. Challenge accepted!
Figure 1: Can you spot the edit made in a long line of text?
Kaushal Modi’s blog post on git diff for minified JS and CSS inspired this idea for all you prose lovers.…
Git pretty graph
Get a cool graph of commits from the command line! For newbies and experts alike, git is a bit hard to visualize. Here’s a handy git command to make understanding git easier.
git log --decorate --oneline --graph Figure 1: Git graph of this repository
This can be made into a git command via an entry your ~/.gitconfig:
[alias] graph = log --decorate --oneline --graph Code Snippet 1: Alias "…
Presenting Android terminal with Termux (EdLug)
Last night, I presented about Termux to EdLug, the Edinburgh Linux User group (see the event page on meetup.com) in a talk titled “The freedom of shelling out on Android”. It was tons of fun showing off how your Android phone/tablet getting a terminal unlocks a powerful tool!
I've made the slides available on this website, click on the slide below. Remember that there are speaker notes for people following at home, press `S` to use them.…
Welcome to my website!
This blog is meant to record some of the thoughts I keep coming back to, or discussions I’ve had over and over again with different people. My rule of thumb for inclusion here is “if you’ve had to say it more than twice, it’s time to write it down”.
See the About page for my background and motivations.
Look forward more posts as I try to condense my thoughts in written form.…