2022

As part of a recent audit that I was doing of Google Font usage in a codebase, I had a need to get a list of all Google Fonts. But, after a bit of searching, I wasn’t finding an easily consumable list of these fonts.

But, it turns out that if you get an Google Fonts API key from the Google Cloud Console, then it’s super simple to query and get a list of fonts. Actually, this is what getting a list of Google Fonts looks like with a single command from the terminal.

curl --location --request GET 'https://www.googleapis.com/webfonts/v1/webfonts?key=...' | jq ".items[] | .family" --raw-output

To get this to work, simply replace the ... with your actual key and then run the command.

Of note, this command does assume that you’ve got jq installed which is used to parse the JSON response. But, if you’d like, you can simply remove the everything after from the | jq and after to echo the json.

Over the past month or so, I spent a bit of time working on setting up a local development environment for a project that I’m working on. As part of that, I also set up a deployment flow using a simple Github webhook handler in PHP.

I modeled this Github webhook handler very much after jplitza’s Gist here, but I simplified it even further since all I really cared about was whether an event happened since I’d already filtered on Github for push events.

This is the handler that I ended up with.

<?php

define( 'LOGFILE', '/DIR' );
define( 'SECRET', '0000000000' );
define( 'PULL_CMD', 'CMD_HERE' );

$post_data = file_get_contents( 'php://input' );
$signature = hash_hmac( 'sha1', $post_data, SECRET );

function log_msg( $message ) {
	file_put_contents( LOGFILE, $message . "\n", FILE_APPEND );
}

if ( empty( $_SERVER['HTTP_X_HUB_SIGNATURE'] ) ) {
	exit;
}

if ( ! hash_equals( 'sha1=' . $signature, $_SERVER['HTTP_X_HUB_SIGNATURE'] ) ) {
	exit;
}

// At this point, we've verified the signature from Github, so we can do things.
$date = date(' m/d/Y h:i:s a', time() );
log_msg( "Deploying at {$date}" );

$output_lines = array();
exec( PULL_CMD, $output_lines );

if ( ! empty( $output_lines ) ) {
	log_msg( implode( "\n", $output_lines ) );
}

exit;

Basically, all this file is doing is:

• Verifying that the request actually comes from Github by creating a signature using our SECRET and then comparing that to the signature that Github sent us with the time constant hash_equals
• Running whatever command is necessary. For me, this command is basically just cd dir && git pull origin main.
• Writing logs so that we can keep track of how often the Github webhook handler is called and whether it’s successful or not.

To get this to work, you’ll roughly do the following:

• Create a log file and add the location to the LOGFILE constant.
• Create a strong secret in the SECRET constant that can be shared on Github so that we can validate the webhook came from Github.
• Update the PULL_CMD constant with the command that you’d like to run.
• Upload this file to your server in a publicly accessible location and make note of what the URL will be.
• Go to Settings > Webhooks for one of the Github repos that you manage and create a new webhook.
• Ensure that you set the URL to the file that we uploaded earlier and ensure that you enter the value from the SECRET constant.

That seems like several steps, but in just a few minutes, you could have your own Github webhook handler in PHP up and running!

I had the need recently to export Github pull requests from a repository to a CSV so that I could do some analysis.

I wasn’t able to find a simple way to this in the Github UI. When I searched, I found several tools. But, the API seemed quite simple, so I just wrote a script that would dump all pull requests from a Github repository to a CSV.

#!/bin/bash

# This script requires jq to be installed and available in the path.

TOKEN=$1
ORG=$2
REPO=$3
OUTPUT_RAW=""

get_pull_requests() {
	curl -s --location --request GET "https://api.github.com/repos/$ORG/$REPO/pulls?state=all&per_page=40&page=$1" \
		--header "Authorization: token $TOKEN" \
		--header "Accept: application/vnd.github+json"
}

get_raw_output() {
	printf '%s' "$1" | jq -r
}

i=1
while [ "$OUTPUT_RAW" != "[]" ] ; do
	OUTPUT=$( get_pull_requests "$i" )
	OUTPUT_RAW=$( get_raw_output "$OUTPUT" )

	i=$((i+1))

	printf '%s' "$OUTPUT" | jq -r '.[] | [ .created_at, .html_url, .user.login, .title ] | @csv'
done

To use the script, you’ll need to have jq. On a mac, you can use brew install jq.

The only other pre-requisite that you’ll need to export pull requests from Github is a personal access token.

From there, you should just need to run the script with something like to export all of your pull requests for a given repository:

sh github_pulls_export.sh TOKEN ORG REPO

If you’d like to change what data gets exported, simply change the fields that are pulled in this section:

[ .created_at, .html_url, .user.login, .title ]

You can modify that printf line to get an idea of what fields are even included that you can pull from

In the past, I’ve often had to generate email lists of users that fit specific conditions. This usually isn’t too difficult with some of our in-house data tools at Automattic. But, when I hit a case where I have to work across systems, it usually results in me dumping the data from each system and then comparing large lists.

Comparing large lists on the command line isn’t that difficult. All it takes is a few commands which I’ll walk you through in this blog post!

So, let’s come up with a theoretical example. Let’s say that I have two separate lists, one of users that have purchased Product A and one of users that have purchased Product B. Second, let’s also agree that these lists contain the user’s email address and the date of purchase. So, in a CSV, the data would look a bit like this:

email_address,date
"user@example.com","2022-03-01"

Prepping the list of users

Later on, we’re going to use the comm command for the actual comparing of the lists. Before we can use that command, there’s a bit of prep work that we first need to complete. Specifically, we need to:

• Remove the CSV header row if there is one
• Filter down the source data to whatever field we want to compare, email addresses in our theoretical example
• Sort the list
• Unique the list

To cut the CSV header from file, we’re going to use the following:

sed 1d product_a.csv

This will remove the first line and print the rest of the file to standard out.

Next, we’re going to pull out the column that contains email address, or the first column in our dataset above. To do this, we’re going to use the following:

cut -d',' -f1 product_a.csv

This command is setting the , as the delimeter and then pulling the first column of the file.

From here, we simply need to sort and unique, which we can do with the sort and uniq commands. If we put all of the above together, we can run it in a single go for each file:

sed 1d product_a.csv | cut -d',' -f1 | sort uniq > product_a_output.csv

At this point, now we just need to take the files and actually compare them. ?

Actually comparing large lists ?

Alright, now that we have two processed files, we can get to the easy part, comparing. The easiest way I?ve found to compare files for these use cases is the comm command. You can man comm to get a detailed view of how that command works, but in summary, you use it like this:

comm file1 file2

That command then outputs three columns, where:

• The first column is all values that are in file1 and not file2
• The second column is all values in file2 and not file1
• The third column is all values in both files

If you’re only interested in the lines that are in both files, you can do the following, which will remove the first and second columns of values:

comm -12 file1 file2

You can then write redirect that output to a file or however you’d like.