til

Running pip install -e .[test] in zsh on macOS Catalina

macOS Catalina uses rather than as the default shell (apparently because Apple don't like GPL 3). I usually set up my Python projects for development like this: datasette % pipenv shell Launching…

Seeing files opened by a process using opensnoop

I decided to try out atuin, a shell extension that writes your history to a SQLite database. It's really neat.

Using Tesseract.js to OCR every image on a page

Pasting this code into a DevTools console should load Tesseract.js from a CDN, loop through every image loaded by that page (every PNG, GIF, JPG or JPEG), run OCR on them and output the result to the…

Mocking subprocess with pytest-subprocess

For apple-notes-to-sqlite I needed to write some tests that simulated executing the command using the Python module. I wanted my tests to run on Linux CI machines, where that command would not exist.

Using the undocumented Fly GraphQL API

Fly has a GraphQL API which is used by some of their own tools - I found it while browsing around their code on GitHub.

sips: Scriptable image processing system

I wanted to convert some images to on my Mac. I asked ChatGPT: On MacOS use CLI to convert webp images to PNG And it told me about : sips -s format png image.webp –out image.png Or to run it against…

Piping echo to a file owned by root using sudo and tee

I wanted to create a file with a shell one-liner where the file lived somewhere owned by root. I tried this but it didn't work: Running using didn't pass through to the bit.

Publish releases to PyPI from GitHub Actions without a password or token

I published a package to PyPI today using their Trusted Publishers mechanism for the first time. Trusted Publishers provides a mechanism for configuring PyPI to allow a specific GitHub Actions…

SQLite aggregate filter clauses

SQLite supports aggregate filter clauses (as of 3.30.0, released 2019-10-04), as described in this SQL Pivot in all databases tutorial.

Turning an array of arrays into objects with jq

Input: I want an array of objects. Here's what I came up with, using jqplay.org: This outputs: If you remove the outer and and use the "Compact output" option you get back this instead:

JSON Pointer

I'm looking at options for representing JSON validation errors in more JSON. The recent RFC 7807: Problem Details for HTTP APIs looks relevant here.

Listen to a web page in Mobile Safari

I found a better way to listen to a whole web page through text-to-speech on Mobile Safari today. Previously I'd been selecting all text on the page manually (an arduous task) and using the "Speak"…

Running different steps on a schedule

Say you have a workflow that runs hourly, but once a day you want the workflow to run slightly differently - without duplicating the entire workflow. Thanks to @BrightRan, here's the solution.

Summing columns in remote Parquet files using DuckDB

vivym/midjourney-messages on Hugging Face is a large ( 8GB) dataset consisting of 55,082,563 Midjourney images - each one with the prompt and a URL to the image hosted on Discord.

TOML in Python

I finally got around to fully learning TOML. Some notes, including how to read and write it from Python. The data structure A TOML file is a document that contains a dictionary of key value pairs.

Running pip install '.[docs]' on ReadTheDocs

I decided to use ReadTheDocs for my in-development datasette-enrichments project. Previously when I've used ReadTheDocs I've had a folder in my project with its own file containing the requirements.

Show files opened by pytest tests

My test suite for Datasette has grown so large that running the whole thing sometimes causes me to run out of file handles.

Simplest possible OAuth authentication with Auth0

Auth0 provides an authentication API which you can use to avoid having to deal with user accounts in your own web application.

Set a GIF to loop using ImageMagick

I managed to accidentally create a GIF that ran once without looping. I think this is because I created it in LICEcap but then deleted some frames and re-saved it using macOS Preview.

Piping from rg to llm to answer questions about code

Here's a trick I've used a couple of times in the past few days. If I have a question about a codebase and I want to answer it using an LLM, I'll use (aka ) to search that codebase for a specific…