til

Working around the size limit for nodeValue in the DOM

TIL that in the DOM has a size limit! I had a table cell element containing HTML-escaped JSON and I was doing this: This was breaking on larger JSON strings.

KNN queries with SpatiaLite

The latest version of SpatiaLite adds KNN support, which makes it easy to efficiently answer the question "what are the X closest records to this point".

Pointing a custom subdomain at Read the Docs

I host the documentation for Datasette on Read the Docs. Until today it lived at https://datasette.readthedocs.io/ but today I moved it to a custom subdomain, https://docs.datasette.io/ Most of this…

The location of the pip cache directory

uses a cache to avoid downloading packages again: The command can be used to find the location of that cache on your system: Wheels are cached in - in a nested set of folders based on a hash, for…

Upgrading Homebrew and avoiding the failed to verify attestation error

I managed to get my Homebrew installation back into shape today. The first problem I was having is that it complained that macOS Sequoia was unsupported: It turns out I was on an older Homebrew…

Using recursive CTEs to explore hierarchical Twitter threads

This TIL adapted from a Gist I put together in 2019, before I started tracking TILs here. My twitter-to-sqlite tool produced a SQLite table with an column that referenced another tweet ID, for…

pytest coverage with context

This tweet from \@Mariatta tipped me off to the ability to measure "contexts" when running coverage - as a way to tell which tests exercise which specific lines of code.

Transferring a GitHub issue from a private to a public repository

I have my own private repository where I sometimes create research threads. Occasionally I want to transfer these to a public repository to publish their contents.

SQLite VACUUM: database or disk is full

I was trying to run against a large SQLite database file ( 7GB) using and I got this error: sqlite3.OperationalError: database or disk is full The volume that the database lived in was 20GB in size…

Running jupyterlab via uv tool install

I tried to get jupyterlab working via today and ran into some sharp edges. You can start like this: That ran for a while and output: It also gave me a warning about my PATH.

Installing different PostgreSQL server versions in GitHub Actions

The GitHub Actions default runner currently includes an installation of PostgreSQL 13. The server is not running by default but you can interact with it like this: You can install alternative…

Publishing a Web Component to npm

I tried this for the first time today with my highly experimental datasette-table Web Component. Here's the source code for version 0.1.0.

Implementing a "copy to clipboard" button

I had to figure this out while building datasette-copyable - demo here. The trick is to run to copy to the clipboard - but only after first ensuring that some selectable content (probably in a…

Quickly testing code in a different Python version using pyenv

I had a bug that was only showing up in CI against Python 3.8. I used the following pattern with pyenv to quickly run the tests against that specific version.

Limited JSON API for Google searches using Programmable Search Engine

I figured out how to use a JSON API to run a very limited Google search today in a legit, non-screen-scraper way.

Start a server in a subprocess during a pytest session

I wanted to start an actual server process, run it for the duration of my pytest session and shut it down at the end. Here's the recipe I came up with.

Using async/await in JavaScript in Selenium

Thanks Stuart Langridge for showing me how to do this:

Using OpenAI functions and their Python library for data extraction

Here's the pattern I figured out for using the openai Python library to extract structured data from text using a single call to the model.

Using Prettier to check JavaScript code style in GitHub Actions

I decided to adopt Prettier as the JavaScript code style for Datasette, based on my success with Black for Python code.

Installing packages from Debian unstable in a Docker image based on stable

For Datasette 1249 I wanted to build a Docker image from the base image ("buster" is the current stable release of Debian) but include a single package from "sid", the unstable Debian distribution.