til

Running gdb against a Python process in a running Docker container

While investigating Datasette issue 1268 I found myself with a Python process that was hanging, and I decided to try running against it based on tips in Debugging of CPython processes with gdb Here's…

Using the tesseract CLI tool

Tesseract OCR has a command-line utility which is woefully under-documented. Thanks to Alexandru Nedelcu I figured out how to use it today.

kubectl proxy

Learned about this today as a way of accessing the Kubernetes REST API. Assuming you have setup and authorized against a cluster (I'm using DigitalOcean K8S) you can start a proxy for talking to the…

Removing a git commit and force pushing to remove it from history

I accidentally triggered a commit which added a big chunk of unwanted data to my repository. I didn't want this to stick around in the history forever, and no-one else was pulling from the repo, so I…

Reading thermometer temperatures over time from a video

Natalie has been experimenting with using a microwave as a kiln for pottery, specifically for Raku. She wanted to understand how long to run the microwave to reach the desired temperature for the Raku…

Testing cookiecutter templates with pytest

I added some unit tests to my datasette-plugin cookiecutter template today, since the latest features involved adding a script. Here's the full test script I wrote.

Importing CSV data into SQLite with .import

I usually use my command to import CSV data into SQLite, but for large CSV files (like a 750MB one) this can take quite a long time - over half an hour in this case.

Increasing the time limit for a Google Cloud Scheduler task

In VIAL issue 724 a Cloud Scheduler job which triggered a Cloud Run hosted export script - by sending an HTTP POST to an endpoint - was returning an error.

Using C_INCLUDE_PATH to install Python packages

I tried to install my datasette-bplist plugin today in a fresh Python 3.10 virtual environment on macOS and got this error: Thanks to StackOverflow I figured out a workaround.

Using psutil to investigate "Too many open files"

I was getting this intermittent error running my Datasette test suite: OSError: Errno 24 Too many open files To figure out what was going on, I used the package and its method.

Lag window function in SQLite

Here's how to use a lag window function to calculate new cases per day when the table just has total cases over time on different dates.

One-liner for running queries against CSV files with SQLite

I figured out how to run a SQL query directly against a CSV file using the command-line utility: sqlite3 :memory: -cmd '.mode csv' -cmd '.import taxi.csv taxi' \ 'SELECT passenger count, COUNT( )…

Running Docker on an M1 Mac

TLDR: Use in and your containers will probably work! I was trying to figure out how to get a development environment for a Django project running inside Docker in a M1 Mac.

Tracing every executed Python statement

Today I learned how to use the Python trace module to output every single executed line of Python code in a program - useful for figuring out exactly when a crash or infinite loop happens.

Using nginx to proxy to a Unix domain socket

I figured this out while adding support to Datasette in 1388. Save the following in : Start against that configuration file - this works without root provided you listen on a high port: nginx -c…

Writing JavaScript that responds to media queries

I needed to apply some changes to the DOM if the users window was smaller than 800px - or if the user resized their window to less than 800px after they had first loaded the page.

Mocking Stripe signature checks in a pytest fixture

I'm writing some code that accepts webhooks from Stripe. I wanted to simulate hits to this endpoint in my Django tests.

Using git-filter-repo to set commit dates to author dates

After rebasing a branch with 60+ commits onto I was disappointed to see that the commit dates on the commits (which are a different thing from the author dates) had all been reset to the same time.

Using the sqlite3 Python module in Pyodide - Python WebAssembly

Pyodide provides "Python with the scientific stack, compiled to WebAssembly" - it's an incredible project which lets you run a full working Jupyter notebook, complete with complex packages such as…

Intercepting fetch in a service worker

I'm learning service workers. I wanted to start with one that intercepts calls to a and returns "Hello World". Here's the initial recipe I came up with.