Notes on software engineering
201–220 of 547 posts, newest first.
provides an easy way to create "reversible" Django SQL migrations, where the reverse operation does nothing (but keeps it possible to reverse back to a previous migration state without being blocked…
While investigating Datasette issue 1268 I found myself with a Python process that was hanging, and I decided to try running against it based on tips in Debugging of CPython processes with gdb Here's…
Tesseract OCR has a command-line utility which is woefully under-documented. Thanks to Alexandru Nedelcu I figured out how to use it today.
I got an unexpected traffic spike to https://russian-ira-facebook-ads.datasettes.com/ - which runs on Cloud Run - and decided to use to block crawlers.
Learned about this today as a way of accessing the Kubernetes REST API. Assuming you have setup and authorized against a cluster (I'm using DigitalOcean K8S) you can start a proxy for talking to the…
Derek Willis has a Datasette instance full of political campaign emails running on Heroku. Matt Hodges pointed out that a lot of these emails include codes, which are used by ActBlue campaigns to…
I accidentally triggered a commit which added a big chunk of unwanted data to my repository. I didn't want this to stick around in the history forever, and no-one else was pulling from the repo, so I…
I helped build the tide chart visualizations for Rocky Beaches. I wanted to generate an SVG line representing 24 hours of tide levels.
Natalie has been experimenting with using a microwave as a kiln for pottery, specifically for Raku. She wanted to understand how long to run the microwave to reach the desired temperature for the Raku…
I was using the Apache Bench command to exercise some new code I'm writing in Datasette and I noticed I was getting a lot of errors: Truncated output: What's with the 953 failed requests?
I deployed https://metmusem.datasettes.com/ by creating a folder on my computer containing a Dockerfile and then shipping that folder up to Google Cloud Run.
I added some unit tests to my datasette-plugin cookiecutter template today, since the latest features involved adding a script. Here's the full test script I wrote.
I ran a survey on Twitter the other day to try and figure out what people mean when they use the term "agents" with respect to AI.
I usually use my command to import CSV data into SQLite, but for large CSV files (like a 750MB one) this can take quite a long time - over half an hour in this case.
In VIAL issue 724 a Cloud Scheduler job which triggered a Cloud Run hosted export script - by sending an HTTP POST to an endpoint - was returning an error.
I tried to install my datasette-bplist plugin today in a fresh Python 3.10 virtual environment on macOS and got this error: Thanks to StackOverflow I figured out a workaround.
I was getting this intermittent error running my Datasette test suite: OSError: Errno 24 Too many open files To figure out what was going on, I used the package and its method.
Here's how to use a lag window function to calculate new cases per day when the table just has total cases over time on different dates.
I figured out how to run a SQL query directly against a CSV file using the command-line utility: sqlite3 :memory: -cmd '.mode csv' -cmd '.import taxi.csv taxi' \ 'SELECT passenger count, COUNT( )…
TLDR: Use in and your containers will probably work! I was trying to figure out how to get a development environment for a Django project running inside Docker in a M1 Mac.