Notes on software engineering

Archive / page 5

81–100 of 547 posts, newest first.

  1. 2025
  2. Compiling and running sqlite3-rsync

    Today I heard about the sqlite3-rsync command, currently available in a branch in the SQLite code repository.

    1 min read
  3. Assigning a custom subdomain to a Fly app

    I deployed an app to Fly and decided to point a custom subdomain to it. My fly app is https://datasette-apache-proxy-demo.fly.dev/ I wanted the URL to be…

    1 min read
  4. Efficiently copying a file

    TLDR: Use I'm writing a Datasette plugin that handles an uploaded file, borrowing the Starlette mechanism for handling file uploads, documented here.

    1 min read
  5. Mocking a Textract LimitExceededException with boto

    For s3-ocr issue 21 I needed to write a test that simulates what happens when Amazon Textract returns a "LimitExceededException".

    1 min read
  6. Compiling to WASM with llvm on macOS

    howto-wasm-minimal by Zalka Ernő (my fork here) is a neat demo of a minimal WASM module. It uses C++ to define functions for simple image manipulation including blurring an image, compiles it to WASM…

    1 min read
  7. My Professional Journey and Passions

    admin only: false Introduction Hello! I’m Farshid Ashouri, Head of Data Engineering at JPMorgan Chase, with over 20 years of experience designing and operating large-scale distributed systems across…

    2 min read
  8. Auto-formatting YAML files with yamlfmt

    I decided to see if there was an equivalent of Black or Prettier for YAML files. I found yamlfmt from Google. Update : It turns out Prettier can format YAML too.

    2 min read
  9. Bundling Python inside an Electron app

    I used python-build-standalone for this, which provides a version of Python that is designed for easy of bundling - it's also used by PyOxidize.

    1 min read
  10. Format code examples in documentation with blacken-docs

    I decided to enforce that all code examples in the Datasette documentation be formatted using Black. Here's issue 1718 where I researched the options for doing this. I found the blacken-docs tool.

    1 min read
  11. Upgrading a Heroku PostgreSQL database with pg:copy

    Figured this out in https://github.com/simonw/simonwillisonblog/issues/132 - I was running PostgreSQL 9.x and I wanted 11.x in order to use the new option in Django 3.1.

    1 min read
  12. Using llama-cpp-python grammars to generate JSON

    llama.cpp recently added the ability to control the output of any model using a grammar. This is an incredibly powerful technique for working with a Large Language Model.

    3 min read
  13. Redirecting a whole domain with Cloudflare

    I had to run this site on for 24 hours due to a domain registration mistake I made. Once I got working again I wanted to permanently redirect the URLs on the temporary site to their equivalent on the…

    1 min read
  14. Customizing my zsh prompt

    I got fed up of the default macOS prompt: simon@Simons-MacBook-Pro % Mainly because I like copying and pasting terminal examples into GitHub issues.

    1 min read
  15. Fixing "compinit: insecure directories" error

    Every time I opened a terminal on my new Mac running Catalina with zsh I got the following annoying error: Based on a tip in https://github.com/zsh-users/zsh-completions/issues/680 I fixed it by doing…

    1 min read
  16. Embedding JavaScript in a Jupyter notebook

    I recently found out modern browsers include a JavaScript API for creating public/private keys for cryptography.

    1 min read
  17. Using io.BufferedReader to peek against a non-peekable stream

    When building the –sniff option for (which attempts to detect the correct CSV delimiter and quote character by looking at the first 2048 bytes of a CSV file) I had the need to peek ahead in an…

    1 min read
  18. axe-core and shot-scraper for accessibility audits

    I just watched a talk by Pamela Fox at North Bay Python on Automated accessibility audits. The video should be up within 24 hours.

    1 min read
  19. Using custom Sphinx templates on Read the Docs

    I wanted to make a small customization to one of my documentation templates on Read the Docs. It turns out you can over-ride the default templates by creating a directory in your documentation folder…

    1 min read
  20. GraphQL fragments

    One of the scripts that builds and deploys datasette.io uses a GraphQL query to retrieve information from GitHub about the repositories used for the various Datasette tools and plugins.

    1 min read
  21. Publishing a simple client-side JavaScript package to npm with GitHub Actions

    The code is in simonw/prompts-js on GitHub. The NPM package is prompts-js. A simple single file client-side package For this project, I wanted to create an old-fashioned JavaScript file that you could…

    2 min read