Farshid/Ashouri

llms

13 posts tagged “llms”.

  1. 2025

  2. Using llama-cpp-python grammars to generate JSON

    llama.cpp recently added the ability to control the output of any model using a grammar. This is an incredibly powerful technique for working with a Large Language Model.

    TIL llms
    3 min read

  3. Training nanoGPT entirely on content from my blog

    This is a follow-up to Running nanoGPT on a MacBook M2 to generate terrible Shakespeare. I used nanoGPT by Andrej Karpathy to train a GPT model entirely against content from my blog!

    TIL llms
    4 min read
  4. 2024

  5. Storing and serving related documents with openai-to-sqlite and embeddings

    I decide to upgrade the related articles feature on my TILs site. Previously I calculated these using full-text search, but I wanted to try out a new trick using OpenAI embeddings for document…

    TIL llms
    5 min read

  6. mlc-chat - RedPajama-INCITE-Chat-3B on macOS

    MLC (Machine Learning Compilation) on May 22nd 2023: Bringing Open Large Language Models to Consumer Devices RedPajama on Apple Silicon is achieved by compiling the LLM using Metal for M1/M2 GPUs The…

    TIL llms
    1 min read

  7. Running a gpt-oss eval suite against LM Studio on a Mac

    OpenAI's gpt-oss models come with an eval suite, which is described in their Verifying gpt-oss implementations cookbook.

    TIL llms
    5 min read

  8. Summarizing Hacker News discussion themes with Claude and LLM

    I've been experimenting with the combination of Claude and my LLM CLI tool to give me quick summaries of long discussions on Hacker News.

    TIL llms
    8 min read

  9. Named Entity Resolution with dslim/distilbert-NER

    I was exploring the original BERT model from 2018, which is mainly useful if you fine-tune a model on top of it for a specific task. dslim/distilbert-NER by David S.

    TIL llms
    1 min read

  10. Piping from rg to llm to answer questions about code

    Here's a trick I've used a couple of times in the past few days. If I have a question about a codebase and I want to answer it using an LLM, I'll use (aka ) to search that codebase for a specific…

    TIL llms
    4 min read

  11. Running Dolly 2.0 on Paperspace

    Dolly 2.0 looks to be a big deal. It calls itself "the first open source, instruction-following LLM, fine-tuned on a human-generated instruction dataset licensed for research and commercial use." I've…

    TIL llms
    2 min read
  12. 2023

  13. Running nanoGPT on a MacBook M2 to generate terrible Shakespeare

    nanoGPT is Andrej Karpathy's "simplest, fastest repository for training/finetuning medium-sized GPTs".

    TIL llms
    2 min read

  14. Running prompts against images, PDFs, audio and video with Google Gemini

    I'm still working towards adding multi-modal support to my LLM tool. In the meantime, here are notes on running prompts against images and PDFs and audio and video files from the command-line using…

    TIL llms
    3 min read

  15. Running LLaMA 7B and 13B on a 64GB M2 MacBook Pro with llama.cpp

    Facebook's LLaMA is a "collection of foundation language models ranging from 7B to 65B parameters", released on February 24th 2023. It claims to be small enough to run on consumer hardware.

    TIL llms
    4 min read

  16. Running OpenAI's large context models using llm

    OpenAI announced new models today. Of particular interest to me is the new model, which provides GPT 3.5 with a 16,000 token context window (up from 4,000) priced at 1/10th of GPT-4 - $0.003 per 1K…

    TIL llms
    2 min read