What are multi-headed self attention blocks and how do they fit into transformers.

Notes on the tokenizer space, what they are, what problems they solve, some example code, and some areas to explore in the future.

Summary of resources I read/watched in May. The focus of this month was to get familiar with terms and concepts underlying Generative AI.

Visit song-sensei.vercel.app to chat with a bot llm to find the music you want on Spotify

Documenting my journey buying the parts for and building a pc.

tldr visit this link to download your saved reddit posts.

In this blog post we will take an in depth look at how Facebook’s C++ String from Folly works. Specifically we will examine how the string class adapts to different operations depending on the size of the string.

In this blog post we take a step lower from our previous blog posts on compilers/virtual machines and operating systems. We will string together how physics and hardware are built and used to execute our abstract thoughts and ideas illustrated in higher level languages.

This blog post introduces the world of compilers with an example in Golang. The code comes from the book Writing An Compiler In Go by Thorsten Ball.

We will go over a solution to the Producer Consumer problem in concurrency with multiple producers and consumers in a buffer of bounded size. The solution is written in C++ and uses mutexes and condition variables. This post is based off of the blog post here by Baptiste Wicht.

This blog post introduces Operating Systems and the pieces involved, specifically: processes and threads, drivers, file systems, and virtual memory.

This blog post introduces the world of interpreters with an example in Golang. The code comes from the book Writing An Interpreter In Go by Thorsten Ball.

This blog post details how C source code is compiled into x86 binary.

An investigation in applying pseudo labeling, a common technique in semi supervised learning where we have limited labeled data, to time series classification.

An exploration of pseudo random number generators and their applications in sampling.

A summary of the paper [Zhirong et al., 2018] as well as a mention of a followup paper MoCo [He et al., 2020] that builds upon the first.

A full walk through of the balls and bins proof, split up into 4 parts.

This post walks through the process of putting a spam detection model using Snorkel into a microservice using Flask, Docker, and AWS.

This post walks through CMU’s ‘bomb’ lab, which involves defusing a ‘bomb’ by finding the correct inputs to successive phases in a binary executable using GDB.

This post walks through CMU’s ‘Attack’ lab, which involves exploiting the stack space of vulnerable binaries.

This post explains the SIR model and includes a Python implementation that generates a graphic describing a population’s infectious status over time.

This post introduces the Clock Replacer algorithm and an implementation in C++.

In this post, I’m going to cover the Perceptron Algorithm and compare its implementation in Python and C++.