Join Me as I Build a Programming Language From Scratch!

There is just something about building a programming language that I find so fulfilling. It is both a deeply artistic and scientific endeavor, as programming languages are the natural interface between the engineering marvels that are computers and humans. I’ve made a few attempts at building toy programming languages in the past, each time learning something new. This time around, I’d like you to join me on my journey. I’ll be building a k-like array programming language in C.

Why am I doing this?

I quickly became fascinated with how programming languages work under the hood after I was exposed to kdb+/q, which is a truly impressive database and programming language maintained by kx systems and created by Arthur Whitney. Over the years, my fascination has taken me on a journey through the art, science, and history of computer architecture. I still have plenty to learn but I think I’ve picked up enough knowledge that I’m confident I can help save you — my fellow programming language junkie — some time and give you a rundown on how the sausage is made in fast interpreted array programming language town.

My hope is that the things you learn from me in this series of articles might just make an impression on how you go about tackling problems in your day job and help keep the software you write simple and fast.

What you’ll be learning about in this series…

This will not be yet another blog post about how to write a lexer, parser and evaluator. Instead, the focus will be on how to use the underlying hardware to its fullest and to take advantage of the design choices that the brilliant hardware engineers at Intel, AMD and other firms have made.

You’ll be learning about simple and effective techniques to implement a grammar, memory allocation, pointer tagging, reference counting, operator dispatch, variable with integer and floating point data types, and dictionary and tabular data structures.

The relevant code that the articles will discuss can be found at https://github.com/rzwiren/lang. The language itself is just a toy, and on top of that it is still a work in progress at the time of writing. I’m trying my best to balance demonstrating the concepts that I want to demonstrate while keeping it small and simple.

In classic array programming language fashion, the code is dense. Maybe not Arthur Whitney dense, but still quite dense. It is my hope that you’ll begin to find an appreciation for this style. Although the code itself is dense, it is heavily commented. The main idea behind the style I use is that it ends up bringing the actual transformations and operations on data to the forefront as opposed to the names of the results of those transformations. In my experience it is often that short variable names can still convey a great deal of information on their purpose.

Something that I’ve enjoyed about this experience is that this is an effort of my own mind. All decisions are my own. I hope that this series of posts will help my readers get to know me a bit more as well.