Mostly intermediate representations and compiler book reviews

Author: dibyendumajumdar

_sources/compiler-books.rst.txt

@@ -15,30 +15,50 @@ I have 3 editions of these.
 These books are criticised today because of the excessive focus on lexical analysis and parsing techniques.
 While this is true, they do cover various aspects of a compiler backend such as intermediate representations and
 optimization techniques including peephole optimization, data flow analysis, register allocation etc.
-I found the description of the lattice in a data flow analysis quite accessible.
+I found the description of the lattice in a data flow analysis quite accessible. 
 
 The 2nd edition adopts a more mathematical presentation style, whereas the earlier editions present
 algorithms using pseudo code. I think the 1986 edition is the best.
 
+The dragon books are a bit dated in that newer techniques such as Static Single Assignment or Graph 
+Coloring Register Allocation etc. are not covered in any detail. I would even say that these books are not 
+useful if your goal is to work with SSA IR. 
+
 For a different take on 2nd edition see `Review of the second addition of the "Dragon Book" <https://gcc.gnu.org/wiki/Review_of_the_second_addition_of_the_Dragon_Book.>`_.
 
 Engineering a Compiler, 2nd Ed. Cooper & Torczon. 2012.
 =======================================================
 This is a more modern version of the Dragon book. It is less focused on the lexical analysis / parsing
-phases, and covers the later phases of a compiler in more detail. Exposition is similar to the Dragon book, i.e. mostly describes
-techniques conceptually, with some high level algorithm descriptions, but like the Dragon book, does not 
-go into detailed descriptions of algorithms.
+phases, and covers the later phases of a compiler in more detail. Exposition is similar to the Dragon book, i.e. describes
+techniques conceptually, and some algorithms are described in more detail using a form of pseudo code.
+
+Defines an intermediate language called ILOC, but this IR does not have support for function calls.
+
+In practice, I found this book helpful when implementing the Dominator algorithm and SSA transformation. However, it left out
+important parts in its coverage of SSA which meant that the algorithms as described do not work. For instance:
+
+* The SSA construction algorithm inserts Phis in blocks where the original variable is dead (semi-pruned SSA). This then
+  causes the renaming phase to fail as there is no available definition of the variable.
+* Liveness analysis does not cover SSA form and does not handle phis correctly.
+* Exiting out of SSA is described conceptually but the algorithms are not described in detail. 
+
+In practice though it is easy to recommend Engineering a Compiler over the Dragon books.
 
 Both this and the Dragon books describe ahead of time compilers and cover topics that are suited for procedural languages
 such as C or traditional Pascal or Fortran. They cover both front-end and back-end techniques; however, on the front-end
-side, interesting topics such as Object Orientation, Closures, Generics, 
-or Semantic analysis of more complex languages such as Java are not covered.
+side, interesting topics such as Object Orientation, Closures, Generics, Semantic analysis of more complex languages 
+such as Java are not covered.
 
 Modern Compiler Implementation in C. Appel. 1998. (Tiger book)
 ==============================================================
 This book takes a hands on tutorial like approach to describing how to implement both the front-end and back-end 
 of a compiler, using a toy language called Tiger as an example. Algorithms are described in pseudo code. 
-If I had to choose between the Dragon book, Engineering a compiler, and this book, I would pick this one.
+If I had to choose from the Dragon book, Engineering a compiler, and this book, I would pick this one and 
+Engineering a Compiler.
+
+It covers a lot of techniques, and usually presents algorithms in pseudo code form. I consulted this book
+when implementing SSA and SCCP, but the descriptions were not sufficiently comprehensive so that I had to 
+consult other material too.
 
 This book covers functional languages, closures, as well as Object Oriented languages such as Java. Type inference is 
 covered too.
@@ -47,49 +67,69 @@ Crafting a Compiler. Fischer, LeBlanc, Cytron. 2010.
 ====================================================
 The last couple of chapters are the most interesting - these focus on code generation and program optimization. 
 
-The 2nd edition of the book (with Cytron as co author) has a description of Static Single assignment that is 
-perhaps the most complete in all the books I cover here. The 1st edition describes data flow analysis in more 
-detail.
+The 2nd edition of the book (with Cytron as co author) has a description of Static Single assignment. However the
+description is based on a statement level IR, rather than one that uses Basic Blocks. Also, the algorithm for exiting
+SSA is not described. 
+
+The 1st edition describes data flow analysis in more detail, but does not cover SSA.
 
 Apart from the final two chapters, the rest of the book is about parsing and semantic analysis.
 
 Building an Optimizing Compiler. Bob Morgan. 1998.
 ==================================================
 I have the kindle edition which is very poor and hard to read. I wish I had a paper copy.
 
-This book is almost completely about the backend of the compiler. 
+This book is almost completely about the backend of the compiler. I consulted the description of SCCP and
+based my implementation at least in part on the descriptions. In particular I found some discussion about how to
+exploit local knowledge in conditional branches to handle null checks, which was useful, and not discussed in
+other books.
 
 Advanced Compiler Design & Implementation. Muchnick. 1997.
 ==========================================================
-I have the kindle edition, which is very poor and hard to read.
+I have the kindle edition, which is very poor quality and hard to read.
 
-This book is also mostly about the backend of a compiler, focusing on optimization.
+This book is mostly about the backend of a compiler, focusing on optimization.
 
 My impression is that this book describes many algorithms in detail. But when I tried to implement one of the
 simpler algorithms (18.1 Unreachable Code Elimination) I found that the description left out a 
-part (No_Path) of the algorithm. 
+part (No_Path) of the algorithm.
 
-This book describes the idea of multiple levels of intermediate representation, HIR, MIR and LIR.
+Introduces the idea of multiple levels of intermediate representation, HIR, MIR and LIR.
 I guess this has influenced many compiler implementations.
 
-Its coverage of SSA is rudimentary - I guess it was written when SSA was still very new.
+Its coverage of SSA is rudimentary - I guess it was written when SSA was still very new. Hence if you are
+working with SSA IR then you will need to consult other material.
+
+The Graph Coloring register allocation algorithm is presented in detail and is based on the paper by
+Preston Briggs.
 
 This book has a reputation of containing many errors, although I assume the latest printings have the errors
-fixed. 
+fixed.
 
 Despite its faults, it is a must have book if you want to learn about compiler construction.
 
 Retargetable C Compiler, A: Design and Implementation. Hanson & Fraser. 1995.
 =============================================================================
-Describes a production C compiler. Detailed dsecription of the actual compiler code.
+Describes a production C compiler. Contains detailed walkthrough of the actual compiler code.
 
-Weak on theoretical aspects, and limited by features of the compiler being described. 
+Weak on theoretical aspects, and limited by features of the compiler being described. The compiler
+implementation is a single pass code generator, hence its optimizing capabilities are limited.
+There is no coverage of data flow analysis or SSA as these weren't used by the implementation.
+
+In short this describes an old school C compiler that generates code fast, but lacks optimizations.
 
 Program Flow Analysis: Theory and Applications. Editors Muchnick, Jones. 1981.
 ==============================================================================
 Collection of essays on program analysis, by various authors. This is pre-SSA, hence a bit
 dated.
 
+SSA-Based Compiler Design - various authors
+===========================================
+An online version of this book is available `here <https://pfalcon.github.io/ssabook/latest/book-full.pdf>`_.
+This book is a collection of articles on various topics related to SSA. As such it presents more
+recent knowledge regarding SSA construction, optimizations based on SSA, and finally destruction and 
+register allocation. I will have more to say about this book as I use it.
+
 Other Book Reviews
 ==================
 * `List of compiler books <https://gcc.gnu.org/wiki/ListOfCompilerBooks>`_

_sources/index.rst.txt

@@ -61,9 +61,11 @@ Basic Front-End techniques
 Basic Back-end techniques
 =========================
 
-* Stack based vs register based Intermediate Representation
-* Control flow graphs and Basic Blocks
-* Bytecode VM with simple garbage collection
+.. toctree::
+   :maxdepth: 1
+   :caption: Backend Basics
+
+   intermediate-representations
 
 Basic Optimization techniques
 =============================

_sources/intermediate-representations.rst.txt

@@ -0,0 +1,204 @@
+============================
+Intermediate Representations
+============================
+
+An input program in the source language may go through many intermediate representations within
+a compiler before it is in a form ready for execution. 
+  
+One of the first such intermediate representations that we have seen is the
+the Abstract Syntax Tree (AST), which is mainly concerned with the grammar of the source language. 
+
+From the AST, we generate a different kind of intermediate representation, one that is more amenable 
+to the manipulations required during optimization and execution. There are many such representations; we will 
+limit ourselves to the following.
+
+* Stack based IR
+* Register based IR
+* Sea of Nodes IR
+
+Stack-Based IR
+==============
+
+The stack based IR encodes stack operations as part of the intermediate representation. Lets look at a simple 
+example::
+
+   func foo(n: Int)->Int {
+      return n+1;
+   }
+   
+Produces::
+
+   L0:
+	   load 0
+	   pushi 1
+	   addi
+	   jump L1
+   L1:
+
+The stack based IR is so called because many of the intructions in the IR push and pop values to/from an evaluation stack at 
+runtime. Above for example, we have the following instructions:
+
+* ``load 0`` - this pushes the value of the input parameter ``n`` to the stack. The ``0`` here identifies the location of the variable ``n``.
+* ``pushi 1`` - pushes the constant ``1`` to the stack.
+* ``addi`` - pops the two topmost values on the stack, and computes the sum and pushes this to the stack
+
+So at the end of the program we are left with the sum of ``n+1`` on the stack, and this forms the return 
+value of the function.
+
+In this IR, control flow can be represented either using labels and branching instructions, or by grouping 
+instructions into basic blocks, and linking basic blocks through jump instructions. These two approaches are
+equivalent, you can think of a label as indicating the start of a basic block, and a jump as ending
+a basic block.
+
+The idea is that inside a basic block, instructions executed linearly one after the other.
+Each basic block ends with a branching instruction, something like a goto or a conditional jump.
+
+Here is a simple example of input source code and the IR you might see::
+
+   func foo()->Int
+   {
+      return 1 == 1 && 2 == 2
+   }
+
+This results in IR that may look like this::
+
+   L0:
+	   pushi 1
+	   pushi 1
+	   eq
+	   cbr L2 L3
+   L2:
+	   pushi 2
+	   pushi 2
+	   eq
+	   jump L4
+   L3:
+	   pushi 0
+	   jump L4
+   L4:
+	   jump L1
+   L1:
+
+Each basic block begins with a label, which is just the unique name of the block.
+
+* The ``jump`` instruction transfers control from a basic block to another.
+* The ``cbr`` instruction is the conditional branch. It consumes the top most value from the stack, 
+  and if this value is true, then control is transferred to the first block, else to the second block.
+* The ``eq`` instruction pops the topmost two values from the stack, and replaces them with integer value
+  ``1`` or ``0``.
+
+Advantages
+----------
+* The IR is compact to represent in stored form as most instructions do not take have operands. 
+  This is a reason why many languages choose to encode their compiled code in
+  this form. Examples are Java, C#, Web Assembly.
+* The IR can be executed easily by an Interpreter.
+* Relatively easy to generate IR from an AST.
+
+Disadvantages
+-------------
+* Not easy to implement optimizations.
+* Harder to analyze the IR, although there are methods available to do so.
+
+Examples
+--------
+* `Example implementation in EeZee Programming Language <https://github.com/CompilerProgramming/ez-lang/tree/main/stackvm>`_.
+* `Java Specifications <https://docs.oracle.com/javase/specs/jvms/se24/html/jvms-6.html>`_.
+* `Web Assembly Specifications <https://webassembly.github.io/spec/core/syntax/instructions.html>`_.
+
+Register Based IR or Three-Address IR
+=====================================
+
+This intermediate representation uses named slots called virtual registers in the instruction when referencing
+values. Lets look at the same example we saw above::
+
+   func foo(n: Int)->Int {
+      return n+1;
+   }
+   
+Produces::
+
+   L0:
+      %t1 = n+1
+      ret %t1
+      goto  L1
+   L1:
+
+The instructions above are as follows:
+
+* ``%t1 = n+1`` - is a typical three-address instruction of the form ``result = value1 operator value2``. The name ``%t1`` 
+  refers to a temporary, whereas ``n`` refers to the input argument ``n``.
+* ``ret %t1`` - is the return instruction, in this instance it references the temporary.
+
+The virtual registers in the IR are so called because they do not map to real registers in the target physical machine.
+Instead these are just named slots in the abstract machine responsible for executing the IR. Typically, the abstract machine
+will assign each virtual register a unique location in its stack frame. So we still end up using the function's
+stack frame, but the IR references locations within the stack frame via these virtual names, rather than implicitly
+through push and pop instructions. During optimization some of the virtual registers will end up in real hardware registers.
+
+Control flow is represented the same way as for the stack IR. Revisiting the same source example from above, we get following 
+IR::
+
+   L0:
+      %t0 = 1==1
+      if %t0 goto L2 else goto L3
+   L2:
+      %t0 = 2==2
+      goto  L4
+   L3:
+      %t0 = 0
+      goto  L4
+   L4:
+      ret %t0
+      goto  L1
+   L1:
+
+
+Advantages
+----------
+* Readability: the flow of values is easier to trace, whereas with a stack IR you need to conceptualize a stack somewhere,
+  and track values being pushed and popped.
+* The IR can be executed easily by an Interpreter.
+* Most optimization algorithms can be applied to this form of IR.
+* The IR can represent Static Single Assignment (SSA) in a natural way.
+
+Disadvantages
+-------------
+* Each instruction has operands, hence representing the IR in serialized form takes more space.
+* Harder to generate the IR during compilation.
+
+Examples
+--------
+* `Example basic register IR in EeZee Programming Language <https://github.com/CompilerProgramming/ez-lang/tree/main/registervm>`_.
+* `Example register IR including SSA form and optimizations in EeZee Programming Language <https://github.com/CompilerProgramming/ez-lang/tree/main/optvm>`_. 
+* `LLVM instruction set <https://llvm.org/docs/LangRef.html#instruction-reference>`_.
+* `Android Dalvik IR <https://source.android.com/docs/core/runtime/dex-format>`_.
+
+Sea of Nodes IR
+===============
+The final example we will look at is known as the Sea of Nodes IR.
+
+It is quite different from the IRs we described above.
+
+The key features of this IR are:
+
+* Instructions are NOT organized into Basic Blocks - instead, intructions form a graph, where
+  each instruction has as its inputs the definitions it uses.
+* Instructions that produce data values are not directly bound to a Basic Block, instead they "float" around,
+  the order being defined purely in terms of the dependencies between the instructions.
+* Control flow is also represented in the same way, and control flows between control flow
+  instructions. Dependencies between data instructions and control intructions occur at few well
+  defined places.
+* The IR as described above cannot be readily executed, because to execute the IR, the instructions
+  must be scheduled; you can think of this as a process that puts the instructions into a traditional
+  Basic Block IR as described earlier.
+
+Describing Sea of Nodes IR is quite involved. For now, I direct you to the `Simple project <https://github.com/SeaOfNodes/Simple/tree/main>`_; this
+is an ongoing effort to explain the Sea of Nodes IR representation and how to implement it.
+
+Beyond how the IR is represented, the main benefits of the Sea of Nodes IR are that:
+
+* It is an SSA IR
+* Various optimizations such as peephole optimizations, value numbering and common subexpressions elimination,
+  dead code elimitation, occur as the IR is built.
+* This makes the SoN IR suitable for quick optimizations, suitable for Just-In-Time (JIT) compilers.

_sources/learning-resources.rst.txt

@@ -5,6 +5,10 @@ Learning Resources
 Courses
 =======
 
+COMP 512: Advanced Compiler Construction - Rice University, K. Cooper
+---------------------------------------------------------------------
+* `COMP 512 Lectures <https://www.clear.rice.edu/comp512/Lectures/>`_. Nice bibliography of important papers related to optimization.
+
 CS 6120: Advanced Compilers: The Self-Guided Online Course
 ----------------------------------------------------------