<?xml version="1.0" encoding="utf-8"?>
<feed xmlns="http://www.w3.org/2005/Atom" xml:base="https://p.ocmatos.com/">
  <title>pmatos rambles</title>
  <subtitle>Programming, sports and music - oh wow - enjoy!</subtitle>
  <link href="https://p.ocmatos.com/igalia.xml" rel="self"/>
  <link href="https://p.ocmatos.com/"/>
  <updated>2026-02-25T10:08:06Z</updated>
  <id>https://p.ocmatos.com/</id>
  <author>
    <name>Paulo Matos</name>
    <email>pmatos@igalia.com</email>
  </author>
  <entry>
    <title>FEX x87 Stack Optimization</title>
    <link href="https://p.ocmatos.com/blog/fex-x87-stack-optimization.html"/>
    <updated>2024-09-25T00:00:00Z</updated>
    <id>https://p.ocmatos.com/blog/fex-x87-stack-optimization.html</id>
    <content xml:lang="en" type="html">&lt;p&gt;In this blog post, I will describe my recent work on improving and optimizing the handling of x87 code in FEX. This work landed on July 22, 2024, through commit &lt;a href=&quot;https://github.com/FEX-Emu/FEX/commit/a1378f94ce8e7843d5a3cd27bc72847973f8e7ec&quot;&gt;https://github.com/FEX-Emu/FEX/commit/a1378f94ce8e7843d5a3cd27bc72847973f8e7ec&lt;/a&gt; (tests and updates to size benchmarks landed separately).&lt;/p&gt;
&lt;p&gt;FEX is an ARM64 emulator of Intel assembly and, as such, needs to be able to handle things as old as x87, which was the way we did floating point pre-SSE.&lt;/p&gt;
&lt;p&gt;Consider a function to calculate the well-known Pythagorean Identity:&lt;/p&gt;
&lt;pre class=&quot;language-c&quot;&gt;&lt;code class=&quot;language-c&quot;&gt;&lt;span class=&quot;token keyword&quot;&gt;float&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;PytIdent&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token keyword&quot;&gt;float&lt;/span&gt; x&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;
  &lt;span class=&quot;token keyword&quot;&gt;return&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;sinf&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;x&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;*&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;sinf&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;x&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;token operator&quot;&gt;+&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;cosf&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;x&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;*&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;cosf&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;x&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
&lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Here is how this can be translated into assembly:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;  sub rsp, 4
  movss [rsp], xmm0
  fld [rsp]
  fsincos
  fmul st0, st0
  fxch
  fmul st0, st0
  faddp
  fstp [rsp]
  movss xmm0, [rsp]
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;These instructions will take &lt;code&gt;x&lt;/code&gt; from register &lt;code&gt;xmm0&lt;/code&gt; and return the result of the identity (the number 1, as we expect) to the same register.&lt;/p&gt;
&lt;h1&gt;A brief visit to the x87 FPU&lt;/h1&gt;
&lt;p&gt;The x87 FPU consists of 8 registers (&lt;code&gt;R0&lt;/code&gt; to &lt;code&gt;R7&lt;/code&gt;), which are aliased to the MMX registers - &lt;code&gt;mm0&lt;/code&gt; to &lt;code&gt;mm7&lt;/code&gt;. The FPU uses them as a stack and tracks the top of the stack through the &lt;code&gt;TOS&lt;/code&gt; (Top of Stack) or &lt;code&gt;TOP&lt;/code&gt; (my preferred way to refer to it), which is kept in the control word. To understand how it all fits together, I will describe the three most important components of this setup for the purpose of this post:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;FPU Stack-based operation&lt;/li&gt;
&lt;li&gt;Status register&lt;/li&gt;
&lt;li&gt;Tag register&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;From the &lt;a href=&quot;https://cdrdv2.intel.com/v1/dl/getContent/671200&quot;&gt;Intel spec&lt;/a&gt;, these components look like this (image labels are from the Intel in case you need to refer to it):&lt;/p&gt;
&lt;div style=&quot;text-align:center;&quot;&gt;
	&lt;img alt=&quot;Patch Details&quot; class=&quot;post-photo&quot; src=&quot;https://p.ocmatos.com/img/2024-09-23-fex-x87-stack-optimization/Screenshot%20From%202024-09-23%2010-54-21.png&quot; /&gt;
&lt;/div&gt;
&lt;p&gt;The 8 registers are labelled as &lt;code&gt;ST0&lt;/code&gt;-&lt;code&gt;ST7&lt;/code&gt; relative to the position of &lt;code&gt;TOP&lt;/code&gt;. In the case above &lt;code&gt;TOP&lt;/code&gt; is 3; therefore, the 3rd register is &lt;code&gt;ST0&lt;/code&gt;. Since the stack grows down, &lt;code&gt;ST1&lt;/code&gt; is the 4th register and so on until it wraps around. When the FPU is initialized &lt;code&gt;TOP&lt;/code&gt; is set to 0. Once a value is loaded, the &lt;code&gt;TOP&lt;/code&gt; is decreased, its value becomes 7 and the first value added to the stack is in the 7th register. Because the MMX registers are aliased to the bottom 64 bits of the stack registers, we can also refer to them as &lt;code&gt;MM0&lt;/code&gt; to &lt;code&gt;MM7&lt;/code&gt;. This is not strictly correct because the MMX registers are 64 bits and the FPU registers are 80 bits but if we squint, they are basically the same.&lt;/p&gt;
&lt;p&gt;The spec provides an example of how the FPU stack works. I copy the image here verbatim for your understanding. Further information can be found in the spec itself.&lt;/p&gt;
&lt;div style=&quot;text-align:center;&quot;&gt;
	&lt;img alt=&quot;Patch Details&quot; class=&quot;post-photo&quot; src=&quot;https://p.ocmatos.com/img/2024-09-23-fex-x87-stack-optimization/Screenshot%20From%202024-09-23%2010-54-43.png&quot; /&gt;
&lt;/div&gt;
&lt;p&gt;The &lt;code&gt;TOP&lt;/code&gt; value is stored along with other flags in the FPU Status Word, which we won&#39;t discuss in detail here.&lt;/p&gt;
&lt;div style=&quot;text-align:center;&quot;&gt;
	&lt;img alt=&quot;Patch Details&quot; class=&quot;post-photo&quot; src=&quot;https://p.ocmatos.com/img/2024-09-23-fex-x87-stack-optimization/Screenshot%20From%202024-09-23%2010-54-53.png&quot; /&gt;
&lt;/div&gt;
&lt;p&gt;And the tag word marks which registers are valid or empty. The Intel spec also allows marking the registers as zero or special. At the moment, tags in FEX are binary, marking the register as valid or invalid. Instead of two bits per register, we only use one bit of information per register.&lt;/p&gt;
&lt;div style=&quot;text-align:center;&quot;&gt;
	&lt;img alt=&quot;Patch Details&quot; class=&quot;post-photo&quot; src=&quot;https://p.ocmatos.com/img/2024-09-23-fex-x87-stack-optimization/Screenshot%20From%202024-09-23%2010-55-21.png&quot; /&gt;
&lt;/div&gt;
&lt;h1&gt;Current issues&lt;/h1&gt;
&lt;p&gt;The way things worked in FEX before this patch landed is that stack based operations would essentially always lower to perform three operations:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Load stack operation argument register values from special memory location;&lt;/li&gt;
&lt;li&gt;Call soft-float library to perform the operation;&lt;/li&gt;
&lt;li&gt;Store result in special memory location;&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;For example, let&#39;s consider the instruction &lt;code&gt;FADDP st4, st0&lt;/code&gt;. This instruction performs &lt;code&gt;st0 &amp;lt;- f80add(st4, st0) &amp;amp;&amp;amp; pop()&lt;/code&gt;. In &lt;code&gt;x87.cpp&lt;/code&gt;, the lowering of this instruction involved the following steps:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Load current value of top from memory;&lt;/li&gt;
&lt;li&gt;With the current value of top, we can load the values of the registers in the instruction, in this case st4 and st0 since they are at a memory offset from top;&lt;/li&gt;
&lt;li&gt;Then we call the soft-float library function to perform 80-bit adds;&lt;/li&gt;
&lt;li&gt;Finally we store the result into st4 (which is a location in memory);&lt;/li&gt;
&lt;li&gt;And pop the stack, which means marking the current top value invalid and incrementing top;&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;In code with a lot of x87 usage, these operations become cumbersome and data movement makes the whole code extremely slow. Therefore, we redesigned the x87 system to cope better with stack operations. FEX support a reduced precision mode that uses 64bit floating points rather than 80 bits, avoiding the soft-float library calls and this work applies equally to the code in reduced precision mode.&lt;/p&gt;
&lt;h1&gt;The new pass&lt;/h1&gt;
&lt;p&gt;The main observation to understand the operation of the new pass is that when compiling a block of instructions, where multiple instructions are x87 instructions, before code generation we have an idea of what the stack will look like, and if we have complete stack information we can generate much better code than the one generated on a per-instruction basis.&lt;/p&gt;
&lt;p&gt;Instead of lowering each x87 instruction in x87.cpp to its components as discussed earlier, we simply generate stack operation IR instructions (all of these added new since there were no stack operations in the IR). Then an optimization pass goes through the IR on a per block basis and optimizes the operations and lowers them.&lt;/p&gt;
&lt;p&gt;Here’s a step-by-step explanation of how this scheme works for the code above. When the above code gets to the OpcodeDispatcher, there’ll be one function a reasonable amount of opcodes, but generally one per functionality, meaning that although there are several opcodes for different versions of fadd, there is a single function in OpcodeDispatcher for x87 (in x87.cpp) implementing it.&lt;/p&gt;
&lt;p&gt;Each of these functions will just transform the incoming instruction into an IR node that deals with the stack. These stack IR operations have Stack in its name. See &lt;a href=&quot;https://github.com/FEX-Emu/FEX/blob/FEX-2409/FEXCore/Source/Interface/IR/IR.json&quot;&gt;IR.json&lt;/a&gt; for a list of these.&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;x86 Asm&lt;/th&gt;
&lt;th&gt;IR nodes&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;fld qword [rsp]&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;LoadMem&lt;/code&gt; + &lt;code&gt;PushStack&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;fsincos&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;F80SinCosStack&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;fmul st0, st0&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;F80MulStack&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;fxch&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;F80StackXchange&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;fmul st0, st0&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;F80MulStack&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;faddp&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;F80AddStack&lt;/code&gt; + &lt;code&gt;PopStackDestroy&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;fstp qword [rsp]&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;StoreStackMemory&lt;/code&gt; + &lt;code&gt;PopStackDestroy&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;The IR will then look like:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;	%9 i64 = LoadRegister #0x4, GPR
	%10 i128 = LoadMem FPR, %9 i64, %Invalid, #0x10, SXTX, #0x1
	(%11 i0) PushStack %10 i128, %10 i128, #0x10, #0x1
	(%12 i0) F80SINCOSStack
	%13 i64 = Constant #0x0
	(%14 i8) StoreContext GPR, %13 i64, #0x3fa
	(%15 i0) F80MulStack #0x0, #0x0
	(%16 i0) F80StackXchange #0x1
	%17 i64 = Constant #0x0
	(%18 i8) StoreContext GPR, %17 i64, #0x3f9
	(%19 i0) F80MulStack #0x0, #0x0
	(%20 i0) F80AddStack #0x1, #0x0
	(%21 i0) PopStackDestroy
	%23 i64 = LoadRegister #0x4, GPR
	(%24 i0) StoreStackMemory %23 i64, i128, #0x1, #0x8
	(%25 i0) PopStackDestroy
&lt;/code&gt;&lt;/pre&gt;
&lt;h2&gt;The Good Case&lt;/h2&gt;
&lt;p&gt;Remember that before this pass, we would have never generated this IR as is. There were no stack operations, therefore each of these &lt;code&gt;F80MulStack&lt;/code&gt; operations, would generate loads from the MMX registers that are mapped into memory for each of the operands, call to &lt;code&gt;F80Mul&lt;/code&gt; softfloat library function and then a write to the memory mapped MMX register to write the result.&lt;/p&gt;
&lt;p&gt;However, once we get to the optimization pass a see a straight line block line this, we can step through the IR nodes and create a virtual stack with pointers to the IR nodes. I have sketched the stacks for the first few loops in the pass.&lt;/p&gt;
&lt;p&gt;The pass will only process &lt;em&gt;x87&lt;/em&gt; instructions, these are the one whose IR nodes are marked with &lt;code&gt;&amp;quot;X87: true&lt;/code&gt; in &lt;code&gt;IR.json&lt;/code&gt;. Therefore the first two instructions above:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;	%9 i64 = LoadRegister #0x4, GPR
	%10 i128 = LoadMem FPR, %9 i64, %Invalid, #0x10, SXTX, #0x1
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;are ignored. But &lt;code&gt;PushStack&lt;/code&gt; is not. &lt;code&gt;PushStack&lt;/code&gt; will literally push the node into the virtual stack.&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;	(%11 i0) PushStack %10 i128, %10 i128, #0x10, #0x1
&lt;/code&gt;&lt;/pre&gt;
&lt;div style=&quot;display: flex; justify-content: flex-end;&quot;&gt;
	&lt;img alt=&quot;Patch Details&quot; class=&quot;post-photo&quot; src=&quot;https://p.ocmatos.com/img/2024-09-23-fex-x87-stack-optimization/X87-example1.svg&quot; width=&quot;35%&quot; /&gt;
&lt;/div&gt;
&lt;p&gt;Internally before we push the node into our internal stack, we update the necessary values like the top of the stack. It starts at zero and it&#39;s decremented to seven, so the value &lt;code&gt;%10 i128&lt;/code&gt; is inserted in what we&#39;ll call &lt;code&gt;R7&lt;/code&gt;, now &lt;code&gt;ST0&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;Next up is the computation of &lt;code&gt;SIN&lt;/code&gt; and &lt;code&gt;COS&lt;/code&gt;. This will result in &lt;code&gt;SIN ST0&lt;/code&gt; replacing &lt;code&gt;ST0&lt;/code&gt; and then pushing &lt;code&gt;COS ST0&lt;/code&gt;, where the value in &lt;code&gt;ST0&lt;/code&gt; here is &lt;code&gt;%10&lt;/code&gt;.&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;	(%12 i0) F80SINCOSStack
&lt;/code&gt;&lt;/pre&gt;
&lt;div style=&quot;display: flex; justify-content: flex-end;&quot;&gt;
	&lt;img alt=&quot;Patch Details&quot; class=&quot;post-photo&quot; src=&quot;https://p.ocmatos.com/img/2024-09-23-fex-x87-stack-optimization/X87-example2.svg&quot; width=&quot;35%&quot; /&gt;
&lt;/div&gt;
&lt;p&gt;Then we have two instructions that we just don&#39;t deal with in the pass, these set the &lt;code&gt;C2&lt;/code&gt; flag to zero - as required by &lt;code&gt;FSINCOS&lt;/code&gt;.&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;	%13 i64 = Constant #0x0
	(%14 i8) StoreContext GPR, %13 i64, #0x3fa
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Followed by a multiplication of the stack element to square them. So we square &lt;code&gt;ST0&lt;/code&gt;.&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;	(%15 i0) F80MulStack #0x0, #0x0
&lt;/code&gt;&lt;/pre&gt;
&lt;div style=&quot;display: flex; justify-content: flex-end;&quot;&gt;
	&lt;img alt=&quot;Patch Details&quot; class=&quot;post-photo&quot; src=&quot;https://p.ocmatos.com/img/2024-09-23-fex-x87-stack-optimization/X87-example3.svg&quot; width=&quot;35%&quot; /&gt;
&lt;/div&gt;
&lt;p&gt;The next instruction swaps &lt;code&gt;ST0&lt;/code&gt; with &lt;code&gt;ST1&lt;/code&gt; so that we can square the sine value.&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;	(%16 i0) F80StackXchange #0x1
&lt;/code&gt;&lt;/pre&gt;
&lt;div style=&quot;display: flex; justify-content: flex-end;&quot;&gt;
	&lt;img alt=&quot;Patch Details&quot; class=&quot;post-photo&quot; src=&quot;https://p.ocmatos.com/img/2024-09-23-fex-x87-stack-optimization/X87-example4.svg&quot; width=&quot;35%&quot; /&gt;
&lt;/div&gt;
&lt;p&gt;Again, similarly two instructions that are ignored by the pass, which set flag &lt;code&gt;C1&lt;/code&gt; to zero - as required by &lt;code&gt;FXCH&lt;/code&gt;.&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;	%17 i64 = Constant #0x0
	(%18 i8) StoreContext GPR, %17 i64, #0x3f9
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;And again we square the value at the top of the stack.&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;	(%19 i0) F80MulStack #0x0, #0x0
&lt;/code&gt;&lt;/pre&gt;
&lt;div style=&quot;display: flex; justify-content: flex-end;&quot;&gt;
	&lt;img alt=&quot;Patch Details&quot; class=&quot;post-photo&quot; src=&quot;https://p.ocmatos.com/img/2024-09-23-fex-x87-stack-optimization/X87-example5.svg&quot; width=&quot;35%&quot; /&gt;
&lt;/div&gt;
&lt;p&gt;We are almost there - to finish off the computation we need to add these two values together.&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;	(%20 i0) F80AddStack #0x1, #0x0
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;In this case we add &lt;code&gt;ST1&lt;/code&gt; to &lt;code&gt;ST0&lt;/code&gt; and store the result in &lt;code&gt;ST1&lt;/code&gt;.&lt;/p&gt;
&lt;div style=&quot;display: flex; justify-content: flex-end;&quot;&gt;
	&lt;img alt=&quot;Patch Details&quot; class=&quot;post-photo&quot; src=&quot;https://p.ocmatos.com/img/2024-09-23-fex-x87-stack-optimization/X87-example6.svg&quot; width=&quot;35%&quot; /&gt;
&lt;/div&gt;
&lt;p&gt;This is followed by a pop of the stack.&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;	(%21 i0) PopStackDestroy
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;As expected this removed the &lt;code&gt;F80Mul&lt;/code&gt; from the top of the stack, leaving us with the result of the computation at the top.&lt;/p&gt;
&lt;div style=&quot;display: flex; justify-content: flex-end;&quot;&gt;
	&lt;img alt=&quot;Patch Details&quot; class=&quot;post-photo&quot; src=&quot;https://p.ocmatos.com/img/2024-09-23-fex-x87-stack-optimization/X87-example7.svg&quot; width=&quot;35%&quot; /&gt;
&lt;/div&gt;
&lt;p&gt;OK - there are three more instructions.&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;	%23 i64 = LoadRegister #0x4, GPR
	(%24 i0) StoreStackMemory %23 i64, i128, #0x1, #0x8
	(%25 i0) PopStackDestroy
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The first two store the top of the stack to memory, which is the result of our computation and which maths assures us it&#39;s 1. And then we pop the stack just to clean it up, although this is not strictly necessary since we are finished  anyway.&lt;/p&gt;
&lt;p&gt;Note that we have finished analysing the block, and played it through in our virtual stack. Now we can generate the instructions that are necessary to calculate the stack values, &lt;strong&gt;without&lt;/strong&gt; always load/stor&#39;ing them back to memory. In addition to generating these instructions we also generate instructions to update the tag word, update the top of stack value and save whatever remaining values in the stack at the end of the block into memory - into their respective memory mapped MMX registers. This is the good case where all the values necessary for the computation were found in the block. However, this is not necessarily always the case.&lt;/p&gt;
&lt;h2&gt;The Bad Case&lt;/h2&gt;
&lt;p&gt;Let&#39;s say that FEX for some reason breaks the beautiful block we had before into two:&lt;/p&gt;
&lt;p&gt;Block 1:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;	%9 i64 = LoadRegister #0x4, GPR
	%10 i128 = LoadMem FPR, %9 i64, %Invalid, #0x10, SXTX, #0x1
	(%11 i0) PushStack %10 i128, %10 i128, #0x10, #0x1
	(%12 i0) F80SINCOSStack
	%13 i64 = Constant #0x0
	(%14 i8) StoreContext GPR, %13 i64, #0x3fa
	(%15 i0) F80MulStack #0x0, #0x0
	(%16 i0) F80StackXchange #0x1
	%17 i64 = Constant #0x0
	(%18 i8) StoreContext GPR, %17 i64, #0x3f9
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Block 2:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;	(%19 i0) F80MulStack #0x0, #0x0
	(%20 i0) F80AddStack #0x1, #0x0
	(%21 i0) PopStackDestroy
	%23 i64 = LoadRegister #0x4, GPR
	(%24 i0) StoreStackMemory %23 i64, i128, #0x1, #0x8
	(%25 i0) PopStackDestroy
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;When we generate code for the first block, there are absolutely no blockers and we run through it exactly as we did before, except that after &lt;code&gt;StoreContext&lt;/code&gt;, we reach the end of the block and our virtual stack is in this state:&lt;/p&gt;
&lt;div style=&quot;display: flex; justify-content: flex-end;&quot;&gt;
	&lt;img alt=&quot;Patch Details&quot; class=&quot;post-photo&quot; src=&quot;https://p.ocmatos.com/img/2024-09-23-fex-x87-stack-optimization/X87-example4.svg&quot; width=&quot;35%&quot; /&gt;
&lt;/div&gt;
&lt;p&gt;At this point, we&#39;ll do what we said we always do before at the end of the block. We save the values in the virtual stack to the respective MMX registers in memory mapped locations: &lt;code&gt;%36&lt;/code&gt; is saved to &lt;code&gt;MM7&lt;/code&gt; and &lt;code&gt;%34&lt;/code&gt; is saved to &lt;code&gt;MM6&lt;/code&gt;. The &lt;code&gt;TOP&lt;/code&gt; is recorded to be 6 and the tags for &lt;code&gt;MM7&lt;/code&gt; and &lt;code&gt;MM6&lt;/code&gt; are marked to be valid. Then we exit the block. And start analyzing the following block.&lt;/p&gt;
&lt;p&gt;When we start the pass for the following block, our virtual stack is empty. We have no knowledge of virtual stacks for previous JITTed blocks, and we see this instruction:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;	(%19 i0) F80MulStack #0x0, #0x0
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;We cannot play through this instruction in our virtual stack because it multiplies the value in &lt;code&gt;ST0&lt;/code&gt; with itself and our virtual stack &lt;strong&gt;does not&lt;/strong&gt; have anything in &lt;code&gt;ST0&lt;/code&gt;. It&#39;s empty after all.&lt;/p&gt;
&lt;p&gt;In this case, we switch to the slow path that does exactly what we used to do before this work. We load the value of &lt;code&gt;ST0&lt;/code&gt; from memory by finding the current value for &lt;code&gt;TOP&lt;/code&gt; and loading the respective register value, issue the &lt;code&gt;F80Mul&lt;/code&gt; on this value and storing it back to the &lt;code&gt;MMX&lt;/code&gt; register. This is then done for the remainder of the block.&lt;/p&gt;
&lt;h2&gt;The Ugly Case&lt;/h2&gt;
&lt;p&gt;The ugly case is when we actually force the slow path, not because we don&#39;t have all the values in the virtual stack but out of necessity.&lt;/p&gt;
&lt;p&gt;There are a few instructions that trigger a &lt;code&gt;_StackForceSlow()&lt;/code&gt; and forces us down the slow path. These are: &lt;code&gt;FLDENV&lt;/code&gt;, &lt;code&gt;FRSTOR&lt;/code&gt;, &lt;code&gt;FLDCW&lt;/code&gt;. All of these load some part or all of the FPU environment from memory and it triggers a switch to the slow path so that values are loaded properly and we can then use those new values in the block. It is not impossible to think we could at least in some cases, load those values from memory, try to rebuild our virtual stack for the current block and continue without switching to the slow path but that hasn&#39;t been yet done.&lt;/p&gt;
&lt;p&gt;There are a few instructions other instructions that trigger a &lt;code&gt;_SyncStackToSlow()&lt;/code&gt;, which doesn&#39;t force us down the slow path but instead just updates the in-memory data from the virtual stack state. These are: &lt;code&gt;FNSTENV&lt;/code&gt;, &lt;code&gt;FNSAVE&lt;/code&gt;, and &lt;code&gt;FNSTSW&lt;/code&gt;. All of these store some part or all of the FPU environment into memory and it ensures that the values in-memory are up-to-date so that the store doesn&#39;t use obsolete values.&lt;/p&gt;
&lt;h1&gt;Results&lt;/h1&gt;
&lt;p&gt;In addition to the huge savings in data movement from the loads and stores of register values, &lt;code&gt;TOP&lt;/code&gt; and tags for each stack instruction, we also optimize memory copies through the stack. So if there&#39;s a bunch of value loaded to the stack, which are then stored to memory, we can just memcopy these values without going through the stack.&lt;/p&gt;
&lt;p&gt;In our code size validation benchmarks from various Steam games, we saw significant reductions in code size. These size code reductions saw a similarly high jump in FPS numbers. For example:&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Game&lt;/th&gt;
&lt;th style=&quot;text-align:center&quot;&gt;Before&lt;/th&gt;
&lt;th&gt;After&lt;/th&gt;
&lt;th&gt;Reduction&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Half-Life (Block 1)&lt;/td&gt;
&lt;td style=&quot;text-align:center&quot;&gt;2034&lt;/td&gt;
&lt;td&gt;1368&lt;/td&gt;
&lt;td&gt;32.7%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Half-Life (Block 2)&lt;/td&gt;
&lt;td style=&quot;text-align:center&quot;&gt;838&lt;/td&gt;
&lt;td&gt;551&lt;/td&gt;
&lt;td&gt;34.2%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Oblivion (Block 1)&lt;/td&gt;
&lt;td style=&quot;text-align:center&quot;&gt;34065&lt;/td&gt;
&lt;td&gt;25782&lt;/td&gt;
&lt;td&gt;24.3%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Oblivion (Block 2)&lt;/td&gt;
&lt;td style=&quot;text-align:center&quot;&gt;22089&lt;/td&gt;
&lt;td&gt;16635&lt;/td&gt;
&lt;td&gt;24.6%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Psychonauts (Block 1)&lt;/td&gt;
&lt;td style=&quot;text-align:center&quot;&gt;20545&lt;/td&gt;
&lt;td&gt;16357&lt;/td&gt;
&lt;td&gt;20.3%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Psychonauts Hot Loop (Matrix Swizzle)&lt;/td&gt;
&lt;td style=&quot;text-align:center&quot;&gt;2340&lt;/td&gt;
&lt;td&gt;165&lt;/td&gt;
&lt;td&gt;92.9%&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;h1&gt;Future work&lt;/h1&gt;
&lt;p&gt;The bulk of this work is now complete, however there are a few details that still need fixing. There&#39;s still the mystery of why &lt;a href=&quot;https://github.com/FEX-Emu/FEX/issues/3685&quot;&gt;Touhou: Luna Nights works with reduced precision&lt;/a&gt; but not normal precision (if anything you would expect it to be the other way around). Then there&#39;s also some work to be done on the interface between the FPU state and the MMX state as described in &amp;quot;&lt;a href=&quot;https://github.com/FEX-Emu/FEX/issues/3850&quot;&gt;MMX/x87 interaction is subtly broken&lt;/a&gt;&amp;quot;.&lt;/p&gt;
&lt;p&gt;This is something I am already focusing on and will continue working on this until complete.&lt;/p&gt;
</content>
  </entry>
  <entry>
    <title>WebAssembly Reference Types in Clang/LLVM</title>
    <link href="https://p.ocmatos.com/blog/webassembly-reference-types-in-clang-llvm.html"/>
    <updated>2023-06-13T00:00:00Z</updated>
    <id>https://p.ocmatos.com/blog/webassembly-reference-types-in-clang-llvm.html</id>
    <content xml:lang="en" type="html">&lt;h1&gt;WebAssembly Reference Types in LLVM&lt;/h1&gt;
&lt;h2&gt;Introduction&lt;/h2&gt;
&lt;p&gt;This post is an overview of the implementation of the &lt;a href=&quot;https://github.com/WebAssembly/reference-types&quot;&gt;WebAssembly Reference Types proposal&lt;/a&gt; in Clang/LLVM. It is a follow-up to two presentations:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href=&quot;https://www.youtube.com/watch?v=UxnUht3uWzc&quot;&gt;&amp;quot;A walk on the weird side: Opaque WebAssembly values and LLVM&amp;quot;&lt;/a&gt; (2021 LLVM Dev Mtg) by Andy Wingo, and&lt;/li&gt;
&lt;li&gt;&lt;a href=&quot;https://www.youtube.com/watch?v=lwPg_Vjs7p4&quot;&gt;&amp;quot;Clang, Clang: Who&#39;s there? WebAssembly!&amp;quot;&lt;/a&gt; (2022 LLVM Dev Mtg) by myself.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;This is the culmination of a couple of years of work started by myself, Andy Wingo and Alex Bradbury at Igalia. It was sponsored in its entirety by Bloomberg.&lt;/p&gt;
&lt;p&gt;The implementation followed a bottom up approach. We worked on the linker first, followed by the WebAssembly backend, MC layer and LLVM IR. Support for reference types has been available in LLVM IR for quite some time, but it was not exposed in Clang. Extending Clang to support values of reference type and tables was the last piece of the puzzle.&lt;/p&gt;
&lt;p&gt;For context for the remainder of the post, I recommend watching the two presentations linked above. In summary, the work consisted of implementing support for the following reference types (read WebAssembly reference types unless stated, not C++ reference types) in LLVM:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;code&gt;externref&lt;/code&gt;, which is a garbage collected reference type, and&lt;/li&gt;
&lt;li&gt;&lt;code&gt;funcref&lt;/code&gt;, which is a non-nullable reference to a function.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;In addition, we implemented support for WebAssembly tables, which are arrays of references. Since values of reference types cannot be stored to linear memory, tables are storage containers for these.&lt;/p&gt;
&lt;h2&gt;Linker and MC Layer&lt;/h2&gt;
&lt;h2&gt;LLVM IR&lt;/h2&gt;
&lt;h2&gt;Clang&lt;/h2&gt;
&lt;h2&gt;Conclusions and Future&lt;/h2&gt;
&lt;p&gt;There was a lot of work done in the last few years to reach this point. Not just by Igalia but kind and patient reviewers and the WebAssembly Tools team at Google who offered a lot of advice on the matter. However, there is still work to be done to completely implement the WebAssembly Reference Types proposal. We don&#39;t support initialization of tables yet and the C++ language extension to deal with these types is not very developer friendly since it&#39;s based on builtins. A more ergonomic approach to solve table access, for example, would be to allow &lt;code&gt;table.get&lt;/code&gt; and &lt;code&gt;table.set&lt;/code&gt; to be deal as array access. For the moment the project is on hold, due to lack of funding. If you are interested in sponsoring this work, please get in touch with me.&lt;/p&gt;
</content>
  </entry>
  <entry>
    <title>A tour of the `for..of` implementation for 32bits JSC</title>
    <link href="https://p.ocmatos.com/blog/a-tour-of-the-for-of-implementation-for-32bits-jsc.html"/>
    <updated>2020-11-20T00:00:00Z</updated>
    <id>https://p.ocmatos.com/blog/a-tour-of-the-for-of-implementation-for-32bits-jsc.html</id>
    <content xml:lang="en" type="html">&lt;h2&gt;Context&lt;/h2&gt;
&lt;p&gt;On April 18, 2020, Keith Miller from Apple committed a change titled &lt;a href=&quot;https://trac.webkit.org/changeset/260323/webkit&quot;&gt;Redesign how we do for-of iteration for JSArrays&lt;/a&gt;. This patch adds an intrinsic for the &lt;code&gt;for-of&lt;/code&gt; javascript construct when they are being iterated with the built-in &lt;code&gt;Symbol.iterator&lt;/code&gt;. However, the implementation is only for 64 bit architectures. At &lt;a href=&quot;https://www.igalia.com/&quot;&gt;Igalia&lt;/a&gt;, we maintain and support javascriptCore 32bits so on July 29, 2020, &lt;a href=&quot;https://trac.webkit.org/changeset/265036/webkit&quot;&gt;I committed support&lt;/a&gt; for the &lt;code&gt;for-of&lt;/code&gt; intrinsics work on 32 bit architectures as joint work with &lt;a href=&quot;https://caiolima.github.io/&quot;&gt;Caio Lima&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;In this blog post, I will attempt to give an overview of the implemented changes in revision &lt;a href=&quot;https://trac.webkit.org/changeset/265036/webkit&quot;&gt;r265036&lt;/a&gt; of WebKit which extends support of the &lt;code&gt;for-of&lt;/code&gt; intrinsics work to 32 bits. For the purposes of brevity I will focus on the iterator open intrinsic.&lt;/p&gt;
&lt;h2&gt;Overview&lt;/h2&gt;
&lt;p&gt;JSC consists of several tiers (compilation layers). These tiers start with LLInt (the Low-Level Interpreter), followed by the Baseline JIT, DFG (Data Flow Graph JIT), and FTL (Faster-Than-Light JIT). We will not discuss FTL here since we do not run the FTL tier for 32 bits.&lt;/p&gt;
&lt;p&gt;The patch for &lt;a href=&quot;https://trac.webkit.org/changeset/265036/webkit&quot;&gt;r265036&lt;/a&gt; touches architecture specific files, like &lt;a href=&quot;https://trac.webkit.org/browser/webkit/trunk/Source/javascriptCore/assembler/MacroAssemblerARMv7.h?rev=265036&quot;&gt;MacroAssemblerARMv7.h&lt;/a&gt; and &lt;a href=&quot;https://trac.webkit.org/browser/webkit/trunk/Source/javascriptCore/assembler/MacroAssemblerMIPS.h?rev=265036&quot;&gt;MacroAssemblerMIPS.h&lt;/a&gt;, however these changes are less relevant as we just add a couple of functions for emitting octet load/store instructions (&lt;code&gt;store8&lt;/code&gt;, &lt;code&gt;or8&lt;/code&gt;, etc). Instead, we will focus on changes to the tiers starting with the changes to LLInt.&lt;/p&gt;
&lt;h2&gt;javascript and for-of&lt;/h2&gt;
&lt;p&gt;I guess as Compiler Engineers we should try to understand what we have to implement first. Lets try to use an iterator implicitly through a &lt;a href=&quot;https://developer.mozilla.org/en-US/docs/Web/javascript/Reference/Statements/for...of&quot;&gt;&lt;code&gt;for-of&lt;/code&gt; loop&lt;/a&gt;.&lt;/p&gt;
&lt;pre class=&quot;language-javascript&quot;&gt;&lt;code class=&quot;language-javascript&quot;&gt;&lt;span class=&quot;token keyword&quot;&gt;const&lt;/span&gt; a &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;token string&quot;&gt;&quot;a&quot;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&quot;b&quot;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&quot;c&quot;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
&lt;span class=&quot;token keyword&quot;&gt;for&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token keyword&quot;&gt;const&lt;/span&gt; value &lt;span class=&quot;token keyword&quot;&gt;of&lt;/span&gt; a&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;
  console&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;log&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;value&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
&lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;
&lt;span class=&quot;token comment&quot;&gt;// =&gt;&lt;/span&gt;
&lt;span class=&quot;token comment&quot;&gt;// &quot;a&quot;&lt;/span&gt;
&lt;span class=&quot;token comment&quot;&gt;// &quot;b&quot;&lt;/span&gt;
&lt;span class=&quot;token comment&quot;&gt;// &quot;c&quot;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The &lt;code&gt;for-of&lt;/code&gt; statement gets an iterator from the array behind the scenes and loops through its values assigning them one by one to the variable &lt;code&gt;value&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;If we were to look at using the iterator explicitly, the above loop would be transformed to:&lt;/p&gt;
&lt;pre class=&quot;language-javascript&quot;&gt;&lt;code class=&quot;language-javascript&quot;&gt;&lt;span class=&quot;token keyword&quot;&gt;const&lt;/span&gt; a &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;token string&quot;&gt;&quot;a&quot;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&quot;b&quot;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&quot;c&quot;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
&lt;span class=&quot;token keyword&quot;&gt;const&lt;/span&gt; it &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; a&lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;Symbol&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;iterator&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
&lt;span class=&quot;token keyword&quot;&gt;let&lt;/span&gt; result &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; it&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;next&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
&lt;span class=&quot;token keyword&quot;&gt;while&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;!&lt;/span&gt;result&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;done&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;
  console&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;log&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;result&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;value&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
  result &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; it&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;next&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
&lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;So the symbol &lt;code&gt;Symbol.iterator&lt;/code&gt; returns a method that when called returns an iterator. This iterator has a &lt;code&gt;next&lt;/code&gt; function to get a result object. The result object has a &lt;code&gt;done&lt;/code&gt; property indicating if we are done iterating and a value which is the current value in the iteration. There&#39;s something to keep in mind though. The &lt;code&gt;done&lt;/code&gt; property is optional if its value would be &lt;code&gt;false&lt;/code&gt; and the &lt;code&gt;value&lt;/code&gt; property is optional if its value would be &lt;code&gt;undefined&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;Armed with this knowledge, &lt;a href=&quot;https://trac.webkit.org/changeset/260323/webkit&quot;&gt;revision 260323&lt;/a&gt; which implements &lt;code&gt;for-of&lt;/code&gt; intrinsics for 64 bits, makes a lot more sense. It implements two intrinsics &lt;code&gt;op_iterator_open&lt;/code&gt; and &lt;code&gt;op_iterator_next&lt;/code&gt;. The commit message explains the semantics of each operator as follows:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;The &lt;code&gt;op_iterator_open&lt;/code&gt; bytecode is semantically the same as:&lt;/li&gt;
&lt;/ul&gt;
&lt;pre&gt;&lt;code&gt;iterator = symbolIterator.@call(iterable);
next = iterator.next;
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;where iterable is the rhs of the &lt;code&gt;for-of&lt;/code&gt; and &lt;code&gt;symbolIterator&lt;/code&gt; is the result of running &lt;code&gt;iterable.symbolIterator&lt;/code&gt;;&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;The &lt;code&gt;op_iterator_next&lt;/code&gt; bytecode is semantically the same as:&lt;/li&gt;
&lt;/ul&gt;
&lt;pre&gt;&lt;code&gt;nextResult = next.@call(iterator);
done = nextResult.done;
value = done ? (undefined / bottom) : nextResult.value;
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;where &lt;code&gt;nextResult&lt;/code&gt; is a temporary (the value &lt;code&gt;VirtualRegister&lt;/code&gt; in the LLInt/Baseline and a &lt;code&gt;tmp&lt;/code&gt; in the DFG);&lt;/p&gt;
&lt;p&gt;We can now proceed to understand how to implement these on 32bits.&lt;/p&gt;
&lt;h2&gt;LLInt&lt;/h2&gt;
&lt;p&gt;LLInt is a bytecode interpreter implemented using a DSL (Domain Specific Language). The compiler for it is &lt;a href=&quot;https://github.com/WebKit/webkit/tree/master/Source/javascriptCore/offlineasm&quot;&gt;implemented in Ruby&lt;/a&gt;. The &lt;code&gt;offlineasm&lt;/code&gt; compiler reads this DSL and outputs a bytecode interpreter in native assembler, for a few supported architectures, or cpp. The cpp backend is known as Cloop and if you enable it, to for example compile JSC for an IBM System Z9, the JSC build system generates the cpp interpreter, compiles it and then links it with JSC. If you do enable Cloop, then none of the remaining JIT tiers are available. This is as much as we will discuss about Cloop as both ARMv7 and MIPS have LLInt support, so from here on out we will assume we are working with the native assembly backend for &lt;code&gt;offlineasm&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;The source code for LLInt (input to &lt;code&gt;offlineasm&lt;/code&gt;) lives in &lt;a href=&quot;https://github.com/WebKit/webkit/tree/master/Source/javascriptCore/llint&quot;&gt;&lt;code&gt;WebKit/Source/javascriptCore&lt;/code&gt;&lt;/a&gt;. The entry point is at &lt;a href=&quot;https://github.com/WebKit/webkit/blob/master/Source/javascriptCore/llint/LowLevelInterpreter.asm&quot;&gt;&lt;code&gt;LowLevelInterpreter.asm&lt;/code&gt;&lt;/a&gt; and it starts with a bunch of constant definitions followed by something you&#39;ll see often: a macro.&lt;/p&gt;
&lt;p&gt;At the time of writing, the first one is &lt;a href=&quot;https://github.com/WebKit/webkit/blob/ce2bff038035bac33352237652d33ff795a69959/Source/javascriptCore/llint/LowLevelInterpreter.asm#L319&quot;&gt;&lt;code&gt;nextInstruction&lt;/code&gt;&lt;/a&gt;:&lt;/p&gt;
&lt;pre class=&quot;language-ruby&quot;&gt;&lt;code class=&quot;language-ruby&quot;&gt;macro nextInstruction&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;
    loadb &lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;token constant&quot;&gt;PB&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;token constant&quot;&gt;PC&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; t0
    leap _g_opcodeMap&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; t1
    jmp &lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;t1&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; t0&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; PtrSize&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; BytecodePtrTag
&lt;span class=&quot;token keyword&quot;&gt;end&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Each time you call elsewhere &lt;code&gt;nextInstruction&lt;/code&gt; (which actually starts processing the next bytecode), the call will be replaced by the three (low-level) instructions in the body of the macro. These instructions &lt;code&gt;loadb&lt;/code&gt;, &lt;code&gt;leap&lt;/code&gt; and &lt;code&gt;jmp&lt;/code&gt; are defined on per-architecture in &lt;a href=&quot;https://github.com/WebKit/webkit/blob/master/Source/javascriptCore/offlineasm/arm.rb&quot;&gt;&lt;code&gt;arm.rb&lt;/code&gt;&lt;/a&gt;, &lt;a href=&quot;https://github.com/WebKit/webkit/blob/master/Source/javascriptCore/offlineasm/arm64.rb&quot;&gt;&lt;code&gt;arm64.rb&lt;/code&gt;&lt;/a&gt;, &lt;a href=&quot;https://github.com/WebKit/webkit/blob/master/Source/javascriptCore/offlineasm/arm64e.rb&quot;&gt;&lt;code&gt;arm64e.rb&lt;/code&gt;&lt;/a&gt;, &lt;a href=&quot;https://github.com/WebKit/webkit/blob/master/Source/javascriptCore/offlineasm/mips.rb&quot;&gt;&lt;code&gt;mips.rb&lt;/code&gt;&lt;/a&gt;, and &lt;a href=&quot;https://github.com/WebKit/webkit/blob/master/Source/javascriptCore/offlineasm/x86.rb&quot;&gt;&lt;code&gt;x86.rb&lt;/code&gt;&lt;/a&gt;. Listing of all available instructions are in &lt;a href=&quot;https://github.com/WebKit/webkit/blob/master/Source/javascriptCore/offlineasm/instructions.rb&quot;&gt;&lt;code&gt;instructions.rb&lt;/code&gt;&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;So for 32bits arm:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;the &lt;code&gt;loadb&lt;/code&gt; will generate a &lt;a href=&quot;https://github.com/WebKit/webkit/blob/b63199a0f04a5f7a435cb91e914419d2fd4f50d7/Source/javascriptCore/offlineasm/arm.rb#L447&quot;&gt;&lt;code&gt;ldrb&lt;/code&gt;&lt;/a&gt;,&lt;/li&gt;
&lt;li&gt;the &lt;code&gt;leap&lt;/code&gt; will generate a &lt;a href=&quot;https://github.com/WebKit/webkit/blob/b63199a0f04a5f7a435cb91e914419d2fd4f50d7/Source/javascriptCore/offlineasm/arm.rb#L659&quot;&gt;load effective address&lt;/a&gt; through an &lt;code&gt;add&lt;/code&gt; whose format depends on the first operand of &lt;code&gt;leap&lt;/code&gt;, and&lt;/li&gt;
&lt;li&gt;the &lt;code&gt;jmp&lt;/code&gt; will generate either a &lt;a href=&quot;https://github.com/WebKit/webkit/blob/b63199a0f04a5f7a435cb91e914419d2fd4f50d7/Source/javascriptCore/offlineasm/arm.rb#L597&quot;&gt;&lt;code&gt;b&lt;/code&gt; or &lt;code&gt;mov&lt;/code&gt; instruction&lt;/a&gt; which depends on if the first operand of &lt;code&gt;jmp&lt;/code&gt; is a label.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;The compilation of this DSL to assembler is done during JSC build-time which means that by the time the interpreter is called there&#39;s no execution of ruby code.&lt;/p&gt;
&lt;p&gt;Lets start by implementing the bytecode &lt;code&gt;op_iterator_open&lt;/code&gt;, which before looked like this in &lt;a href=&quot;https://github.com/WebKit/webkit/blob/99cee626c89e192a5fd211b6ba59be68b40c310b/Source/javascriptCore/llint/LowLevelInterpreter32_64.asm#L2645&quot;&gt;&lt;code&gt;LowLevelInterpreter32_64.asm&lt;/code&gt;&lt;/a&gt;:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;llintOp(op_iterator_open, OpIteratorOpen, macro (size, get, dispatch)
    defineOSRExitReturnLabel(op_iterator_open, size)
    break
    if C_LOOP or C_LOOP_WIN
        # Insert superflous call return labels for Cloop.
        cloopCallJSFunction a0 # symbolIterator
        cloopCallJSFunction a0 # get next
    end
end)
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Before proceeding a clarification of what this is. This is essentially a top-level call to the macro &lt;code&gt;llintOp&lt;/code&gt; with three arguments:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;&lt;code&gt;op_iterator_open&lt;/code&gt;,&lt;/li&gt;
&lt;li&gt;&lt;code&gt;OpIteratorOpen&lt;/code&gt;, and&lt;/li&gt;
&lt;li&gt;&lt;code&gt;macro(size, get, dispatch) ... end&lt;/code&gt;&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;Argument &lt;code&gt;3&lt;/code&gt; is an anonymous macro passed as a first-class citizen to &lt;code&gt;llintOp&lt;/code&gt; which can then call it. The macro &lt;code&gt;llintOp&lt;/code&gt; is defined in &lt;a href=&quot;https://github.com/WebKit/webkit/blob/master/Source/javascriptCore/llint/LowLevelInterpreter.asm&quot;&gt;&lt;code&gt;LowLevelInterpreter.asm&lt;/code&gt;&lt;/a&gt; and the declaration looks like:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;macro llintOp(opcodeName, opcodeStruct, fn)
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The anonymous macro defined in the &lt;code&gt;op_iterator_open&lt;/code&gt; is passed to the &lt;code&gt;llintOp&lt;/code&gt; macro and can be called as &lt;code&gt;fn(...)&lt;/code&gt;. Most of the code organization in LLInt is like this and you should get used to reading it.&lt;/p&gt;
&lt;p&gt;As we require access to the metadata for the implementation of &lt;code&gt;op_iterator_open&lt;/code&gt; we change the declaration slightly and add a few commonly defined macros:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;llintOpWithMetadata(op_iterator_open, OpIteratorOpen, macro (size, get, dispatch, metadata, return)
    macro fastNarrow()
        callSlowPath(_iterator_open_try_fast_narrow)
    end
    macro fastWide16()
        callSlowPath(_iterator_open_try_fast_wide16)
    end
    macro fastWide32()
        callSlowPath(_iterator_open_try_fast_wide32)
    end
    size(fastNarrow, fastWide16, fastWide32, macro (callOp) callOp() end)
    bbeq r1, constexpr IterationMode::Generic, .iteratorOpenGeneric
    dispatch()
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;There are two execution paths that can be taken at this point:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;Fast path: this path is currently implemented in cpp and the code resides in &lt;a href=&quot;https://github.com/WebKit/webkit/blob/89ece25c12467354e836b224948db00fc21edf11/Source/javascriptCore/runtime/CommonSlowPaths.cpp#L856&quot;&gt;&lt;code&gt;CommonSlowPaths.cpp:iterator_open_try_fast&lt;/code&gt;&lt;/a&gt;. If it succeeds then the branch &lt;code&gt;bbeq&lt;/code&gt; is not taken and the control flow dispatches to execute the next bytecode instruction.&lt;/li&gt;
&lt;li&gt;Slow path: the generic path is taken if the fast path fails. If it fails, then it &lt;a href=&quot;https://github.com/WebKit/webkit/blob/89ece25c12467354e836b224948db00fc21edf11/Source/javascriptCore/runtime/CommonSlowPaths.cpp#L898&quot;&gt;sets &lt;code&gt;r1&lt;/code&gt; with the value &lt;code&gt;constexpr IterationMode::Generic&lt;/code&gt;&lt;/a&gt;. In the 32bits ABI &lt;code&gt;r1&lt;/code&gt; is the tag of the return value.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;All the code that follows is part of the slow path and the next step is:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;.iteratorOpenGeneric:
    macro gotoGetByIdCheckpoint()
        jmp .getByIdStart
    end

    macro getCallee(dst)
        get(m_symbolIterator, dst)
    end

    macro getArgumentIncludingThisStart(dst)
        getu(size, OpIteratorOpen, m_stackOffset, dst)
    end

    macro getArgumentIncludingThisCount(dst)
        move 1, dst
    end

    callHelper(op_iterator_open,                    # opcodeName
               _llint_slow_path_iterator_open_call, # slowPath
               OpIteratorOpen,                      # opcodeStruct
               m_iteratorProfile,                   # valueProfileName
               m_iterator,                          # dstVirtualRegister
               prepareForRegularCall,               # prepareCall
               size,                                # size
               gotoGetByIdCheckpoint,               # dispatch
               metadata,                            # metadata
               getCallee,                           # getCallee
               getArgumentIncludingThisStart,       # getArgumentStart
               getArgumentIncludingThisCount)       # getArgumentCountIncludingThis
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Here we define a few helper macros required by &lt;code&gt;callHelper&lt;/code&gt;. I have commented on the &lt;code&gt;callHelper&lt;/code&gt; call with all the argument names to ease understanding what the function does. This is a helper function to simplify the call to the method stored in the &lt;code&gt;Symbol.Iterator&lt;/code&gt; property.&lt;/p&gt;
&lt;p&gt;The implementation of the &lt;code&gt;op_iterator_open&lt;/code&gt; in LLInt ends up with:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;.getByIdStart:
	macro storeNextAndDispatch(valueTag, valuePayload)
        move valueTag, t2
        move valuePayload, t3
        get(m_next, t1)
        storei t2, TagOffset[cfr, t1, 8]
        storei t3, PayloadOffset[cfr, t1, 8]
 	    dispatch()
 	end

    # We need to load m_iterator into t3 because that&#39;s where
 	# performGetByIDHelper expects the base object   
 	loadVariable(get, m_iterator, t3, t0, t3)
 	bineq t0, CellTag, .iteratorOpenGenericGetNextSlow
 	performGetByIDHelper(OpIteratorOpen,                  # opcodeStruct
                         m_modeMetadata,                  # modeMetadataName
                         m_nextProfile,                   # valueProfileName
                         .iteratorOpenGenericGetNextSlow, # slowLabel
                         size,                            # size
                         metadata,                        # metadata
                         storeNextAndDispatch)            # return

.iteratorOpenGenericGetNextSlow:
 	callSlowPath(_llint_slow_path_iterator_open_get_next)
 	dispatch()
end)
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The &lt;code&gt;callHelper&lt;/code&gt; above placed an iterator in register &lt;code&gt;t3&lt;/code&gt;, so we check what type of tag the iterator has. If the iterator is not a cell (tagged with &lt;code&gt;CellTag&lt;/code&gt; - a cell is essentially anything that&#39;s heap allocated) then we jump to the slow path. If, however, the iterator is an object we call &lt;code&gt;storeNextAndDispatch&lt;/code&gt; through &lt;code&gt;performGetByIDHelper&lt;/code&gt; which does a few useful checks. &lt;code&gt;storeNextAndDispatch&lt;/code&gt; initializes the iterator and dispatches to the next bytecode.&lt;/p&gt;
&lt;p&gt;The &lt;code&gt;op_iterator_next&lt;/code&gt; will follow the same pattern but instead implement the semantics of &lt;code&gt;iterator.next&lt;/code&gt;. Next we will see how a similar pattern is implemented in the Baseline JIT compiler.&lt;/p&gt;
&lt;h2&gt;Baseline&lt;/h2&gt;
&lt;p&gt;The Baseline JIT source code lives inside &lt;a href=&quot;https://github.com/WebKit/webkit/tree/master/Source/javascriptCore/jit&quot;&gt;Source/javascriptCore/jit&lt;/a&gt;. The part of the JIT compiler the deals with 32 bit platforms, mostly lives in &lt;a href=&quot;https://github.com/WebKit/webkit/blob/master/Source/javascriptCore/jit/JITCall32_64.cpp&quot;&gt;Source/javascriptCore/jit/JITCall32_64.cpp&lt;/a&gt;. It is responsible to generate assembly code for a given bytecode stream as input. This explains the resemblances we have between LLInt operations, since we need to emit code that has the same semantics as the LLInt operation. This compiler applies barely any compiler optimizations, since the goal is to generate binary code as quickly as it can and its major gain on performance is due to the removal of LLInt instruction dispatch.
The Baseline compiler is a heavy user of JSC’s AssemblerHelpers. This is an essential API that’s used to abstract the code generation for the multiple architectures JSC supports and also abstracts some operations that are quite common on LLInt like &lt;code&gt;branchIfNotCell&lt;/code&gt; or &lt;code&gt;branchIfNotEmpty&lt;/code&gt;.  It also implements a set of operations that are very helpful when we need to implement JIT support for a new opcode, such as &lt;code&gt;emitGetVirtualRegister&lt;/code&gt;, &lt;code&gt;emitPutVirtualRegister&lt;/code&gt;, and &lt;code&gt;callOperation&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;Lets look at how we implement the functions &lt;code&gt;emit_op_iterator_open&lt;/code&gt;, &lt;code&gt;emitSlow_op_iterator_open&lt;/code&gt; and the corresponding &lt;code&gt;next&lt;/code&gt; versions of those.&lt;/p&gt;
&lt;pre class=&quot;language-cpp&quot;&gt;&lt;code class=&quot;language-cpp&quot;&gt;&lt;span class=&quot;token keyword&quot;&gt;void&lt;/span&gt; &lt;span class=&quot;token class-name&quot;&gt;JIT&lt;/span&gt;&lt;span class=&quot;token double-colon punctuation&quot;&gt;::&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;emit_op_iterator_open&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token keyword&quot;&gt;const&lt;/span&gt; Instruction&lt;span class=&quot;token operator&quot;&gt;*&lt;/span&gt; instruction&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;
&lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;
    &lt;span class=&quot;token keyword&quot;&gt;auto&lt;/span&gt; bytecode &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; instruction&lt;span class=&quot;token operator&quot;&gt;-&gt;&lt;/span&gt;&lt;span class=&quot;token generic-function&quot;&gt;&lt;span class=&quot;token function&quot;&gt;as&lt;/span&gt;&lt;span class=&quot;token generic class-name&quot;&gt;&lt;span class=&quot;token operator&quot;&gt;&amp;lt;&lt;/span&gt;OpIteratorOpen&lt;span class=&quot;token operator&quot;&gt;&gt;&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
    &lt;span class=&quot;token keyword&quot;&gt;auto&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;*&lt;/span&gt; tryFastFunction &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;&amp;amp;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;
        &lt;span class=&quot;token keyword&quot;&gt;switch&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;instruction&lt;span class=&quot;token operator&quot;&gt;-&gt;&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;width&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;
        &lt;span class=&quot;token keyword&quot;&gt;case&lt;/span&gt; Narrow&lt;span class=&quot;token operator&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;token keyword&quot;&gt;return&lt;/span&gt; iterator_open_try_fast_narrow&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
        &lt;span class=&quot;token keyword&quot;&gt;case&lt;/span&gt; Wide16&lt;span class=&quot;token operator&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;token keyword&quot;&gt;return&lt;/span&gt; iterator_open_try_fast_wide16&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
        &lt;span class=&quot;token keyword&quot;&gt;case&lt;/span&gt; Wide32&lt;span class=&quot;token operator&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;token keyword&quot;&gt;return&lt;/span&gt; iterator_open_try_fast_wide32&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
        &lt;span class=&quot;token keyword&quot;&gt;default&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;RELEASE_ASSERT_NOT_REACHED&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
        &lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;
    &lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;

    JITSlowPathCall &lt;span class=&quot;token function&quot;&gt;slowPathCall&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token keyword&quot;&gt;this&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; instruction&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; tryFastFunction&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
    slowPathCall&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;call&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
    Jump fastCase &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;branch32&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;NotEqual&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; GPRInfo&lt;span class=&quot;token double-colon punctuation&quot;&gt;::&lt;/span&gt;returnValueGPR2&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;TrustedImm32&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token generic-function&quot;&gt;&lt;span class=&quot;token function&quot;&gt;static_cast&lt;/span&gt;&lt;span class=&quot;token generic class-name&quot;&gt;&lt;span class=&quot;token operator&quot;&gt;&amp;lt;&lt;/span&gt;&lt;span class=&quot;token keyword&quot;&gt;uint32_t&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;&gt;&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;IterationMode&lt;span class=&quot;token double-colon punctuation&quot;&gt;::&lt;/span&gt;Generic&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;It is not hard to see some resemblance between this code and the one shown for LLInt&#39;s &lt;code&gt;op_iterator_open&lt;/code&gt;. Once again we see if we can apply the fast case of this iterator and if not, we call the slow path.&lt;/p&gt;
&lt;pre class=&quot;language-cpp&quot;&gt;&lt;code class=&quot;language-cpp&quot;&gt;    &lt;span class=&quot;token generic-function&quot;&gt;&lt;span class=&quot;token function&quot;&gt;compileOpCall&lt;/span&gt;&lt;span class=&quot;token generic class-name&quot;&gt;&lt;span class=&quot;token operator&quot;&gt;&amp;lt;&lt;/span&gt;OpIteratorOpen&lt;span class=&quot;token operator&quot;&gt;&gt;&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;instruction&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; m_callLinkInfoIndex&lt;span class=&quot;token operator&quot;&gt;++&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;

    &lt;span class=&quot;token function&quot;&gt;advanceToNextCheckpoint&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
    
    &lt;span class=&quot;token comment&quot;&gt;// call result (iterator) is in regT1 (tag)/regT0 (payload)&lt;/span&gt;
    &lt;span class=&quot;token keyword&quot;&gt;const&lt;/span&gt; Identifier&lt;span class=&quot;token operator&quot;&gt;*&lt;/span&gt; ident &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;token operator&quot;&gt;&amp;amp;&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;vm&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;propertyNames&lt;span class=&quot;token operator&quot;&gt;-&gt;&lt;/span&gt;next&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
    
    &lt;span class=&quot;token function&quot;&gt;emitJumpSlowCaseIfNotJSCell&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;regT1&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The result of the call to the &lt;code&gt;Symbol.iterator&lt;/code&gt; method is again split, in 32bits machines, between a tag (in &lt;code&gt;regT1&lt;/code&gt;) and a payload (in &lt;code&gt;regT0&lt;/code&gt;). We jump to the slow case if the result is not a cell.&lt;/p&gt;
&lt;pre class=&quot;language-cpp&quot;&gt;&lt;code class=&quot;language-cpp&quot;&gt;    GPRReg tagIteratorGPR &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; regT1&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
    GPRReg payloadIteratorGPR &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; regT0&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;

    GPRReg tagNextGPR &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; tagIteratorGPR&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
    GPRReg payloadNextGPR &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; payloadIteratorGPR&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;

    JITGetByIdGenerator &lt;span class=&quot;token function&quot;&gt;gen&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;
        m_codeBlock&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt;
        &lt;span class=&quot;token function&quot;&gt;CodeOrigin&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;m_bytecodeIndex&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt;
        &lt;span class=&quot;token function&quot;&gt;CallSiteIndex&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;BytecodeIndex&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;m_bytecodeIndex&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;offset&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt;
        &lt;span class=&quot;token class-name&quot;&gt;RegisterSet&lt;/span&gt;&lt;span class=&quot;token double-colon punctuation&quot;&gt;::&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;stubUnavailableRegisters&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt;
        &lt;span class=&quot;token class-name&quot;&gt;CacheableIdentifier&lt;/span&gt;&lt;span class=&quot;token double-colon punctuation&quot;&gt;::&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;createFromImmortalIdentifier&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;ident&lt;span class=&quot;token operator&quot;&gt;-&gt;&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;impl&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt;
        &lt;span class=&quot;token function&quot;&gt;JSValueRegs&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;tagIteratorGPR&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; payloadIteratorGPR&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt;
        &lt;span class=&quot;token function&quot;&gt;JSValueRegs&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;tagNextGPR&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; payloadNextGPR&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt;
        AccessType&lt;span class=&quot;token double-colon punctuation&quot;&gt;::&lt;/span&gt;GetById&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;At this point we generate the access to the &lt;code&gt;next&lt;/code&gt; property of the iterator and ensure it is initialized.&lt;/p&gt;
&lt;pre class=&quot;language-cpp&quot;&gt;&lt;code class=&quot;language-cpp&quot;&gt;    gen&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;generateFastPath&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;*&lt;/span&gt;&lt;span class=&quot;token keyword&quot;&gt;this&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
    &lt;span class=&quot;token function&quot;&gt;addSlowCase&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;gen&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;slowPathJump&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
    m_getByIds&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;append&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;gen&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;

    &lt;span class=&quot;token function&quot;&gt;emitValueProfilingSite&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;bytecode&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;metadata&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;m_codeBlock&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
    &lt;span class=&quot;token function&quot;&gt;emitPutVirtualRegister&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;bytecode&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;m_next&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;JSValueRegs&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;tagNextGPR&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; payloadNextGPR&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;

    fastCase&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;link&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token keyword&quot;&gt;this&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
&lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The function finishes with some boilerplate code for linking and site profiling. However, most importantly it ensures that the values for the next bytecode are put into the correct registers in order for execution to continue. This function also shows a usage of another important kind of helper: branch instructions. Lets refocus just on the important pieces of &lt;code&gt;emit_op_iterator_open&lt;/code&gt; the branch helper usage.&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;void JIT::emit_op_iterator_open(const Instruction* instruction)
{
    // ...
    JITSlowPathCall slowPathCall(this, instruction, tryFastFunction);
    slowPathCall.call();
    Jump fastCase = branch32(NotEqual, GPRInfo::returnValueGPR2, TrustedImm32(static_cast&amp;lt;uint32_t&amp;gt;(IterationMode::Generic)));
    // ...
    fastCase.link(this);
}
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The code above will generate assembly code that branches to the end of &lt;code&gt;emit_op_iterator_open&lt;/code&gt; if &lt;code&gt;returnValueGPR2&lt;/code&gt; is not equal to &lt;code&gt;IterationMode::Generic&lt;/code&gt;. The branch jumps to where we are linking &lt;code&gt;fastCase&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;Another kind of very useful helpers are &lt;code&gt;JITSlowPathCall slowPathCall(this, instruction, tryFastFunction)&lt;/code&gt;, &lt;code&gt;callOperation&lt;/code&gt; and variants of these. That is because they abstract the ABI of each architecture to perform calls to cpp functions. On JSC we have a set of cpp functions that we call &lt;code&gt;JITOperations&lt;/code&gt; and are used by JIT when something is very complex to be done from assembly directly. For example, &lt;code&gt;emit_op_iterator_open&lt;/code&gt; uses these to call on &lt;code&gt;iterator_open_try_fast_narrow&lt;/code&gt;, &lt;code&gt;iterator_open_try_fast_wide16&lt;/code&gt;, or &lt;code&gt;iterator_open_try_fast_wide32&lt;/code&gt;. Also &lt;code&gt;emitSlow_op_iterator_open&lt;/code&gt; calls on &lt;code&gt;callOperation(operationThrowIteratorResultIsNotObject, TrustedImmPtr(m_codeBlock-&amp;gt;globalObject()));&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;When discussing &lt;code&gt;emit_op_iterator_open&lt;/code&gt; in baseline JIT, we saw above the usage of &lt;code&gt;JITGetByIdGenerator&lt;/code&gt; to generate assembly code to access the property &lt;code&gt;next&lt;/code&gt; from the iterator object. This class is a very useful as it encapsulates the Inline Cache (IC) mechanism for property access on JSC. It takes registers for base object where the &lt;code&gt;Get&lt;/code&gt; operation is going to be performed (&lt;code&gt;tagIteratorGPR&lt;/code&gt; and &lt;code&gt;payloadIteratorGPR&lt;/code&gt;) in addition to where the result of such access will be placed (&lt;code&gt;tagNextGPR&lt;/code&gt; and &lt;code&gt;payloadNextGPR&lt;/code&gt;). The code for fast path is generated by &lt;code&gt;gen.generateFastPath(*this);&lt;/code&gt; and since IC is only filled after its execution, the generation of fast path is a set of “noop” that eventually will  be repatched by fast code that accesses cached properties. We also configure &lt;code&gt;slowPathJump&lt;/code&gt; for this generator and this is important because when IC is empty, we always take the slow path to perform the &lt;code&gt;Get&lt;/code&gt; operation and configure the cache for it.&lt;/p&gt;
&lt;p&gt;The following code shows the &lt;code&gt;iterator_open&lt;/code&gt; slow path and where the &lt;code&gt;JITGetByIdGenerator&lt;/code&gt; is linked into when jumping to it.&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;void JIT::emitSlow_op_iterator_open(const Instruction* instruction, Vector&amp;lt;SlowCaseEntry&amp;gt;::iterator&amp;amp; iter)
{
    // …
    linkAllSlowCases(iter);

    GPRReg tagIteratorGPR = regT1;
    GPRReg payloadIteratorGPR = regT0;

    JumpList notObject;
    notObject.append(branchIfNotCell(tagIteratorGPR));
    notObject.append(branchIfNotObject(payloadIteratorGPR));

    auto bytecode = instruction-&amp;gt;as&amp;lt;OpIteratorOpen&amp;gt;();
    VirtualRegister nextVReg = bytecode.m_next;
    UniquedStringImpl* ident = vm().propertyNames-&amp;gt;next.impl();

    JITGetByIdGenerator&amp;amp; gen = m_getByIds[m_getByIdIndex++];
    
    Label coldPathBegin = label();

    Call call = callOperationWithProfile(
        bytecode.metadata(m_codeBlock),                  // metadata
        operationGetByIdOptimize,                        // operation
        nextVReg,                                        // result
        TrustedImmPtr(m_codeBlock-&amp;gt;globalObject()),      // arg1
        gen.stubInfo(),                                  // arg2
        JSValueRegs(tagIteratorGPR, payloadIteratorGPR), // arg3
        CacheableIdentifier::createFromImmortalIdentifier(ident).rawBits()); // arg4
    
    gen.reportSlowPathCall(coldPathBegin, call);
    auto done = jump();

    notObject.link(this);
    callOperation(operationThrowIteratorResultIsNotObject, TrustedImmPtr(m_codeBlock-&amp;gt;globalObject()));

    done.link(this);
    //…
}
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;During the execution the code above, we first perform a type check to guarantee that the iterator is an object - otherwise we need to throw a “Result is not object” exception. Assuming then that iterator is indeed an object, we perform a call to the JIT operation &lt;code&gt;operationGetByIdOptimize&lt;/code&gt;. This operation is responsible to check if the &lt;code&gt;Get&lt;/code&gt; we are performing is cacheable and then buffer those cases to repatch the fast path once we have enough evidence that this is a hot operation.&lt;/p&gt;
&lt;p&gt;The code for those generators lives on &lt;a href=&quot;https://github.com/WebKit/webkit/blob/master/Source/javascriptCore/jit/JITInlineCacheGenerator.h&quot;&gt;&lt;code&gt;JITInlineCacheGenerator.h&lt;/code&gt;&lt;/a&gt; and &lt;a href=&quot;https://github.com/WebKit/webkit/blob/master/Source/javascriptCore/jit/JITInlineCacheGenerator.cpp&quot;&gt;&lt;code&gt;JITInlineCacheGenerator.cpp&lt;/code&gt;&lt;/a&gt;. It includes generators for &lt;code&gt;op_in&lt;/code&gt;, &lt;code&gt;op_put_by_id&lt;/code&gt; and even for &lt;code&gt;op_get_by_val&lt;/code&gt;. These generators are also used by DFG and FTL code, mainly when there’s not enough type information for some opcodes during compile time.&lt;/p&gt;
&lt;p&gt;Lets look at &lt;code&gt;emit_op_iterator_next&lt;/code&gt; in order to understand &lt;code&gt;emitGetVirtualRegister&lt;/code&gt;.&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;void JIT::emit_op_iterator_next(const Instruction* instruction)
{
    //…
    JSValueRegs nextRegs(regT1, regT0);
    emitGetVirtualRegister(bytecode.m_next, nextRegs);
    //…
}
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This operation loads the &lt;code&gt;next&lt;/code&gt; operand into &lt;code&gt;regT1&lt;/code&gt; and &lt;code&gt;regT0&lt;/code&gt;. The call to &lt;code&gt;emitGetVirtualRegister&lt;/code&gt; generates assembly code that loads a &lt;code&gt;JSValue&lt;/code&gt; stored in a &lt;code&gt;VirtualRegister&lt;/code&gt; into the machine register(s) we pass as a parameter. For both LLInt and Baseline JIT, virtual registers are always assigned to stack slots, which means that &lt;code&gt;emitGetVirtualRegister&lt;/code&gt; will generate code that is going to load values from stack into machine registers.&lt;/p&gt;
&lt;p&gt;It is important to notice that &lt;code&gt;regT1&lt;/code&gt; and &lt;code&gt;regT0&lt;/code&gt; are aliases for registers of the architecture we are targetting. We also see the usage of &lt;code&gt;JSValueRegs&lt;/code&gt;, which is an abstraction created to reuse code among 64-bits and 32-bits code. It abstracts the fact that JSValues are 64-bits long, which means that we need 2 registers (tag and payload) to manage a JSValue on 32-bits code.
Following helpers used by &lt;code&gt;emit_op_iterator_open&lt;/code&gt;, we have &lt;code&gt;emitPutVirtualRegister&lt;/code&gt; that is very similar with &lt;code&gt;emitGetVirtualRegister&lt;/code&gt; but it stores a JSValue into a virtual register. A usage of this helper&#39;s counterpart &lt;code&gt;emitPutVirtualRegister&lt;/code&gt; can be seen in the code for &lt;code&gt;emit_op_iterator_open&lt;/code&gt;, which stores a &lt;code&gt;JSValue&lt;/code&gt; into a virtual register.&lt;/p&gt;
&lt;h2&gt;Conclusions&lt;/h2&gt;
&lt;p&gt;The important bits were the implementation of the intrinsics for LLInt and Baseline JIT. A few other code bits went into the patch but they are just ensuring everything works as expected instead of being large changes to the codebase.&lt;/p&gt;
&lt;p&gt;Although the implementation of the intrinsic is essentially the implementation of the same semantics, they require quite different knowledge since they operate at different levels of abstraction. On the one hand, LLInt is written in a DSL that compiles down to assembler and essentially implements a language interpreter. On the other, the Baseline JIT is written in cpp an implements a JIT Compiler. The API between both is completely different which requires the developer to know both in order to implement a single feature. The implementation of this feature for 64bits required further changes to other tiers of JSC, which we didn&#39;t need to touch for 32bits. In 32bits we do not have FTL enabled and DFG is essentially platform agnostic.&lt;/p&gt;
&lt;h2&gt;Thanks&lt;/h2&gt;
&lt;p&gt;My colleague &lt;a href=&quot;https://caiolima.github.io/&quot;&gt;Caio Lima&lt;/a&gt; has reviewed this post and spent a significant amount of time adding technical details where I initially had few. Any inaccuracies or lack of detail however, are still my own fault.&lt;/p&gt;
</content>
  </entry>
  <entry>
    <title>JSC - what are my options?</title>
    <link href="https://p.ocmatos.com/blog/jsc-what-are-my-options.html"/>
    <updated>2020-06-05T00:00:00Z</updated>
    <id>https://p.ocmatos.com/blog/jsc-what-are-my-options.html</id>
    <content xml:lang="en" type="html">&lt;p&gt;Compilers tend to be large pieces of software that provide an enormous amount of options. We take a quick look at how to find what JavaScriptCore (JSC) provides.&lt;/p&gt;
&lt;p&gt;If you grab a copy of the WebKit source, you can easily build &lt;code&gt;jsc&lt;/code&gt; with &lt;code&gt;Tools/Scripts/build-jsc --release --jsc-only&lt;/code&gt; (from the WebKit root directory). As usual, running the resulting binary will get you into a JavaScript REPL. For example (&lt;code&gt;$&lt;/code&gt; indicates the shell prompt):&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ WebKitBuild/Release/bin/jsc
&lt;span class=&quot;token operator&quot;&gt;&gt;&gt;&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;&gt;&lt;/span&gt; &lt;span class=&quot;token number&quot;&gt;2&lt;/span&gt;+2
&lt;span class=&quot;token number&quot;&gt;4&lt;/span&gt;
&lt;span class=&quot;token operator&quot;&gt;&gt;&gt;&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;&gt;&lt;/span&gt; &lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;That was simple enough, but which options does JSC accept? Using &lt;code&gt;--help&lt;/code&gt; will give you some information but if you&#39;re debuggin JSC, you might be interested in &lt;code&gt;--options&lt;/code&gt;.&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ WebKitBuild/Release/bin/jsc &lt;span class=&quot;token parameter variable&quot;&gt;--options&lt;/span&gt;
All JSC runtime options:
   &lt;span class=&quot;token assign-left variable&quot;&gt;useKernTCSM&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;true   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. Note: this needs to go before other options since they depend on this value.
   &lt;span class=&quot;token assign-left variable&quot;&gt;validateOptions&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. crashes &lt;span class=&quot;token keyword&quot;&gt;if&lt;/span&gt; mis-typed JSC options were passed to the VM
   &lt;span class=&quot;token assign-left variable&quot;&gt;dumpOptions&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;0&lt;/span&gt;   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. dumps JSC options &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;0&lt;/span&gt; &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; None, &lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt; &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; Overridden only, &lt;span class=&quot;token number&quot;&gt;2&lt;/span&gt; &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; All, &lt;span class=&quot;token number&quot;&gt;3&lt;/span&gt; &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; Verbose&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;

&lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;.&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;There is a huge amount of options and parameters to tune the execution of JSC, so here are some filtering hints.&lt;/p&gt;
&lt;p&gt;For verbosity related options:&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ WebKitBuild/Release/bin/jsc &lt;span class=&quot;token parameter variable&quot;&gt;--options&lt;/span&gt; &lt;span class=&quot;token operator&quot;&gt;&lt;span class=&quot;token file-descriptor important&quot;&gt;2&lt;/span&gt;&gt;&lt;/span&gt;&lt;span class=&quot;token file-descriptor important&quot;&gt;&amp;amp;1&lt;/span&gt; &lt;span class=&quot;token operator&quot;&gt;|&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;grep&lt;/span&gt; verbose
   &lt;span class=&quot;token assign-left variable&quot;&gt;verboseDFGBytecodeParsing&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false
   &lt;span class=&quot;token assign-left variable&quot;&gt;verboseCompilation&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false
   &lt;span class=&quot;token assign-left variable&quot;&gt;verboseFTLCompilation&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false
   &lt;span class=&quot;token assign-left variable&quot;&gt;verboseValidationFailure&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false
   &lt;span class=&quot;token assign-left variable&quot;&gt;verboseOSR&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false
   &lt;span class=&quot;token assign-left variable&quot;&gt;verboseDFGOSRExit&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false
   &lt;span class=&quot;token assign-left variable&quot;&gt;verboseFTLOSRExit&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false
   &lt;span class=&quot;token assign-left variable&quot;&gt;verboseCallLink&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false
   &lt;span class=&quot;token assign-left variable&quot;&gt;verboseCompilationQueue&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false
   &lt;span class=&quot;token assign-left variable&quot;&gt;verboseExitProfile&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false
   &lt;span class=&quot;token assign-left variable&quot;&gt;verboseCFA&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false
   &lt;span class=&quot;token assign-left variable&quot;&gt;verboseDFGFailure&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false
   &lt;span class=&quot;token assign-left variable&quot;&gt;verboseFTLToJSThunk&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false
   &lt;span class=&quot;token assign-left variable&quot;&gt;verboseFTLFailure&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false
   &lt;span class=&quot;token assign-left variable&quot;&gt;verboseSanitizeStack&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false
   &lt;span class=&quot;token assign-left variable&quot;&gt;verboseVisitRace&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false
   &lt;span class=&quot;token assign-left variable&quot;&gt;verboseExecutableAllocationFuzz&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;For JIT related options:&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ ~/dev/WebKit/WebKitBuild/Debug/bin/jsc &lt;span class=&quot;token parameter variable&quot;&gt;--options&lt;/span&gt; &lt;span class=&quot;token operator&quot;&gt;&lt;span class=&quot;token file-descriptor important&quot;&gt;2&lt;/span&gt;&gt;&lt;/span&gt;&lt;span class=&quot;token file-descriptor important&quot;&gt;&amp;amp;1&lt;/span&gt; &lt;span class=&quot;token operator&quot;&gt;|&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;grep&lt;/span&gt; JIT    
   &lt;span class=&quot;token assign-left variable&quot;&gt;useJIT&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;true   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. allows the executable pages to be allocated &lt;span class=&quot;token keyword&quot;&gt;for&lt;/span&gt; JIT and thunks &lt;span class=&quot;token keyword&quot;&gt;if&lt;/span&gt; &lt;span class=&quot;token boolean&quot;&gt;true&lt;/span&gt;
   &lt;span class=&quot;token assign-left variable&quot;&gt;useBaselineJIT&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;true   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. allows the baseline JIT to be used &lt;span class=&quot;token keyword&quot;&gt;if&lt;/span&gt; &lt;span class=&quot;token boolean&quot;&gt;true&lt;/span&gt;
   &lt;span class=&quot;token assign-left variable&quot;&gt;useDFGJIT&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;true   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. allows the DFG JIT to be used &lt;span class=&quot;token keyword&quot;&gt;if&lt;/span&gt; &lt;span class=&quot;token boolean&quot;&gt;true&lt;/span&gt;
   &lt;span class=&quot;token assign-left variable&quot;&gt;useRegExpJIT&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;true   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. allows the RegExp JIT to be used &lt;span class=&quot;token keyword&quot;&gt;if&lt;/span&gt; &lt;span class=&quot;token boolean&quot;&gt;true&lt;/span&gt;
   &lt;span class=&quot;token assign-left variable&quot;&gt;useDOMJIT&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;true   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. allows the DOMJIT to be used &lt;span class=&quot;token keyword&quot;&gt;if&lt;/span&gt; &lt;span class=&quot;token boolean&quot;&gt;true&lt;/span&gt;
   &lt;span class=&quot;token assign-left variable&quot;&gt;crashIfCantAllocateJITMemory&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false
   &lt;span class=&quot;token assign-left variable&quot;&gt;dumpDisassembly&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. dumps disassembly of all JIT compiled code upon compilation
   &lt;span class=&quot;token assign-left variable&quot;&gt;logJITCodeForPerf&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false
   &lt;span class=&quot;token assign-left variable&quot;&gt;bytecodeRangeToJITCompile&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;&amp;lt;&lt;/span&gt;null&lt;span class=&quot;token operator&quot;&gt;&gt;&lt;/span&gt;   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. bytecode size range to allow compilation on, e.g. &lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt;:100
   &lt;span class=&quot;token assign-left variable&quot;&gt;reportBaselineCompileTimes&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. dumps JS &lt;span class=&quot;token keyword&quot;&gt;function&lt;/span&gt; signature and the &lt;span class=&quot;token function&quot;&gt;time&lt;/span&gt; it took to BaselineJIT compile
   &lt;span class=&quot;token assign-left variable&quot;&gt;useFTLJIT&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;true   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. allows the FTL JIT to be used &lt;span class=&quot;token keyword&quot;&gt;if&lt;/span&gt; &lt;span class=&quot;token boolean&quot;&gt;true&lt;/span&gt;
   &lt;span class=&quot;token assign-left variable&quot;&gt;enableJITDebugAssertions&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;true
   &lt;span class=&quot;token assign-left variable&quot;&gt;useConcurrentJIT&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;true   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. allows the DFG / FTL compilation &lt;span class=&quot;token keyword&quot;&gt;in&lt;/span&gt; threads other than the executing JS thread
   &lt;span class=&quot;token assign-left variable&quot;&gt;jitPolicyScale&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt;   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. scale JIT thresholds to this specified ratio between &lt;span class=&quot;token number&quot;&gt;0.0&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;compile ASAP&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt; and &lt;span class=&quot;token number&quot;&gt;1.0&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;compile like normal&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;.
   &lt;span class=&quot;token assign-left variable&quot;&gt;thresholdForJITAfterWarmUp&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;500&lt;/span&gt;
   &lt;span class=&quot;token assign-left variable&quot;&gt;thresholdForJITSoon&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;100&lt;/span&gt;
   &lt;span class=&quot;token assign-left variable&quot;&gt;forceGCSlowPaths&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. If true, we will force all JIT fast allocations down their slow paths.
   &lt;span class=&quot;token assign-left variable&quot;&gt;alwaysGeneratePCToCodeOriginMap&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. This will &lt;span class=&quot;token function&quot;&gt;make&lt;/span&gt; sure we always generate a PCToCodeOriginMap &lt;span class=&quot;token keyword&quot;&gt;for&lt;/span&gt; JITed code.
   &lt;span class=&quot;token assign-left variable&quot;&gt;useBBQJIT&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;true   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. allows the BBQ JIT to be used &lt;span class=&quot;token keyword&quot;&gt;if&lt;/span&gt; &lt;span class=&quot;token boolean&quot;&gt;true&lt;/span&gt;
   &lt;span class=&quot;token assign-left variable&quot;&gt;useOMGJIT&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;true   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. allows the OMG JIT to be used &lt;span class=&quot;token keyword&quot;&gt;if&lt;/span&gt; &lt;span class=&quot;token boolean&quot;&gt;true&lt;/span&gt;
   &lt;span class=&quot;token assign-left variable&quot;&gt;traceBaselineJITExecution&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false
   &lt;span class=&quot;token assign-left variable&quot;&gt;dumpJITMemoryPath&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token string&quot;&gt;&quot;&quot;&lt;/span&gt;
   &lt;span class=&quot;token assign-left variable&quot;&gt;dumpJITMemoryFlushInterval&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;10&lt;/span&gt;   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. Maximum &lt;span class=&quot;token function&quot;&gt;time&lt;/span&gt; &lt;span class=&quot;token keyword&quot;&gt;in&lt;/span&gt; between flushes of the JIT memory dump &lt;span class=&quot;token keyword&quot;&gt;in&lt;/span&gt; seconds.&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;And lastly for GC related options:&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ ~/dev/WebKit/WebKitBuild/Debug/bin/jsc &lt;span class=&quot;token parameter variable&quot;&gt;--options&lt;/span&gt; &lt;span class=&quot;token operator&quot;&gt;&lt;span class=&quot;token file-descriptor important&quot;&gt;2&lt;/span&gt;&gt;&lt;/span&gt;&lt;span class=&quot;token file-descriptor important&quot;&gt;&amp;amp;1&lt;/span&gt; &lt;span class=&quot;token operator&quot;&gt;|&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;grep&lt;/span&gt; &lt;span class=&quot;token parameter variable&quot;&gt;-i&lt;/span&gt; gc
   &lt;span class=&quot;token assign-left variable&quot;&gt;repatchBufferingCountdown&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;8&lt;/span&gt;
   &lt;span class=&quot;token assign-left variable&quot;&gt;bytecodeRangeToDFGCompile&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;&amp;lt;&lt;/span&gt;null&lt;span class=&quot;token operator&quot;&gt;&gt;&lt;/span&gt;   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. bytecode size range to allow DFG compilation on, e.g. &lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt;:100
   &lt;span class=&quot;token assign-left variable&quot;&gt;logCompilationChanges&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false
   &lt;span class=&quot;token assign-left variable&quot;&gt;validateDoesGC&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;true
   &lt;span class=&quot;token assign-left variable&quot;&gt;reportDFGCompileTimes&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. dumps JS &lt;span class=&quot;token keyword&quot;&gt;function&lt;/span&gt; signature and the &lt;span class=&quot;token function&quot;&gt;time&lt;/span&gt; it took to DFG and FTL compile
   &lt;span class=&quot;token assign-left variable&quot;&gt;useGenerationalGC&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;true
   &lt;span class=&quot;token assign-left variable&quot;&gt;useConcurrentGC&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;true
   &lt;span class=&quot;token assign-left variable&quot;&gt;criticalGCMemoryThreshold&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;0.8&lt;/span&gt;   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. percent memory &lt;span class=&quot;token keyword&quot;&gt;in&lt;/span&gt; use the GC considers critical.  The collector is much &lt;span class=&quot;token function&quot;&gt;more&lt;/span&gt; aggressive above this threshold
   &lt;span class=&quot;token assign-left variable&quot;&gt;concurrentGCMaxHeadroom&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;1.5&lt;/span&gt;
   &lt;span class=&quot;token assign-left variable&quot;&gt;concurrentGCPeriodMS&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;2&lt;/span&gt;
   &lt;span class=&quot;token assign-left variable&quot;&gt;minimumGCPauseMS&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;0.3&lt;/span&gt;
   &lt;span class=&quot;token assign-left variable&quot;&gt;gcPauseScale&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;0.3&lt;/span&gt;
   &lt;span class=&quot;token assign-left variable&quot;&gt;gcIncrementBytes&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;10000&lt;/span&gt;
   &lt;span class=&quot;token assign-left variable&quot;&gt;gcIncrementMaxBytes&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;100000&lt;/span&gt;
   &lt;span class=&quot;token assign-left variable&quot;&gt;gcIncrementScale&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;0&lt;/span&gt;
   &lt;span class=&quot;token assign-left variable&quot;&gt;numberOfDFGCompilerThreads&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;2&lt;/span&gt;
   &lt;span class=&quot;token assign-left variable&quot;&gt;priorityDeltaOfDFGCompilerThreads&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;0&lt;/span&gt;
   &lt;span class=&quot;token assign-left variable&quot;&gt;maximumInliningCallerBytecodeCost&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;10000&lt;/span&gt;
   &lt;span class=&quot;token assign-left variable&quot;&gt;numberOfGCMarkers&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;8&lt;/span&gt;
   &lt;span class=&quot;token assign-left variable&quot;&gt;useParallelMarkingConstraintSolver&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;true
   &lt;span class=&quot;token assign-left variable&quot;&gt;slowPathAllocsBetweenGCs&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;0&lt;/span&gt;   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. force a GC on every Nth slow path alloc, where N is specified by this option
   &lt;span class=&quot;token assign-left variable&quot;&gt;sweepSynchronously&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. debugging option to sweep all dead objects synchronously at GC end before resuming mutator
   &lt;span class=&quot;token assign-left variable&quot;&gt;logGC&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;None   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. debugging option to log GC activity &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;0&lt;/span&gt; &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; None, &lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt; &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; Basic, &lt;span class=&quot;token number&quot;&gt;2&lt;/span&gt; &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; Verbose&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;
   &lt;span class=&quot;token assign-left variable&quot;&gt;useGC&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;true
   &lt;span class=&quot;token assign-left variable&quot;&gt;gcAtEnd&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. If true, the jsc CLI will &lt;span class=&quot;token keyword&quot;&gt;do&lt;/span&gt; a GC before exiting
   &lt;span class=&quot;token assign-left variable&quot;&gt;forceGCSlowPaths&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. If true, we will force all JIT fast allocations down their slow paths.
   &lt;span class=&quot;token assign-left variable&quot;&gt;forceDidDeferGCWork&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. If true, we will force all DeferGC destructions to perform a GC.
   &lt;span class=&quot;token assign-left variable&quot;&gt;gcMaxHeapSize&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;0&lt;/span&gt;
   &lt;span class=&quot;token assign-left variable&quot;&gt;recordGCPauseTimes&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false
   &lt;span class=&quot;token assign-left variable&quot;&gt;numberOfGCCyclesToRecordForVerification&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;3&lt;/span&gt;
   &lt;span class=&quot;token assign-left variable&quot;&gt;validateDFGClobberize&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. Emits extra validation code &lt;span class=&quot;token keyword&quot;&gt;in&lt;/span&gt; the DFG/FTL &lt;span class=&quot;token keyword&quot;&gt;for&lt;/span&gt; the Clobberize phase&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The options that are relevant to your use case depend on what you want to do, but my hope is that you have already found something useful here or that you now know how to find what you need. Good luck!&lt;/p&gt;
</content>
  </entry>
  <entry>
    <title>CReduce - it&#39;s a kind of magic!</title>
    <link href="https://p.ocmatos.com/blog/creduce-its-a-kind-of-magic.html"/>
    <updated>2020-02-17T00:00:00Z</updated>
    <id>https://p.ocmatos.com/blog/creduce-its-a-kind-of-magic.html</id>
    <content xml:lang="en" type="html">&lt;p&gt;When I started compiler work &lt;a href=&quot;http://delta.tigris.org/&quot;&gt;delta&lt;/a&gt; was all the rage. It would turn a testcase with thousands of lines into a single small one with perhaps no more than 15 crashing your compiler in exactly the same way as the original testcase. I have used and abused this tool but it turns out it could be improved. John Regehr and his team did so with &lt;a href=&quot;https://embed.cs.utah.edu/creduce/&quot;&gt;C-Reduce&lt;/a&gt; (who by he way also developed &lt;a href=&quot;https://embed.cs.utah.edu/csmith/&quot;&gt;C-Smith&lt;/a&gt;, but that&#39;s another store entirely).&lt;/p&gt;
&lt;p&gt;C-Reduce takes things to another level squeezing that testcase ever farther until it&#39;s just a few bytes, and yet still crashing your compiler in the same interesting way it did before. In C and C++ this is a very important tool and its use is not &lt;a href=&quot;https://gcc.gnu.org/wiki/A_guide_to_testcase_reduction&quot;&gt;recommended&lt;/a&gt; often enough. It&#39;s very important and in languages like C and C++ because the preprocessor can a seemingly small example into a behemoth of a source file and therefore reducing the testcase appropriately is extremely important. The recommendation page for &lt;a href=&quot;https://gcc.gnu.org/bugs/&quot;&gt;reporting GCC bugs&lt;/a&gt; goes as far as saying:&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;However, providing a minimal testcase increases the chances of
getting your bug fixed. The only excuses to not send us the
preprocessed sources are (i) if you&#39;ve found a bug in the
preprocessor, (ii) if you&#39;ve reduced the testcase to a small file
that doesn&#39;t include any other file or (iii) if the bug appears only
when using precompiled headers.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;However, one advantage of &lt;code&gt;delta&lt;/code&gt; was that it worked on any text file as opposed to C-Reduce that due its reduction strategy required the source to be C or C++. That is, until I was told I was wrong and that C-Reduce would be able to eat a large &lt;a href=&quot;https://racket-lang.org/&quot;&gt;Racket&lt;/a&gt; and spit out a small one. This stayed in the back of my head until present day.&lt;/p&gt;
&lt;p&gt;Currently I am working on JavaScriptCore, which turns out to have bugs of its own as well. Due to the way these JIT compilers are designed you can go through several compiler tiers, code generations, on stack
replacements and back until you find your bug. How great would it be if you could launch your compiler and the bug would reveal itself straightaway without having to wait for a breakpoint to be hit 60 times, step through 20 machine instructions and two OSR calls?&lt;/p&gt;
&lt;p&gt;Well, that&#39;s what C-Reduce is here for. I remembered being told it worked with Racket and if it works with Racket it must work with JavaScript.&lt;/p&gt;
&lt;p&gt;I develop on a &lt;code&gt;x86_64&lt;/code&gt; but my crash is on &lt;code&gt;MIPS32el&lt;/code&gt; and I can reproduce it on &lt;code&gt;qemu-user&lt;/code&gt; like this:&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ /home/pmatos/roots/br-root.jsc32-mips/host/bin/qemu-mipsel &lt;span class=&quot;token parameter variable&quot;&gt;-L&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;&#92;&lt;/span&gt;
	/home/pmatos/roots/br-root.jsc32-mips/target ./WebKitBuild/Debug/bin/jsc &lt;span class=&quot;token punctuation&quot;&gt;&#92;&lt;/span&gt;
	load-varargs-then-inlined-call-and-exit.js 
qemu: uncaught target signal &lt;span class=&quot;token number&quot;&gt;4&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;Illegal instruction&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt; - core dumped&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The idea here is to reduce &lt;code&gt;load-varargs-then-inlined-call-and-exit.js&lt;/code&gt; but keep the &lt;code&gt;Illegal instruction&lt;/code&gt; error so I developed the simple reducing script:&lt;/p&gt;
&lt;pre class=&quot;language-bash&quot;&gt;&lt;code class=&quot;language-bash&quot;&gt;&lt;span class=&quot;token shebang important&quot;&gt;#!/bin/bash&lt;/span&gt;
/home/pmatos/roots/br-root.jsc32-mips/host/bin/qemu-mipsel &lt;span class=&quot;token parameter variable&quot;&gt;-L&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;&#92;&lt;/span&gt;
	/home/pmatos/roots/br-root.jsc32-mips/target /home/pmatos/dev/WebKit/WebKitBuild/Debug/bin/jsc &lt;span class=&quot;token punctuation&quot;&gt;&#92;&lt;/span&gt;
	load-varargs-then-inlined-call-and-exit.js &lt;span class=&quot;token operator&quot;&gt;&amp;amp;&gt;&lt;/span&gt; err.txt
&lt;span class=&quot;token function&quot;&gt;grep&lt;/span&gt; &lt;span class=&quot;token parameter variable&quot;&gt;-q&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&#39;Illegal instruction&#39;&lt;/span&gt; err.txt&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This simply redirects all output from &lt;code&gt;jsc&lt;/code&gt; into a file and checks that the file still contains the error &lt;code&gt;Illegal instruction&lt;/code&gt;. This file can be as complex as you want to constrain the error you wish to reproduce. For our present situation this is enough.&lt;/p&gt;
&lt;p&gt;Then, call C-Reduce:&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ creduce --not-c &lt;span class=&quot;token parameter variable&quot;&gt;--n&lt;/span&gt; &lt;span class=&quot;token number&quot;&gt;16&lt;/span&gt; ./red.sh load-varargs-then-inlined-call-and-exit.js
&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;&amp;lt;&lt;/span&gt; &lt;span class=&quot;token number&quot;&gt;26334&lt;/span&gt; &lt;span class=&quot;token operator&quot;&gt;&gt;=&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;
running &lt;span class=&quot;token number&quot;&gt;16&lt;/span&gt; interestingness tests &lt;span class=&quot;token keyword&quot;&gt;in&lt;/span&gt; parallel
&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;&amp;lt;&lt;/span&gt; pass_blank :: &lt;span class=&quot;token number&quot;&gt;0&lt;/span&gt; &lt;span class=&quot;token operator&quot;&gt;&gt;=&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;
&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;0.6&lt;/span&gt; %, &lt;span class=&quot;token number&quot;&gt;1046&lt;/span&gt; bytes&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;
&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;&amp;lt;&lt;/span&gt; pass_lines :: &lt;span class=&quot;token number&quot;&gt;0&lt;/span&gt; &lt;span class=&quot;token operator&quot;&gt;&gt;=&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;
&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;-0.3 %, &lt;span class=&quot;token number&quot;&gt;1055&lt;/span&gt; bytes&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;
&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;1.2&lt;/span&gt; %, &lt;span class=&quot;token number&quot;&gt;1039&lt;/span&gt; bytes&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;
&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;&amp;lt;&lt;/span&gt; pass_lines :: &lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt; &lt;span class=&quot;token operator&quot;&gt;&gt;=&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;
&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;-1.9 %, &lt;span class=&quot;token number&quot;&gt;1072&lt;/span&gt; bytes&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;
&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;17.6&lt;/span&gt; %, &lt;span class=&quot;token number&quot;&gt;867&lt;/span&gt; bytes&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;
&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;20.9&lt;/span&gt; %, &lt;span class=&quot;token number&quot;&gt;832&lt;/span&gt; bytes&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;
&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;26.5&lt;/span&gt; %, &lt;span class=&quot;token number&quot;&gt;773&lt;/span&gt; bytes&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;
&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;34.8&lt;/span&gt; %, &lt;span class=&quot;token number&quot;&gt;686&lt;/span&gt; bytes&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;
&lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;.&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;There are many arguments possible to creduce (which you can check with &lt;code&gt;creduce --help&lt;/code&gt;. Here I used:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;code&gt;--not-c&lt;/code&gt;: disable C/C++ specific passes, use for source languages that are, well, not C-like;&lt;/li&gt;
&lt;li&gt;&lt;code&gt;--n 16&lt;/code&gt;: use 16 cores to do the reduction, the more the merrier. If you interestingness script takes a long time to run, grab yourself a coffee;&lt;/li&gt;
&lt;li&gt;&lt;code&gt;./red.sh&lt;/code&gt;: path to the interestingness script (make sure it&#39;s executable);&lt;/li&gt;
&lt;li&gt;&lt;code&gt;load-varargs-then-inlined-call-and-exit.js&lt;/code&gt;: testcase to reduce (this is going to be modified in-place);&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;After a while you&#39;ll be greeted with:&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;token keyword&quot;&gt;done&lt;/span&gt; &lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;

pass statistics:
  method pass_balanced :: curly2 worked &lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt; and failed &lt;span class=&quot;token number&quot;&gt;18&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt;
  method pass_balanced :: curly-inside worked &lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt; and failed &lt;span class=&quot;token number&quot;&gt;12&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt;
  method pass_clex :: rm-toks-11 worked &lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt; and failed &lt;span class=&quot;token number&quot;&gt;105&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt;
  method pass_balanced :: parens-to-zero worked &lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt; and failed &lt;span class=&quot;token number&quot;&gt;25&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt;
  method pass_blank :: &lt;span class=&quot;token number&quot;&gt;0&lt;/span&gt; worked &lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt; and failed &lt;span class=&quot;token number&quot;&gt;0&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt;
  method pass_clex :: rename-toks worked &lt;span class=&quot;token number&quot;&gt;2&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt; and failed &lt;span class=&quot;token number&quot;&gt;4&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt;
  method pass_lines :: &lt;span class=&quot;token number&quot;&gt;8&lt;/span&gt; worked &lt;span class=&quot;token number&quot;&gt;2&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt; and failed &lt;span class=&quot;token number&quot;&gt;123&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt;
  method pass_lines :: &lt;span class=&quot;token number&quot;&gt;4&lt;/span&gt; worked &lt;span class=&quot;token number&quot;&gt;2&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt; and failed &lt;span class=&quot;token number&quot;&gt;123&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt;
  method pass_lines :: &lt;span class=&quot;token number&quot;&gt;10&lt;/span&gt; worked &lt;span class=&quot;token number&quot;&gt;2&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt; and failed &lt;span class=&quot;token number&quot;&gt;123&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt;
  method pass_lines :: &lt;span class=&quot;token number&quot;&gt;3&lt;/span&gt; worked &lt;span class=&quot;token number&quot;&gt;2&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt; and failed &lt;span class=&quot;token number&quot;&gt;123&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt;
  method pass_indent :: regular worked &lt;span class=&quot;token number&quot;&gt;2&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt; and failed &lt;span class=&quot;token number&quot;&gt;0&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt;
  method pass_lines :: &lt;span class=&quot;token number&quot;&gt;6&lt;/span&gt; worked &lt;span class=&quot;token number&quot;&gt;2&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt; and failed &lt;span class=&quot;token number&quot;&gt;123&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt;
  method pass_lines :: &lt;span class=&quot;token number&quot;&gt;2&lt;/span&gt; worked &lt;span class=&quot;token number&quot;&gt;5&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt; and failed &lt;span class=&quot;token number&quot;&gt;134&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt;
  method pass_clex :: rm-tok-pattern-4 worked &lt;span class=&quot;token number&quot;&gt;5&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt; and failed &lt;span class=&quot;token number&quot;&gt;784&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt;
  method pass_lines :: &lt;span class=&quot;token number&quot;&gt;0&lt;/span&gt; worked &lt;span class=&quot;token number&quot;&gt;5&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt; and failed &lt;span class=&quot;token number&quot;&gt;70&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt;
  method pass_balanced :: parens-inside worked &lt;span class=&quot;token number&quot;&gt;6&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt; and failed &lt;span class=&quot;token number&quot;&gt;7&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt;
  method pass_clex :: rm-toks-1 worked &lt;span class=&quot;token number&quot;&gt;7&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt; and failed &lt;span class=&quot;token number&quot;&gt;132&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt;
  method pass_lines :: &lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt; worked &lt;span class=&quot;token number&quot;&gt;7&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt; and failed &lt;span class=&quot;token number&quot;&gt;147&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt;
  method pass_clex :: rm-toks-2 worked &lt;span class=&quot;token number&quot;&gt;8&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt; and failed &lt;span class=&quot;token number&quot;&gt;116&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt;

          ******** /home/pmatos/dev/WebKit/load-varargs-then-inlined-call-and-exit.js ********

&lt;span class=&quot;token keyword&quot;&gt;function&lt;/span&gt; b&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;a&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;
  &lt;span class=&quot;token builtin class-name&quot;&gt;return&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt; a: a + &lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;
&lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;
&lt;span class=&quot;token keyword&quot;&gt;function&lt;/span&gt; &lt;span class=&quot;token function-name function&quot;&gt;bar&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt; b.apply&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;this, array&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;
&lt;span class=&quot;token keyword&quot;&gt;function&lt;/span&gt; &lt;span class=&quot;token function-name function&quot;&gt;c&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;
  bar&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;
  c&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;
&lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;
array &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt; &lt;span class=&quot;token number&quot;&gt;2147483647&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt;
c&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Your test file now contains a much shorter testcase but will still crash your compiler in the same beautiful way.&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ /home/pmatos/roots/br-root.jsc32-mips/host/bin/qemu-mipsel &lt;span class=&quot;token parameter variable&quot;&gt;-L&lt;/span&gt; /home/pmatos/roots/br-root.jsc32-mips/target ./WebKitBuild/Debug/bin/jsc load-varargs-then-inlined-call-and-exit.js 
qemu: uncaught target signal &lt;span class=&quot;token number&quot;&gt;4&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;Illegal instruction&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt; - core dumped&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;For comparison the original file had 41 lines and 1052 bytes and the reduced file has 9 lines and 137 bytes which is an 87% reduction in size. But the big improvement here is not that the file is just smaller but that the number of paths taken by &lt;code&gt;jsc&lt;/code&gt; is much smaller taking me strange to the illegal instruction without JIT compiling functions I don&#39;t need or jumping through tiers I don&#39;t care.&lt;/p&gt;
&lt;p&gt;Of course, this would work just as well in a non cross-platform environment. I just used &lt;code&gt;qemu-user&lt;/code&gt; due to the nature of my bug. The important thing is to have a testcase, and a script to ensure that at each reduction step, you keep the interestingness of the testcase - in other words, a way to reproduce the failure.&lt;/p&gt;
&lt;p&gt;Here are a few examples I can think of as a script for reduction:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;code&gt;jsc&lt;/code&gt; crashes with a specific exit code so you create a script testing that specific exit code;&lt;/li&gt;
&lt;li&gt;&lt;code&gt;jsc&lt;/code&gt; crashes with a specific output (above was &lt;code&gt;Illegal instruction&lt;/code&gt;) but it could be anything else;&lt;/li&gt;
&lt;li&gt;&lt;code&gt;jsc&lt;/code&gt; does not crash but generates some strange JIT code which you noticed by using &lt;code&gt;--dumpDisassembly=true&lt;/code&gt;. Well, ensure in your interestingness script the exit code of &lt;code&gt;jsc&lt;/code&gt; is 0, redirect the output to a file and parse for that specific JIT code block;&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Most cases could probably be reduced (pun-intented) to the above three items. You reduce for a specific exit code, a specific error, or a specific block of text. Or a combination of these.&lt;/p&gt;
&lt;p&gt;In any case, if you implement your reducing script appropriately the result will be a testcase smaller than the original that passes your interestingness script - and C-Reduce will look like it is a kind of magic.&lt;/p&gt;
</content>
  </entry>
  <entry>
    <title>Cross-Arch Reproducibility using Containers</title>
    <link href="https://p.ocmatos.com/blog/cross-arch-reproducibility-using-containers.html"/>
    <updated>2020-01-16T00:00:00Z</updated>
    <id>https://p.ocmatos.com/blog/cross-arch-reproducibility-using-containers.html</id>
    <content xml:lang="en" type="html">&lt;h2&gt;Introduction&lt;/h2&gt;
&lt;p&gt;Part of my work for Igalia is on the 32bit support on MIPS (little endian) and ARM for JSC (JavaScriptCore - the JavaScript compiler in WebKit) and one of the problems we face is that of reproducibility of failures.&lt;/p&gt;
&lt;p&gt;We have boards to test these, namely Raspberry Pi 3 Model B+ boards, running a 32bits ARM kernel and Imagination CI20 boards running a mipsel kernel - all built with buildroot which provides images for these boards out-of-the-box. However, we don&#39;t work on these - most of us have higher performance x86_64 machines and whenever a failure occurs upstream it&#39;s generally time consuming to reproduce.&lt;/p&gt;
&lt;p&gt;I have therefore set out to create an environment where we can easily reproduce cross-architectural failures on JSC. I had worked on similar issues for &lt;a href=&quot;https://racket-lang.org/&quot;&gt;Racket&lt;/a&gt;, when building the cross-architectural Racket chroot environment on GitLab CI.&lt;/p&gt;
&lt;h2&gt;Starting with chroot&lt;/h2&gt;
&lt;p&gt;Let&#39;s start the discussion by talking about &lt;code&gt;chroot&lt;/code&gt;. In any linux system you&#39;ll find a &lt;code&gt;chroot&lt;/code&gt; binary.&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ &lt;span class=&quot;token function&quot;&gt;which&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;chroot&lt;/span&gt;
/usr/sbin/chroot
$ &lt;span class=&quot;token function&quot;&gt;chroot&lt;/span&gt; &lt;span class=&quot;token parameter variable&quot;&gt;--help&lt;/span&gt;
Usage: &lt;span class=&quot;token function&quot;&gt;chroot&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;OPTION&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt; NEWROOT &lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;COMMAND &lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;ARG&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;.&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt;
  or:  &lt;span class=&quot;token function&quot;&gt;chroot&lt;/span&gt; OPTION
Run COMMAND with root directory &lt;span class=&quot;token builtin class-name&quot;&gt;set&lt;/span&gt; to NEWROOT.

  &lt;span class=&quot;token parameter variable&quot;&gt;--groups&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;G_LIST        specify supplementary &lt;span class=&quot;token function&quot;&gt;groups&lt;/span&gt; as g1,g2,&lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;,gN
  &lt;span class=&quot;token parameter variable&quot;&gt;--userspec&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token environment constant&quot;&gt;USER&lt;/span&gt;:GROUP  specify user and group &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;ID or name&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt; to use
  --skip-chdir           &lt;span class=&quot;token keyword&quot;&gt;do&lt;/span&gt; not change working directory to &lt;span class=&quot;token string&quot;&gt;&#39;/&#39;&lt;/span&gt;
      &lt;span class=&quot;token parameter variable&quot;&gt;--help&lt;/span&gt;     display this &lt;span class=&quot;token builtin class-name&quot;&gt;help&lt;/span&gt; and &lt;span class=&quot;token builtin class-name&quot;&gt;exit&lt;/span&gt;
      &lt;span class=&quot;token parameter variable&quot;&gt;--version&lt;/span&gt;  output version information and &lt;span class=&quot;token builtin class-name&quot;&gt;exit&lt;/span&gt;

If no &lt;span class=&quot;token builtin class-name&quot;&gt;command&lt;/span&gt; is given, run &lt;span class=&quot;token string&quot;&gt;&#39;&quot;$SHELL&quot; -i&#39;&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;default: &lt;span class=&quot;token string&quot;&gt;&#39;/bin/sh -i&#39;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;.

GNU coreutils online help: &lt;span class=&quot;token operator&quot;&gt;&amp;lt;&lt;/span&gt;https://www.gnu.org/software/coreutils/&lt;span class=&quot;token operator&quot;&gt;&gt;&lt;/span&gt;
Full documentation at: &lt;span class=&quot;token operator&quot;&gt;&amp;lt;&lt;/span&gt;https://www.gnu.org/software/coreutils/chroot&lt;span class=&quot;token operator&quot;&gt;&gt;&lt;/span&gt;
or available locally via: info &lt;span class=&quot;token string&quot;&gt;&#39;(coreutils) chroot invocation&#39;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;However, at a slightly lower level, &lt;a href=&quot;http://man7.org/linux/man-pages/man2/chroot.2.html&quot;&gt;&lt;code&gt;chroot&lt;/code&gt;&lt;/a&gt; is a system call.&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ &lt;span class=&quot;token function&quot;&gt;man&lt;/span&gt; &lt;span class=&quot;token parameter variable&quot;&gt;-s2&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;chroot&lt;/span&gt;
CHROOT&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;2&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;

NAME
       &lt;span class=&quot;token function&quot;&gt;chroot&lt;/span&gt; - change root directory

SYNOPSIS
       &lt;span class=&quot;token comment&quot;&gt;#include &amp;lt;unistd.h&gt;&lt;/span&gt;

       int chroot&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;const char *path&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;

   Feature Test Macro Requirements &lt;span class=&quot;token keyword&quot;&gt;for&lt;/span&gt; glibc &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;see feature_test_macros&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;7&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;))&lt;/span&gt;:
&lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;.&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;As mentioned in the man page, it changes the root of the file system for a process to the path passed as argument. This new environment created for the process is known as a chroot jail, to which we will refer simply as a &lt;em&gt;jail&lt;/em&gt;.&lt;/p&gt;
&lt;p&gt;The jail allows us to trap a process inside a filesystem, i.e. the process cannot access anything outside the filesystem it is in. So, if it tried to access the root of the filesystem, inside the jail it will  only see the root of the new file system which is the path we passed on to &lt;code&gt;chroot&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;As an example throughout the post I will be using the factorial function. Whenever I refer to &lt;code&gt;factorial.c&lt;/code&gt;, I refer to a file that you might have to create as needed and consists of the following C source code.&lt;/p&gt;
&lt;pre class=&quot;language-c&quot;&gt;&lt;code class=&quot;language-c&quot;&gt;&lt;span class=&quot;token macro property&quot;&gt;&lt;span class=&quot;token directive-hash&quot;&gt;#&lt;/span&gt;&lt;span class=&quot;token directive keyword&quot;&gt;include&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&amp;lt;stdio.h&gt;&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;token macro property&quot;&gt;&lt;span class=&quot;token directive-hash&quot;&gt;#&lt;/span&gt;&lt;span class=&quot;token directive keyword&quot;&gt;include&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&amp;lt;stdint.h&gt;&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;token macro property&quot;&gt;&lt;span class=&quot;token directive-hash&quot;&gt;#&lt;/span&gt;&lt;span class=&quot;token directive keyword&quot;&gt;include&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&amp;lt;inttypes.h&gt;&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;token macro property&quot;&gt;&lt;span class=&quot;token directive-hash&quot;&gt;#&lt;/span&gt;&lt;span class=&quot;token directive keyword&quot;&gt;include&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&amp;lt;stdlib.h&gt;&lt;/span&gt;&lt;/span&gt;

&lt;span class=&quot;token keyword&quot;&gt;int&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;main&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token keyword&quot;&gt;int&lt;/span&gt; argc&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;token keyword&quot;&gt;char&lt;/span&gt; &lt;span class=&quot;token operator&quot;&gt;*&lt;/span&gt;argv&lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;

  &lt;span class=&quot;token class-name&quot;&gt;uint64_t&lt;/span&gt; result &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
  &lt;span class=&quot;token class-name&quot;&gt;uint32_t&lt;/span&gt; arg&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;

  &lt;span class=&quot;token keyword&quot;&gt;if&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;argc &lt;span class=&quot;token operator&quot;&gt;!=&lt;/span&gt; &lt;span class=&quot;token number&quot;&gt;2&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;
    &lt;span class=&quot;token keyword&quot;&gt;return&lt;/span&gt; &lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;

  arg &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;strtoul&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;argv&lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;token constant&quot;&gt;NULL&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;token number&quot;&gt;0&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
  &lt;span class=&quot;token keyword&quot;&gt;while&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;arg&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt; result &lt;span class=&quot;token operator&quot;&gt;*=&lt;/span&gt; arg&lt;span class=&quot;token operator&quot;&gt;--&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;

  &lt;span class=&quot;token function&quot;&gt;printf&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token string&quot;&gt;&quot;Result: %&quot;&lt;/span&gt; PRIu64 &lt;span class=&quot;token string&quot;&gt;&quot;&#92;n&quot;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; result&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
  &lt;span class=&quot;token keyword&quot;&gt;return&lt;/span&gt; &lt;span class=&quot;token number&quot;&gt;0&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
&lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Let&#39;s compile it and run an example.&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ gcc &lt;span class=&quot;token parameter variable&quot;&gt;-Wall&lt;/span&gt; &lt;span class=&quot;token parameter variable&quot;&gt;-Wextra&lt;/span&gt; &lt;span class=&quot;token parameter variable&quot;&gt;-o&lt;/span&gt; factorial factorial.c
$ ./factorial &lt;span class=&quot;token number&quot;&gt;20&lt;/span&gt;
Result: &lt;span class=&quot;token number&quot;&gt;2432902008176640000&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Let&#39;s create a jail for this binary using &lt;code&gt;chroot&lt;/code&gt; and run it.&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ &lt;span class=&quot;token function&quot;&gt;mkdir&lt;/span&gt; jail
$ &lt;span class=&quot;token function&quot;&gt;cp&lt;/span&gt; factorial jail/
$ &lt;span class=&quot;token function&quot;&gt;chroot&lt;/span&gt; jail/ /factorial
chroot: cannot change root directory to &lt;span class=&quot;token string&quot;&gt;&#39;jail/&#39;&lt;/span&gt;&lt;span class=&quot;token builtin class-name&quot;&gt;:&lt;/span&gt; Operation not permitted&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;First lesson here: only &lt;code&gt;root&lt;/code&gt; can &lt;code&gt;chroot&lt;/code&gt;. For security reasons, a normal user cannot &lt;code&gt;chroot&lt;/code&gt;. Being able to do so, would allow the user privilege escalation (&lt;a href=&quot;https://web.archive.org/web/20160127150916/http://www.bpfh.net/simes/computing/chroot-break.html&quot;&gt;further details&lt;/a&gt; on breaking out of the chroot jail and privilege escalation).&lt;/p&gt;
&lt;p&gt;So we &lt;code&gt;sudo&lt;/code&gt;:&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ &lt;span class=&quot;token function&quot;&gt;sudo&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;chroot&lt;/span&gt; jail/ /factorial
chroot: failed to run &lt;span class=&quot;token builtin class-name&quot;&gt;command&lt;/span&gt; ‘/factorial’: No such &lt;span class=&quot;token function&quot;&gt;file&lt;/span&gt; or directory&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Now this really starts to annoy you and you start wondering if the path to &lt;code&gt;factorial&lt;/code&gt; is correct. It is correct, the path is relative to the jail root. Once the root becomes &lt;code&gt;jail/&lt;/code&gt;, the &lt;code&gt;factorial&lt;/code&gt; binary is in the root of the filesystem so &lt;code&gt;/factorial&lt;/code&gt; is correct. The problem is subtle but will teach you an important lesson: dependencies. This is, after all, a dynamically linked executable 💡.&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ ldd factorial
        linux-vdso.so.1 &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;0x00007ffe1f7f9000&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;
        libc.so.6 &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;&gt;&lt;/span&gt; /lib/x86_64-linux-gnu/libc.so.6 &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;0x00007f65bf62e000&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;
        /lib64/ld-linux-x86-64.so.2 &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;0x00007f65bf844000&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Those files exist in your file system, but not in the jail, so you get an error (although a pretty terrible error message at that). Let&#39;s try a static executable instead.&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ gcc &lt;span class=&quot;token parameter variable&quot;&gt;-Wall&lt;/span&gt; &lt;span class=&quot;token parameter variable&quot;&gt;-Wextra&lt;/span&gt; &lt;span class=&quot;token parameter variable&quot;&gt;-static&lt;/span&gt; &lt;span class=&quot;token parameter variable&quot;&gt;-o&lt;/span&gt; factorial factorial.c
$ ldd factorial
        not a dynamic executable
$ &lt;span class=&quot;token function&quot;&gt;sudo&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;chroot&lt;/span&gt; jail/ /factorial &lt;span class=&quot;token number&quot;&gt;20&lt;/span&gt;
Result: &lt;span class=&quot;token number&quot;&gt;2432902008176640000&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This is exactly what we wanted from the beginning - to show that the binary is now jailed in this root and cannot escape without explicitly trying to, something no benign binary is likely to do (see my comment above on escaping jail).&lt;/p&gt;
&lt;p&gt;In chroot jails, the filesystem root changes but it&#39;s still running on the same kernel. This allows us to create a new userspace inside this jail, separate from the host&#39;s, and possibly based on a different
linux distribution. For reproducibility, you can potentially tarball this jail and send it to someone else, who could themselves &lt;code&gt;chroot&lt;/code&gt; into it and reproduce a specific problem.&lt;/p&gt;
&lt;h2&gt;QEMU and binfmt&lt;/h2&gt;
&lt;p&gt;I will now introduce two other essential components to achieve our goal of cross architecture reproducibility: QEMU and binfmt.&lt;/p&gt;
&lt;p&gt;Up until now our jail has contained binaries compiled for our host architecture (in my case x86_64), however this need not be the case. &lt;a href=&quot;https://www.qemu.org/&quot;&gt;QEMU&lt;/a&gt; is an open-source hardware emulator and virtualizer. It implements two execution modes: system mode, and user mode;&lt;/p&gt;
&lt;p&gt;In system mode, QEMU works like &lt;a href=&quot;https://www.virtualbox.org/&quot;&gt;VirtualBox&lt;/a&gt; it emulates a whole system - from the applications, interrupts, and kernel, all the way to the hardware devices. The one I am interested in is user mode. In user mode, QEMU emulates a single binary leaving the rest of the system untouched.&lt;/p&gt;
&lt;p&gt;To demo how it works, lets use a cross-toolchain to compile a program to be run on a different architecture. Given I am on a &lt;code&gt;x86_64&lt;/code&gt;, I will use an &lt;code&gt;armhf&lt;/code&gt; toolchain. You can either download one provided from your distro or compile one with &lt;a href=&quot;https://crosstool-ng.github.io/&quot;&gt;crosstool-ng&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;For the sake of reproducibility, lets compile a pre-configured one with &lt;code&gt;crosstool-ng&lt;/code&gt;. Download it and install it in &lt;code&gt;$PATH&lt;/code&gt; - I used version 1.24.0. Then build the cross toolchain for &lt;code&gt;armv7-rpi2-linux-gnueabihf&lt;/code&gt;.&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ &lt;span class=&quot;token function&quot;&gt;mkdir&lt;/span&gt; build
$ &lt;span class=&quot;token builtin class-name&quot;&gt;cd&lt;/span&gt; build
build/ $ ct-ng armv7-rpi2-linux-gnueabihf
build/ $ ct-ng build&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Let&#39;s once again compile the &lt;code&gt;factorial.c&lt;/code&gt; example, but this time with our new toolchain. This toolchain will be in &lt;code&gt;$HOME/x-tools/armv7-rpi2-linux-gnueabihf&lt;/code&gt; by default.&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ &lt;span class=&quot;token builtin class-name&quot;&gt;export&lt;/span&gt; &lt;span class=&quot;token assign-left variable&quot;&gt;&lt;span class=&quot;token environment constant&quot;&gt;PATH&lt;/span&gt;&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token environment constant&quot;&gt;$HOME&lt;/span&gt;/x-tools/armv7-rpi2-linux-gnueabihf/bin:&lt;span class=&quot;token environment constant&quot;&gt;$PATH&lt;/span&gt;
$ armv7-rpi2-linux-gnueabihf-gcc &lt;span class=&quot;token parameter variable&quot;&gt;-static&lt;/span&gt; &lt;span class=&quot;token parameter variable&quot;&gt;-o&lt;/span&gt; factorial factorial.c
$ ./factorial
zsh: &lt;span class=&quot;token builtin class-name&quot;&gt;exec&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;format&lt;/span&gt; error: ./factorial
$ &lt;span class=&quot;token function&quot;&gt;file&lt;/span&gt; factorial
factorial: ELF &lt;span class=&quot;token number&quot;&gt;32&lt;/span&gt;-bit LSB executable, ARM, EABI5 version &lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;SYSV&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;, &lt;span class=&quot;token punctuation&quot;&gt;&#92;&lt;/span&gt;
dynamically linked, interpreter /lib/ld-linux-armhf.so.3, &lt;span class=&quot;token keyword&quot;&gt;for&lt;/span&gt; GNU/Linux &lt;span class=&quot;token punctuation&quot;&gt;&#92;&lt;/span&gt;
&lt;span class=&quot;token number&quot;&gt;4.19&lt;/span&gt;.21, with debug_info, not stripped&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;We were expecting this error, right? After all, we cannot just execute an arm binary in a x86_64 system. However, we can if we use QEMU in user mode which is what I want to show:&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ qemu-arm ./factorial &lt;span class=&quot;token number&quot;&gt;20&lt;/span&gt; 
Result: &lt;span class=&quot;token number&quot;&gt;2432902008176640000&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Now we have it working - however, there&#39;s a last tool we need to discuss and that&#39;s &lt;code&gt;binfmt&lt;/code&gt;. &lt;code&gt;binfmt_misc&lt;/code&gt; is a linux kernel capability that allows arbitrary executable file formats to be recognized and passed on to an interpreter. This means that we can transparently recognize arm (or any other architecture) executables and request them to be passed to QEMU transparently when we try to execute them.&lt;/p&gt;
&lt;p&gt;Before proceeding, if you wish to follow the examples, verify that your kernel has &lt;code&gt;binfmt&lt;/code&gt; enabled.&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ zcat /proc/config.gz&lt;span class=&quot;token operator&quot;&gt;|&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;grep&lt;/span&gt; BINFMT
&lt;span class=&quot;token assign-left variable&quot;&gt;CONFIG_BINFMT_ELF&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;y
&lt;span class=&quot;token assign-left variable&quot;&gt;CONFIG_COMPAT_BINFMT_ELF&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;y
&lt;span class=&quot;token assign-left variable&quot;&gt;CONFIG_BINFMT_SCRIPT&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;y
&lt;span class=&quot;token assign-left variable&quot;&gt;CONFIG_BINFMT_MISC&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;y&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;If you don&#39;t have &lt;code&gt;/proc/config.gz&lt;/code&gt; try instead &lt;code&gt;grep BINFMT /boot/config-$(uname -r)&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;Let&#39;s go back to our factorial on arm example, but this time install a &lt;code&gt;binfmt&lt;/code&gt; record to call &lt;code&gt;qemu-arm&lt;/code&gt; on the executable whenever we try to execute it directly. A &lt;code&gt;binfmt&lt;/code&gt; record looks like &lt;code&gt;:name:type:offset:magic:mask:interpreter:flags&lt;/code&gt; and needs to be installed by echoing the correct string to &lt;code&gt;/proc/sys/fs/binfmt_misc/register&lt;/code&gt;. For details on this consult the &lt;a href=&quot;https://www.kernel.org/doc/html/latest/admin-guide/binfmt-misc.html&quot;&gt;kernel
documentation&lt;/a&gt;. For &lt;code&gt;armv7&lt;/code&gt; we can register our interpreter and run our binary transparently like this:&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ &lt;span class=&quot;token function&quot;&gt;sudo&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;bash&lt;/span&gt; &lt;span class=&quot;token parameter variable&quot;&gt;-c&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&#39;echo &quot;:qemu-arm:M:0:&#92;&#92;x7f&#92;&#92;x45&#92;&#92;x4c&#92;&#92;x46&#92;&#92;x01&#92;&#92;x01&#92;&#92;x01&#92;&#92;x00&#92;&#92;x00&#92;&#92;x00&#92;&#92;x00&#92;&#92;x00&#92;&#92;x00&#92;
&#92;&#92;x00&#92;&#92;x00&#92;&#92;x00&#92;&#92;x02&#92;&#92;x00&#92;&#92;x28&#92;&#92;x00:&#92;&#92;xff&#92;&#92;xff&#92;&#92;xff&#92;&#92;xff&#92;&#92;xff&#92;&#92;xff&#92;&#92;xff&#92;&#92;x00&#92;&#92;xff&#92;&#92;xff&#92;&#92;xff&#92;&#92;xff&#92;&#92;xff&#92;
&#92;&#92;xff&#92;&#92;xff&#92;&#92;xff&#92;&#92;xfe&#92;&#92;xff&#92;&#92;xff&#92;&#92;xff:/home/pmatos/installs/bin/qemu-arm:OCF&quot; &gt; &#92;
/proc/sys/fs/binfmt_misc/register&#39;&lt;/span&gt;
$ ./factorial &lt;span class=&quot;token number&quot;&gt;20&lt;/span&gt;
Result: &lt;span class=&quot;token number&quot;&gt;2432902008176640000&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The write to &lt;code&gt;/proc/sys/fs/binfmt_misc/register&lt;/code&gt; may fail if you already have a record for &lt;code&gt;qemu-arm&lt;/code&gt; setup. To remove the record run &lt;code&gt;echo -1 &amp;gt; /proc/sys/fs/binfmt_misc/qemu-arm&lt;/code&gt;. Note that the file, as shown above is an ARM binary and yet we transparently run it through QEMU thanks to the &lt;code&gt;binfmt&lt;/code&gt; magic we have initially set up.&lt;/p&gt;
&lt;p&gt;I should note, that while interesting to understand how this works behind the scenes, several people have created images to do the binfmt registration for you. One of those projects is &lt;a href=&quot;https://hub.docker.com/r/docker/binfmt/tags&quot;&gt;docker/binfmt&lt;/a&gt; which you can try to run using the latest tag: &lt;code&gt;docker run --rm --privileged docker/binfmt:66f9012c56a8316f9244ffd7622d7c21c1f6f28d&lt;/code&gt; (another example of a similar project is
&lt;a href=&quot;https://github.com/multiarch/qemu-user-static&quot;&gt;multiarch/qemu-user-static&lt;/a&gt;). If, for some reason, this does not work you should know enough by now to proceed with the registration manually.&lt;/p&gt;
&lt;h2&gt;Creating container base images&lt;/h2&gt;
&lt;p&gt;We have gone through creating chroot jails and transparently executing cross-architecture binaries with QEMU. A base image for a container is based pretty much on what we have just learned. To create a container
base image we will create a jail with a foreign root filesystem and we will use QEMU to execute binaries inside the jail. Once all is working, we import it into docker.&lt;/p&gt;
&lt;p&gt;To help us create a base system, we will use &lt;code&gt;debootstrap&lt;/code&gt; in two stages. We split this into two stages so we can setup QEMU in between.&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ &lt;span class=&quot;token function&quot;&gt;mkdir&lt;/span&gt; rootfs
$ &lt;span class=&quot;token function&quot;&gt;debootstrap&lt;/span&gt; &lt;span class=&quot;token parameter variable&quot;&gt;--foreign&lt;/span&gt; --no-check-gpg &lt;span class=&quot;token parameter variable&quot;&gt;--arch&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;arm buster ./rootfs http://httpredir.debian.org/debian/
$ &lt;span class=&quot;token function&quot;&gt;cp&lt;/span&gt; &lt;span class=&quot;token parameter variable&quot;&gt;-v&lt;/span&gt; /usr/bin/qemu-arm-static ./rootfs/usr/bin/
$ &lt;span class=&quot;token function&quot;&gt;chroot&lt;/span&gt; ./rootfs ./debootstrap/debootstrap --second-stage &lt;span class=&quot;token parameter variable&quot;&gt;--verbose&lt;/span&gt;
$ &lt;span class=&quot;token function&quot;&gt;mount&lt;/span&gt; &lt;span class=&quot;token parameter variable&quot;&gt;-t&lt;/span&gt; devpts devpts ./rootfs/dev/pts
$ &lt;span class=&quot;token function&quot;&gt;mount&lt;/span&gt; &lt;span class=&quot;token parameter variable&quot;&gt;-t&lt;/span&gt; proc proc ./rootfs/proc
$ &lt;span class=&quot;token function&quot;&gt;mount&lt;/span&gt; &lt;span class=&quot;token parameter variable&quot;&gt;-t&lt;/span&gt; sysfs sysfs ./rootfs/sys&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;At this point our system is a debian base system with root at &lt;code&gt;./rootfs&lt;/code&gt;. Note that the &lt;code&gt;qemu&lt;/code&gt; you need to copy into &lt;code&gt;./rootfs/usr/bin&lt;/code&gt; needs to be static in order to avoid dynamic loading issues inside the jail when it is invoked.&lt;/p&gt;
&lt;p&gt;Another important aspect to consider is that depending on your &lt;code&gt;binfmt&lt;/code&gt; setup, you need to put the binary in the proper place inside the jail. For the above to work, your &lt;code&gt;binfmt&lt;/code&gt; interpreter registration has to
point to an interpreter at &lt;code&gt;/usr/bin/qemu-arm-static&lt;/code&gt;, which is the absolute path as seen from inside the jail. Now we can install all dependencies in the system at will. All commands will be using &lt;code&gt;qemu&lt;/code&gt; transparently to execute if &lt;code&gt;binfmt&lt;/code&gt; setup worked correctly. Remember that in the case of chroot jails (and also of containers), the kernel in use is the kernel of your host. Therefore that is the only kernel that needs to be set up with &lt;code&gt;binfmt&lt;/code&gt;. You set it up outside your chroot jail and when the kernel needs to execute a foreign executable, it will look at the current &lt;code&gt;binfmt&lt;/code&gt; setup for the interpreter. However, the kernel cannot access a filesystem outside the jail, so that interpreter needs to exist inside the it.&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ &lt;span class=&quot;token function&quot;&gt;chroot&lt;/span&gt; ./rootfs &lt;span class=&quot;token function&quot;&gt;apt-get&lt;/span&gt; update
$ &lt;span class=&quot;token function&quot;&gt;chroot&lt;/span&gt; ./rootfs &lt;span class=&quot;token function&quot;&gt;apt-get&lt;/span&gt; &lt;span class=&quot;token parameter variable&quot;&gt;-y&lt;/span&gt; upgrade
$ &lt;span class=&quot;token function&quot;&gt;chroot&lt;/span&gt; ./rootfs &lt;span class=&quot;token function&quot;&gt;apt-get&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;install&lt;/span&gt; &lt;span class=&quot;token parameter variable&quot;&gt;-y&lt;/span&gt; g++ cmake libicu-dev &lt;span class=&quot;token function&quot;&gt;git&lt;/span&gt; ruby-highline ruby-json python
&lt;span class=&quot;token function&quot;&gt;chroot&lt;/span&gt; ./rootfs &lt;span class=&quot;token function&quot;&gt;apt-get&lt;/span&gt; &lt;span class=&quot;token parameter variable&quot;&gt;-y&lt;/span&gt; autoremove
&lt;span class=&quot;token function&quot;&gt;chroot&lt;/span&gt; ./rootfs &lt;span class=&quot;token function&quot;&gt;apt-get&lt;/span&gt; clean
&lt;span class=&quot;token function&quot;&gt;chroot&lt;/span&gt; ./rootfs &lt;span class=&quot;token function&quot;&gt;find&lt;/span&gt; /var/lib/apt/lists &lt;span class=&quot;token parameter variable&quot;&gt;-type&lt;/span&gt; f &lt;span class=&quot;token parameter variable&quot;&gt;-delete&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;These are the steps to install the dependencies to build and test JSC. Let&#39;s unmount the filesystems in order to create the docker base image.&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;&lt;span class=&quot;token function&quot;&gt;umount&lt;/span&gt; ./rootfs/dev/pts
&lt;span class=&quot;token function&quot;&gt;umount&lt;/span&gt; ./rootfs/proc
&lt;span class=&quot;token function&quot;&gt;umount&lt;/span&gt; ./rootfs/sys&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Let&#39;s &lt;code&gt;tar&lt;/code&gt; our jail and import it into a docker image.&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ &lt;span class=&quot;token function&quot;&gt;tar&lt;/span&gt; --numeric-owner &lt;span class=&quot;token parameter variable&quot;&gt;-cvf&lt;/span&gt; buster-arm.tar &lt;span class=&quot;token parameter variable&quot;&gt;-C&lt;/span&gt; ./rootfs &lt;span class=&quot;token builtin class-name&quot;&gt;.&lt;/span&gt;
$ &lt;span class=&quot;token function&quot;&gt;docker&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;import&lt;/span&gt; buster-arm.tar jsc-base:arm-raw&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The image &lt;code&gt;jsc-base:arm-raw&lt;/code&gt; now contains our raw system. To be able to add metadata or build on top of this image before releasing we can build a docker image that extends the raw image version.&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ &lt;span class=&quot;token function&quot;&gt;cat&lt;/span&gt; &lt;span class=&quot;token operator&quot;&gt;&amp;lt;&amp;lt;&lt;/span&gt;&lt;span class=&quot;token string&quot;&gt;EOF&lt;span class=&quot;token bash punctuation&quot;&gt; &lt;span class=&quot;token operator&quot;&gt;&gt;&lt;/span&gt; jsc32-base.Dockerfile&lt;/span&gt;
FROM jsc32-base:arm-raw

LABEL description=&quot;Minimal Debian image to reproduce JSC dev&quot;
LABEL maintainer=&quot;Paulo Matos &amp;lt;pmatos@igalia.com&gt;&quot;
 
CMD [&quot;/bin/bash&quot;]
EOF&lt;/span&gt;
$ &lt;span class=&quot;token function&quot;&gt;docker&lt;/span&gt; build &lt;span class=&quot;token parameter variable&quot;&gt;-t&lt;/span&gt; pmatos/jsc32-base:arm &lt;span class=&quot;token parameter variable&quot;&gt;-f&lt;/span&gt; jsc32-base.Dockerfile
$ &lt;span class=&quot;token function&quot;&gt;docker&lt;/span&gt; push pmatos/jsc32-base:arm&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Once we have pushed the image into the repository, we can run it from anywhere as long as &lt;code&gt;binfmt&lt;/code&gt; is setup properly.&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ &lt;span class=&quot;token function&quot;&gt;docker&lt;/span&gt; run pmatos/jsc32-base:arm &lt;span class=&quot;token function&quot;&gt;file&lt;/span&gt; /bin/bash
/bin/bash: ELF &lt;span class=&quot;token number&quot;&gt;32&lt;/span&gt;-bit LSB pie executable, ARM, EABI5 version &lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;SYSV&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;, dynamically linked, &lt;span class=&quot;token punctuation&quot;&gt;&#92;&lt;/span&gt;
interpreter /lib/ld-linux-armhf.so.3, &lt;span class=&quot;token keyword&quot;&gt;for&lt;/span&gt; GNU/Linux &lt;span class=&quot;token number&quot;&gt;3.2&lt;/span&gt;.0, &lt;span class=&quot;token punctuation&quot;&gt;&#92;&lt;/span&gt;
BuildID&lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;sha1&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;40e50d160a6c70d1a4e961200202cf853b4a2145, stripped&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Or to get an interactive shell inside the container, use &lt;code&gt;-it&lt;/code&gt;: &lt;code&gt;docker run -it pmatos/jsc32-base:arm /bin/bash&lt;/code&gt;.&lt;/p&gt;
&lt;h2&gt;Going rootless with podman&lt;/h2&gt;
&lt;p&gt;While &lt;a href=&quot;https://www.docker.com/&quot;&gt;&lt;code&gt;docker&lt;/code&gt;&lt;/a&gt; is a product of Docker Inc with various components and pricing tiers, &lt;a href=&quot;https://podman.io/&quot;&gt;&lt;code&gt;podman&lt;/code&gt;&lt;/a&gt; is a free and open source daemonless engine for developing, managing, and running containers on Linux. One of the main internal differences between &lt;code&gt;docker&lt;/code&gt; and &lt;code&gt;podman&lt;/code&gt; is that &lt;code&gt;podman&lt;/code&gt; uses &lt;a href=&quot;https://en.wikipedia.org/wiki/Cgroups&quot;&gt;cgroups&lt;/a&gt; v2, which &lt;code&gt;docker&lt;/code&gt; doesn&#39;t yet support. Since Fedora 31 ships with cgroups v2, for many users &lt;code&gt;podman&lt;/code&gt; is the only container engine available. Fortunately, &lt;code&gt;podman&lt;/code&gt; is largely &lt;code&gt;docker&lt;/code&gt; compatible meaning that you can &lt;code&gt;alias docker=podman&lt;/code&gt; without any issues.&lt;/p&gt;
&lt;p&gt;So you can do something like this.&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ &lt;span class=&quot;token function&quot;&gt;podman&lt;/span&gt; run pmatos/jsc32-base:arm &lt;span class=&quot;token function&quot;&gt;file&lt;/span&gt; /bin/bash
/bin/bash: ELF &lt;span class=&quot;token number&quot;&gt;32&lt;/span&gt;-bit LSB pie executable, ARM, EABI5 version &lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;SYSV&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;, dynamically linked, &lt;span class=&quot;token punctuation&quot;&gt;&#92;&lt;/span&gt;
interpreter /lib/ld-linux-armhf.so.3, &lt;span class=&quot;token keyword&quot;&gt;for&lt;/span&gt; GNU/Linux &lt;span class=&quot;token number&quot;&gt;3.2&lt;/span&gt;.0, &lt;span class=&quot;token punctuation&quot;&gt;&#92;&lt;/span&gt;
BuildID&lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;sha1&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;40e50d160a6c70d1a4e961200202cf853b4a2145, stripped&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;As long your &lt;code&gt;binfmt&lt;/code&gt; is properly setup, it will run as well as it did with docker.&lt;/p&gt;
&lt;h2&gt;Application to JSC&lt;/h2&gt;
&lt;p&gt;The initial code developed for reproducible JSC32 builds can be found in the &lt;a href=&quot;https://github.com/pmatos/WebKit-misc/&quot;&gt;WebKit-misc&lt;/a&gt; repository and was initially pushed under commit &lt;a href=&quot;https://github.com/pmatos/WebKit-misc/commit/94bf25ebe888ae1824113eeb269fe0c0fcb5f3d4&quot;&gt;94bf25e&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;The main script is in &lt;a href=&quot;https://github.com/pmatos/WebKit-misc/blob/94bf25ebe888ae1824113eeb269fe0c0fcb5f3d4/containers/reprojsc.sh&quot;&gt;&lt;code&gt;containers/reprojsc.sh&lt;/code&gt;&lt;/a&gt; and follows the plan laid out in this blog post.&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;An image is created using &lt;a href=&quot;https://github.com/pmatos/WebKit-misc/blob/94bf25ebe888ae1824113eeb269fe0c0fcb5f3d4/containers/jsc32-base/build-image.sh&quot;&gt;&lt;code&gt;containers/jsc32-base/build-image.sh&lt;/code&gt;&lt;/a&gt; and pushed into docker hub &lt;a href=&quot;https://hub.docker.com/r/pmatos/jsc32-base&quot;&gt;pmatos/jsc32-base&lt;/a&gt;. This only needs to be done when the image is changed.&lt;/li&gt;
&lt;li&gt;The &lt;a href=&quot;https://github.com/pmatos/WebKit-misc/blob/94bf25ebe888ae1824113eeb269fe0c0fcb5f3d4/containers/reprojsc.sh&quot;&gt;&lt;code&gt;containers/reprojsc.sh&lt;/code&gt;&lt;/a&gt; script is run each time one wants to trigger a build and/or test of JSC. This sets up QEMU for the desired architecture, starts the image and issues the necessary commands for the actions laid out in the command line.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;To run it yourself, checkout &lt;a href=&quot;https://github.com/pmatos/WebKit-misc/&quot;&gt;WebKit-misc&lt;/a&gt; and &lt;a href=&quot;https://webkit.org/&quot;&gt;WebKit&lt;/a&gt;, and run &lt;code&gt;reprojsc.sh&lt;/code&gt;.&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ &lt;span class=&quot;token function&quot;&gt;git&lt;/span&gt; clone &lt;span class=&quot;token parameter variable&quot;&gt;--depth&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt; git://git.webkit.org/WebKit.git
$ &lt;span class=&quot;token builtin class-name&quot;&gt;export&lt;/span&gt; &lt;span class=&quot;token assign-left variable&quot;&gt;WEBKIT_PATH&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token environment constant&quot;&gt;$PWD&lt;/span&gt;/WebKit
$ &lt;span class=&quot;token function&quot;&gt;git&lt;/span&gt; clone &lt;span class=&quot;token parameter variable&quot;&gt;--depth&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt; git@github.com:pmatos/WebKit-misc.git
$ &lt;span class=&quot;token builtin class-name&quot;&gt;cd&lt;/span&gt; WebKit-misc/containers
$ ./reprojsc.sh &lt;span class=&quot;token parameter variable&quot;&gt;-a&lt;/span&gt; arm &lt;span class=&quot;token parameter variable&quot;&gt;-b&lt;/span&gt; &lt;span class=&quot;token parameter variable&quot;&gt;-t&lt;/span&gt; &lt;span class=&quot;token parameter variable&quot;&gt;-i&lt;/span&gt; &lt;span class=&quot;token variable&quot;&gt;$WEBKIT_PATH&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This will build, test and send you into an interactive session so you can debug some possibly failed tests. Inside the container, you&#39;ll be in an ARM Cortex-A7 (32bits) environment. The other available architecture at the moment is MIPS which can be chosen with &lt;code&gt;-a mips&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;If you have any issues or requests please open an issue in &lt;a href=&quot;https://github.com/pmatos/WebKit-misc/issues&quot;&gt;GitHub&lt;/a&gt;.&lt;/p&gt;
&lt;h2&gt;What&#39;s missing?&lt;/h2&gt;
&lt;p&gt;There are many things that deserved discussion but I won&#39;t elaborate further. One of them is the difference between the container execution engine and the image builder. When using docker it&#39;s easy to think
that they are the same but it is not the case. For example, docker has a new image builder, called &lt;a href=&quot;https://docs.docker.com/buildx/working-with-buildx/&quot;&gt;&lt;code&gt;buildx&lt;/code&gt;&lt;/a&gt; whichhas in-built support for the creation of multi-architecture images. On the other hand, &lt;code&gt;podman&lt;/code&gt; is a container execution engine but an image creator is, for example, &lt;a href=&quot;https://buildah.io/&quot;&gt;&lt;code&gt;buildah&lt;/code&gt;&lt;/a&gt;, which doesn&#39;t have multi-architecture image support
&lt;a href=&quot;https://github.com/containers/buildah/issues/1590&quot;&gt;yet&lt;/a&gt;. Don&#39;t worry if you are confused - things are moving fast in this area and it&#39;s easy to lose track of all the tools out there.&lt;/p&gt;
&lt;p&gt;With the ability to transparently emulate other architectures QEMU user mode will see more usage patterns. Bugs in QEMU can look like application bugs and debugging is not straightforward, so bear in this
in mind when using this technology.&lt;/p&gt;
&lt;h2&gt;Acknowledgments&lt;/h2&gt;
&lt;p&gt;Thanks to my fellow Igalians Angelos Oikonomopoulos and Philip Chimento for providing suggestions and corrections for this blog post.&lt;/p&gt;
</content>
  </entry>
  <entry>
    <title>A Brief Look at the WebKit Workflow</title>
    <link href="https://p.ocmatos.com/blog/a-brief-look-at-the-webkit-workflow.html"/>
    <updated>2019-10-02T00:00:00Z</updated>
    <id>https://p.ocmatos.com/blog/a-brief-look-at-the-webkit-workflow.html</id>
    <content xml:lang="en" type="html">&lt;p&gt;Since I joined &lt;a href=&quot;https://www.igalia.com/&quot;&gt;Igalia&lt;/a&gt; back in May, I have been working on WebKit. After more than a decade working on GCC and LLVM, moving codebases, workflows, and compilation techniques and optimizations meant I was in very unfamiliar territory.&lt;/p&gt;
&lt;div style=&quot;text-align:center;&quot;&gt;
	&lt;img alt=&quot;Where am I? GIF&quot; class=&quot;post-photo&quot; src=&quot;https://p.ocmatos.com/img/where-am-i.gif&quot; /&gt;
&lt;/div&gt;
&lt;p&gt;One of the main things to understand is the contribution process and each project has their own rules and methodology, so lets start straight into how to contribute to WebKit. In a few places I will focus specifically on JSC -- for other parts of WebKit you might want to contact the &lt;a href=&quot;http://lists.webkit.org/mailman/listinfo/webkit-help&quot;&gt;mailing list&lt;/a&gt; or the &lt;a href=&quot;irc://irc.freenode.net/#webkit&quot;&gt;irc channel&lt;/a&gt; for specific advice.&lt;/p&gt;
&lt;p&gt;The information in this post should be used in addition to what is available in the official &lt;a href=&quot;https://webkit.org/getting-started/&quot;&gt;Getting Started&lt;/a&gt; and &lt;a href=&quot;https://webkit.org/contributing-code/&quot;&gt;Contributing Code&lt;/a&gt; documents.&lt;/p&gt;
&lt;h2&gt;Cloning&lt;/h2&gt;
&lt;p&gt;WebKit uses &lt;code&gt;svn&lt;/code&gt; as the default for version control but it has a &lt;code&gt;git&lt;/code&gt; mirror, therefore given my preference for using &lt;code&gt;git&lt;/code&gt;, we&#39;ll use that in the following examples.&lt;/p&gt;
&lt;p&gt;To clone WebKit we type:&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ &lt;span class=&quot;token function&quot;&gt;git&lt;/span&gt; clone git://git.webkit.org/WebKit.git
&lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;.
$ &lt;span class=&quot;token function&quot;&gt;du&lt;/span&gt; &lt;span class=&quot;token parameter variable&quot;&gt;-hs&lt;/span&gt; WebKit
&lt;span class=&quot;token number&quot;&gt;9&lt;/span&gt;.7G    WebKit&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This will take almost 10Gb of space at time of writing so if you are in a rush or short of space, use the &lt;code&gt;--depth=1&lt;/code&gt; option to &lt;code&gt;git clone&lt;/code&gt;.&lt;/p&gt;
&lt;h2&gt;Building and Testing&lt;/h2&gt;
&lt;p&gt;The most important directory for workflow in WebKit is &lt;code&gt;Tools/Scripts&lt;/code&gt; &lt;sup class=&quot;footnote-ref&quot;&gt;&lt;a href=&quot;https://p.ocmatos.com/blog/a-brief-look-at-the-webkit-workflow.html#fn1&quot; id=&quot;fnref1&quot;&gt;[1]&lt;/a&gt;&lt;/sup&gt;. Here you will find all the scripts we will use in this post.&lt;/p&gt;
&lt;p&gt;There are several scripts to build different parts of WebKit in different modes -- called Ports. For JSC you would use &lt;code&gt;build-jsc&lt;/code&gt; script:&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ Tools/Scripts/build-jsc --jsc-only &lt;span class=&quot;token parameter variable&quot;&gt;--debug&lt;/span&gt;
&lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;.
&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;
 JavaScriptCore is now built &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;02m:44s&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;. 
&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The &lt;code&gt;--jsc-only&lt;/code&gt; flag specifies the WebKit port to build and you probably should use &lt;code&gt;--debug&lt;/code&gt; while developing since it makes debugging much easier. Once you reach a point where you are happy with how it&#39;s going, run some tests. I should also mention that WebKit follows a very specific coding style and has a script to check for style problems. Don&#39;t worry too much about this in the beginning, until you have a patch. After a few contributions, you&#39;ll just get the hang of how to style your code. Make sure, either during development or before creating a patch that you check for style issues with &lt;code&gt;Tools/Scripts/check-webkit-style&lt;/code&gt;:&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ Tools/Scripts/check-webkit-style
ERROR: Source/WTF/wtf/StackBounds.cpp:25:  Alphabetical sorting problem.  &lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;build/include_order&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;4&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt;
ERROR: Source/WTF/wtf/StackBounds.cpp:126:  Place brace on its own line &lt;span class=&quot;token keyword&quot;&gt;for&lt;/span&gt; &lt;span class=&quot;token keyword&quot;&gt;function&lt;/span&gt; definitions.  &lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;whitespace/braces&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;4&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt;
Total errors found: &lt;span class=&quot;token number&quot;&gt;2&lt;/span&gt; &lt;span class=&quot;token keyword&quot;&gt;in&lt;/span&gt; &lt;span class=&quot;token number&quot;&gt;2&lt;/span&gt; files&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The errors are self-explanatory and easy to fix. Go ahead and make the necessary changes.&lt;/p&gt;
&lt;p&gt;Time to run some tests - There are several test scripts and you should probably run the tests most appropriate to the changes you have made. In general, you can list the testing scripts with:&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ &lt;span class=&quot;token function&quot;&gt;ls&lt;/span&gt; &lt;span class=&quot;token parameter variable&quot;&gt;-1&lt;/span&gt; Tools/Scripts/run-*-tests
Tools/Scripts/run-api-tests
Tools/Scripts/run-bindings-tests
Tools/Scripts/run-builtins-generator-tests
Tools/Scripts/run-dashboard-tests
Tools/Scripts/run-gtk-tests
Tools/Scripts/run-iexploder-tests
Tools/Scripts/run-inspector-generator-tests
Tools/Scripts/run-javascriptcore-tests
Tools/Scripts/run-jsc-stress-tests
Tools/Scripts/run-mangleme-tests
Tools/Scripts/run-perf-tests
Tools/Scripts/run-regexp-tests
Tools/Scripts/run-webdriver-tests
Tools/Scripts/run-webkit-tests
Tools/Scripts/run-web-platform-tests
Tools/Scripts/run-wpe-tests&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Scripts tend to have a very good &lt;code&gt;--help&lt;/code&gt; menu so you should give those a try before you run them. For tests in JSC use: &lt;code&gt;run-javascriptcore-tests&lt;/code&gt;.&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ Tools/Scripts/run-javascriptcore-tests --no-build --no-fail-fast &lt;span class=&quot;token parameter variable&quot;&gt;--debug&lt;/span&gt; --jsc-only
&lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;.
    &lt;span class=&quot;token number&quot;&gt;0&lt;/span&gt; failures found.&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;If you run the tests and see some possibly unrelated failures be sure to run them again without your changes since they might be failures that made it into upstream and not necessarily a result of your changes.&lt;/p&gt;
&lt;h2&gt;Submitting a patch&lt;/h2&gt;
&lt;p&gt;Once you have the patch ready for upstream, you can submit it automatically with &lt;code&gt;webkit-patch&lt;/code&gt;, which will automate much of the workflow process. Before proceeding though, make sure you have an account in the &lt;a href=&quot;https://bugs.webkit.org/&quot;&gt;WebKit bug database&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;The process can be summarised as follows:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;Create a patch upstream with &lt;code&gt;webkit-patch upload --no-review&lt;/code&gt; (the script will create a new bug to hold your patch)&lt;/li&gt;
&lt;/ol&gt;
&lt;ul&gt;
&lt;li&gt;This will prompt you to fill at least one &lt;code&gt;ChangeLog&lt;/code&gt; in your favourite editor. If in doubt of the style, run a &lt;code&gt;git log&lt;/code&gt; to see how other logs have been written and follow the same style&lt;/li&gt;
&lt;li&gt;Remember not to change the line with the bug title and URL, these should remain unchanged from what &lt;code&gt;webkit-patch&lt;/code&gt; generated&lt;/li&gt;
&lt;/ul&gt;
&lt;ol start=&quot;2&quot;&gt;
&lt;li&gt;If you are confident about it, set the flag &lt;code&gt;review?&lt;/code&gt; to request a review of your code&lt;/li&gt;
&lt;/ol&gt;
&lt;ul&gt;
&lt;li&gt;Wait for a review... if you end up waiting a week without reply feel free to ping someone on the &lt;a href=&quot;http://lists.webkit.org/mailman/listinfo/webkit-help&quot;&gt;mailing list&lt;/a&gt; or the &lt;a href=&quot;irc://irc.freenode.net/#webkit&quot;&gt;irc channel&lt;/a&gt; about it&lt;/li&gt;
&lt;li&gt;If you get a &lt;code&gt;review-&lt;/code&gt; then, read the comments provided and fix the issues mentioned in the comments. Once the issues have been fixed, upload the new patch with &lt;code&gt;webkit-patch upload&lt;/code&gt;. Referring to the bug number in the command line is optional because the script can read the bug number from the already written &lt;code&gt;ChangeLog&lt;/code&gt;. But to be explicit about it, you can specify the bug number with &lt;code&gt;webkit-patch upload &amp;lt;BUG&amp;gt;&lt;/code&gt;, where &lt;code&gt;&amp;lt;BUG&amp;gt;&lt;/code&gt; was the bug number created in step 1.&lt;/li&gt;
&lt;/ul&gt;
&lt;ol start=&quot;3&quot;&gt;
&lt;li&gt;After the review loop in point 2., you have a &lt;code&gt;review+&lt;/code&gt;. Until you have the correct permissions, you&#39;ll need someone to &lt;code&gt;commit-queue+&lt;/code&gt; your code. In order to request your code to be put into the commit queue, mark it as &lt;code&gt;commit-queue?&lt;/code&gt;. As soon as it is marked &lt;code&gt;commit-queue+&lt;/code&gt;, the bots will check your code, run the necessary tests, commit the patch upstream and close the bug.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;That&#39;s it. You got your contribution upstream -- Congratulations!&lt;/p&gt;
&lt;h2&gt;Further notes on the review/commit-queue cycle&lt;/h2&gt;
&lt;p&gt;In order not to make the above three-point loop more complicated, I am using this section to drill down on the review and commit-queue flags. As first time contributors, all of our patches need to be reviewed and put into the commit-queue. These flags are known as &lt;code&gt;review&lt;/code&gt; and &lt;code&gt;commit-queue&lt;/code&gt; or by the shorthands &lt;code&gt;r&lt;/code&gt; and &lt;code&gt;cq&lt;/code&gt;. To these you can append the symbols &lt;code&gt;-&lt;/code&gt;, &lt;code&gt;?&lt;/code&gt; or &lt;code&gt;+&lt;/code&gt;. We can only set &lt;code&gt;r?&lt;/code&gt; and &lt;code&gt;cq?&lt;/code&gt; and wait for someone with more permissions to change these accordingly.&lt;/p&gt;
&lt;p&gt;There are two ways to change these flags. In the web interface of your bug, you can click on the &amp;quot;Details&amp;quot; link for your patch and then change the flags accordingly.&lt;/p&gt;
&lt;div style=&quot;text-align:center;&quot;&gt;
	&lt;img alt=&quot;Patch Details&quot; class=&quot;post-photo&quot; src=&quot;https://p.ocmatos.com/img/2019-10-02-WebKitWorkflow-PatchDetails.png&quot; /&gt;
&lt;/div&gt;
&lt;p&gt;Once in the patch details view, you can see the patch flags.&lt;/p&gt;
&lt;div style=&quot;text-align:center;&quot;&gt;
	&lt;img alt=&quot;Patch Flags&quot; class=&quot;post-photo&quot; src=&quot;https://p.ocmatos.com/img/2019-10-02-WebKitWorkflow-Flags.png&quot; /&gt;
&lt;/div&gt;
&lt;p&gt;When you submit a bug with &lt;code&gt;webkit-patch upload&lt;/code&gt;, if confident,  you can set the &lt;code&gt;r?&lt;/code&gt; flag requesting a review by &lt;em&gt;not&lt;/em&gt; using &lt;code&gt;--no-review&lt;/code&gt; in the command. However, if you want to &lt;em&gt;in addition&lt;/em&gt; request the patch to be sent to the commit queue after review, you can add to the &lt;code&gt;upload&lt;/code&gt; command &lt;code&gt;--request-commit&lt;/code&gt;. This will add the flags &lt;code&gt;r?&lt;/code&gt; and &lt;code&gt;cq?&lt;/code&gt;. If you don&#39;t use &lt;code&gt;cq?&lt;/code&gt;, then you can manually set it once you get an &lt;code&gt;r+&lt;/code&gt;.&lt;/p&gt;
&lt;h2&gt;The review process&lt;/h2&gt;
&lt;p&gt;There are a couple of situations that can happen during the review process (point 2. above) that are worth mentioning.&lt;/p&gt;
&lt;h3&gt;Wrong bug title&lt;/h3&gt;
&lt;p&gt;Imagine that you got your patch reviewed and you realised that the bug title, which you created when you first submitted your patch was based on incorrect knowledge and you wish to change it? Also, this needs to be reflected in the patch.&lt;/p&gt;
&lt;p&gt;When you modify your patch locally and keep the ChangeLog unmodified, &lt;code&gt;webkit-patch upload --no-review&lt;/code&gt; will automatically use the &lt;code&gt;ChangeLog&lt;/code&gt; that you have locally. The best way to change the title of your bug is to do the following:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;Update the bug title in your bug page using the web interface, and;&lt;/li&gt;
&lt;li&gt;Execute &lt;code&gt;webkit-patch upload&lt;/code&gt; whenever you are ready to re-submit your fixed patch but add &lt;code&gt;--update-changelog&lt;/code&gt;. This will automatically grab the new title and update the &lt;code&gt;ChangeLog&lt;/code&gt; for you.&lt;/li&gt;
&lt;/ol&gt;
&lt;h3&gt;Apply patch locally&lt;/h3&gt;
&lt;p&gt;Imagine a collaborator improved your patch, uploaded it to the bug but it is still not quite right. You need for some reason to apply the patch locally to keep working on it. You can download it manually and apply it, however you can also automatically do this by checking the patch id (also known as attachment id) on the web interface and running on your WebKit checkout &lt;code&gt;webkit-patch apply-attachment --force-clean &amp;lt;ATTACH&amp;gt;&lt;/code&gt;, where &lt;code&gt;&amp;lt;ATTACH&amp;gt;&lt;/code&gt; is your patch id. Do note that &lt;code&gt;--force-clean&lt;/code&gt; will remove all your local changes and commit.&lt;/p&gt;
&lt;p&gt;There is a simpler alternative, where you don&#39;t need to look up the attachment id. If all you need is the latest patch uploaded to a bug, you can use &lt;code&gt;webkit-patch apply-from-bug &amp;lt;BUG&amp;gt;&lt;/code&gt;. As with &lt;code&gt;apply-attachment&lt;/code&gt;, the command &lt;code&gt;apply-from-bug&lt;/code&gt; also accepts &lt;code&gt;--force-clean&lt;/code&gt;&lt;/p&gt;
&lt;h2&gt;Command summary&lt;/h2&gt;
&lt;h3&gt;Help&lt;/h3&gt;
&lt;h4&gt;Global Help&lt;/h4&gt;
&lt;p&gt;&lt;code&gt;webkit-patch help --all-commands&lt;/code&gt;&lt;/p&gt;
&lt;h4&gt;Command Help&lt;/h4&gt;
&lt;p&gt;Take &lt;code&gt;upload&lt;/code&gt; as an example:&lt;/p&gt;
&lt;p&gt;&lt;code&gt;webkit-patch help upload&lt;/code&gt;&lt;/p&gt;
&lt;h3&gt;Upload patch&lt;/h3&gt;
&lt;p&gt;&lt;code&gt;webkit-patch upload&lt;/code&gt;&lt;/p&gt;
&lt;p&gt;This will upload your current local changes as a patch upstream. It will show you the patch it is going to upload before it actually does so it&#39;s safe to do to so what&#39;s included in the patch before the actual upload takes place.&lt;/p&gt;
&lt;p&gt;Relevant flags:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;code&gt;--no-review&lt;/code&gt;: by default, the &lt;code&gt;upload&lt;/code&gt; command will set that patch to &lt;code&gt;review?&lt;/code&gt; (commonly shortedned to &lt;code&gt;r?&lt;/code&gt;). Use &lt;code&gt;--no-review&lt;/code&gt; if you think the patch is not ready to review, or if you want to confirm the patch is not breaking any tests by using the upstream build workers.&lt;/li&gt;
&lt;li&gt;&lt;code&gt;--request-review&lt;/code&gt;: this will set the &lt;code&gt;cq?&lt;/code&gt; flag on your patch so that it is added to the commit queue after review.&lt;/li&gt;
&lt;li&gt;&lt;code&gt;--update-changelog&lt;/code&gt;: by default if a &lt;code&gt;ChangeLog&lt;/code&gt; already exists it will use that, otherwise it will ask you to write one by opening your favourite editor. Sometimes, however you might have written a &lt;code&gt;ChangeLog&lt;/code&gt; but you want to modify it. Use &lt;code&gt;--update-changelog&lt;/code&gt; to request that.&lt;/li&gt;
&lt;li&gt;&lt;code&gt;--comment=&amp;lt;COMMENT&amp;gt;&lt;/code&gt;: you can specify a comment to add to the post along with the patch upload. If you are uploading a new patch with changes requested by the reviewer, you should post a comment with &lt;code&gt;--comment=&amp;quot;Here is a new patch that addresses the commands in the review&amp;quot;&lt;/code&gt;.&lt;/li&gt;
&lt;/ul&gt;
&lt;h2&gt;Apply upstream attachment&lt;/h2&gt;
&lt;p&gt;&lt;code&gt;webkit-patch apply-attachment&lt;/code&gt; or &lt;code&gt;webkit-patch apply-from-bug&lt;/code&gt;&lt;/p&gt;
&lt;p&gt;This will apply an upstream patch locally. The command &lt;code&gt;apply-attachment&lt;/code&gt; requires the patch (attachment) identifier so you might have to look that up beforehand. If you only need the latest patch to a bug, you can use &lt;code&gt;apply-from-bug&lt;/code&gt; with the bug identifier.&lt;/p&gt;
&lt;p&gt;Relevant flags:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;code&gt;--force-clean&lt;/code&gt;: if you have changes locally the patch might not apply cleanly. Use this flag to clean everything locally before attempting to apply patch. You might want to use &lt;code&gt;git stash&lt;/code&gt; to stash your changes, if they are important.&lt;/li&gt;
&lt;/ul&gt;
&lt;h2&gt;Bonus -- WebKit on GitHub&lt;/h2&gt;
&lt;p&gt;I prefer to use GitHub to keep my project forks however, the WebKit repository is so large that you won&#39;t be able to import it straight into GitHub (either through the UI or manually). If you try, you&#39;ll get an error about packs being too large.&lt;/p&gt;
&lt;p&gt;The easiest way to get WebKit into GitHub is by forking the &lt;a href=&quot;https://github.com/webkit/webkit&quot;&gt;unofficial mirror here&lt;/a&gt;.&lt;/p&gt;
&lt;h2&gt;Thanks&lt;/h2&gt;
&lt;p&gt;Thanks to my Igalian colleagues Caio Lima and Guillaume Emont for reading a draft of this post and providing valuable suggestions.&lt;/p&gt;
&lt;h2&gt;Corrections or Comments?&lt;/h2&gt;
&lt;p&gt;Feel free to send any corrections or comments &lt;a href=&quot;mailto:pmatos@igalia.com&quot;&gt;straight into my inbox&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;Edits since publication (02.10.2019):&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;02.10.2019: my colleague Guillaume Emont found a typo in the &amp;quot;Upload patch&amp;quot; subsection, where I wrote &lt;code&gt;r+&lt;/code&gt; instead of &lt;code&gt;r?&lt;/code&gt;;&lt;/li&gt;
&lt;/ul&gt;
&lt;hr class=&quot;footnotes-sep&quot; /&gt;
&lt;section class=&quot;footnotes&quot;&gt;
&lt;ol class=&quot;footnotes-list&quot;&gt;
&lt;li id=&quot;fn1&quot; class=&quot;footnote-item&quot;&gt;&lt;p&gt;These scripts require Python2, therefore if you are using Python3 by default, you&#39;ll run into troubles running them. Try creating a virtual environment with &lt;code&gt;$ python2 -m virtualenv venv &amp;amp;&amp;amp; source venv/bin/activate&lt;/code&gt; at the root directory of WebKit. &lt;a href=&quot;https://p.ocmatos.com/blog/a-brief-look-at-the-webkit-workflow.html#fnref1&quot; class=&quot;footnote-backref&quot;&gt;↩︎&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;/section&gt;
</content>
  </entry>
</feed>
