<?xml version="1.0" encoding="utf-8"?>
<feed xmlns="http://www.w3.org/2005/Atom" xml:base="https://p.ocmatos.com/">
  <title>pmatos rambles</title>
  <subtitle>Programming, sports and music - oh wow - enjoy!</subtitle>
  <link href="https://p.ocmatos.com/feed.xml" rel="self"/>
  <link href="https://p.ocmatos.com/"/>
  <updated>2026-02-25T10:08:06Z</updated>
  <id>https://p.ocmatos.com/</id>
  <author>
    <name>Paulo Matos</name>
    <email>p@ocmatos.com</email>
  </author>
  <entry>
    <title>Introducing Musiker: An Interactive Timeline of Classical Music</title>
    <link href="https://p.ocmatos.com/blog/introducing-musiker-an-interactive-timeline-of-classical-music.html"/>
    <updated>2026-02-23T00:00:00Z</updated>
    <id>https://p.ocmatos.com/blog/introducing-musiker-an-interactive-timeline-of-classical-music.html</id>
    <content xml:lang="en" type="html">&lt;p&gt;&lt;a href=&quot;https://musiker.page/&quot;&gt;Musiker&lt;/a&gt; is an interactive timeline that maps the lifetimes and connections of classical music composers and players — from William Byrd in the 1540s to Lang Lang today.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://p.ocmatos.com/img/2026/02/musiker-piano-timeline.png&quot; alt=&quot;Piano timeline on musiker.page&quot; /&gt;&lt;/p&gt;
&lt;h2&gt;What You Can Explore&lt;/h2&gt;
&lt;p&gt;The site currently covers three instruments: &lt;strong&gt;piano&lt;/strong&gt;, &lt;strong&gt;violin&lt;/strong&gt;, and &lt;strong&gt;trombone&lt;/strong&gt;. Each timeline lays out musicians as horizontal bars spanning their lifetimes, set against color-coded era backgrounds — Baroque, Classical, Romantic, Modern, and Contemporary.&lt;/p&gt;
&lt;p&gt;Click on any person to open a detail panel with their portrait, biography, Wikipedia link, and a list of related people. Connection lines between musicians show teacher-student relationships (dashed blue) and family ties (solid red), making it easy to trace how musical traditions passed from one generation to the next.&lt;/p&gt;
&lt;p&gt;You can zoom in with Ctrl+scroll to focus on a specific period, or scroll horizontally to explore the full span of centuries.&lt;/p&gt;
&lt;div style=&quot;position: relative; padding-bottom: 56.25%; height: 0; overflow: hidden; max-width: 100%; margin: 2rem 0;&quot;&gt;
&lt;iframe style=&quot;position: absolute; top: 0; left: 0; width: 100%; height: 100%;&quot; src=&quot;https://www.youtube.com/embed/Nyh2OJA5UxY&quot; title=&quot;Musiker promo video&quot; frameborder=&quot;0&quot; allow=&quot;accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture&quot; allowfullscreen=&quot;&quot;&gt;&lt;/iframe&gt;
&lt;/div&gt;
&lt;h2&gt;Under the Hood&lt;/h2&gt;
&lt;p&gt;Musiker is a React + TypeScript application that renders everything as SVG — no charting libraries involved. The main challenge was fitting dozens of overlapping lifetimes into a readable layout. A greedy lane-packing algorithm assigns each person to the first available vertical lane where they don&#39;t overlap with anyone else, keeping the timeline compact without losing clarity.&lt;/p&gt;
&lt;p&gt;The data is split into three concerns: a shared list of people with bios and metadata, a shared list of connections between them, and per-instrument configuration files that define which people appear and what eras to show. Adding a new instrument is just a matter of creating a new JSON file and listing the relevant people.&lt;/p&gt;
&lt;h2&gt;Part of Rightkey&lt;/h2&gt;
&lt;p&gt;Musiker is built in collaboration with &lt;a href=&quot;https://rightkey.app/&quot;&gt;Rightkey.app&lt;/a&gt;, a music education platform. It&#39;s one piece of a broader effort to make classical music more accessible and easier to explore — whether you&#39;re a student, a teacher, or just curious about how Beethoven connects to Czerny connects to Liszt.&lt;/p&gt;
&lt;h2&gt;Try It Out&lt;/h2&gt;
&lt;p&gt;Head over to &lt;a href=&quot;https://musiker.page/&quot;&gt;musiker.page&lt;/a&gt; and explore. If you notice a missing composer, a wrong connection, or an instrument you&#39;d like to see added, use the feedback links at the top of the page — suggestions and corrections are very welcome.&lt;/p&gt;
</content>
  </entry>
  <entry>
    <title>ESBMC Plugin Update: Python Support, a New Home, and Where This Is Going</title>
    <link href="https://p.ocmatos.com/blog/esbmc-plugin-update-python-support-a-new-home-and-where-this-is-going.html"/>
    <updated>2026-02-23T00:00:00Z</updated>
    <id>https://p.ocmatos.com/blog/esbmc-plugin-update-python-support-a-new-home-and-where-this-is-going.html</id>
    <content xml:lang="en" type="html">&lt;p&gt;A couple of weeks ago I &lt;a href=&quot;https://p.ocmatos.com/blog/formal-verification-in-your-terminal-esbmc-meets-claude-code.html&quot;&gt;wrote about&lt;/a&gt; the ESBMC plugin for Claude Code — a way to bring bounded model checking into your terminal through an AI coding assistant. The response was encouraging, and things have moved quickly since then. This is a short update on what changed, and a first sketch of where I think this is heading.&lt;/p&gt;
&lt;h2&gt;What&#39;s New&lt;/h2&gt;
&lt;h3&gt;Python Support&lt;/h3&gt;
&lt;p&gt;Professor Cordeiro, ESBMC&#39;s lead, picked up the plugin and improved Python verification support. ESBMC has had a Python frontend for a while, but the plugin wasn&#39;t tuned for it. Now it is — you can point it at Python code with type annotations and get the same kind of exhaustive property checking that C users have had. Professor Cordeiro &lt;a href=&quot;https://www.linkedin.com/feed/update/urn:li:activity:7431460883609989120/&quot;&gt;announced this on LinkedIn&lt;/a&gt; along with a demo of verifying Python code for common issues.&lt;/p&gt;
&lt;p&gt;This matters because Python is where a lot of AI-generated code lands. If your LLM writes a function with type annotations, you can now formally verify properties about it without switching tools or contexts.&lt;/p&gt;
&lt;h3&gt;The Agent Marketplace&lt;/h3&gt;
&lt;p&gt;The plugin has moved to its own repository: &lt;a href=&quot;https://github.com/esbmc/agent-marketplace&quot;&gt;esbmc/agent-marketplace&lt;/a&gt;. This is the new home for ESBMC&#39;s Claude Code integration, and I maintain it. The separation from the main ESBMC repo is deliberate — the plugin evolves on a different cadence than the verifier itself, and having a dedicated repo makes it easier to iterate on the agent-facing interface without coupling to ESBMC&#39;s release cycle.&lt;/p&gt;
&lt;p&gt;Installation now uses the new repository names:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;$ claude
&amp;gt; /plugin marketplace add esbmc/agent-marketplace
&amp;gt; /plugin install esbmc-plugin@esbmc-marketplace
&lt;/code&gt;&lt;/pre&gt;
&lt;h2&gt;The Idea I Keep Coming Back To&lt;/h2&gt;
&lt;p&gt;I want to share something that&#39;s been shaping how I think about this plugin, even if I&#39;m not ready to lay out the full argument yet.&lt;/p&gt;
&lt;p&gt;Most of the time, verification is something you do &lt;em&gt;after&lt;/em&gt; writing code. You finish a function, maybe write some tests, and if you&#39;re particularly disciplined, you might run a static analyzer or a model checker before merging. Verification is a gate at the end of a pipeline.&lt;/p&gt;
&lt;p&gt;But what if it wasn&#39;t? What if verification happened &lt;em&gt;while&lt;/em&gt; you write code, the same way &lt;code&gt;cargo check&lt;/code&gt; gives you type errors as you go in Rust? Not as a final gate, but as a co-temporal activity — verification and implementation advancing together, each informing the other.&lt;/p&gt;
&lt;p&gt;This is what excites me about having ESBMC inside a coding agent. The agent doesn&#39;t need to &amp;quot;finish&amp;quot; the code and then &amp;quot;run verification&amp;quot; as a separate step. It can write a function, immediately verify a property, discover a counterexample, and revise — all in one fluid loop. The verification isn&#39;t a post-hoc audit; it&#39;s part of the writing process. Think of it as the difference between a spell checker you run on a finished document and one that underlines words as you type.&lt;/p&gt;
&lt;p&gt;The ESBMC plugin is a first step toward this. Today it&#39;s mostly used reactively — you write code, then ask Claude to verify it. But the skill system means Claude can reach for ESBMC proactively during code generation, and the pieces are falling into place for a tighter loop.&lt;/p&gt;
&lt;p&gt;I have more to say about what this looks like concretely — strategy cascading, counterexample-driven refinement, how you&#39;d encode verification profiles for different domains — but I&#39;ll save that for a dedicated post. For now, I wanted to plant the flag: verification should be co-temporal with coding, not a separate phase, and AI coding agents are the mechanism that makes this practical.&lt;/p&gt;
&lt;h2&gt;Try It&lt;/h2&gt;
&lt;p&gt;If you have ESBMC installed and use Claude Code, give the plugin a spin. It now works with both C and Python, and the &lt;code&gt;/verify&lt;/code&gt; and &lt;code&gt;/audit&lt;/code&gt; commands are the fastest way to get started. File issues on the &lt;a href=&quot;https://github.com/esbmc/agent-marketplace&quot;&gt;agent-marketplace repo&lt;/a&gt; — I want to hear what works and what doesn&#39;t.&lt;/p&gt;
</content>
  </entry>
  <entry>
    <title>Formal Verification in Your Terminal: ESBMC meets Claude Code</title>
    <link href="https://p.ocmatos.com/blog/formal-verification-in-your-terminal-esbmc-meets-claude-code.html"/>
    <updated>2026-02-13T00:00:00Z</updated>
    <id>https://p.ocmatos.com/blog/formal-verification-in-your-terminal-esbmc-meets-claude-code.html</id>
    <content xml:lang="en" type="html">&lt;p&gt;&lt;em&gt;Edit (2026-02-23): Updated plugin installation instructions to use the new repository names.&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;&lt;a href=&quot;https://esbmc.org/&quot;&gt;ESBMC&lt;/a&gt; is an open-source bounded model checker for C, C++, Python, Solidity, and CUDA. Unlike traditional testing, which only explores the paths you think to test, bounded model checking exhaustively explores all execution paths up to a given bound and produces mathematically verified counterexamples when it finds a bug. It&#39;s a powerful technique that catches bugs no amount of unit testing would find.&lt;/p&gt;
&lt;p&gt;&lt;a href=&quot;https://docs.anthropic.com/en/docs/claude-code/overview&quot;&gt;Claude Code&lt;/a&gt; is Anthropic&#39;s agentic coding assistant that lives in your terminal. It supports a plugin system that lets third-party tools integrate deeply into the AI workflow through custom slash commands and skills.&lt;/p&gt;
&lt;p&gt;A new &lt;a href=&quot;https://github.com/esbmc/esbmc/commit/4b79ba4&quot;&gt;Claude Code plugin for ESBMC&lt;/a&gt;, now part of ESBMC upstream, bridges the two. Let me show you what that looks like.&lt;/p&gt;
&lt;h2&gt;Setup&lt;/h2&gt;
&lt;p&gt;You need ESBMC installed and available somewhere on your system (the plugin will find it). Then add the ESBMC marketplace and install the plugin:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;$ claude
&amp;gt; /plugin marketplace add esbmc/agent-marketplace
&amp;gt; /plugin install esbmc-plugin@esbmc-marketplace
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Restart Claude Code and you&#39;re ready to go.&lt;/p&gt;
&lt;h2&gt;How It Works&lt;/h2&gt;
&lt;p&gt;The plugin provides two slash commands -- &lt;code&gt;/verify&lt;/code&gt; for quick checks and &lt;code&gt;/audit&lt;/code&gt; for comprehensive multi-pass security audits -- but you don&#39;t actually need to use them. The plugin registers a &lt;em&gt;skill&lt;/em&gt; that Claude activates automatically when you mention anything related to verification: &amp;quot;check this code for bugs&amp;quot;, &amp;quot;find memory leaks&amp;quot;, &amp;quot;prove correctness&amp;quot;, &amp;quot;detect buffer overflows&amp;quot;, and so on. Just talk to Claude naturally and it will reach for ESBMC when appropriate.&lt;/p&gt;
&lt;p&gt;Under the hood, the skill carries extensive reference material: CLI options, verification strategies, language-specific features, and ESBMC&#39;s intrinsics API. This means Claude knows how to pick the right solver, set appropriate unwind bounds, use k-induction for unbounded proofs, and even add &lt;code&gt;__ESBMC_assume&lt;/code&gt; / &lt;code&gt;__ESBMC_assert&lt;/code&gt; annotations to your source code when needed. You get the full power of a formal verification tool without having to memorize its command-line interface.&lt;/p&gt;
&lt;h2&gt;A Quick Demo&lt;/h2&gt;
&lt;script src=&quot;https://asciinema.org/a/VO4lFSXM8pje8oB8.js&quot; id=&quot;asciicast-VO4lFSXM8pje8oB8&quot; async=&quot;true&quot;&gt;&lt;/script&gt;
&lt;p&gt;Consider this small C program (&lt;a href=&quot;https://gist.github.com/pmatos/a01a86770caf0252bfb490608a4ec2dd&quot;&gt;gist&lt;/a&gt;):&lt;/p&gt;
&lt;pre class=&quot;language-c&quot;&gt;&lt;code class=&quot;language-c&quot;&gt;&lt;span class=&quot;token macro property&quot;&gt;&lt;span class=&quot;token directive-hash&quot;&gt;#&lt;/span&gt;&lt;span class=&quot;token directive keyword&quot;&gt;include&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&amp;lt;stdlib.h&gt;&lt;/span&gt;&lt;/span&gt;

&lt;span class=&quot;token keyword&quot;&gt;int&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;main&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;
    &lt;span class=&quot;token keyword&quot;&gt;int&lt;/span&gt; buf&lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;10&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;

    &lt;span class=&quot;token keyword&quot;&gt;for&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token keyword&quot;&gt;int&lt;/span&gt; i &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;token number&quot;&gt;0&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt; i &lt;span class=&quot;token operator&quot;&gt;&amp;lt;=&lt;/span&gt; &lt;span class=&quot;token number&quot;&gt;10&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt; i&lt;span class=&quot;token operator&quot;&gt;++&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;
        buf&lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;i&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt; &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; i &lt;span class=&quot;token operator&quot;&gt;*&lt;/span&gt; &lt;span class=&quot;token number&quot;&gt;2&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;

    &lt;span class=&quot;token keyword&quot;&gt;int&lt;/span&gt; &lt;span class=&quot;token operator&quot;&gt;*&lt;/span&gt;p &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;malloc&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token keyword&quot;&gt;sizeof&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token keyword&quot;&gt;int&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
    &lt;span class=&quot;token operator&quot;&gt;*&lt;/span&gt;p &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; buf&lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;9&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt; &lt;span class=&quot;token operator&quot;&gt;+&lt;/span&gt; &lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;

    &lt;span class=&quot;token keyword&quot;&gt;int&lt;/span&gt; total &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;token number&quot;&gt;0&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
    &lt;span class=&quot;token keyword&quot;&gt;for&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token keyword&quot;&gt;int&lt;/span&gt; i &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;token number&quot;&gt;0&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt; i &lt;span class=&quot;token operator&quot;&gt;&amp;lt;&lt;/span&gt; &lt;span class=&quot;token number&quot;&gt;10&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt; i&lt;span class=&quot;token operator&quot;&gt;++&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;
        total &lt;span class=&quot;token operator&quot;&gt;+=&lt;/span&gt; buf&lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;i&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;

    &lt;span class=&quot;token function&quot;&gt;free&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;p&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
    &lt;span class=&quot;token keyword&quot;&gt;return&lt;/span&gt; total&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
&lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;There are two bugs here. Can you spot them? They&#39;re the kind that compilers won&#39;t warn about and that can easily slip through code review.&lt;/p&gt;
&lt;p&gt;I asked Claude to &amp;quot;formally verify the code in demo.c using state of the art techniques&amp;quot;. It recognized this as a verification task, activated the ESBMC skill, and ran the model checker with array bounds checking, overflow detection, memory leak analysis, and NULL pointer checks. Here&#39;s what it found.&lt;/p&gt;
&lt;h3&gt;Bug 1: Array out-of-bounds write&lt;/h3&gt;
&lt;p&gt;ESBMC immediately finds that the first loop&#39;s condition &lt;code&gt;i &amp;lt;= 10&lt;/code&gt; causes an off-by-one error. With an array of size 10, valid indices are 0-9, but the loop writes to &lt;code&gt;buf[10]&lt;/code&gt;:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;Violated property:
  file demo.c line 10 function main
  array bounds violated: array `buf&#39; upper bound
  (signed long int)i &amp;lt; 10
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This isn&#39;t a heuristic or a guess -- it&#39;s a formal proof that the property is violated, backed by a concrete counterexample trace showing exactly the state that triggers the bug.&lt;/p&gt;
&lt;h3&gt;Bug 2: NULL pointer dereference&lt;/h3&gt;
&lt;p&gt;The model checker stops at the first violation, so after fixing the off-by-one, a second run catches the &lt;code&gt;malloc&lt;/code&gt; without a NULL check:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;State 23 file demo.c line 12 function main
  p = (signed int *)0

Violated property:
  file demo.c line 13 function main
  dereference failure: NULL pointer
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;ESBMC models &lt;code&gt;malloc&lt;/code&gt; as potentially returning NULL (as the C standard permits), and proves that the dereference at line 13 is reachable in that case. Again, a concrete counterexample, not a warning.&lt;/p&gt;
&lt;h2&gt;Why This Matters&lt;/h2&gt;
&lt;p&gt;Static analyzers flag potential issues. Bounded model checkers &lt;em&gt;prove&lt;/em&gt; them. When ESBMC says &amp;quot;verification failed&amp;quot;, it means there is a real, reachable execution path that triggers the bug, and it shows you exactly what that path is.&lt;/p&gt;
&lt;p&gt;Having this integrated into Claude Code means you can ask it to verify your code as naturally as you&#39;d ask it to write a test. The AI handles constructing the right ESBMC invocation and interpreting the counterexample traces, while the mathematical rigor comes from the model checker itself -- a nice separation of concerns.&lt;/p&gt;
&lt;h2&gt;Formal Methods and the Age of Generated Code&lt;/h2&gt;
&lt;p&gt;I&#39;ve been a fan of formal methods for a long time, but I&#39;ll be honest: I rarely used them in practice. The barrier was always friction. Setting up the tooling, learning the command-line options, figuring out the right unwind bounds, interpreting the output -- it was a lot of overhead for finding corner-case bugs that might not surface for months or years. The payoff was real but the upfront cost was hard to justify in day-to-day work.&lt;/p&gt;
&lt;p&gt;I think that&#39;s about to change, and the reason is LLM-generated code. When an AI writes most of your code, you gain speed but you lose something: the deep familiarity with every line that comes from having typed it yourself. Subtle bugs -- off-by-one errors, missing NULL checks, integer overflows in edge cases -- are exactly the kind of thing that LLMs produce confidently and that humans skim past during review.&lt;/p&gt;
&lt;p&gt;Formal verification is a natural counterpart to code generation. The LLM generates code fast; the model checker proves properties about it exhaustively. Together they give you something neither can offer alone: rapid development with mathematical guarantees that entire classes of bugs are absent. The LLM lowers the barrier to writing code, and the model checker lowers the barrier to trusting it.&lt;/p&gt;
&lt;p&gt;This plugin is a small step in that direction -- making formal verification as easy as asking a question in your terminal. I hope it&#39;s the beginning of a much broader trend.&lt;/p&gt;
&lt;h2&gt;Beyond the Basics&lt;/h2&gt;
&lt;p&gt;The demo above only scratches the surface. The plugin also supports:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;Multi-language verification&lt;/strong&gt; -- C++, Python (with type annotations), Solidity smart contracts, and CUDA kernels.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Comprehensive audits&lt;/strong&gt; -- the &lt;code&gt;/audit&lt;/code&gt; command runs six verification passes: a quick scan, memory safety, integer safety, concurrency checks, deep verification with higher unwind bounds, and a k-induction proof attempt.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Verification annotations&lt;/strong&gt; -- Claude knows ESBMC&#39;s intrinsics API and can add &lt;code&gt;__ESBMC_nondet_*()&lt;/code&gt; for symbolic inputs, &lt;code&gt;__ESBMC_assume()&lt;/code&gt; for preconditions, and &lt;code&gt;__ESBMC_assert()&lt;/code&gt; for custom properties directly to your code.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Strategy selection&lt;/strong&gt; -- from incremental BMC for quick bug hunting to k-induction for unbounded proofs of correctness.&lt;/li&gt;
&lt;/ul&gt;
&lt;h2&gt;Current Status&lt;/h2&gt;
&lt;p&gt;The plugin has landed upstream in the ESBMC repository (&lt;a href=&quot;https://github.com/esbmc/esbmc/commit/4b79ba4&quot;&gt;commit 4b79ba4&lt;/a&gt;). Give it a try if you have ESBMC installed -- I&#39;d love to hear how it works for you.&lt;/p&gt;
</content>
  </entry>
  <entry>
    <title>Claude Code Usage in Waybar</title>
    <link href="https://p.ocmatos.com/blog/claude-code-usage-in-waybar.html"/>
    <updated>2026-02-11T00:00:00Z</updated>
    <id>https://p.ocmatos.com/blog/claude-code-usage-in-waybar.html</id>
    <content xml:lang="en" type="html">&lt;p&gt;If you&#39;re a heavy Claude Code user, you&#39;ve probably been caught off guard by hitting your session or weekly usage cap mid-flow. I wrote a small script that shows your current Claude Code API usage right in &lt;a href=&quot;https://github.com/Alexays/Waybar&quot;&gt;Waybar&lt;/a&gt;, so you can always see where you stand at a glance.&lt;/p&gt;
&lt;p&gt;Here&#39;s what it looks like:&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://p.ocmatos.com/img/2026/02/waybar-claude-usage.png&quot; alt=&quot;Waybar with Claude Code usage module&quot; /&gt;&lt;/p&gt;
&lt;p&gt;The module on the left shows session and weekly utilization percentages. It color-codes based on session utilization — neutral brown under 50%, orange at 50-79%, and red at 80%+. Hovering shows the reset times.&lt;/p&gt;
&lt;h2&gt;The Script&lt;/h2&gt;
&lt;p&gt;The full script is available as a &lt;a href=&quot;https://gist.github.com/pmatos/70f18d396f8e21804f8a437184de1292&quot;&gt;gist&lt;/a&gt;. It reads your Claude Code OAuth credentials, hits the Anthropic usage API, and outputs JSON that Waybar can consume directly.&lt;/p&gt;
&lt;p&gt;It requires &lt;code&gt;jq&lt;/code&gt; and &lt;code&gt;curl&lt;/code&gt; — both likely already on your system.&lt;/p&gt;
&lt;h2&gt;Waybar Configuration&lt;/h2&gt;
&lt;p&gt;Add a custom module to your Waybar config:&lt;/p&gt;
&lt;pre class=&quot;language-json&quot;&gt;&lt;code class=&quot;language-json&quot;&gt;&lt;span class=&quot;token property&quot;&gt;&quot;custom/claude&quot;&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;
    &lt;span class=&quot;token property&quot;&gt;&quot;exec&quot;&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&quot;~/.config/waybar/claude-usage.sh&quot;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt;
    &lt;span class=&quot;token property&quot;&gt;&quot;return-type&quot;&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&quot;json&quot;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt;
    &lt;span class=&quot;token property&quot;&gt;&quot;interval&quot;&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;token number&quot;&gt;120&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt;
    &lt;span class=&quot;token property&quot;&gt;&quot;format&quot;&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&quot;🤖 {}&quot;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt;
    &lt;span class=&quot;token property&quot;&gt;&quot;tooltip&quot;&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;token boolean&quot;&gt;true&lt;/span&gt;
&lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Then include &lt;code&gt;&amp;quot;custom/claude&amp;quot;&lt;/code&gt; in whichever module section you prefer (I have it in &lt;code&gt;modules-right&lt;/code&gt;).&lt;/p&gt;
&lt;p&gt;For styling, add something like this to your &lt;code&gt;style.css&lt;/code&gt;:&lt;/p&gt;
&lt;pre class=&quot;language-css&quot;&gt;&lt;code class=&quot;language-css&quot;&gt;&lt;span class=&quot;token selector&quot;&gt;#custom-claude&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;
    &lt;span class=&quot;token property&quot;&gt;background-color&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; #d4a574&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
    &lt;span class=&quot;token property&quot;&gt;color&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; #000000&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
    &lt;span class=&quot;token property&quot;&gt;font-weight&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; bold&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
&lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;

&lt;span class=&quot;token selector&quot;&gt;#custom-claude.claude-mid&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;
    &lt;span class=&quot;token property&quot;&gt;background-color&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; #f39c12&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
    &lt;span class=&quot;token property&quot;&gt;color&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; #000000&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
&lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;

&lt;span class=&quot;token selector&quot;&gt;#custom-claude.claude-high&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;
    &lt;span class=&quot;token property&quot;&gt;background-color&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; #e74c3c&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
    &lt;span class=&quot;token property&quot;&gt;color&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; #ffffff&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
&lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;

&lt;span class=&quot;token selector&quot;&gt;#custom-claude.claude-error&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;
    &lt;span class=&quot;token property&quot;&gt;background-color&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; #7f8c8d&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
    &lt;span class=&quot;token property&quot;&gt;color&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; #ffffff&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
&lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The CSS classes are set by the script based on the session utilization percentage, so the module changes color as you approach your session limit.&lt;/p&gt;
&lt;p&gt;That&#39;s it — a simple way to keep an eye on your Claude Code budget without leaving your workflow.&lt;/p&gt;
</content>
  </entry>
  <entry>
    <title>Daring to imagine...</title>
    <link href="https://p.ocmatos.com/blog/daring-to-imagine.html"/>
    <updated>2026-01-19T00:00:00Z</updated>
    <id>https://p.ocmatos.com/blog/daring-to-imagine.html</id>
    <content xml:lang="en" type="html">&lt;p&gt;It&#39;s hard to even imagine what things will look like in 2-3 years time.&lt;/p&gt;
&lt;p&gt;I have been submerged in the day-to-day, pushing philosophical questions of the future aside. But when I do look up, I get the shivers. Here&#39;s what I see: A world where software materializes faster than we can read it — code blooming into existence like time-lapse flowers, flawless and strange. Where errors dissolve before we even notice them, perfection arriving faster than doubt.&lt;/p&gt;
&lt;p&gt;A world whose full implications I can barely grasp, like standing at the edge of something vast and inevitable, my stomach dropping, my mind racing to catch up.&lt;/p&gt;
</content>
  </entry>
  <entry>
    <title>Life on GenAI in 2025 (part 1 - Intro and Education)</title>
    <link href="https://p.ocmatos.com/blog/life-on-genai-in-2025-part-1-intro-and-education.html"/>
    <updated>2026-01-09T00:00:00Z</updated>
    <id>https://p.ocmatos.com/blog/life-on-genai-in-2025-part-1-intro-and-education.html</id>
    <content xml:lang="en" type="html">&lt;p&gt;I was already following closely the developments in GenAI, but watching my kids navigate these tools pushed me to keep exploring the landscape and following the trends.
Late 2024 and early 2025, I progressively moved from ChatGPT to Anthropic&#39;s Claude and Google&#39;s Gemini. Claude stood out for its conversational depth and ability to maintain context, while Gemini made it easy to experiment with both simultaneously and was already included with my Google One Subscription.&lt;/p&gt;
&lt;p&gt;Before I share specific use cases, though, it&#39;s important to talk about how I approach reliability when using these tools. LLMs are much better than they once were but still make mistakes. Different LLMs have different strengths and weaknesses—two models might perform equally well on certain benchmarks while struggling in completely different areas. For sensitive subjects like health or where I want to be more certain of something, I try to request science-backed responses that link to trusted websites and query multiple LLMs, so that I can compare the answers. Gemini itself will present its sources in an answer but you can also request a fact-check. When requested to do so Gemini will highlight the parts of the answer it&#39;s able to fact check, which is extremely useful.&lt;/p&gt;
&lt;p&gt;Let me show you what I mean with a quick example—even for straightforward historical facts. So, was D. Afonso Henriques the first king of Portugal? Gemini 3 Pro seems to think it was.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://p.ocmatos.com/img/2026/01/5beb0d68.png&quot; alt=&quot;image&quot; /&gt;&lt;/p&gt;
&lt;p&gt;But you&#39;re not convinced... so now you can actually ask it to double-check the answer.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://p.ocmatos.com/img/2026/01/badc1c74.png&quot; alt=&quot;image&quot; /&gt;&lt;/p&gt;
&lt;p&gt;Within moments, you get the answer highlighted and the source for each stated fact.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://p.ocmatos.com/img/2026/01/a4c066e3.png&quot; alt=&quot;image&quot; /&gt;&lt;/p&gt;
&lt;p&gt;Clearly there&#39;s already a concern to try to establish the facts in answers given by Gemini 3 Pro and this will continue to improve so that hopefully answers will be grounded on facts that you can manually check if you want.&lt;/p&gt;
&lt;p&gt;With these fact-checking tools in hand, I feel more confident using LLMs for learning new skills and helping my family. So with those caveats in mind, let me share how I&#39;ve been using these tools, starting with education.&lt;/p&gt;
&lt;h1&gt;Education&lt;/h1&gt;
&lt;p&gt;If I were studying full-time today, the experience would be dramatically more personal and exciting than when I was in school. Here are a few things that are possible now that simply weren&#39;t back in the day:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Scan a few pages of your textbook, feed them to &lt;a href=&quot;https://notebooklm.google.com/&quot;&gt;NotebookLM&lt;/a&gt;, generate a quiz, a podcast, a video presentation or a slide deck.&lt;/li&gt;
&lt;li&gt;Ask Claude using the Learning style to explain a topic to you.&lt;/li&gt;
&lt;li&gt;Use Suno to generate a song with lyrics which include the verb you want to learn to conjugate in French.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;The possibilities are pretty much endless depending on your learning preference. Kids these days will learn and want to use these technologies to get one step ahead. I have personally used these for my studies in Music Theory. Given that I am self-studying for the ABRSM exams, they have been extremely useful to help go through the curriculum without someone to ask.&lt;/p&gt;
&lt;p&gt;Beyond my own learning, what really excites me about LLMs is their potential to democratize education—helping those without access to a tutor, or complementing traditional tutoring with material presented from different perspectives.&lt;/p&gt;
&lt;p&gt;Let me get specific about how I organize these conversations, because the structure makes a huge difference.&lt;/p&gt;
&lt;p&gt;I use Claude Projects a lot as you&#39;ll see—these are workspaces where you can group related conversations and attach reference materials that persist across all chats. I tend to have projects to group a topic or set of chats. I have a project for &amp;quot;Piano Playing and Music Theory&amp;quot;. There I group all my chats about:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Goals - &amp;quot;are my goals for the next 6 months achievable given my progress in the last 6 months?&amp;quot;&lt;/li&gt;
&lt;li&gt;Doubts - &amp;quot;I really cannot understand compound time signatures. What&#39;s the point? How do I identify them?&amp;quot;&lt;/li&gt;
&lt;li&gt;Progress: &amp;quot;Given my current repertoire, what should I add next that takes me to the next level?&amp;quot;&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;As project knowledge I add all my practice sessions, my repertoire and the knowledge I started with. This means that at any time I can prompt Claude and discuss my practice methodology, my pieces, and repertoire and I will know that Claude is answering me with the full picture rather than with partial context.&lt;/p&gt;
&lt;p&gt;For example, when I asked about compound time signatures, Claude didn&#39;t just give me the textbook definition—it connected it to pieces I was already playing and explained how recognizing these patterns would help me with my current repertoire.&lt;/p&gt;
&lt;p&gt;Another exciting avenue for education is creation of material for studying. A recent example is the upcoming Physics exam of my daughter. With a few photo snaps of her Physics book, my wife &lt;a href=&quot;https://gemini.google.com/share/2e23260cb830&quot;&gt;queried Gemini for a learning plan&lt;/a&gt;. I then used this chat as a basis for Claude Code to create a &lt;a href=&quot;https://drive.google.com/file/d/1O8c8m3uOodzAFZDPMQ6fx1a0x7SEFIHP/view?usp=drive_link&quot;&gt;whole workbook&lt;/a&gt; with &lt;a href=&quot;https://drive.google.com/file/d/14Q8Pwv1sDvqu4aeuyIwLaDdfFT2QAft4/view?usp=drive_link&quot;&gt;solutions&lt;/a&gt; using Typst for my daughter.&lt;/p&gt;
&lt;p&gt;But education is just one piece of the puzzle—I&#39;ve found LLMs equally transformative for health and fitness goals, which I&#39;ll dive into next.&lt;/p&gt;
</content>
  </entry>
  <entry>
    <title>FEX x87 Stack Optimization</title>
    <link href="https://p.ocmatos.com/blog/fex-x87-stack-optimization.html"/>
    <updated>2024-09-25T00:00:00Z</updated>
    <id>https://p.ocmatos.com/blog/fex-x87-stack-optimization.html</id>
    <content xml:lang="en" type="html">&lt;p&gt;In this blog post, I will describe my recent work on improving and optimizing the handling of x87 code in FEX. This work landed on July 22, 2024, through commit &lt;a href=&quot;https://github.com/FEX-Emu/FEX/commit/a1378f94ce8e7843d5a3cd27bc72847973f8e7ec&quot;&gt;https://github.com/FEX-Emu/FEX/commit/a1378f94ce8e7843d5a3cd27bc72847973f8e7ec&lt;/a&gt; (tests and updates to size benchmarks landed separately).&lt;/p&gt;
&lt;p&gt;FEX is an ARM64 emulator of Intel assembly and, as such, needs to be able to handle things as old as x87, which was the way we did floating point pre-SSE.&lt;/p&gt;
&lt;p&gt;Consider a function to calculate the well-known Pythagorean Identity:&lt;/p&gt;
&lt;pre class=&quot;language-c&quot;&gt;&lt;code class=&quot;language-c&quot;&gt;&lt;span class=&quot;token keyword&quot;&gt;float&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;PytIdent&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token keyword&quot;&gt;float&lt;/span&gt; x&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;
  &lt;span class=&quot;token keyword&quot;&gt;return&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;sinf&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;x&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;*&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;sinf&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;x&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;token operator&quot;&gt;+&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;cosf&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;x&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;*&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;cosf&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;x&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
&lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Here is how this can be translated into assembly:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;  sub rsp, 4
  movss [rsp], xmm0
  fld [rsp]
  fsincos
  fmul st0, st0
  fxch
  fmul st0, st0
  faddp
  fstp [rsp]
  movss xmm0, [rsp]
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;These instructions will take &lt;code&gt;x&lt;/code&gt; from register &lt;code&gt;xmm0&lt;/code&gt; and return the result of the identity (the number 1, as we expect) to the same register.&lt;/p&gt;
&lt;h1&gt;A brief visit to the x87 FPU&lt;/h1&gt;
&lt;p&gt;The x87 FPU consists of 8 registers (&lt;code&gt;R0&lt;/code&gt; to &lt;code&gt;R7&lt;/code&gt;), which are aliased to the MMX registers - &lt;code&gt;mm0&lt;/code&gt; to &lt;code&gt;mm7&lt;/code&gt;. The FPU uses them as a stack and tracks the top of the stack through the &lt;code&gt;TOS&lt;/code&gt; (Top of Stack) or &lt;code&gt;TOP&lt;/code&gt; (my preferred way to refer to it), which is kept in the control word. To understand how it all fits together, I will describe the three most important components of this setup for the purpose of this post:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;FPU Stack-based operation&lt;/li&gt;
&lt;li&gt;Status register&lt;/li&gt;
&lt;li&gt;Tag register&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;From the &lt;a href=&quot;https://cdrdv2.intel.com/v1/dl/getContent/671200&quot;&gt;Intel spec&lt;/a&gt;, these components look like this (image labels are from the Intel in case you need to refer to it):&lt;/p&gt;
&lt;div style=&quot;text-align:center;&quot;&gt;
	&lt;img alt=&quot;Patch Details&quot; class=&quot;post-photo&quot; src=&quot;https://p.ocmatos.com/img/2024-09-23-fex-x87-stack-optimization/Screenshot%20From%202024-09-23%2010-54-21.png&quot; /&gt;
&lt;/div&gt;
&lt;p&gt;The 8 registers are labelled as &lt;code&gt;ST0&lt;/code&gt;-&lt;code&gt;ST7&lt;/code&gt; relative to the position of &lt;code&gt;TOP&lt;/code&gt;. In the case above &lt;code&gt;TOP&lt;/code&gt; is 3; therefore, the 3rd register is &lt;code&gt;ST0&lt;/code&gt;. Since the stack grows down, &lt;code&gt;ST1&lt;/code&gt; is the 4th register and so on until it wraps around. When the FPU is initialized &lt;code&gt;TOP&lt;/code&gt; is set to 0. Once a value is loaded, the &lt;code&gt;TOP&lt;/code&gt; is decreased, its value becomes 7 and the first value added to the stack is in the 7th register. Because the MMX registers are aliased to the bottom 64 bits of the stack registers, we can also refer to them as &lt;code&gt;MM0&lt;/code&gt; to &lt;code&gt;MM7&lt;/code&gt;. This is not strictly correct because the MMX registers are 64 bits and the FPU registers are 80 bits but if we squint, they are basically the same.&lt;/p&gt;
&lt;p&gt;The spec provides an example of how the FPU stack works. I copy the image here verbatim for your understanding. Further information can be found in the spec itself.&lt;/p&gt;
&lt;div style=&quot;text-align:center;&quot;&gt;
	&lt;img alt=&quot;Patch Details&quot; class=&quot;post-photo&quot; src=&quot;https://p.ocmatos.com/img/2024-09-23-fex-x87-stack-optimization/Screenshot%20From%202024-09-23%2010-54-43.png&quot; /&gt;
&lt;/div&gt;
&lt;p&gt;The &lt;code&gt;TOP&lt;/code&gt; value is stored along with other flags in the FPU Status Word, which we won&#39;t discuss in detail here.&lt;/p&gt;
&lt;div style=&quot;text-align:center;&quot;&gt;
	&lt;img alt=&quot;Patch Details&quot; class=&quot;post-photo&quot; src=&quot;https://p.ocmatos.com/img/2024-09-23-fex-x87-stack-optimization/Screenshot%20From%202024-09-23%2010-54-53.png&quot; /&gt;
&lt;/div&gt;
&lt;p&gt;And the tag word marks which registers are valid or empty. The Intel spec also allows marking the registers as zero or special. At the moment, tags in FEX are binary, marking the register as valid or invalid. Instead of two bits per register, we only use one bit of information per register.&lt;/p&gt;
&lt;div style=&quot;text-align:center;&quot;&gt;
	&lt;img alt=&quot;Patch Details&quot; class=&quot;post-photo&quot; src=&quot;https://p.ocmatos.com/img/2024-09-23-fex-x87-stack-optimization/Screenshot%20From%202024-09-23%2010-55-21.png&quot; /&gt;
&lt;/div&gt;
&lt;h1&gt;Current issues&lt;/h1&gt;
&lt;p&gt;The way things worked in FEX before this patch landed is that stack based operations would essentially always lower to perform three operations:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Load stack operation argument register values from special memory location;&lt;/li&gt;
&lt;li&gt;Call soft-float library to perform the operation;&lt;/li&gt;
&lt;li&gt;Store result in special memory location;&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;For example, let&#39;s consider the instruction &lt;code&gt;FADDP st4, st0&lt;/code&gt;. This instruction performs &lt;code&gt;st0 &amp;lt;- f80add(st4, st0) &amp;amp;&amp;amp; pop()&lt;/code&gt;. In &lt;code&gt;x87.cpp&lt;/code&gt;, the lowering of this instruction involved the following steps:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Load current value of top from memory;&lt;/li&gt;
&lt;li&gt;With the current value of top, we can load the values of the registers in the instruction, in this case st4 and st0 since they are at a memory offset from top;&lt;/li&gt;
&lt;li&gt;Then we call the soft-float library function to perform 80-bit adds;&lt;/li&gt;
&lt;li&gt;Finally we store the result into st4 (which is a location in memory);&lt;/li&gt;
&lt;li&gt;And pop the stack, which means marking the current top value invalid and incrementing top;&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;In code with a lot of x87 usage, these operations become cumbersome and data movement makes the whole code extremely slow. Therefore, we redesigned the x87 system to cope better with stack operations. FEX support a reduced precision mode that uses 64bit floating points rather than 80 bits, avoiding the soft-float library calls and this work applies equally to the code in reduced precision mode.&lt;/p&gt;
&lt;h1&gt;The new pass&lt;/h1&gt;
&lt;p&gt;The main observation to understand the operation of the new pass is that when compiling a block of instructions, where multiple instructions are x87 instructions, before code generation we have an idea of what the stack will look like, and if we have complete stack information we can generate much better code than the one generated on a per-instruction basis.&lt;/p&gt;
&lt;p&gt;Instead of lowering each x87 instruction in x87.cpp to its components as discussed earlier, we simply generate stack operation IR instructions (all of these added new since there were no stack operations in the IR). Then an optimization pass goes through the IR on a per block basis and optimizes the operations and lowers them.&lt;/p&gt;
&lt;p&gt;Here’s a step-by-step explanation of how this scheme works for the code above. When the above code gets to the OpcodeDispatcher, there’ll be one function a reasonable amount of opcodes, but generally one per functionality, meaning that although there are several opcodes for different versions of fadd, there is a single function in OpcodeDispatcher for x87 (in x87.cpp) implementing it.&lt;/p&gt;
&lt;p&gt;Each of these functions will just transform the incoming instruction into an IR node that deals with the stack. These stack IR operations have Stack in its name. See &lt;a href=&quot;https://github.com/FEX-Emu/FEX/blob/FEX-2409/FEXCore/Source/Interface/IR/IR.json&quot;&gt;IR.json&lt;/a&gt; for a list of these.&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;x86 Asm&lt;/th&gt;
&lt;th&gt;IR nodes&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;fld qword [rsp]&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;LoadMem&lt;/code&gt; + &lt;code&gt;PushStack&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;fsincos&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;F80SinCosStack&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;fmul st0, st0&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;F80MulStack&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;fxch&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;F80StackXchange&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;fmul st0, st0&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;F80MulStack&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;faddp&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;F80AddStack&lt;/code&gt; + &lt;code&gt;PopStackDestroy&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;fstp qword [rsp]&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;StoreStackMemory&lt;/code&gt; + &lt;code&gt;PopStackDestroy&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;The IR will then look like:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;	%9 i64 = LoadRegister #0x4, GPR
	%10 i128 = LoadMem FPR, %9 i64, %Invalid, #0x10, SXTX, #0x1
	(%11 i0) PushStack %10 i128, %10 i128, #0x10, #0x1
	(%12 i0) F80SINCOSStack
	%13 i64 = Constant #0x0
	(%14 i8) StoreContext GPR, %13 i64, #0x3fa
	(%15 i0) F80MulStack #0x0, #0x0
	(%16 i0) F80StackXchange #0x1
	%17 i64 = Constant #0x0
	(%18 i8) StoreContext GPR, %17 i64, #0x3f9
	(%19 i0) F80MulStack #0x0, #0x0
	(%20 i0) F80AddStack #0x1, #0x0
	(%21 i0) PopStackDestroy
	%23 i64 = LoadRegister #0x4, GPR
	(%24 i0) StoreStackMemory %23 i64, i128, #0x1, #0x8
	(%25 i0) PopStackDestroy
&lt;/code&gt;&lt;/pre&gt;
&lt;h2&gt;The Good Case&lt;/h2&gt;
&lt;p&gt;Remember that before this pass, we would have never generated this IR as is. There were no stack operations, therefore each of these &lt;code&gt;F80MulStack&lt;/code&gt; operations, would generate loads from the MMX registers that are mapped into memory for each of the operands, call to &lt;code&gt;F80Mul&lt;/code&gt; softfloat library function and then a write to the memory mapped MMX register to write the result.&lt;/p&gt;
&lt;p&gt;However, once we get to the optimization pass a see a straight line block line this, we can step through the IR nodes and create a virtual stack with pointers to the IR nodes. I have sketched the stacks for the first few loops in the pass.&lt;/p&gt;
&lt;p&gt;The pass will only process &lt;em&gt;x87&lt;/em&gt; instructions, these are the one whose IR nodes are marked with &lt;code&gt;&amp;quot;X87: true&lt;/code&gt; in &lt;code&gt;IR.json&lt;/code&gt;. Therefore the first two instructions above:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;	%9 i64 = LoadRegister #0x4, GPR
	%10 i128 = LoadMem FPR, %9 i64, %Invalid, #0x10, SXTX, #0x1
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;are ignored. But &lt;code&gt;PushStack&lt;/code&gt; is not. &lt;code&gt;PushStack&lt;/code&gt; will literally push the node into the virtual stack.&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;	(%11 i0) PushStack %10 i128, %10 i128, #0x10, #0x1
&lt;/code&gt;&lt;/pre&gt;
&lt;div style=&quot;display: flex; justify-content: flex-end;&quot;&gt;
	&lt;img alt=&quot;Patch Details&quot; class=&quot;post-photo&quot; src=&quot;https://p.ocmatos.com/img/2024-09-23-fex-x87-stack-optimization/X87-example1.svg&quot; width=&quot;35%&quot; /&gt;
&lt;/div&gt;
&lt;p&gt;Internally before we push the node into our internal stack, we update the necessary values like the top of the stack. It starts at zero and it&#39;s decremented to seven, so the value &lt;code&gt;%10 i128&lt;/code&gt; is inserted in what we&#39;ll call &lt;code&gt;R7&lt;/code&gt;, now &lt;code&gt;ST0&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;Next up is the computation of &lt;code&gt;SIN&lt;/code&gt; and &lt;code&gt;COS&lt;/code&gt;. This will result in &lt;code&gt;SIN ST0&lt;/code&gt; replacing &lt;code&gt;ST0&lt;/code&gt; and then pushing &lt;code&gt;COS ST0&lt;/code&gt;, where the value in &lt;code&gt;ST0&lt;/code&gt; here is &lt;code&gt;%10&lt;/code&gt;.&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;	(%12 i0) F80SINCOSStack
&lt;/code&gt;&lt;/pre&gt;
&lt;div style=&quot;display: flex; justify-content: flex-end;&quot;&gt;
	&lt;img alt=&quot;Patch Details&quot; class=&quot;post-photo&quot; src=&quot;https://p.ocmatos.com/img/2024-09-23-fex-x87-stack-optimization/X87-example2.svg&quot; width=&quot;35%&quot; /&gt;
&lt;/div&gt;
&lt;p&gt;Then we have two instructions that we just don&#39;t deal with in the pass, these set the &lt;code&gt;C2&lt;/code&gt; flag to zero - as required by &lt;code&gt;FSINCOS&lt;/code&gt;.&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;	%13 i64 = Constant #0x0
	(%14 i8) StoreContext GPR, %13 i64, #0x3fa
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Followed by a multiplication of the stack element to square them. So we square &lt;code&gt;ST0&lt;/code&gt;.&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;	(%15 i0) F80MulStack #0x0, #0x0
&lt;/code&gt;&lt;/pre&gt;
&lt;div style=&quot;display: flex; justify-content: flex-end;&quot;&gt;
	&lt;img alt=&quot;Patch Details&quot; class=&quot;post-photo&quot; src=&quot;https://p.ocmatos.com/img/2024-09-23-fex-x87-stack-optimization/X87-example3.svg&quot; width=&quot;35%&quot; /&gt;
&lt;/div&gt;
&lt;p&gt;The next instruction swaps &lt;code&gt;ST0&lt;/code&gt; with &lt;code&gt;ST1&lt;/code&gt; so that we can square the sine value.&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;	(%16 i0) F80StackXchange #0x1
&lt;/code&gt;&lt;/pre&gt;
&lt;div style=&quot;display: flex; justify-content: flex-end;&quot;&gt;
	&lt;img alt=&quot;Patch Details&quot; class=&quot;post-photo&quot; src=&quot;https://p.ocmatos.com/img/2024-09-23-fex-x87-stack-optimization/X87-example4.svg&quot; width=&quot;35%&quot; /&gt;
&lt;/div&gt;
&lt;p&gt;Again, similarly two instructions that are ignored by the pass, which set flag &lt;code&gt;C1&lt;/code&gt; to zero - as required by &lt;code&gt;FXCH&lt;/code&gt;.&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;	%17 i64 = Constant #0x0
	(%18 i8) StoreContext GPR, %17 i64, #0x3f9
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;And again we square the value at the top of the stack.&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;	(%19 i0) F80MulStack #0x0, #0x0
&lt;/code&gt;&lt;/pre&gt;
&lt;div style=&quot;display: flex; justify-content: flex-end;&quot;&gt;
	&lt;img alt=&quot;Patch Details&quot; class=&quot;post-photo&quot; src=&quot;https://p.ocmatos.com/img/2024-09-23-fex-x87-stack-optimization/X87-example5.svg&quot; width=&quot;35%&quot; /&gt;
&lt;/div&gt;
&lt;p&gt;We are almost there - to finish off the computation we need to add these two values together.&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;	(%20 i0) F80AddStack #0x1, #0x0
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;In this case we add &lt;code&gt;ST1&lt;/code&gt; to &lt;code&gt;ST0&lt;/code&gt; and store the result in &lt;code&gt;ST1&lt;/code&gt;.&lt;/p&gt;
&lt;div style=&quot;display: flex; justify-content: flex-end;&quot;&gt;
	&lt;img alt=&quot;Patch Details&quot; class=&quot;post-photo&quot; src=&quot;https://p.ocmatos.com/img/2024-09-23-fex-x87-stack-optimization/X87-example6.svg&quot; width=&quot;35%&quot; /&gt;
&lt;/div&gt;
&lt;p&gt;This is followed by a pop of the stack.&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;	(%21 i0) PopStackDestroy
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;As expected this removed the &lt;code&gt;F80Mul&lt;/code&gt; from the top of the stack, leaving us with the result of the computation at the top.&lt;/p&gt;
&lt;div style=&quot;display: flex; justify-content: flex-end;&quot;&gt;
	&lt;img alt=&quot;Patch Details&quot; class=&quot;post-photo&quot; src=&quot;https://p.ocmatos.com/img/2024-09-23-fex-x87-stack-optimization/X87-example7.svg&quot; width=&quot;35%&quot; /&gt;
&lt;/div&gt;
&lt;p&gt;OK - there are three more instructions.&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;	%23 i64 = LoadRegister #0x4, GPR
	(%24 i0) StoreStackMemory %23 i64, i128, #0x1, #0x8
	(%25 i0) PopStackDestroy
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The first two store the top of the stack to memory, which is the result of our computation and which maths assures us it&#39;s 1. And then we pop the stack just to clean it up, although this is not strictly necessary since we are finished  anyway.&lt;/p&gt;
&lt;p&gt;Note that we have finished analysing the block, and played it through in our virtual stack. Now we can generate the instructions that are necessary to calculate the stack values, &lt;strong&gt;without&lt;/strong&gt; always load/stor&#39;ing them back to memory. In addition to generating these instructions we also generate instructions to update the tag word, update the top of stack value and save whatever remaining values in the stack at the end of the block into memory - into their respective memory mapped MMX registers. This is the good case where all the values necessary for the computation were found in the block. However, this is not necessarily always the case.&lt;/p&gt;
&lt;h2&gt;The Bad Case&lt;/h2&gt;
&lt;p&gt;Let&#39;s say that FEX for some reason breaks the beautiful block we had before into two:&lt;/p&gt;
&lt;p&gt;Block 1:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;	%9 i64 = LoadRegister #0x4, GPR
	%10 i128 = LoadMem FPR, %9 i64, %Invalid, #0x10, SXTX, #0x1
	(%11 i0) PushStack %10 i128, %10 i128, #0x10, #0x1
	(%12 i0) F80SINCOSStack
	%13 i64 = Constant #0x0
	(%14 i8) StoreContext GPR, %13 i64, #0x3fa
	(%15 i0) F80MulStack #0x0, #0x0
	(%16 i0) F80StackXchange #0x1
	%17 i64 = Constant #0x0
	(%18 i8) StoreContext GPR, %17 i64, #0x3f9
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Block 2:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;	(%19 i0) F80MulStack #0x0, #0x0
	(%20 i0) F80AddStack #0x1, #0x0
	(%21 i0) PopStackDestroy
	%23 i64 = LoadRegister #0x4, GPR
	(%24 i0) StoreStackMemory %23 i64, i128, #0x1, #0x8
	(%25 i0) PopStackDestroy
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;When we generate code for the first block, there are absolutely no blockers and we run through it exactly as we did before, except that after &lt;code&gt;StoreContext&lt;/code&gt;, we reach the end of the block and our virtual stack is in this state:&lt;/p&gt;
&lt;div style=&quot;display: flex; justify-content: flex-end;&quot;&gt;
	&lt;img alt=&quot;Patch Details&quot; class=&quot;post-photo&quot; src=&quot;https://p.ocmatos.com/img/2024-09-23-fex-x87-stack-optimization/X87-example4.svg&quot; width=&quot;35%&quot; /&gt;
&lt;/div&gt;
&lt;p&gt;At this point, we&#39;ll do what we said we always do before at the end of the block. We save the values in the virtual stack to the respective MMX registers in memory mapped locations: &lt;code&gt;%36&lt;/code&gt; is saved to &lt;code&gt;MM7&lt;/code&gt; and &lt;code&gt;%34&lt;/code&gt; is saved to &lt;code&gt;MM6&lt;/code&gt;. The &lt;code&gt;TOP&lt;/code&gt; is recorded to be 6 and the tags for &lt;code&gt;MM7&lt;/code&gt; and &lt;code&gt;MM6&lt;/code&gt; are marked to be valid. Then we exit the block. And start analyzing the following block.&lt;/p&gt;
&lt;p&gt;When we start the pass for the following block, our virtual stack is empty. We have no knowledge of virtual stacks for previous JITTed blocks, and we see this instruction:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;	(%19 i0) F80MulStack #0x0, #0x0
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;We cannot play through this instruction in our virtual stack because it multiplies the value in &lt;code&gt;ST0&lt;/code&gt; with itself and our virtual stack &lt;strong&gt;does not&lt;/strong&gt; have anything in &lt;code&gt;ST0&lt;/code&gt;. It&#39;s empty after all.&lt;/p&gt;
&lt;p&gt;In this case, we switch to the slow path that does exactly what we used to do before this work. We load the value of &lt;code&gt;ST0&lt;/code&gt; from memory by finding the current value for &lt;code&gt;TOP&lt;/code&gt; and loading the respective register value, issue the &lt;code&gt;F80Mul&lt;/code&gt; on this value and storing it back to the &lt;code&gt;MMX&lt;/code&gt; register. This is then done for the remainder of the block.&lt;/p&gt;
&lt;h2&gt;The Ugly Case&lt;/h2&gt;
&lt;p&gt;The ugly case is when we actually force the slow path, not because we don&#39;t have all the values in the virtual stack but out of necessity.&lt;/p&gt;
&lt;p&gt;There are a few instructions that trigger a &lt;code&gt;_StackForceSlow()&lt;/code&gt; and forces us down the slow path. These are: &lt;code&gt;FLDENV&lt;/code&gt;, &lt;code&gt;FRSTOR&lt;/code&gt;, &lt;code&gt;FLDCW&lt;/code&gt;. All of these load some part or all of the FPU environment from memory and it triggers a switch to the slow path so that values are loaded properly and we can then use those new values in the block. It is not impossible to think we could at least in some cases, load those values from memory, try to rebuild our virtual stack for the current block and continue without switching to the slow path but that hasn&#39;t been yet done.&lt;/p&gt;
&lt;p&gt;There are a few instructions other instructions that trigger a &lt;code&gt;_SyncStackToSlow()&lt;/code&gt;, which doesn&#39;t force us down the slow path but instead just updates the in-memory data from the virtual stack state. These are: &lt;code&gt;FNSTENV&lt;/code&gt;, &lt;code&gt;FNSAVE&lt;/code&gt;, and &lt;code&gt;FNSTSW&lt;/code&gt;. All of these store some part or all of the FPU environment into memory and it ensures that the values in-memory are up-to-date so that the store doesn&#39;t use obsolete values.&lt;/p&gt;
&lt;h1&gt;Results&lt;/h1&gt;
&lt;p&gt;In addition to the huge savings in data movement from the loads and stores of register values, &lt;code&gt;TOP&lt;/code&gt; and tags for each stack instruction, we also optimize memory copies through the stack. So if there&#39;s a bunch of value loaded to the stack, which are then stored to memory, we can just memcopy these values without going through the stack.&lt;/p&gt;
&lt;p&gt;In our code size validation benchmarks from various Steam games, we saw significant reductions in code size. These size code reductions saw a similarly high jump in FPS numbers. For example:&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Game&lt;/th&gt;
&lt;th style=&quot;text-align:center&quot;&gt;Before&lt;/th&gt;
&lt;th&gt;After&lt;/th&gt;
&lt;th&gt;Reduction&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Half-Life (Block 1)&lt;/td&gt;
&lt;td style=&quot;text-align:center&quot;&gt;2034&lt;/td&gt;
&lt;td&gt;1368&lt;/td&gt;
&lt;td&gt;32.7%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Half-Life (Block 2)&lt;/td&gt;
&lt;td style=&quot;text-align:center&quot;&gt;838&lt;/td&gt;
&lt;td&gt;551&lt;/td&gt;
&lt;td&gt;34.2%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Oblivion (Block 1)&lt;/td&gt;
&lt;td style=&quot;text-align:center&quot;&gt;34065&lt;/td&gt;
&lt;td&gt;25782&lt;/td&gt;
&lt;td&gt;24.3%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Oblivion (Block 2)&lt;/td&gt;
&lt;td style=&quot;text-align:center&quot;&gt;22089&lt;/td&gt;
&lt;td&gt;16635&lt;/td&gt;
&lt;td&gt;24.6%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Psychonauts (Block 1)&lt;/td&gt;
&lt;td style=&quot;text-align:center&quot;&gt;20545&lt;/td&gt;
&lt;td&gt;16357&lt;/td&gt;
&lt;td&gt;20.3%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Psychonauts Hot Loop (Matrix Swizzle)&lt;/td&gt;
&lt;td style=&quot;text-align:center&quot;&gt;2340&lt;/td&gt;
&lt;td&gt;165&lt;/td&gt;
&lt;td&gt;92.9%&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;h1&gt;Future work&lt;/h1&gt;
&lt;p&gt;The bulk of this work is now complete, however there are a few details that still need fixing. There&#39;s still the mystery of why &lt;a href=&quot;https://github.com/FEX-Emu/FEX/issues/3685&quot;&gt;Touhou: Luna Nights works with reduced precision&lt;/a&gt; but not normal precision (if anything you would expect it to be the other way around). Then there&#39;s also some work to be done on the interface between the FPU state and the MMX state as described in &amp;quot;&lt;a href=&quot;https://github.com/FEX-Emu/FEX/issues/3850&quot;&gt;MMX/x87 interaction is subtly broken&lt;/a&gt;&amp;quot;.&lt;/p&gt;
&lt;p&gt;This is something I am already focusing on and will continue working on this until complete.&lt;/p&gt;
</content>
  </entry>
  <entry>
    <title>WebAssembly Reference Types in Clang/LLVM</title>
    <link href="https://p.ocmatos.com/blog/webassembly-reference-types-in-clang-llvm.html"/>
    <updated>2023-06-13T00:00:00Z</updated>
    <id>https://p.ocmatos.com/blog/webassembly-reference-types-in-clang-llvm.html</id>
    <content xml:lang="en" type="html">&lt;h1&gt;WebAssembly Reference Types in LLVM&lt;/h1&gt;
&lt;h2&gt;Introduction&lt;/h2&gt;
&lt;p&gt;This post is an overview of the implementation of the &lt;a href=&quot;https://github.com/WebAssembly/reference-types&quot;&gt;WebAssembly Reference Types proposal&lt;/a&gt; in Clang/LLVM. It is a follow-up to two presentations:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href=&quot;https://www.youtube.com/watch?v=UxnUht3uWzc&quot;&gt;&amp;quot;A walk on the weird side: Opaque WebAssembly values and LLVM&amp;quot;&lt;/a&gt; (2021 LLVM Dev Mtg) by Andy Wingo, and&lt;/li&gt;
&lt;li&gt;&lt;a href=&quot;https://www.youtube.com/watch?v=lwPg_Vjs7p4&quot;&gt;&amp;quot;Clang, Clang: Who&#39;s there? WebAssembly!&amp;quot;&lt;/a&gt; (2022 LLVM Dev Mtg) by myself.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;This is the culmination of a couple of years of work started by myself, Andy Wingo and Alex Bradbury at Igalia. It was sponsored in its entirety by Bloomberg.&lt;/p&gt;
&lt;p&gt;The implementation followed a bottom up approach. We worked on the linker first, followed by the WebAssembly backend, MC layer and LLVM IR. Support for reference types has been available in LLVM IR for quite some time, but it was not exposed in Clang. Extending Clang to support values of reference type and tables was the last piece of the puzzle.&lt;/p&gt;
&lt;p&gt;For context for the remainder of the post, I recommend watching the two presentations linked above. In summary, the work consisted of implementing support for the following reference types (read WebAssembly reference types unless stated, not C++ reference types) in LLVM:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;code&gt;externref&lt;/code&gt;, which is a garbage collected reference type, and&lt;/li&gt;
&lt;li&gt;&lt;code&gt;funcref&lt;/code&gt;, which is a non-nullable reference to a function.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;In addition, we implemented support for WebAssembly tables, which are arrays of references. Since values of reference types cannot be stored to linear memory, tables are storage containers for these.&lt;/p&gt;
&lt;h2&gt;Linker and MC Layer&lt;/h2&gt;
&lt;h2&gt;LLVM IR&lt;/h2&gt;
&lt;h2&gt;Clang&lt;/h2&gt;
&lt;h2&gt;Conclusions and Future&lt;/h2&gt;
&lt;p&gt;There was a lot of work done in the last few years to reach this point. Not just by Igalia but kind and patient reviewers and the WebAssembly Tools team at Google who offered a lot of advice on the matter. However, there is still work to be done to completely implement the WebAssembly Reference Types proposal. We don&#39;t support initialization of tables yet and the C++ language extension to deal with these types is not very developer friendly since it&#39;s based on builtins. A more ergonomic approach to solve table access, for example, would be to allow &lt;code&gt;table.get&lt;/code&gt; and &lt;code&gt;table.set&lt;/code&gt; to be deal as array access. For the moment the project is on hold, due to lack of funding. If you are interested in sponsoring this work, please get in touch with me.&lt;/p&gt;
</content>
  </entry>
  <entry>
    <title>From Middle-C to Concert</title>
    <link href="https://p.ocmatos.com/blog/from-middle-c-to-concert.html"/>
    <updated>2022-10-24T00:00:00Z</updated>
    <id>https://p.ocmatos.com/blog/from-middle-c-to-concert.html</id>
    <content xml:lang="en" type="html">&lt;p&gt;Last Saturday, on the 24th of September of 2022 I played piano publicly for the first time in a local concert organized for students of my piano teacher - &lt;a href=&quot;http://www.takako-ono.com/&quot;&gt;Takako Ono&lt;/a&gt;. My performance was far from perfect, but I am proud of it. This is a brief overview of my lessons and practice over the last 8 months.&lt;/p&gt;
&lt;p&gt;Earlier this year, on January 18, I started playing the piano with a 30-minute lesson. What I mean by &amp;quot;started playing&amp;quot;, is learning the location of the middle C on the piano. Or that when reading sheet music all of the following notes are the same - indeed, these are all middle C.&lt;/p&gt;
&lt;svg xmlns=&quot;http://www.w3.org/2000/svg&quot; xmlns:xlink=&quot;http://www.w3.org/1999/xlink&quot; version=&quot;1.2&quot; width=&quot;85.75mm&quot; height=&quot;26.16mm&quot; viewBox=&quot;-0.3115 -0.0210 48.7956 14.8867&quot;&gt;
&lt;style type=&quot;text/css&quot;&gt;
&lt;![CDATA[
tspan { white-space: pre; }
]]&gt;
&lt;/style&gt;
&lt;g transform=&quot;translate(47.8841, -0.0000)&quot;&gt;
&lt;rect x=&quot;0.0000&quot; y=&quot;5.8290&quot; width=&quot;0.6000&quot; height=&quot;4.9500&quot; ry=&quot;0.0000&quot; fill=&quot;currentColor&quot;&gt;&lt;/rect&gt;
&lt;/g&gt;
&lt;g transform=&quot;translate(47.3941, -0.0000)&quot;&gt;
&lt;rect x=&quot;0.0000&quot; y=&quot;5.8290&quot; width=&quot;0.1900&quot; height=&quot;4.9500&quot; ry=&quot;0.0000&quot; fill=&quot;currentColor&quot;&gt;&lt;/rect&gt;
&lt;/g&gt;
&lt;g transform=&quot;translate(28.4453, 3.7790)&quot;&gt;
&lt;rect x=&quot;0.0000&quot; y=&quot;-2.0000&quot; width=&quot;0.1900&quot; height=&quot;4.0500&quot; ry=&quot;0.0000&quot; fill=&quot;currentColor&quot;&gt;&lt;/rect&gt;
&lt;/g&gt;
&lt;g transform=&quot;translate(40.7449, 12.7790)&quot;&gt;
&lt;rect x=&quot;0.0000&quot; y=&quot;-2.0500&quot; width=&quot;0.1900&quot; height=&quot;4.0500&quot; ry=&quot;0.0000&quot; fill=&quot;currentColor&quot;&gt;&lt;/rect&gt;
&lt;/g&gt;
&lt;g transform=&quot;translate(47.8841, 3.7790)&quot;&gt;
&lt;rect x=&quot;0.0000&quot; y=&quot;-2.0000&quot; width=&quot;0.6000&quot; height=&quot;4.0500&quot; ry=&quot;0.0000&quot; fill=&quot;currentColor&quot;&gt;&lt;/rect&gt;
&lt;/g&gt;
&lt;g transform=&quot;translate(47.3941, 3.7790)&quot;&gt;
&lt;rect x=&quot;0.0000&quot; y=&quot;-2.0000&quot; width=&quot;0.1900&quot; height=&quot;4.0500&quot; ry=&quot;0.0000&quot; fill=&quot;currentColor&quot;&gt;&lt;/rect&gt;
&lt;/g&gt;
&lt;g transform=&quot;translate(47.8841, 12.7790)&quot;&gt;
&lt;rect x=&quot;0.0000&quot; y=&quot;-2.0500&quot; width=&quot;0.6000&quot; height=&quot;4.0500&quot; ry=&quot;0.0000&quot; fill=&quot;currentColor&quot;&gt;&lt;/rect&gt;
&lt;/g&gt;
&lt;g transform=&quot;translate(47.3941, 12.7790)&quot;&gt;
&lt;rect x=&quot;0.0000&quot; y=&quot;-2.0500&quot; width=&quot;0.1900&quot; height=&quot;4.0500&quot; ry=&quot;0.0000&quot; fill=&quot;currentColor&quot;&gt;&lt;/rect&gt;
&lt;/g&gt;
&lt;g transform=&quot;translate(40.7449, -0.0000)&quot;&gt;
&lt;rect x=&quot;0.0000&quot; y=&quot;5.8290&quot; width=&quot;0.1900&quot; height=&quot;4.9500&quot; ry=&quot;0.0000&quot; fill=&quot;currentColor&quot;&gt;&lt;/rect&gt;
&lt;/g&gt;
&lt;g transform=&quot;translate(28.4453, 12.7790)&quot;&gt;
&lt;rect x=&quot;0.0000&quot; y=&quot;-2.0500&quot; width=&quot;0.1900&quot; height=&quot;4.0500&quot; ry=&quot;0.0000&quot; fill=&quot;currentColor&quot;&gt;&lt;/rect&gt;
&lt;/g&gt;
&lt;g transform=&quot;translate(40.7449, 3.7790)&quot;&gt;
&lt;rect x=&quot;0.0000&quot; y=&quot;-2.0000&quot; width=&quot;0.1900&quot; height=&quot;4.0500&quot; ry=&quot;0.0000&quot; fill=&quot;currentColor&quot;&gt;&lt;/rect&gt;
&lt;/g&gt;
&lt;g transform=&quot;translate(28.4453, -0.0000)&quot;&gt;
&lt;rect x=&quot;0.0000&quot; y=&quot;5.8290&quot; width=&quot;0.1900&quot; height=&quot;4.9500&quot; ry=&quot;0.0000&quot; fill=&quot;currentColor&quot;&gt;&lt;/rect&gt;
&lt;/g&gt;
&lt;g transform=&quot;translate(8.5358, 14.7790)&quot;&gt;
&lt;line stroke-linejoin=&quot;round&quot; stroke-linecap=&quot;round&quot; stroke-width=&quot;0.1000&quot; stroke=&quot;currentColor&quot; x1=&quot;0.0500&quot; y1=&quot;0&quot; x2=&quot;39.8983&quot; y2=&quot;0&quot;&gt;&lt;/line&gt;
&lt;/g&gt;
&lt;g transform=&quot;translate(8.5358, 13.7790)&quot;&gt;
&lt;line stroke-linejoin=&quot;round&quot; stroke-linecap=&quot;round&quot; stroke-width=&quot;0.1000&quot; stroke=&quot;currentColor&quot; x1=&quot;0.0500&quot; y1=&quot;0&quot; x2=&quot;39.8983&quot; y2=&quot;0&quot;&gt;&lt;/line&gt;
&lt;/g&gt;
&lt;g transform=&quot;translate(8.5358, 12.7790)&quot;&gt;
&lt;line stroke-linejoin=&quot;round&quot; stroke-linecap=&quot;round&quot; stroke-width=&quot;0.1000&quot; stroke=&quot;currentColor&quot; x1=&quot;0.0500&quot; y1=&quot;0&quot; x2=&quot;39.8983&quot; y2=&quot;0&quot;&gt;&lt;/line&gt;
&lt;/g&gt;
&lt;g transform=&quot;translate(8.5358, 11.7790)&quot;&gt;
&lt;line stroke-linejoin=&quot;round&quot; stroke-linecap=&quot;round&quot; stroke-width=&quot;0.1000&quot; stroke=&quot;currentColor&quot; x1=&quot;0.0500&quot; y1=&quot;0&quot; x2=&quot;39.8983&quot; y2=&quot;0&quot;&gt;&lt;/line&gt;
&lt;/g&gt;
&lt;g transform=&quot;translate(8.5358, 10.7790)&quot;&gt;
&lt;line stroke-linejoin=&quot;round&quot; stroke-linecap=&quot;round&quot; stroke-width=&quot;0.1000&quot; stroke=&quot;currentColor&quot; x1=&quot;0.0500&quot; y1=&quot;0&quot; x2=&quot;39.8983&quot; y2=&quot;0&quot;&gt;&lt;/line&gt;
&lt;/g&gt;
&lt;g transform=&quot;translate(0.0000, 9.7790)&quot;&gt;
&lt;rect x=&quot;44.3340&quot; y=&quot;-0.1000&quot; width=&quot;1.9563&quot; height=&quot;0.2000&quot; ry=&quot;0.1000&quot; fill=&quot;currentColor&quot;&gt;&lt;/rect&gt;
&lt;/g&gt;
&lt;g transform=&quot;translate(0.0000, 9.7790)&quot;&gt;
&lt;rect x=&quot;37.6847&quot; y=&quot;-0.1000&quot; width=&quot;1.9563&quot; height=&quot;0.2000&quot; ry=&quot;0.1000&quot; fill=&quot;currentColor&quot;&gt;&lt;/rect&gt;
&lt;/g&gt;
&lt;g transform=&quot;translate(0.0000, 9.7790)&quot;&gt;
&lt;rect x=&quot;32.0344&quot; y=&quot;-0.1000&quot; width=&quot;1.9563&quot; height=&quot;0.2000&quot; ry=&quot;0.1000&quot; fill=&quot;currentColor&quot;&gt;&lt;/rect&gt;
&lt;/g&gt;
&lt;g transform=&quot;translate(0.0000, 9.7790)&quot;&gt;
&lt;rect x=&quot;25.3852&quot; y=&quot;-0.1000&quot; width=&quot;1.9563&quot; height=&quot;0.2000&quot; ry=&quot;0.1000&quot; fill=&quot;currentColor&quot;&gt;&lt;/rect&gt;
&lt;/g&gt;
&lt;g transform=&quot;translate(0.0000, 9.7790)&quot;&gt;
&lt;rect x=&quot;19.7349&quot; y=&quot;-0.1000&quot; width=&quot;1.9563&quot; height=&quot;0.2000&quot; ry=&quot;0.1000&quot; fill=&quot;currentColor&quot;&gt;&lt;/rect&gt;
&lt;/g&gt;
&lt;g transform=&quot;translate(0.0000, 6.7790)&quot;&gt;
&lt;rect x=&quot;41.5088&quot; y=&quot;-0.1000&quot; width=&quot;1.9563&quot; height=&quot;0.2000&quot; ry=&quot;0.1000&quot; fill=&quot;currentColor&quot;&gt;&lt;/rect&gt;
&lt;/g&gt;
&lt;g transform=&quot;translate(0.0000, 6.7790)&quot;&gt;
&lt;rect x=&quot;34.8596&quot; y=&quot;-0.1000&quot; width=&quot;1.9563&quot; height=&quot;0.2000&quot; ry=&quot;0.1000&quot; fill=&quot;currentColor&quot;&gt;&lt;/rect&gt;
&lt;/g&gt;
&lt;g transform=&quot;translate(0.0000, 6.7790)&quot;&gt;
&lt;rect x=&quot;29.2093&quot; y=&quot;-0.1000&quot; width=&quot;1.9563&quot; height=&quot;0.2000&quot; ry=&quot;0.1000&quot; fill=&quot;currentColor&quot;&gt;&lt;/rect&gt;
&lt;/g&gt;
&lt;g transform=&quot;translate(0.0000, 6.7790)&quot;&gt;
&lt;rect x=&quot;22.5601&quot; y=&quot;-0.1000&quot; width=&quot;1.9563&quot; height=&quot;0.2000&quot; ry=&quot;0.1000&quot; fill=&quot;currentColor&quot;&gt;&lt;/rect&gt;
&lt;/g&gt;
&lt;g transform=&quot;translate(0.0000, 6.7790)&quot;&gt;
&lt;rect x=&quot;16.9098&quot; y=&quot;-0.1000&quot; width=&quot;1.9563&quot; height=&quot;0.2000&quot; ry=&quot;0.1000&quot; fill=&quot;currentColor&quot;&gt;&lt;/rect&gt;
&lt;/g&gt;
&lt;g transform=&quot;translate(8.5358, 5.7790)&quot;&gt;
&lt;line stroke-linejoin=&quot;round&quot; stroke-linecap=&quot;round&quot; stroke-width=&quot;0.1000&quot; stroke=&quot;currentColor&quot; x1=&quot;0.0500&quot; y1=&quot;0&quot; x2=&quot;39.8983&quot; y2=&quot;0&quot;&gt;&lt;/line&gt;
&lt;/g&gt;
&lt;g transform=&quot;translate(8.5358, 4.7790)&quot;&gt;
&lt;line stroke-linejoin=&quot;round&quot; stroke-linecap=&quot;round&quot; stroke-width=&quot;0.1000&quot; stroke=&quot;currentColor&quot; x1=&quot;0.0500&quot; y1=&quot;0&quot; x2=&quot;39.8983&quot; y2=&quot;0&quot;&gt;&lt;/line&gt;
&lt;/g&gt;
&lt;g transform=&quot;translate(8.5358, 3.7790)&quot;&gt;
&lt;line stroke-linejoin=&quot;round&quot; stroke-linecap=&quot;round&quot; stroke-width=&quot;0.1000&quot; stroke=&quot;currentColor&quot; x1=&quot;0.0500&quot; y1=&quot;0&quot; x2=&quot;39.8983&quot; y2=&quot;0&quot;&gt;&lt;/line&gt;
&lt;/g&gt;
&lt;g transform=&quot;translate(8.5358, 2.7790)&quot;&gt;
&lt;line stroke-linejoin=&quot;round&quot; stroke-linecap=&quot;round&quot; stroke-width=&quot;0.1000&quot; stroke=&quot;currentColor&quot; x1=&quot;0.0500&quot; y1=&quot;0&quot; x2=&quot;39.8983&quot; y2=&quot;0&quot;&gt;&lt;/line&gt;
&lt;/g&gt;
&lt;g transform=&quot;translate(8.5358, 1.7790)&quot;&gt;
&lt;line stroke-linejoin=&quot;round&quot; stroke-linecap=&quot;round&quot; stroke-width=&quot;0.1000&quot; stroke=&quot;currentColor&quot; x1=&quot;0.0500&quot; y1=&quot;0&quot; x2=&quot;39.8983&quot; y2=&quot;0&quot;&gt;&lt;/line&gt;
&lt;/g&gt;
&lt;a style=&quot;color:inherit;&quot; xlink:href=&quot;textedit:///tmp/lyInsert-2026125-4206-1x7a9yz.lb7s.ly:12:73:74&quot;&gt;
&lt;g transform=&quot;translate(38.0108, 3.7790)&quot;&gt;
&lt;path transform=&quot;scale(0.0040, -0.0040)&quot; d=&quot;M-23 -116c0 28 11 42 40 42c33 0 78 -13 119 -31l-132 158c-7 9 -10 17 -10 25c0 34 50 66 87 99c25 22 37 52 37 83c0 24 -8 49 -25 69l-35 42c-3 3 -4 7 -4 10c0 9 9 15 17 15c4 0 8 -1 11 -5l151 -180c7 -9 10 -17 10 -25c0 -34 -50 -66 -87 -99
c-25 -22 -37 -52 -37 -83c0 -24 7 -49 24 -69l84 -99c3 -3 4 -7 4 -10c0 -9 -8 -16 -16 -16c-4 0 -9 2 -12 6c-18 21 -63 38 -97 38c-41 0 -53 -26 -53 -67c0 -35 11 -74 28 -94c7 -8 -5 -19 -12 -11c-45 54 -92 148 -92 202z&quot; fill=&quot;currentColor&quot;&gt;&lt;/path&gt;
&lt;/g&gt;
&lt;/a&gt;
&lt;a style=&quot;color:inherit;&quot; xlink:href=&quot;textedit:///tmp/lyInsert-2026125-4206-1x7a9yz.lb7s.ly:13:73:74&quot;&gt;
&lt;g transform=&quot;translate(38.0108, 9.7790)&quot;&gt;
&lt;path transform=&quot;scale(0.0040, -0.0040)&quot; d=&quot;M218 136c55 0 108 -28 108 -89c0 -71 -55 -121 -102 -149c-35 -21 -75 -34 -116 -34c-55 0 -108 28 -108 89c0 71 55 121 102 149c35 21 75 34 116 34z&quot; fill=&quot;currentColor&quot;&gt;&lt;/path&gt;
&lt;/g&gt;
&lt;/a&gt;
&lt;g transform=&quot;translate(38.0758, 12.7790)&quot;&gt;
&lt;rect x=&quot;-0.0650&quot; y=&quot;-2.8139&quot; width=&quot;0.1300&quot; height=&quot;3.3139&quot; ry=&quot;0.0400&quot; fill=&quot;currentColor&quot;&gt;&lt;/rect&gt;
&lt;/g&gt;
&lt;a style=&quot;color:inherit;&quot; xlink:href=&quot;textedit:///tmp/lyInsert-2026125-4206-1x7a9yz.lb7s.ly:12:75:76&quot;&gt;
&lt;g transform=&quot;translate(41.8349, 6.7790)&quot;&gt;
&lt;path transform=&quot;scale(0.0040, -0.0040)&quot; d=&quot;M218 136c55 0 108 -28 108 -89c0 -71 -55 -121 -102 -149c-35 -21 -75 -34 -116 -34c-55 0 -108 28 -108 89c0 71 55 121 102 149c35 21 75 34 116 34z&quot; fill=&quot;currentColor&quot;&gt;&lt;/path&gt;
&lt;/g&gt;
&lt;/a&gt;
&lt;a style=&quot;color:inherit;&quot; xlink:href=&quot;textedit:///tmp/lyInsert-2026125-4206-1x7a9yz.lb7s.ly:13:75:76&quot;&gt;
&lt;g transform=&quot;translate(41.8349, 12.7790)&quot;&gt;
&lt;path transform=&quot;scale(0.0040, -0.0040)&quot; d=&quot;M-23 -116c0 28 11 42 40 42c33 0 78 -13 119 -31l-132 158c-7 9 -10 17 -10 25c0 34 50 66 87 99c25 22 37 52 37 83c0 24 -8 49 -25 69l-35 42c-3 3 -4 7 -4 10c0 9 9 15 17 15c4 0 8 -1 11 -5l151 -180c7 -9 10 -17 10 -25c0 -34 -50 -66 -87 -99
c-25 -22 -37 -52 -37 -83c0 -24 7 -49 24 -69l84 -99c3 -3 4 -7 4 -10c0 -9 -8 -16 -16 -16c-4 0 -9 2 -12 6c-18 21 -63 38 -97 38c-41 0 -53 -26 -53 -67c0 -35 11 -74 28 -94c7 -8 -5 -19 -12 -11c-45 54 -92 148 -92 202z&quot; fill=&quot;currentColor&quot;&gt;&lt;/path&gt;
&lt;/g&gt;
&lt;/a&gt;
&lt;a style=&quot;color:inherit;&quot; xlink:href=&quot;textedit:///tmp/lyInsert-2026125-4206-1x7a9yz.lb7s.ly:12:69:70&quot;&gt;
&lt;g transform=&quot;translate(32.3605, 3.7790)&quot;&gt;
&lt;path transform=&quot;scale(0.0040, -0.0040)&quot; d=&quot;M-23 -116c0 28 11 42 40 42c33 0 78 -13 119 -31l-132 158c-7 9 -10 17 -10 25c0 34 50 66 87 99c25 22 37 52 37 83c0 24 -8 49 -25 69l-35 42c-3 3 -4 7 -4 10c0 9 9 15 17 15c4 0 8 -1 11 -5l151 -180c7 -9 10 -17 10 -25c0 -34 -50 -66 -87 -99
c-25 -22 -37 -52 -37 -83c0 -24 7 -49 24 -69l84 -99c3 -3 4 -7 4 -10c0 -9 -8 -16 -16 -16c-4 0 -9 2 -12 6c-18 21 -63 38 -97 38c-41 0 -53 -26 -53 -67c0 -35 11 -74 28 -94c7 -8 -5 -19 -12 -11c-45 54 -92 148 -92 202z&quot; fill=&quot;currentColor&quot;&gt;&lt;/path&gt;
&lt;/g&gt;
&lt;/a&gt;
&lt;a style=&quot;color:inherit;&quot; xlink:href=&quot;textedit:///tmp/lyInsert-2026125-4206-1x7a9yz.lb7s.ly:13:69:70&quot;&gt;
&lt;g transform=&quot;translate(32.3605, 9.7790)&quot;&gt;
&lt;path transform=&quot;scale(0.0040, -0.0040)&quot; d=&quot;M218 136c55 0 108 -28 108 -89c0 -71 -55 -121 -102 -149c-35 -21 -75 -34 -116 -34c-55 0 -108 28 -108 89c0 71 55 121 102 149c35 21 75 34 116 34z&quot; fill=&quot;currentColor&quot;&gt;&lt;/path&gt;
&lt;/g&gt;
&lt;/a&gt;
&lt;g transform=&quot;translate(32.4255, 12.7790)&quot;&gt;
&lt;rect x=&quot;-0.0650&quot; y=&quot;-2.8139&quot; width=&quot;0.1300&quot; height=&quot;3.3139&quot; ry=&quot;0.0400&quot; fill=&quot;currentColor&quot;&gt;&lt;/rect&gt;
&lt;/g&gt;
&lt;a style=&quot;color:inherit;&quot; xlink:href=&quot;textedit:///tmp/lyInsert-2026125-4206-1x7a9yz.lb7s.ly:12:71:72&quot;&gt;
&lt;g transform=&quot;translate(35.1856, 6.7790)&quot;&gt;
&lt;path transform=&quot;scale(0.0040, -0.0040)&quot; d=&quot;M218 136c55 0 108 -28 108 -89c0 -71 -55 -121 -102 -149c-35 -21 -75 -34 -116 -34c-55 0 -108 28 -108 89c0 71 55 121 102 149c35 21 75 34 116 34z&quot; fill=&quot;currentColor&quot;&gt;&lt;/path&gt;
&lt;/g&gt;
&lt;/a&gt;
&lt;a style=&quot;color:inherit;&quot; xlink:href=&quot;textedit:///tmp/lyInsert-2026125-4206-1x7a9yz.lb7s.ly:13:71:72&quot;&gt;
&lt;g transform=&quot;translate(35.1856, 12.7790)&quot;&gt;
&lt;path transform=&quot;scale(0.0040, -0.0040)&quot; d=&quot;M-23 -116c0 28 11 42 40 42c33 0 78 -13 119 -31l-132 158c-7 9 -10 17 -10 25c0 34 50 66 87 99c25 22 37 52 37 83c0 24 -8 49 -25 69l-35 42c-3 3 -4 7 -4 10c0 9 9 15 17 15c4 0 8 -1 11 -5l151 -180c7 -9 10 -17 10 -25c0 -34 -50 -66 -87 -99
c-25 -22 -37 -52 -37 -83c0 -24 7 -49 24 -69l84 -99c3 -3 4 -7 4 -10c0 -9 -8 -16 -16 -16c-4 0 -9 2 -12 6c-18 21 -63 38 -97 38c-41 0 -53 -26 -53 -67c0 -35 11 -74 28 -94c7 -8 -5 -19 -12 -11c-45 54 -92 148 -92 202z&quot; fill=&quot;currentColor&quot;&gt;&lt;/path&gt;
&lt;/g&gt;
&lt;/a&gt;
&lt;g transform=&quot;translate(36.4249, 3.7790)&quot;&gt;
&lt;rect x=&quot;-0.0650&quot; y=&quot;-0.5000&quot; width=&quot;0.1300&quot; height=&quot;3.3139&quot; ry=&quot;0.0400&quot; fill=&quot;currentColor&quot;&gt;&lt;/rect&gt;
&lt;/g&gt;
&lt;a style=&quot;color:inherit;&quot; xlink:href=&quot;textedit:///tmp/lyInsert-2026125-4206-1x7a9yz.lb7s.ly:13:77:78&quot;&gt;
&lt;g transform=&quot;translate(44.6600, 9.7790)&quot;&gt;
&lt;path transform=&quot;scale(0.0040, -0.0040)&quot; d=&quot;M218 136c55 0 108 -28 108 -89c0 -71 -55 -121 -102 -149c-35 -21 -75 -34 -116 -34c-55 0 -108 28 -108 89c0 71 55 121 102 149c35 21 75 34 116 34z&quot; fill=&quot;currentColor&quot;&gt;&lt;/path&gt;
&lt;/g&gt;
&lt;/a&gt;
&lt;g transform=&quot;translate(44.7250, 12.7790)&quot;&gt;
&lt;rect x=&quot;-0.0650&quot; y=&quot;-2.8139&quot; width=&quot;0.1300&quot; height=&quot;3.3139&quot; ry=&quot;0.0400&quot; fill=&quot;currentColor&quot;&gt;&lt;/rect&gt;
&lt;/g&gt;
&lt;a style=&quot;color:inherit;&quot; xlink:href=&quot;textedit:///tmp/lyInsert-2026125-4206-1x7a9yz.lb7s.ly:12:77:78&quot;&gt;
&lt;g transform=&quot;translate(44.6600, 3.7790)&quot;&gt;
&lt;path transform=&quot;scale(0.0040, -0.0040)&quot; d=&quot;M-23 -116c0 28 11 42 40 42c33 0 78 -13 119 -31l-132 158c-7 9 -10 17 -10 25c0 34 50 66 87 99c25 22 37 52 37 83c0 24 -8 49 -25 69l-35 42c-3 3 -4 7 -4 10c0 9 9 15 17 15c4 0 8 -1 11 -5l151 -180c7 -9 10 -17 10 -25c0 -34 -50 -66 -87 -99
c-25 -22 -37 -52 -37 -83c0 -24 7 -49 24 -69l84 -99c3 -3 4 -7 4 -10c0 -9 -8 -16 -16 -16c-4 0 -9 2 -12 6c-18 21 -63 38 -97 38c-41 0 -53 -26 -53 -67c0 -35 11 -74 28 -94c7 -8 -5 -19 -12 -11c-45 54 -92 148 -92 202z&quot; fill=&quot;currentColor&quot;&gt;&lt;/path&gt;
&lt;/g&gt;
&lt;/a&gt;
&lt;g transform=&quot;translate(43.0741, 3.7790)&quot;&gt;
&lt;rect x=&quot;-0.0650&quot; y=&quot;-0.5000&quot; width=&quot;0.1300&quot; height=&quot;3.3139&quot; ry=&quot;0.0400&quot; fill=&quot;currentColor&quot;&gt;&lt;/rect&gt;
&lt;/g&gt;
&lt;a style=&quot;color:inherit;&quot; xlink:href=&quot;textedit:///tmp/lyInsert-2026125-4206-1x7a9yz.lb7s.ly:12:61:62&quot;&gt;
&lt;g transform=&quot;translate(20.0610, 3.7790)&quot;&gt;
&lt;path transform=&quot;scale(0.0040, -0.0040)&quot; d=&quot;M-23 -116c0 28 11 42 40 42c33 0 78 -13 119 -31l-132 158c-7 9 -10 17 -10 25c0 34 50 66 87 99c25 22 37 52 37 83c0 24 -8 49 -25 69l-35 42c-3 3 -4 7 -4 10c0 9 9 15 17 15c4 0 8 -1 11 -5l151 -180c7 -9 10 -17 10 -25c0 -34 -50 -66 -87 -99
c-25 -22 -37 -52 -37 -83c0 -24 7 -49 24 -69l84 -99c3 -3 4 -7 4 -10c0 -9 -8 -16 -16 -16c-4 0 -9 2 -12 6c-18 21 -63 38 -97 38c-41 0 -53 -26 -53 -67c0 -35 11 -74 28 -94c7 -8 -5 -19 -12 -11c-45 54 -92 148 -92 202z&quot; fill=&quot;currentColor&quot;&gt;&lt;/path&gt;
&lt;/g&gt;
&lt;/a&gt;
&lt;a style=&quot;color:inherit;&quot; xlink:href=&quot;textedit:///tmp/lyInsert-2026125-4206-1x7a9yz.lb7s.ly:13:61:62&quot;&gt;
&lt;g transform=&quot;translate(20.0610, 9.7790)&quot;&gt;
&lt;path transform=&quot;scale(0.0040, -0.0040)&quot; d=&quot;M218 136c55 0 108 -28 108 -89c0 -71 -55 -121 -102 -149c-35 -21 -75 -34 -116 -34c-55 0 -108 28 -108 89c0 71 55 121 102 149c35 21 75 34 116 34z&quot; fill=&quot;currentColor&quot;&gt;&lt;/path&gt;
&lt;/g&gt;
&lt;/a&gt;
&lt;g transform=&quot;translate(20.1260, 12.7790)&quot;&gt;
&lt;rect x=&quot;-0.0650&quot; y=&quot;-2.8139&quot; width=&quot;0.1300&quot; height=&quot;3.3139&quot; ry=&quot;0.0400&quot; fill=&quot;currentColor&quot;&gt;&lt;/rect&gt;
&lt;/g&gt;
&lt;a style=&quot;color:inherit;&quot; xlink:href=&quot;textedit:///tmp/lyInsert-2026125-4206-1x7a9yz.lb7s.ly:13:63:64&quot;&gt;
&lt;g transform=&quot;translate(22.8861, 12.7790)&quot;&gt;
&lt;path transform=&quot;scale(0.0040, -0.0040)&quot; d=&quot;M-23 -116c0 28 11 42 40 42c33 0 78 -13 119 -31l-132 158c-7 9 -10 17 -10 25c0 34 50 66 87 99c25 22 37 52 37 83c0 24 -8 49 -25 69l-35 42c-3 3 -4 7 -4 10c0 9 9 15 17 15c4 0 8 -1 11 -5l151 -180c7 -9 10 -17 10 -25c0 -34 -50 -66 -87 -99
c-25 -22 -37 -52 -37 -83c0 -24 7 -49 24 -69l84 -99c3 -3 4 -7 4 -10c0 -9 -8 -16 -16 -16c-4 0 -9 2 -12 6c-18 21 -63 38 -97 38c-41 0 -53 -26 -53 -67c0 -35 11 -74 28 -94c7 -8 -5 -19 -12 -11c-45 54 -92 148 -92 202z&quot; fill=&quot;currentColor&quot;&gt;&lt;/path&gt;
&lt;/g&gt;
&lt;/a&gt;
&lt;g transform=&quot;translate(24.1253, 3.7790)&quot;&gt;
&lt;rect x=&quot;-0.0650&quot; y=&quot;-0.5000&quot; width=&quot;0.1300&quot; height=&quot;3.3139&quot; ry=&quot;0.0400&quot; fill=&quot;currentColor&quot;&gt;&lt;/rect&gt;
&lt;/g&gt;
&lt;g transform=&quot;translate(9.3358, 4.7790)&quot;&gt;
&lt;path transform=&quot;scale(0.0040, -0.0040)&quot; d=&quot;M266 -635h-6c-108 0 -195 88 -195 197c0 58 53 103 112 103c54 0 95 -47 95 -103c0 -52 -43 -95 -95 -95c-11 0 -21 2 -31 6c26 -39 68 -65 117 -65h4zM461 -203c68 24 113 95 113 164c0 90 -66 179 -173 190c24 -116 46 -231 60 -354zM74 28c0 -135 129 -247 264 -247
c28 0 55 2 82 6c-14 127 -37 245 -63 364c-79 -8 -124 -61 -124 -119c0 -44 25 -91 81 -123c5 -5 7 -10 7 -15c0 -11 -10 -22 -22 -22c-3 0 -6 1 -9 2c-80 43 -117 115 -117 185c0 88 58 174 160 197c-14 58 -29 117 -46 175c-107 -121 -213 -243 -213 -403zM335 -262
c-188 0 -333 172 -333 374c0 177 131 306 248 441c-19 62 -37 125 -45 190c-6 52 -7 104 -7 156c0 115 55 224 149 292c6 5 14 5 20 0c71 -84 133 -245 133 -358c0 -143 -86 -255 -180 -364c21 -68 39 -138 56 -207c4 0 9 1 13 1c155 0 256 -128 256 -261
c0 -76 -33 -154 -107 -210c-22 -17 -47 -28 -73 -36c3 -35 5 -70 5 -105c0 -19 -1 -39 -2 -58c-7 -119 -88 -225 -202 -228l1 43c93 2 153 92 159 191c1 18 2 37 2 55c0 31 -1 61 -4 92c-29 -5 -58 -8 -89 -8zM428 916c0 55 -4 79 -20 129c-99 -48 -162 -149 -162 -259
c0 -74 18 -133 36 -194c80 97 146 198 146 324z&quot; fill=&quot;currentColor&quot;&gt;&lt;/path&gt;
&lt;/g&gt;
&lt;g transform=&quot;translate(-0.3115, 9.0728)&quot;&gt;
&lt;text font-family=&quot;serif&quot; font-size=&quot;2.2001&quot; text-anchor=&quot;start&quot; fill=&quot;currentColor&quot;&gt;
&lt;tspan&gt;Piano&lt;/tspan&gt;
&lt;/text&gt;
&lt;/g&gt;
&lt;g transform=&quot;translate(8.1758, 8.2790)&quot;&gt;
&lt;path transform=&quot;scale(0.0040, -0.0040)&quot; d=&quot;M-189 -1146c0 267 108 528 108 780c0 135 -33 261 -129 360c-3 3 -3 3 -3 6s0 3 3 6c96 99 129 225 129 360c0 252 -108 513 -108 780c0 189 45 366 177 501c15 15 39 -9 24 -24c-99 -99 -129 -225 -129 -360c0 -249 102 -504 102 -765c0 -186 -45 -363 -174 -498
c129 -135 174 -312 174 -498c0 -261 -102 -516 -102 -765c0 -135 30 -261 129 -360c15 -15 -9 -39 -24 -24c-132 135 -177 312 -177 501z&quot; fill=&quot;currentColor&quot;&gt;&lt;/path&gt;
&lt;/g&gt;
&lt;g transform=&quot;translate(8.4758, 8.2790)&quot;&gt;
&lt;rect x=&quot;0.0000&quot; y=&quot;-6.5500&quot; width=&quot;0.1600&quot; height=&quot;13.1000&quot; ry=&quot;0.0500&quot; fill=&quot;currentColor&quot;&gt;&lt;/rect&gt;
&lt;/g&gt;
&lt;a style=&quot;color:inherit;&quot; xlink:href=&quot;textedit:///tmp/lyInsert-2026125-4206-1x7a9yz.lb7s.ly:12:59:60&quot;&gt;
&lt;g transform=&quot;translate(17.2358, 6.7790)&quot;&gt;
&lt;path transform=&quot;scale(0.0040, -0.0040)&quot; d=&quot;M218 136c55 0 108 -28 108 -89c0 -71 -55 -121 -102 -149c-35 -21 -75 -34 -116 -34c-55 0 -108 28 -108 89c0 71 55 121 102 149c35 21 75 34 116 34z&quot; fill=&quot;currentColor&quot;&gt;&lt;/path&gt;
&lt;/g&gt;
&lt;/a&gt;
&lt;a style=&quot;color:inherit;&quot; xlink:href=&quot;textedit:///tmp/lyInsert-2026125-4206-1x7a9yz.lb7s.ly:12:63:64&quot;&gt;
&lt;g transform=&quot;translate(22.8861, 6.7790)&quot;&gt;
&lt;path transform=&quot;scale(0.0040, -0.0040)&quot; d=&quot;M218 136c55 0 108 -28 108 -89c0 -71 -55 -121 -102 -149c-35 -21 -75 -34 -116 -34c-55 0 -108 28 -108 89c0 71 55 121 102 149c35 21 75 34 116 34z&quot; fill=&quot;currentColor&quot;&gt;&lt;/path&gt;
&lt;/g&gt;
&lt;/a&gt;
&lt;g transform=&quot;translate(13.5358, 3.7790)&quot;&gt;
&lt;path transform=&quot;scale(0.0040, -0.0040)&quot; d=&quot;M359 27c-49 0 -75 42 -75 75c0 38 27 77 72 77c4 0 9 0 14 -1c-28 37 -72 59 -120 59c-106 0 -113 -73 -113 -186v-51v-51c0 -113 7 -187 113 -187c80 0 139 70 158 151c2 7 7 10 12 10c6 0 13 -4 13 -12c0 -94 -105 -174 -183 -174c-68 0 -137 21 -184 70
c-49 51 -66 122 -66 193s17 142 66 193c47 49 116 69 184 69c87 0 160 -63 175 -149c1 -5 1 -10 1 -14c0 -40 -30 -72 -67 -72z&quot; fill=&quot;currentColor&quot;&gt;&lt;/path&gt;
&lt;/g&gt;
&lt;a style=&quot;color:inherit;&quot; xlink:href=&quot;textedit:///tmp/lyInsert-2026125-4206-1x7a9yz.lb7s.ly:13:59:60&quot;&gt;
&lt;g transform=&quot;translate(17.2358, 12.7790)&quot;&gt;
&lt;path transform=&quot;scale(0.0040, -0.0040)&quot; d=&quot;M-23 -116c0 28 11 42 40 42c33 0 78 -13 119 -31l-132 158c-7 9 -10 17 -10 25c0 34 50 66 87 99c25 22 37 52 37 83c0 24 -8 49 -25 69l-35 42c-3 3 -4 7 -4 10c0 9 9 15 17 15c4 0 8 -1 11 -5l151 -180c7 -9 10 -17 10 -25c0 -34 -50 -66 -87 -99
c-25 -22 -37 -52 -37 -83c0 -24 7 -49 24 -69l84 -99c3 -3 4 -7 4 -10c0 -9 -8 -16 -16 -16c-4 0 -9 2 -12 6c-18 21 -63 38 -97 38c-41 0 -53 -26 -53 -67c0 -35 11 -74 28 -94c7 -8 -5 -19 -12 -11c-45 54 -92 148 -92 202z&quot; fill=&quot;currentColor&quot;&gt;&lt;/path&gt;
&lt;/g&gt;
&lt;/a&gt;
&lt;g transform=&quot;translate(9.3358, 11.7790)&quot;&gt;
&lt;path transform=&quot;scale(0.0040, -0.0040)&quot; d=&quot;M557 -125c0 28 23 51 51 51s51 -23 51 -51s-23 -51 -51 -51s-51 23 -51 51zM557 125c0 28 23 51 51 51s51 -23 51 -51s-23 -51 -51 -51s-51 23 -51 51zM232 263c172 0 293 -88 293 -251c0 -263 -263 -414 -516 -521c-3 -3 -6 -4 -9 -4c-7 0 -13 6 -13 13c0 3 1 6 4 9
c202 118 412 265 412 493c0 120 -63 235 -171 235c-74 0 -129 -54 -154 -126c11 5 22 8 34 8c55 0 100 -45 100 -100c0 -58 -44 -106 -100 -106c-60 0 -112 47 -112 106c0 133 102 244 232 244z&quot; fill=&quot;currentColor&quot;&gt;&lt;/path&gt;
&lt;/g&gt;
&lt;g transform=&quot;translate(13.5358, 12.7790)&quot;&gt;
&lt;path transform=&quot;scale(0.0040, -0.0040)&quot; d=&quot;M359 27c-49 0 -75 42 -75 75c0 38 27 77 72 77c4 0 9 0 14 -1c-28 37 -72 59 -120 59c-106 0 -113 -73 -113 -186v-51v-51c0 -113 7 -187 113 -187c80 0 139 70 158 151c2 7 7 10 12 10c6 0 13 -4 13 -12c0 -94 -105 -174 -183 -174c-68 0 -137 21 -184 70
c-49 51 -66 122 -66 193s17 142 66 193c47 49 116 69 184 69c87 0 160 -63 175 -149c1 -5 1 -10 1 -14c0 -40 -30 -72 -67 -72z&quot; fill=&quot;currentColor&quot;&gt;&lt;/path&gt;
&lt;/g&gt;
&lt;g transform=&quot;translate(18.4750, 3.7790)&quot;&gt;
&lt;rect x=&quot;-0.0650&quot; y=&quot;-0.5000&quot; width=&quot;0.1300&quot; height=&quot;3.3139&quot; ry=&quot;0.0400&quot; fill=&quot;currentColor&quot;&gt;&lt;/rect&gt;
&lt;/g&gt;
&lt;a style=&quot;color:inherit;&quot; xlink:href=&quot;textedit:///tmp/lyInsert-2026125-4206-1x7a9yz.lb7s.ly:12:67:68&quot;&gt;
&lt;g transform=&quot;translate(29.5353, 6.7790)&quot;&gt;
&lt;path transform=&quot;scale(0.0040, -0.0040)&quot; d=&quot;M218 136c55 0 108 -28 108 -89c0 -71 -55 -121 -102 -149c-35 -21 -75 -34 -116 -34c-55 0 -108 28 -108 89c0 71 55 121 102 149c35 21 75 34 116 34z&quot; fill=&quot;currentColor&quot;&gt;&lt;/path&gt;
&lt;/g&gt;
&lt;/a&gt;
&lt;a style=&quot;color:inherit;&quot; xlink:href=&quot;textedit:///tmp/lyInsert-2026125-4206-1x7a9yz.lb7s.ly:13:67:68&quot;&gt;
&lt;g transform=&quot;translate(29.5353, 12.7790)&quot;&gt;
&lt;path transform=&quot;scale(0.0040, -0.0040)&quot; d=&quot;M-23 -116c0 28 11 42 40 42c33 0 78 -13 119 -31l-132 158c-7 9 -10 17 -10 25c0 34 50 66 87 99c25 22 37 52 37 83c0 24 -8 49 -25 69l-35 42c-3 3 -4 7 -4 10c0 9 9 15 17 15c4 0 8 -1 11 -5l151 -180c7 -9 10 -17 10 -25c0 -34 -50 -66 -87 -99
c-25 -22 -37 -52 -37 -83c0 -24 7 -49 24 -69l84 -99c3 -3 4 -7 4 -10c0 -9 -8 -16 -16 -16c-4 0 -9 2 -12 6c-18 21 -63 38 -97 38c-41 0 -53 -26 -53 -67c0 -35 11 -74 28 -94c7 -8 -5 -19 -12 -11c-45 54 -92 148 -92 202z&quot; fill=&quot;currentColor&quot;&gt;&lt;/path&gt;
&lt;/g&gt;
&lt;/a&gt;
&lt;g transform=&quot;translate(30.7746, 3.7790)&quot;&gt;
&lt;rect x=&quot;-0.0650&quot; y=&quot;-0.5000&quot; width=&quot;0.1300&quot; height=&quot;3.3139&quot; ry=&quot;0.0400&quot; fill=&quot;currentColor&quot;&gt;&lt;/rect&gt;
&lt;/g&gt;
&lt;a style=&quot;color:inherit;&quot; xlink:href=&quot;textedit:///tmp/lyInsert-2026125-4206-1x7a9yz.lb7s.ly:12:65:66&quot;&gt;
&lt;g transform=&quot;translate(25.7113, 3.7790)&quot;&gt;
&lt;path transform=&quot;scale(0.0040, -0.0040)&quot; d=&quot;M-23 -116c0 28 11 42 40 42c33 0 78 -13 119 -31l-132 158c-7 9 -10 17 -10 25c0 34 50 66 87 99c25 22 37 52 37 83c0 24 -8 49 -25 69l-35 42c-3 3 -4 7 -4 10c0 9 9 15 17 15c4 0 8 -1 11 -5l151 -180c7 -9 10 -17 10 -25c0 -34 -50 -66 -87 -99
c-25 -22 -37 -52 -37 -83c0 -24 7 -49 24 -69l84 -99c3 -3 4 -7 4 -10c0 -9 -8 -16 -16 -16c-4 0 -9 2 -12 6c-18 21 -63 38 -97 38c-41 0 -53 -26 -53 -67c0 -35 11 -74 28 -94c7 -8 -5 -19 -12 -11c-45 54 -92 148 -92 202z&quot; fill=&quot;currentColor&quot;&gt;&lt;/path&gt;
&lt;/g&gt;
&lt;/a&gt;
&lt;a style=&quot;color:inherit;&quot; xlink:href=&quot;textedit:///tmp/lyInsert-2026125-4206-1x7a9yz.lb7s.ly:13:65:66&quot;&gt;
&lt;g transform=&quot;translate(25.7113, 9.7790)&quot;&gt;
&lt;path transform=&quot;scale(0.0040, -0.0040)&quot; d=&quot;M218 136c55 0 108 -28 108 -89c0 -71 -55 -121 -102 -149c-35 -21 -75 -34 -116 -34c-55 0 -108 28 -108 89c0 71 55 121 102 149c35 21 75 34 116 34z&quot; fill=&quot;currentColor&quot;&gt;&lt;/path&gt;
&lt;/g&gt;
&lt;/a&gt;
&lt;g transform=&quot;translate(25.7763, 12.7790)&quot;&gt;
&lt;rect x=&quot;-0.0650&quot; y=&quot;-2.8139&quot; width=&quot;0.1300&quot; height=&quot;3.3139&quot; ry=&quot;0.0400&quot; fill=&quot;currentColor&quot;&gt;&lt;/rect&gt;
&lt;/g&gt;
&lt;/svg&gt;
&lt;p&gt;I knew what middle C was... I had been playing Ukulele for a while so I knew it was the note on the second string of the Ukulele, if it was tuned in standard tuning but was unaware of its relationship to the piano.&lt;/p&gt;
&lt;p&gt;I quickly learned to love the uniformity of the piano. If middle C is here, then the next note D is the white key next to it, and the E is the next one after. Also, there&#39;s only one place to play a given note.&lt;/p&gt;
&lt;p&gt;I was beyond excited to be learning the piano. For a long time, I had been wanting to learn the piano, which I always saw as the king of all instruments. Taking the plunge and showing up to piano lessons was a dream come true. We started using the book &lt;a href=&quot;https://www.alfred.com/alfreds-basic-adult-piano-course-lesson-book-1/p/00-2236/&quot;&gt;Alfred&#39;s Basic Adult Piano Course: Level 1&lt;/a&gt; and over the months that followed we made steady progress on that. We would practice a couple of pieces one hand at a time during the lesson and I would perfect those at home to perform them in the following lesson. If all was ok, great - we moved ahead. If not, we would practice more together and I would perfect them again at home. We quickly got to June and my teacher asked me if I would like to play publicly in a student concert. I immediately said &amp;quot;Yes&amp;quot;, which seemed to take her by surprise. Apparently, adult students and especially beginner students are not keen on joining concerts. At this point, we chose two pieces: Allegretto by Vanhal and Nächtliche Reise by Gurlitt. She played the pieces for me and I loved them so being able to play three months later seemed amazing. Of course, now I needed to be able to focus on them as they were slightly trickier than my weekly ones from Alfred&#39;s book. So for the following lessons, I focused on them and at home, I did the same. Every lesson was focused on perfecting some parts of each piece. And on Saturday it all came together.&lt;/p&gt;
&lt;p&gt;My wife, both of my children, and my parents-in-law came to the concert and although I was anxious the whole day, I got slightly less anxious before. I didn&#39;t get nervous until pretty much it was my turn to play, but I didn&#39;t have a meltdown. That felt like it came later. The Vanhal piece was always the one I had the most trouble with and Gurlitt was the one that seemed easier. However, I played Vanhal first after the tradition of playing older pieces first. Vanhal was going very well and the first repetition was flawless. During the second repetition I thought that &amp;quot;it&#39;s just a matter of finishing the second repetition and then the easy piece is coming.&amp;quot;. I deviated my look a bit from the keys as it occurred to me and as I looked into the keys, I forgot where in the scale I was and missed a beat. Damn! I continued as if nothing happened and I grew increasingly nervous wondering what else was going to happen. My brain went haywire as I started the easier piece and I missed a couple of beats. This was supposed to be the easy piece but I was murdering it. But like everything else, it came to an end. As I get up from the piano, everyone was clapping and afterward many people came to congratulate me but I was disappointed and couldn&#39;t shake the feeling I had screwed up.&lt;/p&gt;
&lt;p&gt;My wife filmed it and I later watched, half covering my face in embarrassment, but noticed that actually, it sounded quite nice. And although, yes, there were a couple of lapses, unless you know the piece you wouldn&#39;t necessarily notice them.&lt;/p&gt;
&lt;p&gt;I am just very excited I had the opportunity to join this concert and can&#39;t wait to do better, on harder pieces, next time.&lt;/p&gt;
&lt;p&gt;When I decide to take on a new hobby, I want to give it all and be the best amateur I can be. I believe that you need a strategy instead of going at it Adhoc given you&#39;re doing it in your spare time and you have to fit things in with your family life.
So I will finish off with a few points and tips for absolute beginners from my journey that might be more interesting to read than the above memory dump. These are things that worked very well for me and that I would advise you to consider but they are in no way gospel or remotely generalizable. But if you see you&#39;re in a position to try them out, please do.&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;Immerse yourself. If you are going to do a hobby make sure you stay up to date on developments and what&#39;s going on. Personally, I subscribed to Pianist, subscribed to &lt;a href=&quot;https://www.reddit.com/r/piano&quot;&gt;r/piano&lt;/a&gt; and &lt;a href=&quot;https://www.reddit.com/r/pianolearning&quot;&gt;r/pianolearning&lt;/a&gt;, follow several pianists on Twitter like &lt;a href=&quot;https://twitter.com/lang_lang&quot;&gt;@lang_lang&lt;/a&gt;, &lt;a href=&quot;https://twitter.com/lars_vogt&quot;&gt;@lars_vogt&lt;/a&gt;, or &lt;a href=&quot;https://twitter.com/daniil_trifonov&quot;&gt;@daniil_trifonov&lt;/a&gt;, and started listening to podcasts related to piano playing like the &lt;a href=&quot;https://open.spotify.com/show/72OQO63V0y3jZ0fh4tVlwj&quot;&gt;Josh Wright Piano Podcast&lt;/a&gt;. In addition to that my knowledge and feeling for classical music were zero. Since then, I have been regularly putting on some classical music, looking at the scores (even if all pieces are impossible for me to play), and trying to appreciate how hard it must be to play them. I don&#39;t read a huge amount except on holidays when I tend to disconnect and read quite a bit. So these last Summer holidays saw me going through 3 books related to Piano playing that I recommend: &lt;a href=&quot;https://www.goodreads.com/book/show/17332356-play-it-again&quot;&gt;Play it Again: An Amateur Against the Impossible&lt;/a&gt; by Alan Rusbridger, &lt;a href=&quot;https://www.goodreads.com/book/show/2727587-journey-of-a-thousand-miles&quot;&gt;Journey of a Thousand Miles: My Story&lt;/a&gt; by Lang Lang and David Ritz, and &lt;a href=&quot;https://www.goodreads.com/book/show/589577.Piano_Notes&quot;&gt;Piano Notes: The World of the Pianist&lt;/a&gt; by Charles Rosen.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Measure your progress. It&#39;s hard to know how much time you&#39;re actually putting in if you don&#39;t measure. You might guess but most likely you&#39;ll be wrong. I use a file where I write down all post-lesson notes about the lesson and track all my time in front of the piano at home using &lt;a href=&quot;https://instrumentive.com/&quot;&gt;Instrumentive&lt;/a&gt;.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Set a long-term goal to challenge yourself. Hard long-term goals are important in my opinion. They keep you focused with something to look forward to. If you were running, maybe a marathon could be a long-term goal. In piano, maybe playing a certain piece publicly (and this can mean to others or even just to your family who you invite for a concert). Some people recommend that you share that with others, or even post online for further motivation. I generally prefer to keep those (sometimes ridiculous goals) within a small circle of people. But I guess that&#39;s just a personal preference.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Research the best. Within your chosen hobby there are always some who are at the top of it. Follow them on social media, read about them, and see how they practice, train, etc. How they got to where they are, etc. It&#39;s likely that as a hobbyist you might not be able to emulate everything they do or how they do it. However, I think that just knowing what&#39;s possible is incredibly motivating. At the moment, I always keep an eye on Lang Lang, Daniil Trifonov, and Josh Wright.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Find support. Sometimes you have hobbies that are very social like playing football, poker, etc. Some other times it can be lonely like triathlon racing or piano playing. In any case, you should try to find some support... a little fan base. That can be your family or close circle of friends. Plan to play a piece for them, and ask them to criticize you. Keep playing, and improving. During the two weeks preceding my first public performance, I asked my wife and children to listen to me play every single evening before bedtime. They were and are my support, my fan base, and those who&#39;ll cheer for me even if I screw up. Find those people!&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Lastly: Prioritize, prioritize, prioritize! You&#39;ll need to prioritize a lot. If you need to take one or two hours of your day, you&#39;ll need to prioritize. Do I need to take that coffee break? Can I optimize my commute? Do I really need to watch that series on Netflix, etc? In the end, we all have 24 hours per day. What matters, is how we spend those 24 hours. There&#39;s no magic!&lt;/p&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;h2&gt;Book references:&lt;/h2&gt;
&lt;ul&gt;
&lt;li&gt;Alfred&#39;s Basic Adult Piano Course : Level 1 (ISBN: 978-0739082416).&lt;/li&gt;
&lt;li&gt;Play it Again: An Amateur Against the Impossible (ISBN: 978-1250875402)&lt;/li&gt;
&lt;li&gt;Journey of a Thousand Miles: My Story by Lang Land and David Ritz (ISBN: 978-0385524575)&lt;/li&gt;
&lt;li&gt;Piano Notes: The World of the Pianist by Charles Rosen (ISBN: 978-0140298635)&lt;/li&gt;
&lt;/ul&gt;
&lt;h2&gt;Music references:&lt;/h2&gt;
&lt;ul&gt;
&lt;li&gt;Allegretto by J. B. Vanhal&lt;/li&gt;
&lt;li&gt;Nächtliche Reise by C. Gurlitt&lt;/li&gt;
&lt;/ul&gt;
&lt;h2&gt;Online Forums:&lt;/h2&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href=&quot;https://www.reddit.com/r/piano&quot;&gt;&lt;code&gt;r/piano&lt;/code&gt;&lt;/a&gt;, take a look especially at the &lt;a href=&quot;https://www.reddit.com/r/piano/wiki/faq/&quot;&gt;FAQ&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href=&quot;https://www.reddit.com/r/pianolearning&quot;&gt;&lt;code&gt;r/pianolearning&lt;/code&gt;&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;h2&gt;Others:&lt;/h2&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href=&quot;https://www.pianistmagazine.com/&quot;&gt;Pianist Magazine&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href=&quot;https://open.spotify.com/show/72OQO63V0y3jZ0fh4tVlwj&quot;&gt;Josh Wright Piano Podcast&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href=&quot;https://instrumentive.com/&quot;&gt;Instrumentive Progress Tracking App&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
</content>
  </entry>
  <entry>
    <title>A tour of the `for..of` implementation for 32bits JSC</title>
    <link href="https://p.ocmatos.com/blog/a-tour-of-the-for-of-implementation-for-32bits-jsc.html"/>
    <updated>2020-11-20T00:00:00Z</updated>
    <id>https://p.ocmatos.com/blog/a-tour-of-the-for-of-implementation-for-32bits-jsc.html</id>
    <content xml:lang="en" type="html">&lt;h2&gt;Context&lt;/h2&gt;
&lt;p&gt;On April 18, 2020, Keith Miller from Apple committed a change titled &lt;a href=&quot;https://trac.webkit.org/changeset/260323/webkit&quot;&gt;Redesign how we do for-of iteration for JSArrays&lt;/a&gt;. This patch adds an intrinsic for the &lt;code&gt;for-of&lt;/code&gt; javascript construct when they are being iterated with the built-in &lt;code&gt;Symbol.iterator&lt;/code&gt;. However, the implementation is only for 64 bit architectures. At &lt;a href=&quot;https://www.igalia.com/&quot;&gt;Igalia&lt;/a&gt;, we maintain and support javascriptCore 32bits so on July 29, 2020, &lt;a href=&quot;https://trac.webkit.org/changeset/265036/webkit&quot;&gt;I committed support&lt;/a&gt; for the &lt;code&gt;for-of&lt;/code&gt; intrinsics work on 32 bit architectures as joint work with &lt;a href=&quot;https://caiolima.github.io/&quot;&gt;Caio Lima&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;In this blog post, I will attempt to give an overview of the implemented changes in revision &lt;a href=&quot;https://trac.webkit.org/changeset/265036/webkit&quot;&gt;r265036&lt;/a&gt; of WebKit which extends support of the &lt;code&gt;for-of&lt;/code&gt; intrinsics work to 32 bits. For the purposes of brevity I will focus on the iterator open intrinsic.&lt;/p&gt;
&lt;h2&gt;Overview&lt;/h2&gt;
&lt;p&gt;JSC consists of several tiers (compilation layers). These tiers start with LLInt (the Low-Level Interpreter), followed by the Baseline JIT, DFG (Data Flow Graph JIT), and FTL (Faster-Than-Light JIT). We will not discuss FTL here since we do not run the FTL tier for 32 bits.&lt;/p&gt;
&lt;p&gt;The patch for &lt;a href=&quot;https://trac.webkit.org/changeset/265036/webkit&quot;&gt;r265036&lt;/a&gt; touches architecture specific files, like &lt;a href=&quot;https://trac.webkit.org/browser/webkit/trunk/Source/javascriptCore/assembler/MacroAssemblerARMv7.h?rev=265036&quot;&gt;MacroAssemblerARMv7.h&lt;/a&gt; and &lt;a href=&quot;https://trac.webkit.org/browser/webkit/trunk/Source/javascriptCore/assembler/MacroAssemblerMIPS.h?rev=265036&quot;&gt;MacroAssemblerMIPS.h&lt;/a&gt;, however these changes are less relevant as we just add a couple of functions for emitting octet load/store instructions (&lt;code&gt;store8&lt;/code&gt;, &lt;code&gt;or8&lt;/code&gt;, etc). Instead, we will focus on changes to the tiers starting with the changes to LLInt.&lt;/p&gt;
&lt;h2&gt;javascript and for-of&lt;/h2&gt;
&lt;p&gt;I guess as Compiler Engineers we should try to understand what we have to implement first. Lets try to use an iterator implicitly through a &lt;a href=&quot;https://developer.mozilla.org/en-US/docs/Web/javascript/Reference/Statements/for...of&quot;&gt;&lt;code&gt;for-of&lt;/code&gt; loop&lt;/a&gt;.&lt;/p&gt;
&lt;pre class=&quot;language-javascript&quot;&gt;&lt;code class=&quot;language-javascript&quot;&gt;&lt;span class=&quot;token keyword&quot;&gt;const&lt;/span&gt; a &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;token string&quot;&gt;&quot;a&quot;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&quot;b&quot;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&quot;c&quot;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
&lt;span class=&quot;token keyword&quot;&gt;for&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token keyword&quot;&gt;const&lt;/span&gt; value &lt;span class=&quot;token keyword&quot;&gt;of&lt;/span&gt; a&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;
  console&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;log&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;value&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
&lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;
&lt;span class=&quot;token comment&quot;&gt;// =&gt;&lt;/span&gt;
&lt;span class=&quot;token comment&quot;&gt;// &quot;a&quot;&lt;/span&gt;
&lt;span class=&quot;token comment&quot;&gt;// &quot;b&quot;&lt;/span&gt;
&lt;span class=&quot;token comment&quot;&gt;// &quot;c&quot;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The &lt;code&gt;for-of&lt;/code&gt; statement gets an iterator from the array behind the scenes and loops through its values assigning them one by one to the variable &lt;code&gt;value&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;If we were to look at using the iterator explicitly, the above loop would be transformed to:&lt;/p&gt;
&lt;pre class=&quot;language-javascript&quot;&gt;&lt;code class=&quot;language-javascript&quot;&gt;&lt;span class=&quot;token keyword&quot;&gt;const&lt;/span&gt; a &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;token string&quot;&gt;&quot;a&quot;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&quot;b&quot;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&quot;c&quot;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
&lt;span class=&quot;token keyword&quot;&gt;const&lt;/span&gt; it &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; a&lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;Symbol&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;iterator&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
&lt;span class=&quot;token keyword&quot;&gt;let&lt;/span&gt; result &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; it&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;next&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
&lt;span class=&quot;token keyword&quot;&gt;while&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;!&lt;/span&gt;result&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;done&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;
  console&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;log&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;result&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;value&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
  result &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; it&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;next&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
&lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;So the symbol &lt;code&gt;Symbol.iterator&lt;/code&gt; returns a method that when called returns an iterator. This iterator has a &lt;code&gt;next&lt;/code&gt; function to get a result object. The result object has a &lt;code&gt;done&lt;/code&gt; property indicating if we are done iterating and a value which is the current value in the iteration. There&#39;s something to keep in mind though. The &lt;code&gt;done&lt;/code&gt; property is optional if its value would be &lt;code&gt;false&lt;/code&gt; and the &lt;code&gt;value&lt;/code&gt; property is optional if its value would be &lt;code&gt;undefined&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;Armed with this knowledge, &lt;a href=&quot;https://trac.webkit.org/changeset/260323/webkit&quot;&gt;revision 260323&lt;/a&gt; which implements &lt;code&gt;for-of&lt;/code&gt; intrinsics for 64 bits, makes a lot more sense. It implements two intrinsics &lt;code&gt;op_iterator_open&lt;/code&gt; and &lt;code&gt;op_iterator_next&lt;/code&gt;. The commit message explains the semantics of each operator as follows:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;The &lt;code&gt;op_iterator_open&lt;/code&gt; bytecode is semantically the same as:&lt;/li&gt;
&lt;/ul&gt;
&lt;pre&gt;&lt;code&gt;iterator = symbolIterator.@call(iterable);
next = iterator.next;
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;where iterable is the rhs of the &lt;code&gt;for-of&lt;/code&gt; and &lt;code&gt;symbolIterator&lt;/code&gt; is the result of running &lt;code&gt;iterable.symbolIterator&lt;/code&gt;;&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;The &lt;code&gt;op_iterator_next&lt;/code&gt; bytecode is semantically the same as:&lt;/li&gt;
&lt;/ul&gt;
&lt;pre&gt;&lt;code&gt;nextResult = next.@call(iterator);
done = nextResult.done;
value = done ? (undefined / bottom) : nextResult.value;
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;where &lt;code&gt;nextResult&lt;/code&gt; is a temporary (the value &lt;code&gt;VirtualRegister&lt;/code&gt; in the LLInt/Baseline and a &lt;code&gt;tmp&lt;/code&gt; in the DFG);&lt;/p&gt;
&lt;p&gt;We can now proceed to understand how to implement these on 32bits.&lt;/p&gt;
&lt;h2&gt;LLInt&lt;/h2&gt;
&lt;p&gt;LLInt is a bytecode interpreter implemented using a DSL (Domain Specific Language). The compiler for it is &lt;a href=&quot;https://github.com/WebKit/webkit/tree/master/Source/javascriptCore/offlineasm&quot;&gt;implemented in Ruby&lt;/a&gt;. The &lt;code&gt;offlineasm&lt;/code&gt; compiler reads this DSL and outputs a bytecode interpreter in native assembler, for a few supported architectures, or cpp. The cpp backend is known as Cloop and if you enable it, to for example compile JSC for an IBM System Z9, the JSC build system generates the cpp interpreter, compiles it and then links it with JSC. If you do enable Cloop, then none of the remaining JIT tiers are available. This is as much as we will discuss about Cloop as both ARMv7 and MIPS have LLInt support, so from here on out we will assume we are working with the native assembly backend for &lt;code&gt;offlineasm&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;The source code for LLInt (input to &lt;code&gt;offlineasm&lt;/code&gt;) lives in &lt;a href=&quot;https://github.com/WebKit/webkit/tree/master/Source/javascriptCore/llint&quot;&gt;&lt;code&gt;WebKit/Source/javascriptCore&lt;/code&gt;&lt;/a&gt;. The entry point is at &lt;a href=&quot;https://github.com/WebKit/webkit/blob/master/Source/javascriptCore/llint/LowLevelInterpreter.asm&quot;&gt;&lt;code&gt;LowLevelInterpreter.asm&lt;/code&gt;&lt;/a&gt; and it starts with a bunch of constant definitions followed by something you&#39;ll see often: a macro.&lt;/p&gt;
&lt;p&gt;At the time of writing, the first one is &lt;a href=&quot;https://github.com/WebKit/webkit/blob/ce2bff038035bac33352237652d33ff795a69959/Source/javascriptCore/llint/LowLevelInterpreter.asm#L319&quot;&gt;&lt;code&gt;nextInstruction&lt;/code&gt;&lt;/a&gt;:&lt;/p&gt;
&lt;pre class=&quot;language-ruby&quot;&gt;&lt;code class=&quot;language-ruby&quot;&gt;macro nextInstruction&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;
    loadb &lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;token constant&quot;&gt;PB&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;token constant&quot;&gt;PC&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; t0
    leap _g_opcodeMap&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; t1
    jmp &lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;t1&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; t0&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; PtrSize&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; BytecodePtrTag
&lt;span class=&quot;token keyword&quot;&gt;end&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Each time you call elsewhere &lt;code&gt;nextInstruction&lt;/code&gt; (which actually starts processing the next bytecode), the call will be replaced by the three (low-level) instructions in the body of the macro. These instructions &lt;code&gt;loadb&lt;/code&gt;, &lt;code&gt;leap&lt;/code&gt; and &lt;code&gt;jmp&lt;/code&gt; are defined on per-architecture in &lt;a href=&quot;https://github.com/WebKit/webkit/blob/master/Source/javascriptCore/offlineasm/arm.rb&quot;&gt;&lt;code&gt;arm.rb&lt;/code&gt;&lt;/a&gt;, &lt;a href=&quot;https://github.com/WebKit/webkit/blob/master/Source/javascriptCore/offlineasm/arm64.rb&quot;&gt;&lt;code&gt;arm64.rb&lt;/code&gt;&lt;/a&gt;, &lt;a href=&quot;https://github.com/WebKit/webkit/blob/master/Source/javascriptCore/offlineasm/arm64e.rb&quot;&gt;&lt;code&gt;arm64e.rb&lt;/code&gt;&lt;/a&gt;, &lt;a href=&quot;https://github.com/WebKit/webkit/blob/master/Source/javascriptCore/offlineasm/mips.rb&quot;&gt;&lt;code&gt;mips.rb&lt;/code&gt;&lt;/a&gt;, and &lt;a href=&quot;https://github.com/WebKit/webkit/blob/master/Source/javascriptCore/offlineasm/x86.rb&quot;&gt;&lt;code&gt;x86.rb&lt;/code&gt;&lt;/a&gt;. Listing of all available instructions are in &lt;a href=&quot;https://github.com/WebKit/webkit/blob/master/Source/javascriptCore/offlineasm/instructions.rb&quot;&gt;&lt;code&gt;instructions.rb&lt;/code&gt;&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;So for 32bits arm:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;the &lt;code&gt;loadb&lt;/code&gt; will generate a &lt;a href=&quot;https://github.com/WebKit/webkit/blob/b63199a0f04a5f7a435cb91e914419d2fd4f50d7/Source/javascriptCore/offlineasm/arm.rb#L447&quot;&gt;&lt;code&gt;ldrb&lt;/code&gt;&lt;/a&gt;,&lt;/li&gt;
&lt;li&gt;the &lt;code&gt;leap&lt;/code&gt; will generate a &lt;a href=&quot;https://github.com/WebKit/webkit/blob/b63199a0f04a5f7a435cb91e914419d2fd4f50d7/Source/javascriptCore/offlineasm/arm.rb#L659&quot;&gt;load effective address&lt;/a&gt; through an &lt;code&gt;add&lt;/code&gt; whose format depends on the first operand of &lt;code&gt;leap&lt;/code&gt;, and&lt;/li&gt;
&lt;li&gt;the &lt;code&gt;jmp&lt;/code&gt; will generate either a &lt;a href=&quot;https://github.com/WebKit/webkit/blob/b63199a0f04a5f7a435cb91e914419d2fd4f50d7/Source/javascriptCore/offlineasm/arm.rb#L597&quot;&gt;&lt;code&gt;b&lt;/code&gt; or &lt;code&gt;mov&lt;/code&gt; instruction&lt;/a&gt; which depends on if the first operand of &lt;code&gt;jmp&lt;/code&gt; is a label.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;The compilation of this DSL to assembler is done during JSC build-time which means that by the time the interpreter is called there&#39;s no execution of ruby code.&lt;/p&gt;
&lt;p&gt;Lets start by implementing the bytecode &lt;code&gt;op_iterator_open&lt;/code&gt;, which before looked like this in &lt;a href=&quot;https://github.com/WebKit/webkit/blob/99cee626c89e192a5fd211b6ba59be68b40c310b/Source/javascriptCore/llint/LowLevelInterpreter32_64.asm#L2645&quot;&gt;&lt;code&gt;LowLevelInterpreter32_64.asm&lt;/code&gt;&lt;/a&gt;:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;llintOp(op_iterator_open, OpIteratorOpen, macro (size, get, dispatch)
    defineOSRExitReturnLabel(op_iterator_open, size)
    break
    if C_LOOP or C_LOOP_WIN
        # Insert superflous call return labels for Cloop.
        cloopCallJSFunction a0 # symbolIterator
        cloopCallJSFunction a0 # get next
    end
end)
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Before proceeding a clarification of what this is. This is essentially a top-level call to the macro &lt;code&gt;llintOp&lt;/code&gt; with three arguments:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;&lt;code&gt;op_iterator_open&lt;/code&gt;,&lt;/li&gt;
&lt;li&gt;&lt;code&gt;OpIteratorOpen&lt;/code&gt;, and&lt;/li&gt;
&lt;li&gt;&lt;code&gt;macro(size, get, dispatch) ... end&lt;/code&gt;&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;Argument &lt;code&gt;3&lt;/code&gt; is an anonymous macro passed as a first-class citizen to &lt;code&gt;llintOp&lt;/code&gt; which can then call it. The macro &lt;code&gt;llintOp&lt;/code&gt; is defined in &lt;a href=&quot;https://github.com/WebKit/webkit/blob/master/Source/javascriptCore/llint/LowLevelInterpreter.asm&quot;&gt;&lt;code&gt;LowLevelInterpreter.asm&lt;/code&gt;&lt;/a&gt; and the declaration looks like:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;macro llintOp(opcodeName, opcodeStruct, fn)
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The anonymous macro defined in the &lt;code&gt;op_iterator_open&lt;/code&gt; is passed to the &lt;code&gt;llintOp&lt;/code&gt; macro and can be called as &lt;code&gt;fn(...)&lt;/code&gt;. Most of the code organization in LLInt is like this and you should get used to reading it.&lt;/p&gt;
&lt;p&gt;As we require access to the metadata for the implementation of &lt;code&gt;op_iterator_open&lt;/code&gt; we change the declaration slightly and add a few commonly defined macros:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;llintOpWithMetadata(op_iterator_open, OpIteratorOpen, macro (size, get, dispatch, metadata, return)
    macro fastNarrow()
        callSlowPath(_iterator_open_try_fast_narrow)
    end
    macro fastWide16()
        callSlowPath(_iterator_open_try_fast_wide16)
    end
    macro fastWide32()
        callSlowPath(_iterator_open_try_fast_wide32)
    end
    size(fastNarrow, fastWide16, fastWide32, macro (callOp) callOp() end)
    bbeq r1, constexpr IterationMode::Generic, .iteratorOpenGeneric
    dispatch()
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;There are two execution paths that can be taken at this point:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;Fast path: this path is currently implemented in cpp and the code resides in &lt;a href=&quot;https://github.com/WebKit/webkit/blob/89ece25c12467354e836b224948db00fc21edf11/Source/javascriptCore/runtime/CommonSlowPaths.cpp#L856&quot;&gt;&lt;code&gt;CommonSlowPaths.cpp:iterator_open_try_fast&lt;/code&gt;&lt;/a&gt;. If it succeeds then the branch &lt;code&gt;bbeq&lt;/code&gt; is not taken and the control flow dispatches to execute the next bytecode instruction.&lt;/li&gt;
&lt;li&gt;Slow path: the generic path is taken if the fast path fails. If it fails, then it &lt;a href=&quot;https://github.com/WebKit/webkit/blob/89ece25c12467354e836b224948db00fc21edf11/Source/javascriptCore/runtime/CommonSlowPaths.cpp#L898&quot;&gt;sets &lt;code&gt;r1&lt;/code&gt; with the value &lt;code&gt;constexpr IterationMode::Generic&lt;/code&gt;&lt;/a&gt;. In the 32bits ABI &lt;code&gt;r1&lt;/code&gt; is the tag of the return value.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;All the code that follows is part of the slow path and the next step is:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;.iteratorOpenGeneric:
    macro gotoGetByIdCheckpoint()
        jmp .getByIdStart
    end

    macro getCallee(dst)
        get(m_symbolIterator, dst)
    end

    macro getArgumentIncludingThisStart(dst)
        getu(size, OpIteratorOpen, m_stackOffset, dst)
    end

    macro getArgumentIncludingThisCount(dst)
        move 1, dst
    end

    callHelper(op_iterator_open,                    # opcodeName
               _llint_slow_path_iterator_open_call, # slowPath
               OpIteratorOpen,                      # opcodeStruct
               m_iteratorProfile,                   # valueProfileName
               m_iterator,                          # dstVirtualRegister
               prepareForRegularCall,               # prepareCall
               size,                                # size
               gotoGetByIdCheckpoint,               # dispatch
               metadata,                            # metadata
               getCallee,                           # getCallee
               getArgumentIncludingThisStart,       # getArgumentStart
               getArgumentIncludingThisCount)       # getArgumentCountIncludingThis
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Here we define a few helper macros required by &lt;code&gt;callHelper&lt;/code&gt;. I have commented on the &lt;code&gt;callHelper&lt;/code&gt; call with all the argument names to ease understanding what the function does. This is a helper function to simplify the call to the method stored in the &lt;code&gt;Symbol.Iterator&lt;/code&gt; property.&lt;/p&gt;
&lt;p&gt;The implementation of the &lt;code&gt;op_iterator_open&lt;/code&gt; in LLInt ends up with:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;.getByIdStart:
	macro storeNextAndDispatch(valueTag, valuePayload)
        move valueTag, t2
        move valuePayload, t3
        get(m_next, t1)
        storei t2, TagOffset[cfr, t1, 8]
        storei t3, PayloadOffset[cfr, t1, 8]
 	    dispatch()
 	end

    # We need to load m_iterator into t3 because that&#39;s where
 	# performGetByIDHelper expects the base object   
 	loadVariable(get, m_iterator, t3, t0, t3)
 	bineq t0, CellTag, .iteratorOpenGenericGetNextSlow
 	performGetByIDHelper(OpIteratorOpen,                  # opcodeStruct
                         m_modeMetadata,                  # modeMetadataName
                         m_nextProfile,                   # valueProfileName
                         .iteratorOpenGenericGetNextSlow, # slowLabel
                         size,                            # size
                         metadata,                        # metadata
                         storeNextAndDispatch)            # return

.iteratorOpenGenericGetNextSlow:
 	callSlowPath(_llint_slow_path_iterator_open_get_next)
 	dispatch()
end)
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The &lt;code&gt;callHelper&lt;/code&gt; above placed an iterator in register &lt;code&gt;t3&lt;/code&gt;, so we check what type of tag the iterator has. If the iterator is not a cell (tagged with &lt;code&gt;CellTag&lt;/code&gt; - a cell is essentially anything that&#39;s heap allocated) then we jump to the slow path. If, however, the iterator is an object we call &lt;code&gt;storeNextAndDispatch&lt;/code&gt; through &lt;code&gt;performGetByIDHelper&lt;/code&gt; which does a few useful checks. &lt;code&gt;storeNextAndDispatch&lt;/code&gt; initializes the iterator and dispatches to the next bytecode.&lt;/p&gt;
&lt;p&gt;The &lt;code&gt;op_iterator_next&lt;/code&gt; will follow the same pattern but instead implement the semantics of &lt;code&gt;iterator.next&lt;/code&gt;. Next we will see how a similar pattern is implemented in the Baseline JIT compiler.&lt;/p&gt;
&lt;h2&gt;Baseline&lt;/h2&gt;
&lt;p&gt;The Baseline JIT source code lives inside &lt;a href=&quot;https://github.com/WebKit/webkit/tree/master/Source/javascriptCore/jit&quot;&gt;Source/javascriptCore/jit&lt;/a&gt;. The part of the JIT compiler the deals with 32 bit platforms, mostly lives in &lt;a href=&quot;https://github.com/WebKit/webkit/blob/master/Source/javascriptCore/jit/JITCall32_64.cpp&quot;&gt;Source/javascriptCore/jit/JITCall32_64.cpp&lt;/a&gt;. It is responsible to generate assembly code for a given bytecode stream as input. This explains the resemblances we have between LLInt operations, since we need to emit code that has the same semantics as the LLInt operation. This compiler applies barely any compiler optimizations, since the goal is to generate binary code as quickly as it can and its major gain on performance is due to the removal of LLInt instruction dispatch.
The Baseline compiler is a heavy user of JSC’s AssemblerHelpers. This is an essential API that’s used to abstract the code generation for the multiple architectures JSC supports and also abstracts some operations that are quite common on LLInt like &lt;code&gt;branchIfNotCell&lt;/code&gt; or &lt;code&gt;branchIfNotEmpty&lt;/code&gt;.  It also implements a set of operations that are very helpful when we need to implement JIT support for a new opcode, such as &lt;code&gt;emitGetVirtualRegister&lt;/code&gt;, &lt;code&gt;emitPutVirtualRegister&lt;/code&gt;, and &lt;code&gt;callOperation&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;Lets look at how we implement the functions &lt;code&gt;emit_op_iterator_open&lt;/code&gt;, &lt;code&gt;emitSlow_op_iterator_open&lt;/code&gt; and the corresponding &lt;code&gt;next&lt;/code&gt; versions of those.&lt;/p&gt;
&lt;pre class=&quot;language-cpp&quot;&gt;&lt;code class=&quot;language-cpp&quot;&gt;&lt;span class=&quot;token keyword&quot;&gt;void&lt;/span&gt; &lt;span class=&quot;token class-name&quot;&gt;JIT&lt;/span&gt;&lt;span class=&quot;token double-colon punctuation&quot;&gt;::&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;emit_op_iterator_open&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token keyword&quot;&gt;const&lt;/span&gt; Instruction&lt;span class=&quot;token operator&quot;&gt;*&lt;/span&gt; instruction&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;
&lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;
    &lt;span class=&quot;token keyword&quot;&gt;auto&lt;/span&gt; bytecode &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; instruction&lt;span class=&quot;token operator&quot;&gt;-&gt;&lt;/span&gt;&lt;span class=&quot;token generic-function&quot;&gt;&lt;span class=&quot;token function&quot;&gt;as&lt;/span&gt;&lt;span class=&quot;token generic class-name&quot;&gt;&lt;span class=&quot;token operator&quot;&gt;&amp;lt;&lt;/span&gt;OpIteratorOpen&lt;span class=&quot;token operator&quot;&gt;&gt;&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
    &lt;span class=&quot;token keyword&quot;&gt;auto&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;*&lt;/span&gt; tryFastFunction &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;&amp;amp;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;
        &lt;span class=&quot;token keyword&quot;&gt;switch&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;instruction&lt;span class=&quot;token operator&quot;&gt;-&gt;&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;width&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;
        &lt;span class=&quot;token keyword&quot;&gt;case&lt;/span&gt; Narrow&lt;span class=&quot;token operator&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;token keyword&quot;&gt;return&lt;/span&gt; iterator_open_try_fast_narrow&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
        &lt;span class=&quot;token keyword&quot;&gt;case&lt;/span&gt; Wide16&lt;span class=&quot;token operator&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;token keyword&quot;&gt;return&lt;/span&gt; iterator_open_try_fast_wide16&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
        &lt;span class=&quot;token keyword&quot;&gt;case&lt;/span&gt; Wide32&lt;span class=&quot;token operator&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;token keyword&quot;&gt;return&lt;/span&gt; iterator_open_try_fast_wide32&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
        &lt;span class=&quot;token keyword&quot;&gt;default&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;RELEASE_ASSERT_NOT_REACHED&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
        &lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;
    &lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;

    JITSlowPathCall &lt;span class=&quot;token function&quot;&gt;slowPathCall&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token keyword&quot;&gt;this&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; instruction&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; tryFastFunction&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
    slowPathCall&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;call&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
    Jump fastCase &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;branch32&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;NotEqual&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; GPRInfo&lt;span class=&quot;token double-colon punctuation&quot;&gt;::&lt;/span&gt;returnValueGPR2&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;TrustedImm32&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token generic-function&quot;&gt;&lt;span class=&quot;token function&quot;&gt;static_cast&lt;/span&gt;&lt;span class=&quot;token generic class-name&quot;&gt;&lt;span class=&quot;token operator&quot;&gt;&amp;lt;&lt;/span&gt;&lt;span class=&quot;token keyword&quot;&gt;uint32_t&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;&gt;&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;IterationMode&lt;span class=&quot;token double-colon punctuation&quot;&gt;::&lt;/span&gt;Generic&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;It is not hard to see some resemblance between this code and the one shown for LLInt&#39;s &lt;code&gt;op_iterator_open&lt;/code&gt;. Once again we see if we can apply the fast case of this iterator and if not, we call the slow path.&lt;/p&gt;
&lt;pre class=&quot;language-cpp&quot;&gt;&lt;code class=&quot;language-cpp&quot;&gt;    &lt;span class=&quot;token generic-function&quot;&gt;&lt;span class=&quot;token function&quot;&gt;compileOpCall&lt;/span&gt;&lt;span class=&quot;token generic class-name&quot;&gt;&lt;span class=&quot;token operator&quot;&gt;&amp;lt;&lt;/span&gt;OpIteratorOpen&lt;span class=&quot;token operator&quot;&gt;&gt;&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;instruction&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; m_callLinkInfoIndex&lt;span class=&quot;token operator&quot;&gt;++&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;

    &lt;span class=&quot;token function&quot;&gt;advanceToNextCheckpoint&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
    
    &lt;span class=&quot;token comment&quot;&gt;// call result (iterator) is in regT1 (tag)/regT0 (payload)&lt;/span&gt;
    &lt;span class=&quot;token keyword&quot;&gt;const&lt;/span&gt; Identifier&lt;span class=&quot;token operator&quot;&gt;*&lt;/span&gt; ident &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;token operator&quot;&gt;&amp;amp;&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;vm&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;propertyNames&lt;span class=&quot;token operator&quot;&gt;-&gt;&lt;/span&gt;next&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
    
    &lt;span class=&quot;token function&quot;&gt;emitJumpSlowCaseIfNotJSCell&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;regT1&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The result of the call to the &lt;code&gt;Symbol.iterator&lt;/code&gt; method is again split, in 32bits machines, between a tag (in &lt;code&gt;regT1&lt;/code&gt;) and a payload (in &lt;code&gt;regT0&lt;/code&gt;). We jump to the slow case if the result is not a cell.&lt;/p&gt;
&lt;pre class=&quot;language-cpp&quot;&gt;&lt;code class=&quot;language-cpp&quot;&gt;    GPRReg tagIteratorGPR &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; regT1&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
    GPRReg payloadIteratorGPR &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; regT0&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;

    GPRReg tagNextGPR &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; tagIteratorGPR&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
    GPRReg payloadNextGPR &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; payloadIteratorGPR&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;

    JITGetByIdGenerator &lt;span class=&quot;token function&quot;&gt;gen&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;
        m_codeBlock&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt;
        &lt;span class=&quot;token function&quot;&gt;CodeOrigin&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;m_bytecodeIndex&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt;
        &lt;span class=&quot;token function&quot;&gt;CallSiteIndex&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;BytecodeIndex&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;m_bytecodeIndex&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;offset&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt;
        &lt;span class=&quot;token class-name&quot;&gt;RegisterSet&lt;/span&gt;&lt;span class=&quot;token double-colon punctuation&quot;&gt;::&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;stubUnavailableRegisters&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt;
        &lt;span class=&quot;token class-name&quot;&gt;CacheableIdentifier&lt;/span&gt;&lt;span class=&quot;token double-colon punctuation&quot;&gt;::&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;createFromImmortalIdentifier&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;ident&lt;span class=&quot;token operator&quot;&gt;-&gt;&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;impl&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt;
        &lt;span class=&quot;token function&quot;&gt;JSValueRegs&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;tagIteratorGPR&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; payloadIteratorGPR&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt;
        &lt;span class=&quot;token function&quot;&gt;JSValueRegs&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;tagNextGPR&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; payloadNextGPR&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt;
        AccessType&lt;span class=&quot;token double-colon punctuation&quot;&gt;::&lt;/span&gt;GetById&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;At this point we generate the access to the &lt;code&gt;next&lt;/code&gt; property of the iterator and ensure it is initialized.&lt;/p&gt;
&lt;pre class=&quot;language-cpp&quot;&gt;&lt;code class=&quot;language-cpp&quot;&gt;    gen&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;generateFastPath&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;*&lt;/span&gt;&lt;span class=&quot;token keyword&quot;&gt;this&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
    &lt;span class=&quot;token function&quot;&gt;addSlowCase&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;gen&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;slowPathJump&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
    m_getByIds&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;append&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;gen&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;

    &lt;span class=&quot;token function&quot;&gt;emitValueProfilingSite&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;bytecode&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;metadata&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;m_codeBlock&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
    &lt;span class=&quot;token function&quot;&gt;emitPutVirtualRegister&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;bytecode&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;m_next&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;JSValueRegs&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;tagNextGPR&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; payloadNextGPR&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;

    fastCase&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;link&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token keyword&quot;&gt;this&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
&lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The function finishes with some boilerplate code for linking and site profiling. However, most importantly it ensures that the values for the next bytecode are put into the correct registers in order for execution to continue. This function also shows a usage of another important kind of helper: branch instructions. Lets refocus just on the important pieces of &lt;code&gt;emit_op_iterator_open&lt;/code&gt; the branch helper usage.&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;void JIT::emit_op_iterator_open(const Instruction* instruction)
{
    // ...
    JITSlowPathCall slowPathCall(this, instruction, tryFastFunction);
    slowPathCall.call();
    Jump fastCase = branch32(NotEqual, GPRInfo::returnValueGPR2, TrustedImm32(static_cast&amp;lt;uint32_t&amp;gt;(IterationMode::Generic)));
    // ...
    fastCase.link(this);
}
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The code above will generate assembly code that branches to the end of &lt;code&gt;emit_op_iterator_open&lt;/code&gt; if &lt;code&gt;returnValueGPR2&lt;/code&gt; is not equal to &lt;code&gt;IterationMode::Generic&lt;/code&gt;. The branch jumps to where we are linking &lt;code&gt;fastCase&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;Another kind of very useful helpers are &lt;code&gt;JITSlowPathCall slowPathCall(this, instruction, tryFastFunction)&lt;/code&gt;, &lt;code&gt;callOperation&lt;/code&gt; and variants of these. That is because they abstract the ABI of each architecture to perform calls to cpp functions. On JSC we have a set of cpp functions that we call &lt;code&gt;JITOperations&lt;/code&gt; and are used by JIT when something is very complex to be done from assembly directly. For example, &lt;code&gt;emit_op_iterator_open&lt;/code&gt; uses these to call on &lt;code&gt;iterator_open_try_fast_narrow&lt;/code&gt;, &lt;code&gt;iterator_open_try_fast_wide16&lt;/code&gt;, or &lt;code&gt;iterator_open_try_fast_wide32&lt;/code&gt;. Also &lt;code&gt;emitSlow_op_iterator_open&lt;/code&gt; calls on &lt;code&gt;callOperation(operationThrowIteratorResultIsNotObject, TrustedImmPtr(m_codeBlock-&amp;gt;globalObject()));&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;When discussing &lt;code&gt;emit_op_iterator_open&lt;/code&gt; in baseline JIT, we saw above the usage of &lt;code&gt;JITGetByIdGenerator&lt;/code&gt; to generate assembly code to access the property &lt;code&gt;next&lt;/code&gt; from the iterator object. This class is a very useful as it encapsulates the Inline Cache (IC) mechanism for property access on JSC. It takes registers for base object where the &lt;code&gt;Get&lt;/code&gt; operation is going to be performed (&lt;code&gt;tagIteratorGPR&lt;/code&gt; and &lt;code&gt;payloadIteratorGPR&lt;/code&gt;) in addition to where the result of such access will be placed (&lt;code&gt;tagNextGPR&lt;/code&gt; and &lt;code&gt;payloadNextGPR&lt;/code&gt;). The code for fast path is generated by &lt;code&gt;gen.generateFastPath(*this);&lt;/code&gt; and since IC is only filled after its execution, the generation of fast path is a set of “noop” that eventually will  be repatched by fast code that accesses cached properties. We also configure &lt;code&gt;slowPathJump&lt;/code&gt; for this generator and this is important because when IC is empty, we always take the slow path to perform the &lt;code&gt;Get&lt;/code&gt; operation and configure the cache for it.&lt;/p&gt;
&lt;p&gt;The following code shows the &lt;code&gt;iterator_open&lt;/code&gt; slow path and where the &lt;code&gt;JITGetByIdGenerator&lt;/code&gt; is linked into when jumping to it.&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;void JIT::emitSlow_op_iterator_open(const Instruction* instruction, Vector&amp;lt;SlowCaseEntry&amp;gt;::iterator&amp;amp; iter)
{
    // …
    linkAllSlowCases(iter);

    GPRReg tagIteratorGPR = regT1;
    GPRReg payloadIteratorGPR = regT0;

    JumpList notObject;
    notObject.append(branchIfNotCell(tagIteratorGPR));
    notObject.append(branchIfNotObject(payloadIteratorGPR));

    auto bytecode = instruction-&amp;gt;as&amp;lt;OpIteratorOpen&amp;gt;();
    VirtualRegister nextVReg = bytecode.m_next;
    UniquedStringImpl* ident = vm().propertyNames-&amp;gt;next.impl();

    JITGetByIdGenerator&amp;amp; gen = m_getByIds[m_getByIdIndex++];
    
    Label coldPathBegin = label();

    Call call = callOperationWithProfile(
        bytecode.metadata(m_codeBlock),                  // metadata
        operationGetByIdOptimize,                        // operation
        nextVReg,                                        // result
        TrustedImmPtr(m_codeBlock-&amp;gt;globalObject()),      // arg1
        gen.stubInfo(),                                  // arg2
        JSValueRegs(tagIteratorGPR, payloadIteratorGPR), // arg3
        CacheableIdentifier::createFromImmortalIdentifier(ident).rawBits()); // arg4
    
    gen.reportSlowPathCall(coldPathBegin, call);
    auto done = jump();

    notObject.link(this);
    callOperation(operationThrowIteratorResultIsNotObject, TrustedImmPtr(m_codeBlock-&amp;gt;globalObject()));

    done.link(this);
    //…
}
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;During the execution the code above, we first perform a type check to guarantee that the iterator is an object - otherwise we need to throw a “Result is not object” exception. Assuming then that iterator is indeed an object, we perform a call to the JIT operation &lt;code&gt;operationGetByIdOptimize&lt;/code&gt;. This operation is responsible to check if the &lt;code&gt;Get&lt;/code&gt; we are performing is cacheable and then buffer those cases to repatch the fast path once we have enough evidence that this is a hot operation.&lt;/p&gt;
&lt;p&gt;The code for those generators lives on &lt;a href=&quot;https://github.com/WebKit/webkit/blob/master/Source/javascriptCore/jit/JITInlineCacheGenerator.h&quot;&gt;&lt;code&gt;JITInlineCacheGenerator.h&lt;/code&gt;&lt;/a&gt; and &lt;a href=&quot;https://github.com/WebKit/webkit/blob/master/Source/javascriptCore/jit/JITInlineCacheGenerator.cpp&quot;&gt;&lt;code&gt;JITInlineCacheGenerator.cpp&lt;/code&gt;&lt;/a&gt;. It includes generators for &lt;code&gt;op_in&lt;/code&gt;, &lt;code&gt;op_put_by_id&lt;/code&gt; and even for &lt;code&gt;op_get_by_val&lt;/code&gt;. These generators are also used by DFG and FTL code, mainly when there’s not enough type information for some opcodes during compile time.&lt;/p&gt;
&lt;p&gt;Lets look at &lt;code&gt;emit_op_iterator_next&lt;/code&gt; in order to understand &lt;code&gt;emitGetVirtualRegister&lt;/code&gt;.&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;void JIT::emit_op_iterator_next(const Instruction* instruction)
{
    //…
    JSValueRegs nextRegs(regT1, regT0);
    emitGetVirtualRegister(bytecode.m_next, nextRegs);
    //…
}
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This operation loads the &lt;code&gt;next&lt;/code&gt; operand into &lt;code&gt;regT1&lt;/code&gt; and &lt;code&gt;regT0&lt;/code&gt;. The call to &lt;code&gt;emitGetVirtualRegister&lt;/code&gt; generates assembly code that loads a &lt;code&gt;JSValue&lt;/code&gt; stored in a &lt;code&gt;VirtualRegister&lt;/code&gt; into the machine register(s) we pass as a parameter. For both LLInt and Baseline JIT, virtual registers are always assigned to stack slots, which means that &lt;code&gt;emitGetVirtualRegister&lt;/code&gt; will generate code that is going to load values from stack into machine registers.&lt;/p&gt;
&lt;p&gt;It is important to notice that &lt;code&gt;regT1&lt;/code&gt; and &lt;code&gt;regT0&lt;/code&gt; are aliases for registers of the architecture we are targetting. We also see the usage of &lt;code&gt;JSValueRegs&lt;/code&gt;, which is an abstraction created to reuse code among 64-bits and 32-bits code. It abstracts the fact that JSValues are 64-bits long, which means that we need 2 registers (tag and payload) to manage a JSValue on 32-bits code.
Following helpers used by &lt;code&gt;emit_op_iterator_open&lt;/code&gt;, we have &lt;code&gt;emitPutVirtualRegister&lt;/code&gt; that is very similar with &lt;code&gt;emitGetVirtualRegister&lt;/code&gt; but it stores a JSValue into a virtual register. A usage of this helper&#39;s counterpart &lt;code&gt;emitPutVirtualRegister&lt;/code&gt; can be seen in the code for &lt;code&gt;emit_op_iterator_open&lt;/code&gt;, which stores a &lt;code&gt;JSValue&lt;/code&gt; into a virtual register.&lt;/p&gt;
&lt;h2&gt;Conclusions&lt;/h2&gt;
&lt;p&gt;The important bits were the implementation of the intrinsics for LLInt and Baseline JIT. A few other code bits went into the patch but they are just ensuring everything works as expected instead of being large changes to the codebase.&lt;/p&gt;
&lt;p&gt;Although the implementation of the intrinsic is essentially the implementation of the same semantics, they require quite different knowledge since they operate at different levels of abstraction. On the one hand, LLInt is written in a DSL that compiles down to assembler and essentially implements a language interpreter. On the other, the Baseline JIT is written in cpp an implements a JIT Compiler. The API between both is completely different which requires the developer to know both in order to implement a single feature. The implementation of this feature for 64bits required further changes to other tiers of JSC, which we didn&#39;t need to touch for 32bits. In 32bits we do not have FTL enabled and DFG is essentially platform agnostic.&lt;/p&gt;
&lt;h2&gt;Thanks&lt;/h2&gt;
&lt;p&gt;My colleague &lt;a href=&quot;https://caiolima.github.io/&quot;&gt;Caio Lima&lt;/a&gt; has reviewed this post and spent a significant amount of time adding technical details where I initially had few. Any inaccuracies or lack of detail however, are still my own fault.&lt;/p&gt;
</content>
  </entry>
  <entry>
    <title>JSC - what are my options?</title>
    <link href="https://p.ocmatos.com/blog/jsc-what-are-my-options.html"/>
    <updated>2020-06-05T00:00:00Z</updated>
    <id>https://p.ocmatos.com/blog/jsc-what-are-my-options.html</id>
    <content xml:lang="en" type="html">&lt;p&gt;Compilers tend to be large pieces of software that provide an enormous amount of options. We take a quick look at how to find what JavaScriptCore (JSC) provides.&lt;/p&gt;
&lt;p&gt;If you grab a copy of the WebKit source, you can easily build &lt;code&gt;jsc&lt;/code&gt; with &lt;code&gt;Tools/Scripts/build-jsc --release --jsc-only&lt;/code&gt; (from the WebKit root directory). As usual, running the resulting binary will get you into a JavaScript REPL. For example (&lt;code&gt;$&lt;/code&gt; indicates the shell prompt):&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ WebKitBuild/Release/bin/jsc
&lt;span class=&quot;token operator&quot;&gt;&gt;&gt;&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;&gt;&lt;/span&gt; &lt;span class=&quot;token number&quot;&gt;2&lt;/span&gt;+2
&lt;span class=&quot;token number&quot;&gt;4&lt;/span&gt;
&lt;span class=&quot;token operator&quot;&gt;&gt;&gt;&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;&gt;&lt;/span&gt; &lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;That was simple enough, but which options does JSC accept? Using &lt;code&gt;--help&lt;/code&gt; will give you some information but if you&#39;re debuggin JSC, you might be interested in &lt;code&gt;--options&lt;/code&gt;.&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ WebKitBuild/Release/bin/jsc &lt;span class=&quot;token parameter variable&quot;&gt;--options&lt;/span&gt;
All JSC runtime options:
   &lt;span class=&quot;token assign-left variable&quot;&gt;useKernTCSM&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;true   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. Note: this needs to go before other options since they depend on this value.
   &lt;span class=&quot;token assign-left variable&quot;&gt;validateOptions&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. crashes &lt;span class=&quot;token keyword&quot;&gt;if&lt;/span&gt; mis-typed JSC options were passed to the VM
   &lt;span class=&quot;token assign-left variable&quot;&gt;dumpOptions&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;0&lt;/span&gt;   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. dumps JSC options &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;0&lt;/span&gt; &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; None, &lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt; &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; Overridden only, &lt;span class=&quot;token number&quot;&gt;2&lt;/span&gt; &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; All, &lt;span class=&quot;token number&quot;&gt;3&lt;/span&gt; &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; Verbose&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;

&lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;.&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;There is a huge amount of options and parameters to tune the execution of JSC, so here are some filtering hints.&lt;/p&gt;
&lt;p&gt;For verbosity related options:&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ WebKitBuild/Release/bin/jsc &lt;span class=&quot;token parameter variable&quot;&gt;--options&lt;/span&gt; &lt;span class=&quot;token operator&quot;&gt;&lt;span class=&quot;token file-descriptor important&quot;&gt;2&lt;/span&gt;&gt;&lt;/span&gt;&lt;span class=&quot;token file-descriptor important&quot;&gt;&amp;amp;1&lt;/span&gt; &lt;span class=&quot;token operator&quot;&gt;|&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;grep&lt;/span&gt; verbose
   &lt;span class=&quot;token assign-left variable&quot;&gt;verboseDFGBytecodeParsing&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false
   &lt;span class=&quot;token assign-left variable&quot;&gt;verboseCompilation&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false
   &lt;span class=&quot;token assign-left variable&quot;&gt;verboseFTLCompilation&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false
   &lt;span class=&quot;token assign-left variable&quot;&gt;verboseValidationFailure&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false
   &lt;span class=&quot;token assign-left variable&quot;&gt;verboseOSR&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false
   &lt;span class=&quot;token assign-left variable&quot;&gt;verboseDFGOSRExit&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false
   &lt;span class=&quot;token assign-left variable&quot;&gt;verboseFTLOSRExit&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false
   &lt;span class=&quot;token assign-left variable&quot;&gt;verboseCallLink&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false
   &lt;span class=&quot;token assign-left variable&quot;&gt;verboseCompilationQueue&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false
   &lt;span class=&quot;token assign-left variable&quot;&gt;verboseExitProfile&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false
   &lt;span class=&quot;token assign-left variable&quot;&gt;verboseCFA&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false
   &lt;span class=&quot;token assign-left variable&quot;&gt;verboseDFGFailure&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false
   &lt;span class=&quot;token assign-left variable&quot;&gt;verboseFTLToJSThunk&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false
   &lt;span class=&quot;token assign-left variable&quot;&gt;verboseFTLFailure&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false
   &lt;span class=&quot;token assign-left variable&quot;&gt;verboseSanitizeStack&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false
   &lt;span class=&quot;token assign-left variable&quot;&gt;verboseVisitRace&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false
   &lt;span class=&quot;token assign-left variable&quot;&gt;verboseExecutableAllocationFuzz&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;For JIT related options:&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ ~/dev/WebKit/WebKitBuild/Debug/bin/jsc &lt;span class=&quot;token parameter variable&quot;&gt;--options&lt;/span&gt; &lt;span class=&quot;token operator&quot;&gt;&lt;span class=&quot;token file-descriptor important&quot;&gt;2&lt;/span&gt;&gt;&lt;/span&gt;&lt;span class=&quot;token file-descriptor important&quot;&gt;&amp;amp;1&lt;/span&gt; &lt;span class=&quot;token operator&quot;&gt;|&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;grep&lt;/span&gt; JIT    
   &lt;span class=&quot;token assign-left variable&quot;&gt;useJIT&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;true   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. allows the executable pages to be allocated &lt;span class=&quot;token keyword&quot;&gt;for&lt;/span&gt; JIT and thunks &lt;span class=&quot;token keyword&quot;&gt;if&lt;/span&gt; &lt;span class=&quot;token boolean&quot;&gt;true&lt;/span&gt;
   &lt;span class=&quot;token assign-left variable&quot;&gt;useBaselineJIT&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;true   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. allows the baseline JIT to be used &lt;span class=&quot;token keyword&quot;&gt;if&lt;/span&gt; &lt;span class=&quot;token boolean&quot;&gt;true&lt;/span&gt;
   &lt;span class=&quot;token assign-left variable&quot;&gt;useDFGJIT&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;true   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. allows the DFG JIT to be used &lt;span class=&quot;token keyword&quot;&gt;if&lt;/span&gt; &lt;span class=&quot;token boolean&quot;&gt;true&lt;/span&gt;
   &lt;span class=&quot;token assign-left variable&quot;&gt;useRegExpJIT&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;true   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. allows the RegExp JIT to be used &lt;span class=&quot;token keyword&quot;&gt;if&lt;/span&gt; &lt;span class=&quot;token boolean&quot;&gt;true&lt;/span&gt;
   &lt;span class=&quot;token assign-left variable&quot;&gt;useDOMJIT&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;true   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. allows the DOMJIT to be used &lt;span class=&quot;token keyword&quot;&gt;if&lt;/span&gt; &lt;span class=&quot;token boolean&quot;&gt;true&lt;/span&gt;
   &lt;span class=&quot;token assign-left variable&quot;&gt;crashIfCantAllocateJITMemory&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false
   &lt;span class=&quot;token assign-left variable&quot;&gt;dumpDisassembly&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. dumps disassembly of all JIT compiled code upon compilation
   &lt;span class=&quot;token assign-left variable&quot;&gt;logJITCodeForPerf&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false
   &lt;span class=&quot;token assign-left variable&quot;&gt;bytecodeRangeToJITCompile&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;&amp;lt;&lt;/span&gt;null&lt;span class=&quot;token operator&quot;&gt;&gt;&lt;/span&gt;   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. bytecode size range to allow compilation on, e.g. &lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt;:100
   &lt;span class=&quot;token assign-left variable&quot;&gt;reportBaselineCompileTimes&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. dumps JS &lt;span class=&quot;token keyword&quot;&gt;function&lt;/span&gt; signature and the &lt;span class=&quot;token function&quot;&gt;time&lt;/span&gt; it took to BaselineJIT compile
   &lt;span class=&quot;token assign-left variable&quot;&gt;useFTLJIT&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;true   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. allows the FTL JIT to be used &lt;span class=&quot;token keyword&quot;&gt;if&lt;/span&gt; &lt;span class=&quot;token boolean&quot;&gt;true&lt;/span&gt;
   &lt;span class=&quot;token assign-left variable&quot;&gt;enableJITDebugAssertions&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;true
   &lt;span class=&quot;token assign-left variable&quot;&gt;useConcurrentJIT&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;true   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. allows the DFG / FTL compilation &lt;span class=&quot;token keyword&quot;&gt;in&lt;/span&gt; threads other than the executing JS thread
   &lt;span class=&quot;token assign-left variable&quot;&gt;jitPolicyScale&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt;   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. scale JIT thresholds to this specified ratio between &lt;span class=&quot;token number&quot;&gt;0.0&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;compile ASAP&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt; and &lt;span class=&quot;token number&quot;&gt;1.0&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;compile like normal&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;.
   &lt;span class=&quot;token assign-left variable&quot;&gt;thresholdForJITAfterWarmUp&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;500&lt;/span&gt;
   &lt;span class=&quot;token assign-left variable&quot;&gt;thresholdForJITSoon&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;100&lt;/span&gt;
   &lt;span class=&quot;token assign-left variable&quot;&gt;forceGCSlowPaths&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. If true, we will force all JIT fast allocations down their slow paths.
   &lt;span class=&quot;token assign-left variable&quot;&gt;alwaysGeneratePCToCodeOriginMap&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. This will &lt;span class=&quot;token function&quot;&gt;make&lt;/span&gt; sure we always generate a PCToCodeOriginMap &lt;span class=&quot;token keyword&quot;&gt;for&lt;/span&gt; JITed code.
   &lt;span class=&quot;token assign-left variable&quot;&gt;useBBQJIT&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;true   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. allows the BBQ JIT to be used &lt;span class=&quot;token keyword&quot;&gt;if&lt;/span&gt; &lt;span class=&quot;token boolean&quot;&gt;true&lt;/span&gt;
   &lt;span class=&quot;token assign-left variable&quot;&gt;useOMGJIT&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;true   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. allows the OMG JIT to be used &lt;span class=&quot;token keyword&quot;&gt;if&lt;/span&gt; &lt;span class=&quot;token boolean&quot;&gt;true&lt;/span&gt;
   &lt;span class=&quot;token assign-left variable&quot;&gt;traceBaselineJITExecution&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false
   &lt;span class=&quot;token assign-left variable&quot;&gt;dumpJITMemoryPath&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token string&quot;&gt;&quot;&quot;&lt;/span&gt;
   &lt;span class=&quot;token assign-left variable&quot;&gt;dumpJITMemoryFlushInterval&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;10&lt;/span&gt;   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. Maximum &lt;span class=&quot;token function&quot;&gt;time&lt;/span&gt; &lt;span class=&quot;token keyword&quot;&gt;in&lt;/span&gt; between flushes of the JIT memory dump &lt;span class=&quot;token keyword&quot;&gt;in&lt;/span&gt; seconds.&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;And lastly for GC related options:&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ ~/dev/WebKit/WebKitBuild/Debug/bin/jsc &lt;span class=&quot;token parameter variable&quot;&gt;--options&lt;/span&gt; &lt;span class=&quot;token operator&quot;&gt;&lt;span class=&quot;token file-descriptor important&quot;&gt;2&lt;/span&gt;&gt;&lt;/span&gt;&lt;span class=&quot;token file-descriptor important&quot;&gt;&amp;amp;1&lt;/span&gt; &lt;span class=&quot;token operator&quot;&gt;|&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;grep&lt;/span&gt; &lt;span class=&quot;token parameter variable&quot;&gt;-i&lt;/span&gt; gc
   &lt;span class=&quot;token assign-left variable&quot;&gt;repatchBufferingCountdown&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;8&lt;/span&gt;
   &lt;span class=&quot;token assign-left variable&quot;&gt;bytecodeRangeToDFGCompile&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;&amp;lt;&lt;/span&gt;null&lt;span class=&quot;token operator&quot;&gt;&gt;&lt;/span&gt;   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. bytecode size range to allow DFG compilation on, e.g. &lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt;:100
   &lt;span class=&quot;token assign-left variable&quot;&gt;logCompilationChanges&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false
   &lt;span class=&quot;token assign-left variable&quot;&gt;validateDoesGC&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;true
   &lt;span class=&quot;token assign-left variable&quot;&gt;reportDFGCompileTimes&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. dumps JS &lt;span class=&quot;token keyword&quot;&gt;function&lt;/span&gt; signature and the &lt;span class=&quot;token function&quot;&gt;time&lt;/span&gt; it took to DFG and FTL compile
   &lt;span class=&quot;token assign-left variable&quot;&gt;useGenerationalGC&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;true
   &lt;span class=&quot;token assign-left variable&quot;&gt;useConcurrentGC&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;true
   &lt;span class=&quot;token assign-left variable&quot;&gt;criticalGCMemoryThreshold&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;0.8&lt;/span&gt;   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. percent memory &lt;span class=&quot;token keyword&quot;&gt;in&lt;/span&gt; use the GC considers critical.  The collector is much &lt;span class=&quot;token function&quot;&gt;more&lt;/span&gt; aggressive above this threshold
   &lt;span class=&quot;token assign-left variable&quot;&gt;concurrentGCMaxHeadroom&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;1.5&lt;/span&gt;
   &lt;span class=&quot;token assign-left variable&quot;&gt;concurrentGCPeriodMS&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;2&lt;/span&gt;
   &lt;span class=&quot;token assign-left variable&quot;&gt;minimumGCPauseMS&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;0.3&lt;/span&gt;
   &lt;span class=&quot;token assign-left variable&quot;&gt;gcPauseScale&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;0.3&lt;/span&gt;
   &lt;span class=&quot;token assign-left variable&quot;&gt;gcIncrementBytes&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;10000&lt;/span&gt;
   &lt;span class=&quot;token assign-left variable&quot;&gt;gcIncrementMaxBytes&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;100000&lt;/span&gt;
   &lt;span class=&quot;token assign-left variable&quot;&gt;gcIncrementScale&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;0&lt;/span&gt;
   &lt;span class=&quot;token assign-left variable&quot;&gt;numberOfDFGCompilerThreads&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;2&lt;/span&gt;
   &lt;span class=&quot;token assign-left variable&quot;&gt;priorityDeltaOfDFGCompilerThreads&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;0&lt;/span&gt;
   &lt;span class=&quot;token assign-left variable&quot;&gt;maximumInliningCallerBytecodeCost&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;10000&lt;/span&gt;
   &lt;span class=&quot;token assign-left variable&quot;&gt;numberOfGCMarkers&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;8&lt;/span&gt;
   &lt;span class=&quot;token assign-left variable&quot;&gt;useParallelMarkingConstraintSolver&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;true
   &lt;span class=&quot;token assign-left variable&quot;&gt;slowPathAllocsBetweenGCs&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;0&lt;/span&gt;   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. force a GC on every Nth slow path alloc, where N is specified by this option
   &lt;span class=&quot;token assign-left variable&quot;&gt;sweepSynchronously&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. debugging option to sweep all dead objects synchronously at GC end before resuming mutator
   &lt;span class=&quot;token assign-left variable&quot;&gt;logGC&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;None   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. debugging option to log GC activity &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;0&lt;/span&gt; &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; None, &lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt; &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; Basic, &lt;span class=&quot;token number&quot;&gt;2&lt;/span&gt; &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; Verbose&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;
   &lt;span class=&quot;token assign-left variable&quot;&gt;useGC&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;true
   &lt;span class=&quot;token assign-left variable&quot;&gt;gcAtEnd&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. If true, the jsc CLI will &lt;span class=&quot;token keyword&quot;&gt;do&lt;/span&gt; a GC before exiting
   &lt;span class=&quot;token assign-left variable&quot;&gt;forceGCSlowPaths&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. If true, we will force all JIT fast allocations down their slow paths.
   &lt;span class=&quot;token assign-left variable&quot;&gt;forceDidDeferGCWork&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. If true, we will force all DeferGC destructions to perform a GC.
   &lt;span class=&quot;token assign-left variable&quot;&gt;gcMaxHeapSize&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;0&lt;/span&gt;
   &lt;span class=&quot;token assign-left variable&quot;&gt;recordGCPauseTimes&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false
   &lt;span class=&quot;token assign-left variable&quot;&gt;numberOfGCCyclesToRecordForVerification&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;3&lt;/span&gt;
   &lt;span class=&quot;token assign-left variable&quot;&gt;validateDFGClobberize&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. Emits extra validation code &lt;span class=&quot;token keyword&quot;&gt;in&lt;/span&gt; the DFG/FTL &lt;span class=&quot;token keyword&quot;&gt;for&lt;/span&gt; the Clobberize phase&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The options that are relevant to your use case depend on what you want to do, but my hope is that you have already found something useful here or that you now know how to find what you need. Good luck!&lt;/p&gt;
</content>
  </entry>
  <entry>
    <title>Announcing HTML examples (`html-examples`)</title>
    <link href="https://p.ocmatos.com/blog/announcing-html-examples-html-examples.html"/>
    <updated>2020-06-01T00:00:00Z</updated>
    <id>https://p.ocmatos.com/blog/announcing-html-examples-html-examples.html</id>
    <content xml:lang="en" type="html">&lt;p&gt;Announcing a new racket package &lt;a href=&quot;https://pkgs.racket-lang.org/package/html-examples&quot;&gt;&lt;code&gt;html-examples&lt;/code&gt;&lt;/a&gt; to convert an &lt;a href=&quot;https://docs.racket-lang.org/scribble/eval.html#(form._((lib._scribble%2Fexample..rkt)._examples))&quot;&gt;&lt;code&gt;examples&lt;/code&gt;&lt;/a&gt; block to HTML for inclusion in your own page.&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ raco pkg &lt;span class=&quot;token function&quot;&gt;install&lt;/span&gt; html-examples&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;a href=&quot;https://pkgs.racket-lang.org/package/html-examples&quot;&gt;&lt;code&gt;html-examples&lt;/code&gt;&lt;/a&gt; is a simple, one function, package to convert an &lt;a href=&quot;https://docs.racket-lang.org/scribble/eval.html&quot;&gt;&lt;code&gt;scribble/example&lt;/code&gt;&lt;/a&gt; block into HTML. I tried to find a pre-existing function to do this but all I could do was to get scribble to generate a whole page together with a bunch of assets for the page to render properly. Therefore I created this package which fishes out the correct &lt;code&gt;div&lt;/code&gt; from the page generated by &lt;code&gt;scribble&lt;/code&gt;, so you can typeset beautiful examples on your web page like:&lt;/p&gt;
&lt;p&gt;As usual, documentation can be &lt;a href=&quot;https://docs.racket-lang.org/html-examples/index.html&quot;&gt;found online&lt;/a&gt; and code can be found in the &lt;a href=&quot;https://github.com/pmatos/html-examples&quot;&gt;project&#39;s page on GitHub&lt;/a&gt; where I am happy to receive any bug reports and feature requests you might have.&lt;/p&gt;
</content>
  </entry>
  <entry>
    <title>Announcing Racket Bitvectors (`bv`)</title>
    <link href="https://p.ocmatos.com/blog/announcing-racket-bitvectors-bv.html"/>
    <updated>2020-05-31T00:00:00Z</updated>
    <id>https://p.ocmatos.com/blog/announcing-racket-bitvectors-bv.html</id>
    <content xml:lang="en" type="html">&lt;p&gt;Announcing a new racket package &lt;a href=&quot;https://pkgs.racket-lang.org/package/bv&quot;&gt;&lt;code&gt;bv&lt;/code&gt;&lt;/a&gt; for manipulating bitvectors with a &lt;code&gt;rosette&lt;/code&gt;-style API.&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ raco pkg &lt;span class=&quot;token function&quot;&gt;install&lt;/span&gt; bv&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Racket &lt;a href=&quot;https://pkgs.racket-lang.org/package/bv&quot;&gt;&lt;code&gt;bv&lt;/code&gt;&lt;/a&gt; (&lt;a href=&quot;https://github.com/pmatos/racket-bv&quot;&gt;source&lt;/a&gt; and &lt;a href=&quot;https://docs.racket-lang.org/bv/index.html&quot;&gt;documentation&lt;/a&gt;) is a package for bitvector manipulation. The interface follows the names and conventions defined by &lt;a href=&quot;https://pkgs.racket-lang.org/package/rosette&quot;&gt;&lt;code&gt;rosette&lt;/code&gt;&lt;/a&gt; and its long-term goal is to be a high-performance bitvector library.&lt;/p&gt;
&lt;p&gt;I am happy to receive any bug reports and feature requests you might have in the &lt;a href=&quot;https://github.com/pmatos/racket-bv/issues/new&quot;&gt;project&#39;s GitHub page&lt;/a&gt;.&lt;/p&gt;
&lt;h1&gt;Why?&lt;/h1&gt;
&lt;p&gt;This package release has been extremely overdue. It was first developed in 2017 and much improved through 2018 as part of &lt;a href=&quot;https://linki.tools/s10.html&quot;&gt;s10&lt;/a&gt;, it hasn&#39;t seen the light of open source until now. Partially because it was intertwined with a lot of other code and partially because I never bothered to sit down and write some documentation.&lt;/p&gt;
&lt;p&gt;I am slowly untangling &lt;a href=&quot;https://p.ocmatos.com/s10.html&quot;&gt;s10&lt;/a&gt; into libraries and documenting them and hope to release them as open source sooner or later. The reason the library has a &lt;code&gt;rosette&lt;/code&gt; API because I needed a fast bitvector library with the same API rosette uses due to constraints on how I engineered the equivalence solvers and search techniques within &lt;a href=&quot;https://p.ocmatos.com/s10.html&quot;&gt;s10&lt;/a&gt; however, I think it turned out to be a good decision in any case because the API is intuitive and it has great naming for functions operating on bitvectors.&lt;/p&gt;
</content>
  </entry>
  <entry>
    <title>CReduce - it&#39;s a kind of magic!</title>
    <link href="https://p.ocmatos.com/blog/creduce-its-a-kind-of-magic.html"/>
    <updated>2020-02-17T00:00:00Z</updated>
    <id>https://p.ocmatos.com/blog/creduce-its-a-kind-of-magic.html</id>
    <content xml:lang="en" type="html">&lt;p&gt;When I started compiler work &lt;a href=&quot;http://delta.tigris.org/&quot;&gt;delta&lt;/a&gt; was all the rage. It would turn a testcase with thousands of lines into a single small one with perhaps no more than 15 crashing your compiler in exactly the same way as the original testcase. I have used and abused this tool but it turns out it could be improved. John Regehr and his team did so with &lt;a href=&quot;https://embed.cs.utah.edu/creduce/&quot;&gt;C-Reduce&lt;/a&gt; (who by he way also developed &lt;a href=&quot;https://embed.cs.utah.edu/csmith/&quot;&gt;C-Smith&lt;/a&gt;, but that&#39;s another store entirely).&lt;/p&gt;
&lt;p&gt;C-Reduce takes things to another level squeezing that testcase ever farther until it&#39;s just a few bytes, and yet still crashing your compiler in the same interesting way it did before. In C and C++ this is a very important tool and its use is not &lt;a href=&quot;https://gcc.gnu.org/wiki/A_guide_to_testcase_reduction&quot;&gt;recommended&lt;/a&gt; often enough. It&#39;s very important and in languages like C and C++ because the preprocessor can a seemingly small example into a behemoth of a source file and therefore reducing the testcase appropriately is extremely important. The recommendation page for &lt;a href=&quot;https://gcc.gnu.org/bugs/&quot;&gt;reporting GCC bugs&lt;/a&gt; goes as far as saying:&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;However, providing a minimal testcase increases the chances of
getting your bug fixed. The only excuses to not send us the
preprocessed sources are (i) if you&#39;ve found a bug in the
preprocessor, (ii) if you&#39;ve reduced the testcase to a small file
that doesn&#39;t include any other file or (iii) if the bug appears only
when using precompiled headers.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;However, one advantage of &lt;code&gt;delta&lt;/code&gt; was that it worked on any text file as opposed to C-Reduce that due its reduction strategy required the source to be C or C++. That is, until I was told I was wrong and that C-Reduce would be able to eat a large &lt;a href=&quot;https://racket-lang.org/&quot;&gt;Racket&lt;/a&gt; and spit out a small one. This stayed in the back of my head until present day.&lt;/p&gt;
&lt;p&gt;Currently I am working on JavaScriptCore, which turns out to have bugs of its own as well. Due to the way these JIT compilers are designed you can go through several compiler tiers, code generations, on stack
replacements and back until you find your bug. How great would it be if you could launch your compiler and the bug would reveal itself straightaway without having to wait for a breakpoint to be hit 60 times, step through 20 machine instructions and two OSR calls?&lt;/p&gt;
&lt;p&gt;Well, that&#39;s what C-Reduce is here for. I remembered being told it worked with Racket and if it works with Racket it must work with JavaScript.&lt;/p&gt;
&lt;p&gt;I develop on a &lt;code&gt;x86_64&lt;/code&gt; but my crash is on &lt;code&gt;MIPS32el&lt;/code&gt; and I can reproduce it on &lt;code&gt;qemu-user&lt;/code&gt; like this:&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ /home/pmatos/roots/br-root.jsc32-mips/host/bin/qemu-mipsel &lt;span class=&quot;token parameter variable&quot;&gt;-L&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;&#92;&lt;/span&gt;
	/home/pmatos/roots/br-root.jsc32-mips/target ./WebKitBuild/Debug/bin/jsc &lt;span class=&quot;token punctuation&quot;&gt;&#92;&lt;/span&gt;
	load-varargs-then-inlined-call-and-exit.js 
qemu: uncaught target signal &lt;span class=&quot;token number&quot;&gt;4&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;Illegal instruction&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt; - core dumped&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The idea here is to reduce &lt;code&gt;load-varargs-then-inlined-call-and-exit.js&lt;/code&gt; but keep the &lt;code&gt;Illegal instruction&lt;/code&gt; error so I developed the simple reducing script:&lt;/p&gt;
&lt;pre class=&quot;language-bash&quot;&gt;&lt;code class=&quot;language-bash&quot;&gt;&lt;span class=&quot;token shebang important&quot;&gt;#!/bin/bash&lt;/span&gt;
/home/pmatos/roots/br-root.jsc32-mips/host/bin/qemu-mipsel &lt;span class=&quot;token parameter variable&quot;&gt;-L&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;&#92;&lt;/span&gt;
	/home/pmatos/roots/br-root.jsc32-mips/target /home/pmatos/dev/WebKit/WebKitBuild/Debug/bin/jsc &lt;span class=&quot;token punctuation&quot;&gt;&#92;&lt;/span&gt;
	load-varargs-then-inlined-call-and-exit.js &lt;span class=&quot;token operator&quot;&gt;&amp;amp;&gt;&lt;/span&gt; err.txt
&lt;span class=&quot;token function&quot;&gt;grep&lt;/span&gt; &lt;span class=&quot;token parameter variable&quot;&gt;-q&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&#39;Illegal instruction&#39;&lt;/span&gt; err.txt&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This simply redirects all output from &lt;code&gt;jsc&lt;/code&gt; into a file and checks that the file still contains the error &lt;code&gt;Illegal instruction&lt;/code&gt;. This file can be as complex as you want to constrain the error you wish to reproduce. For our present situation this is enough.&lt;/p&gt;
&lt;p&gt;Then, call C-Reduce:&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ creduce --not-c &lt;span class=&quot;token parameter variable&quot;&gt;--n&lt;/span&gt; &lt;span class=&quot;token number&quot;&gt;16&lt;/span&gt; ./red.sh load-varargs-then-inlined-call-and-exit.js
&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;&amp;lt;&lt;/span&gt; &lt;span class=&quot;token number&quot;&gt;26334&lt;/span&gt; &lt;span class=&quot;token operator&quot;&gt;&gt;=&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;
running &lt;span class=&quot;token number&quot;&gt;16&lt;/span&gt; interestingness tests &lt;span class=&quot;token keyword&quot;&gt;in&lt;/span&gt; parallel
&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;&amp;lt;&lt;/span&gt; pass_blank :: &lt;span class=&quot;token number&quot;&gt;0&lt;/span&gt; &lt;span class=&quot;token operator&quot;&gt;&gt;=&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;
&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;0.6&lt;/span&gt; %, &lt;span class=&quot;token number&quot;&gt;1046&lt;/span&gt; bytes&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;
&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;&amp;lt;&lt;/span&gt; pass_lines :: &lt;span class=&quot;token number&quot;&gt;0&lt;/span&gt; &lt;span class=&quot;token operator&quot;&gt;&gt;=&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;
&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;-0.3 %, &lt;span class=&quot;token number&quot;&gt;1055&lt;/span&gt; bytes&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;
&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;1.2&lt;/span&gt; %, &lt;span class=&quot;token number&quot;&gt;1039&lt;/span&gt; bytes&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;
&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;&amp;lt;&lt;/span&gt; pass_lines :: &lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt; &lt;span class=&quot;token operator&quot;&gt;&gt;=&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;
&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;-1.9 %, &lt;span class=&quot;token number&quot;&gt;1072&lt;/span&gt; bytes&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;
&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;17.6&lt;/span&gt; %, &lt;span class=&quot;token number&quot;&gt;867&lt;/span&gt; bytes&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;
&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;20.9&lt;/span&gt; %, &lt;span class=&quot;token number&quot;&gt;832&lt;/span&gt; bytes&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;
&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;26.5&lt;/span&gt; %, &lt;span class=&quot;token number&quot;&gt;773&lt;/span&gt; bytes&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;
&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;34.8&lt;/span&gt; %, &lt;span class=&quot;token number&quot;&gt;686&lt;/span&gt; bytes&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;
&lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;.&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;There are many arguments possible to creduce (which you can check with &lt;code&gt;creduce --help&lt;/code&gt;. Here I used:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;code&gt;--not-c&lt;/code&gt;: disable C/C++ specific passes, use for source languages that are, well, not C-like;&lt;/li&gt;
&lt;li&gt;&lt;code&gt;--n 16&lt;/code&gt;: use 16 cores to do the reduction, the more the merrier. If you interestingness script takes a long time to run, grab yourself a coffee;&lt;/li&gt;
&lt;li&gt;&lt;code&gt;./red.sh&lt;/code&gt;: path to the interestingness script (make sure it&#39;s executable);&lt;/li&gt;
&lt;li&gt;&lt;code&gt;load-varargs-then-inlined-call-and-exit.js&lt;/code&gt;: testcase to reduce (this is going to be modified in-place);&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;After a while you&#39;ll be greeted with:&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;token keyword&quot;&gt;done&lt;/span&gt; &lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;

pass statistics:
  method pass_balanced :: curly2 worked &lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt; and failed &lt;span class=&quot;token number&quot;&gt;18&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt;
  method pass_balanced :: curly-inside worked &lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt; and failed &lt;span class=&quot;token number&quot;&gt;12&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt;
  method pass_clex :: rm-toks-11 worked &lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt; and failed &lt;span class=&quot;token number&quot;&gt;105&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt;
  method pass_balanced :: parens-to-zero worked &lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt; and failed &lt;span class=&quot;token number&quot;&gt;25&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt;
  method pass_blank :: &lt;span class=&quot;token number&quot;&gt;0&lt;/span&gt; worked &lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt; and failed &lt;span class=&quot;token number&quot;&gt;0&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt;
  method pass_clex :: rename-toks worked &lt;span class=&quot;token number&quot;&gt;2&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt; and failed &lt;span class=&quot;token number&quot;&gt;4&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt;
  method pass_lines :: &lt;span class=&quot;token number&quot;&gt;8&lt;/span&gt; worked &lt;span class=&quot;token number&quot;&gt;2&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt; and failed &lt;span class=&quot;token number&quot;&gt;123&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt;
  method pass_lines :: &lt;span class=&quot;token number&quot;&gt;4&lt;/span&gt; worked &lt;span class=&quot;token number&quot;&gt;2&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt; and failed &lt;span class=&quot;token number&quot;&gt;123&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt;
  method pass_lines :: &lt;span class=&quot;token number&quot;&gt;10&lt;/span&gt; worked &lt;span class=&quot;token number&quot;&gt;2&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt; and failed &lt;span class=&quot;token number&quot;&gt;123&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt;
  method pass_lines :: &lt;span class=&quot;token number&quot;&gt;3&lt;/span&gt; worked &lt;span class=&quot;token number&quot;&gt;2&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt; and failed &lt;span class=&quot;token number&quot;&gt;123&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt;
  method pass_indent :: regular worked &lt;span class=&quot;token number&quot;&gt;2&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt; and failed &lt;span class=&quot;token number&quot;&gt;0&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt;
  method pass_lines :: &lt;span class=&quot;token number&quot;&gt;6&lt;/span&gt; worked &lt;span class=&quot;token number&quot;&gt;2&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt; and failed &lt;span class=&quot;token number&quot;&gt;123&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt;
  method pass_lines :: &lt;span class=&quot;token number&quot;&gt;2&lt;/span&gt; worked &lt;span class=&quot;token number&quot;&gt;5&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt; and failed &lt;span class=&quot;token number&quot;&gt;134&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt;
  method pass_clex :: rm-tok-pattern-4 worked &lt;span class=&quot;token number&quot;&gt;5&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt; and failed &lt;span class=&quot;token number&quot;&gt;784&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt;
  method pass_lines :: &lt;span class=&quot;token number&quot;&gt;0&lt;/span&gt; worked &lt;span class=&quot;token number&quot;&gt;5&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt; and failed &lt;span class=&quot;token number&quot;&gt;70&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt;
  method pass_balanced :: parens-inside worked &lt;span class=&quot;token number&quot;&gt;6&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt; and failed &lt;span class=&quot;token number&quot;&gt;7&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt;
  method pass_clex :: rm-toks-1 worked &lt;span class=&quot;token number&quot;&gt;7&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt; and failed &lt;span class=&quot;token number&quot;&gt;132&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt;
  method pass_lines :: &lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt; worked &lt;span class=&quot;token number&quot;&gt;7&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt; and failed &lt;span class=&quot;token number&quot;&gt;147&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt;
  method pass_clex :: rm-toks-2 worked &lt;span class=&quot;token number&quot;&gt;8&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt; and failed &lt;span class=&quot;token number&quot;&gt;116&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt;

          ******** /home/pmatos/dev/WebKit/load-varargs-then-inlined-call-and-exit.js ********

&lt;span class=&quot;token keyword&quot;&gt;function&lt;/span&gt; b&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;a&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;
  &lt;span class=&quot;token builtin class-name&quot;&gt;return&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt; a: a + &lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;
&lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;
&lt;span class=&quot;token keyword&quot;&gt;function&lt;/span&gt; &lt;span class=&quot;token function-name function&quot;&gt;bar&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt; b.apply&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;this, array&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;
&lt;span class=&quot;token keyword&quot;&gt;function&lt;/span&gt; &lt;span class=&quot;token function-name function&quot;&gt;c&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;
  bar&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;
  c&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;
&lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;
array &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt; &lt;span class=&quot;token number&quot;&gt;2147483647&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt;
c&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Your test file now contains a much shorter testcase but will still crash your compiler in the same beautiful way.&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ /home/pmatos/roots/br-root.jsc32-mips/host/bin/qemu-mipsel &lt;span class=&quot;token parameter variable&quot;&gt;-L&lt;/span&gt; /home/pmatos/roots/br-root.jsc32-mips/target ./WebKitBuild/Debug/bin/jsc load-varargs-then-inlined-call-and-exit.js 
qemu: uncaught target signal &lt;span class=&quot;token number&quot;&gt;4&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;Illegal instruction&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt; - core dumped&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;For comparison the original file had 41 lines and 1052 bytes and the reduced file has 9 lines and 137 bytes which is an 87% reduction in size. But the big improvement here is not that the file is just smaller but that the number of paths taken by &lt;code&gt;jsc&lt;/code&gt; is much smaller taking me strange to the illegal instruction without JIT compiling functions I don&#39;t need or jumping through tiers I don&#39;t care.&lt;/p&gt;
&lt;p&gt;Of course, this would work just as well in a non cross-platform environment. I just used &lt;code&gt;qemu-user&lt;/code&gt; due to the nature of my bug. The important thing is to have a testcase, and a script to ensure that at each reduction step, you keep the interestingness of the testcase - in other words, a way to reproduce the failure.&lt;/p&gt;
&lt;p&gt;Here are a few examples I can think of as a script for reduction:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;code&gt;jsc&lt;/code&gt; crashes with a specific exit code so you create a script testing that specific exit code;&lt;/li&gt;
&lt;li&gt;&lt;code&gt;jsc&lt;/code&gt; crashes with a specific output (above was &lt;code&gt;Illegal instruction&lt;/code&gt;) but it could be anything else;&lt;/li&gt;
&lt;li&gt;&lt;code&gt;jsc&lt;/code&gt; does not crash but generates some strange JIT code which you noticed by using &lt;code&gt;--dumpDisassembly=true&lt;/code&gt;. Well, ensure in your interestingness script the exit code of &lt;code&gt;jsc&lt;/code&gt; is 0, redirect the output to a file and parse for that specific JIT code block;&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Most cases could probably be reduced (pun-intented) to the above three items. You reduce for a specific exit code, a specific error, or a specific block of text. Or a combination of these.&lt;/p&gt;
&lt;p&gt;In any case, if you implement your reducing script appropriately the result will be a testcase smaller than the original that passes your interestingness script - and C-Reduce will look like it is a kind of magic.&lt;/p&gt;
</content>
  </entry>
  <entry>
    <title>Cross-Arch Reproducibility using Containers</title>
    <link href="https://p.ocmatos.com/blog/cross-arch-reproducibility-using-containers.html"/>
    <updated>2020-01-16T00:00:00Z</updated>
    <id>https://p.ocmatos.com/blog/cross-arch-reproducibility-using-containers.html</id>
    <content xml:lang="en" type="html">&lt;h2&gt;Introduction&lt;/h2&gt;
&lt;p&gt;Part of my work for Igalia is on the 32bit support on MIPS (little endian) and ARM for JSC (JavaScriptCore - the JavaScript compiler in WebKit) and one of the problems we face is that of reproducibility of failures.&lt;/p&gt;
&lt;p&gt;We have boards to test these, namely Raspberry Pi 3 Model B+ boards, running a 32bits ARM kernel and Imagination CI20 boards running a mipsel kernel - all built with buildroot which provides images for these boards out-of-the-box. However, we don&#39;t work on these - most of us have higher performance x86_64 machines and whenever a failure occurs upstream it&#39;s generally time consuming to reproduce.&lt;/p&gt;
&lt;p&gt;I have therefore set out to create an environment where we can easily reproduce cross-architectural failures on JSC. I had worked on similar issues for &lt;a href=&quot;https://racket-lang.org/&quot;&gt;Racket&lt;/a&gt;, when building the cross-architectural Racket chroot environment on GitLab CI.&lt;/p&gt;
&lt;h2&gt;Starting with chroot&lt;/h2&gt;
&lt;p&gt;Let&#39;s start the discussion by talking about &lt;code&gt;chroot&lt;/code&gt;. In any linux system you&#39;ll find a &lt;code&gt;chroot&lt;/code&gt; binary.&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ &lt;span class=&quot;token function&quot;&gt;which&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;chroot&lt;/span&gt;
/usr/sbin/chroot
$ &lt;span class=&quot;token function&quot;&gt;chroot&lt;/span&gt; &lt;span class=&quot;token parameter variable&quot;&gt;--help&lt;/span&gt;
Usage: &lt;span class=&quot;token function&quot;&gt;chroot&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;OPTION&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt; NEWROOT &lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;COMMAND &lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;ARG&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;.&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt;
  or:  &lt;span class=&quot;token function&quot;&gt;chroot&lt;/span&gt; OPTION
Run COMMAND with root directory &lt;span class=&quot;token builtin class-name&quot;&gt;set&lt;/span&gt; to NEWROOT.

  &lt;span class=&quot;token parameter variable&quot;&gt;--groups&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;G_LIST        specify supplementary &lt;span class=&quot;token function&quot;&gt;groups&lt;/span&gt; as g1,g2,&lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;,gN
  &lt;span class=&quot;token parameter variable&quot;&gt;--userspec&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token environment constant&quot;&gt;USER&lt;/span&gt;:GROUP  specify user and group &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;ID or name&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt; to use
  --skip-chdir           &lt;span class=&quot;token keyword&quot;&gt;do&lt;/span&gt; not change working directory to &lt;span class=&quot;token string&quot;&gt;&#39;/&#39;&lt;/span&gt;
      &lt;span class=&quot;token parameter variable&quot;&gt;--help&lt;/span&gt;     display this &lt;span class=&quot;token builtin class-name&quot;&gt;help&lt;/span&gt; and &lt;span class=&quot;token builtin class-name&quot;&gt;exit&lt;/span&gt;
      &lt;span class=&quot;token parameter variable&quot;&gt;--version&lt;/span&gt;  output version information and &lt;span class=&quot;token builtin class-name&quot;&gt;exit&lt;/span&gt;

If no &lt;span class=&quot;token builtin class-name&quot;&gt;command&lt;/span&gt; is given, run &lt;span class=&quot;token string&quot;&gt;&#39;&quot;$SHELL&quot; -i&#39;&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;default: &lt;span class=&quot;token string&quot;&gt;&#39;/bin/sh -i&#39;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;.

GNU coreutils online help: &lt;span class=&quot;token operator&quot;&gt;&amp;lt;&lt;/span&gt;https://www.gnu.org/software/coreutils/&lt;span class=&quot;token operator&quot;&gt;&gt;&lt;/span&gt;
Full documentation at: &lt;span class=&quot;token operator&quot;&gt;&amp;lt;&lt;/span&gt;https://www.gnu.org/software/coreutils/chroot&lt;span class=&quot;token operator&quot;&gt;&gt;&lt;/span&gt;
or available locally via: info &lt;span class=&quot;token string&quot;&gt;&#39;(coreutils) chroot invocation&#39;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;However, at a slightly lower level, &lt;a href=&quot;http://man7.org/linux/man-pages/man2/chroot.2.html&quot;&gt;&lt;code&gt;chroot&lt;/code&gt;&lt;/a&gt; is a system call.&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ &lt;span class=&quot;token function&quot;&gt;man&lt;/span&gt; &lt;span class=&quot;token parameter variable&quot;&gt;-s2&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;chroot&lt;/span&gt;
CHROOT&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;2&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;

NAME
       &lt;span class=&quot;token function&quot;&gt;chroot&lt;/span&gt; - change root directory

SYNOPSIS
       &lt;span class=&quot;token comment&quot;&gt;#include &amp;lt;unistd.h&gt;&lt;/span&gt;

       int chroot&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;const char *path&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;

   Feature Test Macro Requirements &lt;span class=&quot;token keyword&quot;&gt;for&lt;/span&gt; glibc &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;see feature_test_macros&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;7&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;))&lt;/span&gt;:
&lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;.&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;As mentioned in the man page, it changes the root of the file system for a process to the path passed as argument. This new environment created for the process is known as a chroot jail, to which we will refer simply as a &lt;em&gt;jail&lt;/em&gt;.&lt;/p&gt;
&lt;p&gt;The jail allows us to trap a process inside a filesystem, i.e. the process cannot access anything outside the filesystem it is in. So, if it tried to access the root of the filesystem, inside the jail it will  only see the root of the new file system which is the path we passed on to &lt;code&gt;chroot&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;As an example throughout the post I will be using the factorial function. Whenever I refer to &lt;code&gt;factorial.c&lt;/code&gt;, I refer to a file that you might have to create as needed and consists of the following C source code.&lt;/p&gt;
&lt;pre class=&quot;language-c&quot;&gt;&lt;code class=&quot;language-c&quot;&gt;&lt;span class=&quot;token macro property&quot;&gt;&lt;span class=&quot;token directive-hash&quot;&gt;#&lt;/span&gt;&lt;span class=&quot;token directive keyword&quot;&gt;include&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&amp;lt;stdio.h&gt;&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;token macro property&quot;&gt;&lt;span class=&quot;token directive-hash&quot;&gt;#&lt;/span&gt;&lt;span class=&quot;token directive keyword&quot;&gt;include&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&amp;lt;stdint.h&gt;&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;token macro property&quot;&gt;&lt;span class=&quot;token directive-hash&quot;&gt;#&lt;/span&gt;&lt;span class=&quot;token directive keyword&quot;&gt;include&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&amp;lt;inttypes.h&gt;&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;token macro property&quot;&gt;&lt;span class=&quot;token directive-hash&quot;&gt;#&lt;/span&gt;&lt;span class=&quot;token directive keyword&quot;&gt;include&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&amp;lt;stdlib.h&gt;&lt;/span&gt;&lt;/span&gt;

&lt;span class=&quot;token keyword&quot;&gt;int&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;main&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token keyword&quot;&gt;int&lt;/span&gt; argc&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;token keyword&quot;&gt;char&lt;/span&gt; &lt;span class=&quot;token operator&quot;&gt;*&lt;/span&gt;argv&lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;

  &lt;span class=&quot;token class-name&quot;&gt;uint64_t&lt;/span&gt; result &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
  &lt;span class=&quot;token class-name&quot;&gt;uint32_t&lt;/span&gt; arg&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;

  &lt;span class=&quot;token keyword&quot;&gt;if&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;argc &lt;span class=&quot;token operator&quot;&gt;!=&lt;/span&gt; &lt;span class=&quot;token number&quot;&gt;2&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;
    &lt;span class=&quot;token keyword&quot;&gt;return&lt;/span&gt; &lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;

  arg &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;strtoul&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;argv&lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;token constant&quot;&gt;NULL&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;token number&quot;&gt;0&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
  &lt;span class=&quot;token keyword&quot;&gt;while&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;arg&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt; result &lt;span class=&quot;token operator&quot;&gt;*=&lt;/span&gt; arg&lt;span class=&quot;token operator&quot;&gt;--&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;

  &lt;span class=&quot;token function&quot;&gt;printf&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token string&quot;&gt;&quot;Result: %&quot;&lt;/span&gt; PRIu64 &lt;span class=&quot;token string&quot;&gt;&quot;&#92;n&quot;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; result&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
  &lt;span class=&quot;token keyword&quot;&gt;return&lt;/span&gt; &lt;span class=&quot;token number&quot;&gt;0&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
&lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Let&#39;s compile it and run an example.&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ gcc &lt;span class=&quot;token parameter variable&quot;&gt;-Wall&lt;/span&gt; &lt;span class=&quot;token parameter variable&quot;&gt;-Wextra&lt;/span&gt; &lt;span class=&quot;token parameter variable&quot;&gt;-o&lt;/span&gt; factorial factorial.c
$ ./factorial &lt;span class=&quot;token number&quot;&gt;20&lt;/span&gt;
Result: &lt;span class=&quot;token number&quot;&gt;2432902008176640000&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Let&#39;s create a jail for this binary using &lt;code&gt;chroot&lt;/code&gt; and run it.&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ &lt;span class=&quot;token function&quot;&gt;mkdir&lt;/span&gt; jail
$ &lt;span class=&quot;token function&quot;&gt;cp&lt;/span&gt; factorial jail/
$ &lt;span class=&quot;token function&quot;&gt;chroot&lt;/span&gt; jail/ /factorial
chroot: cannot change root directory to &lt;span class=&quot;token string&quot;&gt;&#39;jail/&#39;&lt;/span&gt;&lt;span class=&quot;token builtin class-name&quot;&gt;:&lt;/span&gt; Operation not permitted&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;First lesson here: only &lt;code&gt;root&lt;/code&gt; can &lt;code&gt;chroot&lt;/code&gt;. For security reasons, a normal user cannot &lt;code&gt;chroot&lt;/code&gt;. Being able to do so, would allow the user privilege escalation (&lt;a href=&quot;https://web.archive.org/web/20160127150916/http://www.bpfh.net/simes/computing/chroot-break.html&quot;&gt;further details&lt;/a&gt; on breaking out of the chroot jail and privilege escalation).&lt;/p&gt;
&lt;p&gt;So we &lt;code&gt;sudo&lt;/code&gt;:&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ &lt;span class=&quot;token function&quot;&gt;sudo&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;chroot&lt;/span&gt; jail/ /factorial
chroot: failed to run &lt;span class=&quot;token builtin class-name&quot;&gt;command&lt;/span&gt; ‘/factorial’: No such &lt;span class=&quot;token function&quot;&gt;file&lt;/span&gt; or directory&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Now this really starts to annoy you and you start wondering if the path to &lt;code&gt;factorial&lt;/code&gt; is correct. It is correct, the path is relative to the jail root. Once the root becomes &lt;code&gt;jail/&lt;/code&gt;, the &lt;code&gt;factorial&lt;/code&gt; binary is in the root of the filesystem so &lt;code&gt;/factorial&lt;/code&gt; is correct. The problem is subtle but will teach you an important lesson: dependencies. This is, after all, a dynamically linked executable 💡.&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ ldd factorial
        linux-vdso.so.1 &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;0x00007ffe1f7f9000&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;
        libc.so.6 &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;&gt;&lt;/span&gt; /lib/x86_64-linux-gnu/libc.so.6 &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;0x00007f65bf62e000&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;
        /lib64/ld-linux-x86-64.so.2 &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;0x00007f65bf844000&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Those files exist in your file system, but not in the jail, so you get an error (although a pretty terrible error message at that). Let&#39;s try a static executable instead.&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ gcc &lt;span class=&quot;token parameter variable&quot;&gt;-Wall&lt;/span&gt; &lt;span class=&quot;token parameter variable&quot;&gt;-Wextra&lt;/span&gt; &lt;span class=&quot;token parameter variable&quot;&gt;-static&lt;/span&gt; &lt;span class=&quot;token parameter variable&quot;&gt;-o&lt;/span&gt; factorial factorial.c
$ ldd factorial
        not a dynamic executable
$ &lt;span class=&quot;token function&quot;&gt;sudo&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;chroot&lt;/span&gt; jail/ /factorial &lt;span class=&quot;token number&quot;&gt;20&lt;/span&gt;
Result: &lt;span class=&quot;token number&quot;&gt;2432902008176640000&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This is exactly what we wanted from the beginning - to show that the binary is now jailed in this root and cannot escape without explicitly trying to, something no benign binary is likely to do (see my comment above on escaping jail).&lt;/p&gt;
&lt;p&gt;In chroot jails, the filesystem root changes but it&#39;s still running on the same kernel. This allows us to create a new userspace inside this jail, separate from the host&#39;s, and possibly based on a different
linux distribution. For reproducibility, you can potentially tarball this jail and send it to someone else, who could themselves &lt;code&gt;chroot&lt;/code&gt; into it and reproduce a specific problem.&lt;/p&gt;
&lt;h2&gt;QEMU and binfmt&lt;/h2&gt;
&lt;p&gt;I will now introduce two other essential components to achieve our goal of cross architecture reproducibility: QEMU and binfmt.&lt;/p&gt;
&lt;p&gt;Up until now our jail has contained binaries compiled for our host architecture (in my case x86_64), however this need not be the case. &lt;a href=&quot;https://www.qemu.org/&quot;&gt;QEMU&lt;/a&gt; is an open-source hardware emulator and virtualizer. It implements two execution modes: system mode, and user mode;&lt;/p&gt;
&lt;p&gt;In system mode, QEMU works like &lt;a href=&quot;https://www.virtualbox.org/&quot;&gt;VirtualBox&lt;/a&gt; it emulates a whole system - from the applications, interrupts, and kernel, all the way to the hardware devices. The one I am interested in is user mode. In user mode, QEMU emulates a single binary leaving the rest of the system untouched.&lt;/p&gt;
&lt;p&gt;To demo how it works, lets use a cross-toolchain to compile a program to be run on a different architecture. Given I am on a &lt;code&gt;x86_64&lt;/code&gt;, I will use an &lt;code&gt;armhf&lt;/code&gt; toolchain. You can either download one provided from your distro or compile one with &lt;a href=&quot;https://crosstool-ng.github.io/&quot;&gt;crosstool-ng&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;For the sake of reproducibility, lets compile a pre-configured one with &lt;code&gt;crosstool-ng&lt;/code&gt;. Download it and install it in &lt;code&gt;$PATH&lt;/code&gt; - I used version 1.24.0. Then build the cross toolchain for &lt;code&gt;armv7-rpi2-linux-gnueabihf&lt;/code&gt;.&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ &lt;span class=&quot;token function&quot;&gt;mkdir&lt;/span&gt; build
$ &lt;span class=&quot;token builtin class-name&quot;&gt;cd&lt;/span&gt; build
build/ $ ct-ng armv7-rpi2-linux-gnueabihf
build/ $ ct-ng build&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Let&#39;s once again compile the &lt;code&gt;factorial.c&lt;/code&gt; example, but this time with our new toolchain. This toolchain will be in &lt;code&gt;$HOME/x-tools/armv7-rpi2-linux-gnueabihf&lt;/code&gt; by default.&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ &lt;span class=&quot;token builtin class-name&quot;&gt;export&lt;/span&gt; &lt;span class=&quot;token assign-left variable&quot;&gt;&lt;span class=&quot;token environment constant&quot;&gt;PATH&lt;/span&gt;&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token environment constant&quot;&gt;$HOME&lt;/span&gt;/x-tools/armv7-rpi2-linux-gnueabihf/bin:&lt;span class=&quot;token environment constant&quot;&gt;$PATH&lt;/span&gt;
$ armv7-rpi2-linux-gnueabihf-gcc &lt;span class=&quot;token parameter variable&quot;&gt;-static&lt;/span&gt; &lt;span class=&quot;token parameter variable&quot;&gt;-o&lt;/span&gt; factorial factorial.c
$ ./factorial
zsh: &lt;span class=&quot;token builtin class-name&quot;&gt;exec&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;format&lt;/span&gt; error: ./factorial
$ &lt;span class=&quot;token function&quot;&gt;file&lt;/span&gt; factorial
factorial: ELF &lt;span class=&quot;token number&quot;&gt;32&lt;/span&gt;-bit LSB executable, ARM, EABI5 version &lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;SYSV&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;, &lt;span class=&quot;token punctuation&quot;&gt;&#92;&lt;/span&gt;
dynamically linked, interpreter /lib/ld-linux-armhf.so.3, &lt;span class=&quot;token keyword&quot;&gt;for&lt;/span&gt; GNU/Linux &lt;span class=&quot;token punctuation&quot;&gt;&#92;&lt;/span&gt;
&lt;span class=&quot;token number&quot;&gt;4.19&lt;/span&gt;.21, with debug_info, not stripped&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;We were expecting this error, right? After all, we cannot just execute an arm binary in a x86_64 system. However, we can if we use QEMU in user mode which is what I want to show:&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ qemu-arm ./factorial &lt;span class=&quot;token number&quot;&gt;20&lt;/span&gt; 
Result: &lt;span class=&quot;token number&quot;&gt;2432902008176640000&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Now we have it working - however, there&#39;s a last tool we need to discuss and that&#39;s &lt;code&gt;binfmt&lt;/code&gt;. &lt;code&gt;binfmt_misc&lt;/code&gt; is a linux kernel capability that allows arbitrary executable file formats to be recognized and passed on to an interpreter. This means that we can transparently recognize arm (or any other architecture) executables and request them to be passed to QEMU transparently when we try to execute them.&lt;/p&gt;
&lt;p&gt;Before proceeding, if you wish to follow the examples, verify that your kernel has &lt;code&gt;binfmt&lt;/code&gt; enabled.&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ zcat /proc/config.gz&lt;span class=&quot;token operator&quot;&gt;|&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;grep&lt;/span&gt; BINFMT
&lt;span class=&quot;token assign-left variable&quot;&gt;CONFIG_BINFMT_ELF&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;y
&lt;span class=&quot;token assign-left variable&quot;&gt;CONFIG_COMPAT_BINFMT_ELF&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;y
&lt;span class=&quot;token assign-left variable&quot;&gt;CONFIG_BINFMT_SCRIPT&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;y
&lt;span class=&quot;token assign-left variable&quot;&gt;CONFIG_BINFMT_MISC&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;y&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;If you don&#39;t have &lt;code&gt;/proc/config.gz&lt;/code&gt; try instead &lt;code&gt;grep BINFMT /boot/config-$(uname -r)&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;Let&#39;s go back to our factorial on arm example, but this time install a &lt;code&gt;binfmt&lt;/code&gt; record to call &lt;code&gt;qemu-arm&lt;/code&gt; on the executable whenever we try to execute it directly. A &lt;code&gt;binfmt&lt;/code&gt; record looks like &lt;code&gt;:name:type:offset:magic:mask:interpreter:flags&lt;/code&gt; and needs to be installed by echoing the correct string to &lt;code&gt;/proc/sys/fs/binfmt_misc/register&lt;/code&gt;. For details on this consult the &lt;a href=&quot;https://www.kernel.org/doc/html/latest/admin-guide/binfmt-misc.html&quot;&gt;kernel
documentation&lt;/a&gt;. For &lt;code&gt;armv7&lt;/code&gt; we can register our interpreter and run our binary transparently like this:&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ &lt;span class=&quot;token function&quot;&gt;sudo&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;bash&lt;/span&gt; &lt;span class=&quot;token parameter variable&quot;&gt;-c&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&#39;echo &quot;:qemu-arm:M:0:&#92;&#92;x7f&#92;&#92;x45&#92;&#92;x4c&#92;&#92;x46&#92;&#92;x01&#92;&#92;x01&#92;&#92;x01&#92;&#92;x00&#92;&#92;x00&#92;&#92;x00&#92;&#92;x00&#92;&#92;x00&#92;&#92;x00&#92;
&#92;&#92;x00&#92;&#92;x00&#92;&#92;x00&#92;&#92;x02&#92;&#92;x00&#92;&#92;x28&#92;&#92;x00:&#92;&#92;xff&#92;&#92;xff&#92;&#92;xff&#92;&#92;xff&#92;&#92;xff&#92;&#92;xff&#92;&#92;xff&#92;&#92;x00&#92;&#92;xff&#92;&#92;xff&#92;&#92;xff&#92;&#92;xff&#92;&#92;xff&#92;
&#92;&#92;xff&#92;&#92;xff&#92;&#92;xff&#92;&#92;xfe&#92;&#92;xff&#92;&#92;xff&#92;&#92;xff:/home/pmatos/installs/bin/qemu-arm:OCF&quot; &gt; &#92;
/proc/sys/fs/binfmt_misc/register&#39;&lt;/span&gt;
$ ./factorial &lt;span class=&quot;token number&quot;&gt;20&lt;/span&gt;
Result: &lt;span class=&quot;token number&quot;&gt;2432902008176640000&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The write to &lt;code&gt;/proc/sys/fs/binfmt_misc/register&lt;/code&gt; may fail if you already have a record for &lt;code&gt;qemu-arm&lt;/code&gt; setup. To remove the record run &lt;code&gt;echo -1 &amp;gt; /proc/sys/fs/binfmt_misc/qemu-arm&lt;/code&gt;. Note that the file, as shown above is an ARM binary and yet we transparently run it through QEMU thanks to the &lt;code&gt;binfmt&lt;/code&gt; magic we have initially set up.&lt;/p&gt;
&lt;p&gt;I should note, that while interesting to understand how this works behind the scenes, several people have created images to do the binfmt registration for you. One of those projects is &lt;a href=&quot;https://hub.docker.com/r/docker/binfmt/tags&quot;&gt;docker/binfmt&lt;/a&gt; which you can try to run using the latest tag: &lt;code&gt;docker run --rm --privileged docker/binfmt:66f9012c56a8316f9244ffd7622d7c21c1f6f28d&lt;/code&gt; (another example of a similar project is
&lt;a href=&quot;https://github.com/multiarch/qemu-user-static&quot;&gt;multiarch/qemu-user-static&lt;/a&gt;). If, for some reason, this does not work you should know enough by now to proceed with the registration manually.&lt;/p&gt;
&lt;h2&gt;Creating container base images&lt;/h2&gt;
&lt;p&gt;We have gone through creating chroot jails and transparently executing cross-architecture binaries with QEMU. A base image for a container is based pretty much on what we have just learned. To create a container
base image we will create a jail with a foreign root filesystem and we will use QEMU to execute binaries inside the jail. Once all is working, we import it into docker.&lt;/p&gt;
&lt;p&gt;To help us create a base system, we will use &lt;code&gt;debootstrap&lt;/code&gt; in two stages. We split this into two stages so we can setup QEMU in between.&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ &lt;span class=&quot;token function&quot;&gt;mkdir&lt;/span&gt; rootfs
$ &lt;span class=&quot;token function&quot;&gt;debootstrap&lt;/span&gt; &lt;span class=&quot;token parameter variable&quot;&gt;--foreign&lt;/span&gt; --no-check-gpg &lt;span class=&quot;token parameter variable&quot;&gt;--arch&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;arm buster ./rootfs http://httpredir.debian.org/debian/
$ &lt;span class=&quot;token function&quot;&gt;cp&lt;/span&gt; &lt;span class=&quot;token parameter variable&quot;&gt;-v&lt;/span&gt; /usr/bin/qemu-arm-static ./rootfs/usr/bin/
$ &lt;span class=&quot;token function&quot;&gt;chroot&lt;/span&gt; ./rootfs ./debootstrap/debootstrap --second-stage &lt;span class=&quot;token parameter variable&quot;&gt;--verbose&lt;/span&gt;
$ &lt;span class=&quot;token function&quot;&gt;mount&lt;/span&gt; &lt;span class=&quot;token parameter variable&quot;&gt;-t&lt;/span&gt; devpts devpts ./rootfs/dev/pts
$ &lt;span class=&quot;token function&quot;&gt;mount&lt;/span&gt; &lt;span class=&quot;token parameter variable&quot;&gt;-t&lt;/span&gt; proc proc ./rootfs/proc
$ &lt;span class=&quot;token function&quot;&gt;mount&lt;/span&gt; &lt;span class=&quot;token parameter variable&quot;&gt;-t&lt;/span&gt; sysfs sysfs ./rootfs/sys&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;At this point our system is a debian base system with root at &lt;code&gt;./rootfs&lt;/code&gt;. Note that the &lt;code&gt;qemu&lt;/code&gt; you need to copy into &lt;code&gt;./rootfs/usr/bin&lt;/code&gt; needs to be static in order to avoid dynamic loading issues inside the jail when it is invoked.&lt;/p&gt;
&lt;p&gt;Another important aspect to consider is that depending on your &lt;code&gt;binfmt&lt;/code&gt; setup, you need to put the binary in the proper place inside the jail. For the above to work, your &lt;code&gt;binfmt&lt;/code&gt; interpreter registration has to
point to an interpreter at &lt;code&gt;/usr/bin/qemu-arm-static&lt;/code&gt;, which is the absolute path as seen from inside the jail. Now we can install all dependencies in the system at will. All commands will be using &lt;code&gt;qemu&lt;/code&gt; transparently to execute if &lt;code&gt;binfmt&lt;/code&gt; setup worked correctly. Remember that in the case of chroot jails (and also of containers), the kernel in use is the kernel of your host. Therefore that is the only kernel that needs to be set up with &lt;code&gt;binfmt&lt;/code&gt;. You set it up outside your chroot jail and when the kernel needs to execute a foreign executable, it will look at the current &lt;code&gt;binfmt&lt;/code&gt; setup for the interpreter. However, the kernel cannot access a filesystem outside the jail, so that interpreter needs to exist inside the it.&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ &lt;span class=&quot;token function&quot;&gt;chroot&lt;/span&gt; ./rootfs &lt;span class=&quot;token function&quot;&gt;apt-get&lt;/span&gt; update
$ &lt;span class=&quot;token function&quot;&gt;chroot&lt;/span&gt; ./rootfs &lt;span class=&quot;token function&quot;&gt;apt-get&lt;/span&gt; &lt;span class=&quot;token parameter variable&quot;&gt;-y&lt;/span&gt; upgrade
$ &lt;span class=&quot;token function&quot;&gt;chroot&lt;/span&gt; ./rootfs &lt;span class=&quot;token function&quot;&gt;apt-get&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;install&lt;/span&gt; &lt;span class=&quot;token parameter variable&quot;&gt;-y&lt;/span&gt; g++ cmake libicu-dev &lt;span class=&quot;token function&quot;&gt;git&lt;/span&gt; ruby-highline ruby-json python
&lt;span class=&quot;token function&quot;&gt;chroot&lt;/span&gt; ./rootfs &lt;span class=&quot;token function&quot;&gt;apt-get&lt;/span&gt; &lt;span class=&quot;token parameter variable&quot;&gt;-y&lt;/span&gt; autoremove
&lt;span class=&quot;token function&quot;&gt;chroot&lt;/span&gt; ./rootfs &lt;span class=&quot;token function&quot;&gt;apt-get&lt;/span&gt; clean
&lt;span class=&quot;token function&quot;&gt;chroot&lt;/span&gt; ./rootfs &lt;span class=&quot;token function&quot;&gt;find&lt;/span&gt; /var/lib/apt/lists &lt;span class=&quot;token parameter variable&quot;&gt;-type&lt;/span&gt; f &lt;span class=&quot;token parameter variable&quot;&gt;-delete&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;These are the steps to install the dependencies to build and test JSC. Let&#39;s unmount the filesystems in order to create the docker base image.&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;&lt;span class=&quot;token function&quot;&gt;umount&lt;/span&gt; ./rootfs/dev/pts
&lt;span class=&quot;token function&quot;&gt;umount&lt;/span&gt; ./rootfs/proc
&lt;span class=&quot;token function&quot;&gt;umount&lt;/span&gt; ./rootfs/sys&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Let&#39;s &lt;code&gt;tar&lt;/code&gt; our jail and import it into a docker image.&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ &lt;span class=&quot;token function&quot;&gt;tar&lt;/span&gt; --numeric-owner &lt;span class=&quot;token parameter variable&quot;&gt;-cvf&lt;/span&gt; buster-arm.tar &lt;span class=&quot;token parameter variable&quot;&gt;-C&lt;/span&gt; ./rootfs &lt;span class=&quot;token builtin class-name&quot;&gt;.&lt;/span&gt;
$ &lt;span class=&quot;token function&quot;&gt;docker&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;import&lt;/span&gt; buster-arm.tar jsc-base:arm-raw&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The image &lt;code&gt;jsc-base:arm-raw&lt;/code&gt; now contains our raw system. To be able to add metadata or build on top of this image before releasing we can build a docker image that extends the raw image version.&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ &lt;span class=&quot;token function&quot;&gt;cat&lt;/span&gt; &lt;span class=&quot;token operator&quot;&gt;&amp;lt;&amp;lt;&lt;/span&gt;&lt;span class=&quot;token string&quot;&gt;EOF&lt;span class=&quot;token bash punctuation&quot;&gt; &lt;span class=&quot;token operator&quot;&gt;&gt;&lt;/span&gt; jsc32-base.Dockerfile&lt;/span&gt;
FROM jsc32-base:arm-raw

LABEL description=&quot;Minimal Debian image to reproduce JSC dev&quot;
LABEL maintainer=&quot;Paulo Matos &amp;lt;pmatos@igalia.com&gt;&quot;
 
CMD [&quot;/bin/bash&quot;]
EOF&lt;/span&gt;
$ &lt;span class=&quot;token function&quot;&gt;docker&lt;/span&gt; build &lt;span class=&quot;token parameter variable&quot;&gt;-t&lt;/span&gt; pmatos/jsc32-base:arm &lt;span class=&quot;token parameter variable&quot;&gt;-f&lt;/span&gt; jsc32-base.Dockerfile
$ &lt;span class=&quot;token function&quot;&gt;docker&lt;/span&gt; push pmatos/jsc32-base:arm&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Once we have pushed the image into the repository, we can run it from anywhere as long as &lt;code&gt;binfmt&lt;/code&gt; is setup properly.&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ &lt;span class=&quot;token function&quot;&gt;docker&lt;/span&gt; run pmatos/jsc32-base:arm &lt;span class=&quot;token function&quot;&gt;file&lt;/span&gt; /bin/bash
/bin/bash: ELF &lt;span class=&quot;token number&quot;&gt;32&lt;/span&gt;-bit LSB pie executable, ARM, EABI5 version &lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;SYSV&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;, dynamically linked, &lt;span class=&quot;token punctuation&quot;&gt;&#92;&lt;/span&gt;
interpreter /lib/ld-linux-armhf.so.3, &lt;span class=&quot;token keyword&quot;&gt;for&lt;/span&gt; GNU/Linux &lt;span class=&quot;token number&quot;&gt;3.2&lt;/span&gt;.0, &lt;span class=&quot;token punctuation&quot;&gt;&#92;&lt;/span&gt;
BuildID&lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;sha1&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;40e50d160a6c70d1a4e961200202cf853b4a2145, stripped&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Or to get an interactive shell inside the container, use &lt;code&gt;-it&lt;/code&gt;: &lt;code&gt;docker run -it pmatos/jsc32-base:arm /bin/bash&lt;/code&gt;.&lt;/p&gt;
&lt;h2&gt;Going rootless with podman&lt;/h2&gt;
&lt;p&gt;While &lt;a href=&quot;https://www.docker.com/&quot;&gt;&lt;code&gt;docker&lt;/code&gt;&lt;/a&gt; is a product of Docker Inc with various components and pricing tiers, &lt;a href=&quot;https://podman.io/&quot;&gt;&lt;code&gt;podman&lt;/code&gt;&lt;/a&gt; is a free and open source daemonless engine for developing, managing, and running containers on Linux. One of the main internal differences between &lt;code&gt;docker&lt;/code&gt; and &lt;code&gt;podman&lt;/code&gt; is that &lt;code&gt;podman&lt;/code&gt; uses &lt;a href=&quot;https://en.wikipedia.org/wiki/Cgroups&quot;&gt;cgroups&lt;/a&gt; v2, which &lt;code&gt;docker&lt;/code&gt; doesn&#39;t yet support. Since Fedora 31 ships with cgroups v2, for many users &lt;code&gt;podman&lt;/code&gt; is the only container engine available. Fortunately, &lt;code&gt;podman&lt;/code&gt; is largely &lt;code&gt;docker&lt;/code&gt; compatible meaning that you can &lt;code&gt;alias docker=podman&lt;/code&gt; without any issues.&lt;/p&gt;
&lt;p&gt;So you can do something like this.&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ &lt;span class=&quot;token function&quot;&gt;podman&lt;/span&gt; run pmatos/jsc32-base:arm &lt;span class=&quot;token function&quot;&gt;file&lt;/span&gt; /bin/bash
/bin/bash: ELF &lt;span class=&quot;token number&quot;&gt;32&lt;/span&gt;-bit LSB pie executable, ARM, EABI5 version &lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;SYSV&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;, dynamically linked, &lt;span class=&quot;token punctuation&quot;&gt;&#92;&lt;/span&gt;
interpreter /lib/ld-linux-armhf.so.3, &lt;span class=&quot;token keyword&quot;&gt;for&lt;/span&gt; GNU/Linux &lt;span class=&quot;token number&quot;&gt;3.2&lt;/span&gt;.0, &lt;span class=&quot;token punctuation&quot;&gt;&#92;&lt;/span&gt;
BuildID&lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;sha1&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;40e50d160a6c70d1a4e961200202cf853b4a2145, stripped&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;As long your &lt;code&gt;binfmt&lt;/code&gt; is properly setup, it will run as well as it did with docker.&lt;/p&gt;
&lt;h2&gt;Application to JSC&lt;/h2&gt;
&lt;p&gt;The initial code developed for reproducible JSC32 builds can be found in the &lt;a href=&quot;https://github.com/pmatos/WebKit-misc/&quot;&gt;WebKit-misc&lt;/a&gt; repository and was initially pushed under commit &lt;a href=&quot;https://github.com/pmatos/WebKit-misc/commit/94bf25ebe888ae1824113eeb269fe0c0fcb5f3d4&quot;&gt;94bf25e&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;The main script is in &lt;a href=&quot;https://github.com/pmatos/WebKit-misc/blob/94bf25ebe888ae1824113eeb269fe0c0fcb5f3d4/containers/reprojsc.sh&quot;&gt;&lt;code&gt;containers/reprojsc.sh&lt;/code&gt;&lt;/a&gt; and follows the plan laid out in this blog post.&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;An image is created using &lt;a href=&quot;https://github.com/pmatos/WebKit-misc/blob/94bf25ebe888ae1824113eeb269fe0c0fcb5f3d4/containers/jsc32-base/build-image.sh&quot;&gt;&lt;code&gt;containers/jsc32-base/build-image.sh&lt;/code&gt;&lt;/a&gt; and pushed into docker hub &lt;a href=&quot;https://hub.docker.com/r/pmatos/jsc32-base&quot;&gt;pmatos/jsc32-base&lt;/a&gt;. This only needs to be done when the image is changed.&lt;/li&gt;
&lt;li&gt;The &lt;a href=&quot;https://github.com/pmatos/WebKit-misc/blob/94bf25ebe888ae1824113eeb269fe0c0fcb5f3d4/containers/reprojsc.sh&quot;&gt;&lt;code&gt;containers/reprojsc.sh&lt;/code&gt;&lt;/a&gt; script is run each time one wants to trigger a build and/or test of JSC. This sets up QEMU for the desired architecture, starts the image and issues the necessary commands for the actions laid out in the command line.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;To run it yourself, checkout &lt;a href=&quot;https://github.com/pmatos/WebKit-misc/&quot;&gt;WebKit-misc&lt;/a&gt; and &lt;a href=&quot;https://webkit.org/&quot;&gt;WebKit&lt;/a&gt;, and run &lt;code&gt;reprojsc.sh&lt;/code&gt;.&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ &lt;span class=&quot;token function&quot;&gt;git&lt;/span&gt; clone &lt;span class=&quot;token parameter variable&quot;&gt;--depth&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt; git://git.webkit.org/WebKit.git
$ &lt;span class=&quot;token builtin class-name&quot;&gt;export&lt;/span&gt; &lt;span class=&quot;token assign-left variable&quot;&gt;WEBKIT_PATH&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token environment constant&quot;&gt;$PWD&lt;/span&gt;/WebKit
$ &lt;span class=&quot;token function&quot;&gt;git&lt;/span&gt; clone &lt;span class=&quot;token parameter variable&quot;&gt;--depth&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt; git@github.com:pmatos/WebKit-misc.git
$ &lt;span class=&quot;token builtin class-name&quot;&gt;cd&lt;/span&gt; WebKit-misc/containers
$ ./reprojsc.sh &lt;span class=&quot;token parameter variable&quot;&gt;-a&lt;/span&gt; arm &lt;span class=&quot;token parameter variable&quot;&gt;-b&lt;/span&gt; &lt;span class=&quot;token parameter variable&quot;&gt;-t&lt;/span&gt; &lt;span class=&quot;token parameter variable&quot;&gt;-i&lt;/span&gt; &lt;span class=&quot;token variable&quot;&gt;$WEBKIT_PATH&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This will build, test and send you into an interactive session so you can debug some possibly failed tests. Inside the container, you&#39;ll be in an ARM Cortex-A7 (32bits) environment. The other available architecture at the moment is MIPS which can be chosen with &lt;code&gt;-a mips&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;If you have any issues or requests please open an issue in &lt;a href=&quot;https://github.com/pmatos/WebKit-misc/issues&quot;&gt;GitHub&lt;/a&gt;.&lt;/p&gt;
&lt;h2&gt;What&#39;s missing?&lt;/h2&gt;
&lt;p&gt;There are many things that deserved discussion but I won&#39;t elaborate further. One of them is the difference between the container execution engine and the image builder. When using docker it&#39;s easy to think
that they are the same but it is not the case. For example, docker has a new image builder, called &lt;a href=&quot;https://docs.docker.com/buildx/working-with-buildx/&quot;&gt;&lt;code&gt;buildx&lt;/code&gt;&lt;/a&gt; whichhas in-built support for the creation of multi-architecture images. On the other hand, &lt;code&gt;podman&lt;/code&gt; is a container execution engine but an image creator is, for example, &lt;a href=&quot;https://buildah.io/&quot;&gt;&lt;code&gt;buildah&lt;/code&gt;&lt;/a&gt;, which doesn&#39;t have multi-architecture image support
&lt;a href=&quot;https://github.com/containers/buildah/issues/1590&quot;&gt;yet&lt;/a&gt;. Don&#39;t worry if you are confused - things are moving fast in this area and it&#39;s easy to lose track of all the tools out there.&lt;/p&gt;
&lt;p&gt;With the ability to transparently emulate other architectures QEMU user mode will see more usage patterns. Bugs in QEMU can look like application bugs and debugging is not straightforward, so bear in this
in mind when using this technology.&lt;/p&gt;
&lt;h2&gt;Acknowledgments&lt;/h2&gt;
&lt;p&gt;Thanks to my fellow Igalians Angelos Oikonomopoulos and Philip Chimento for providing suggestions and corrections for this blog post.&lt;/p&gt;
</content>
  </entry>
  <entry>
    <title>A Brief Look at the WebKit Workflow</title>
    <link href="https://p.ocmatos.com/blog/a-brief-look-at-the-webkit-workflow.html"/>
    <updated>2019-10-02T00:00:00Z</updated>
    <id>https://p.ocmatos.com/blog/a-brief-look-at-the-webkit-workflow.html</id>
    <content xml:lang="en" type="html">&lt;p&gt;Since I joined &lt;a href=&quot;https://www.igalia.com/&quot;&gt;Igalia&lt;/a&gt; back in May, I have been working on WebKit. After more than a decade working on GCC and LLVM, moving codebases, workflows, and compilation techniques and optimizations meant I was in very unfamiliar territory.&lt;/p&gt;
&lt;div style=&quot;text-align:center;&quot;&gt;
	&lt;img alt=&quot;Where am I? GIF&quot; class=&quot;post-photo&quot; src=&quot;https://p.ocmatos.com/img/where-am-i.gif&quot; /&gt;
&lt;/div&gt;
&lt;p&gt;One of the main things to understand is the contribution process and each project has their own rules and methodology, so lets start straight into how to contribute to WebKit. In a few places I will focus specifically on JSC -- for other parts of WebKit you might want to contact the &lt;a href=&quot;http://lists.webkit.org/mailman/listinfo/webkit-help&quot;&gt;mailing list&lt;/a&gt; or the &lt;a href=&quot;irc://irc.freenode.net/#webkit&quot;&gt;irc channel&lt;/a&gt; for specific advice.&lt;/p&gt;
&lt;p&gt;The information in this post should be used in addition to what is available in the official &lt;a href=&quot;https://webkit.org/getting-started/&quot;&gt;Getting Started&lt;/a&gt; and &lt;a href=&quot;https://webkit.org/contributing-code/&quot;&gt;Contributing Code&lt;/a&gt; documents.&lt;/p&gt;
&lt;h2&gt;Cloning&lt;/h2&gt;
&lt;p&gt;WebKit uses &lt;code&gt;svn&lt;/code&gt; as the default for version control but it has a &lt;code&gt;git&lt;/code&gt; mirror, therefore given my preference for using &lt;code&gt;git&lt;/code&gt;, we&#39;ll use that in the following examples.&lt;/p&gt;
&lt;p&gt;To clone WebKit we type:&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ &lt;span class=&quot;token function&quot;&gt;git&lt;/span&gt; clone git://git.webkit.org/WebKit.git
&lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;.
$ &lt;span class=&quot;token function&quot;&gt;du&lt;/span&gt; &lt;span class=&quot;token parameter variable&quot;&gt;-hs&lt;/span&gt; WebKit
&lt;span class=&quot;token number&quot;&gt;9&lt;/span&gt;.7G    WebKit&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This will take almost 10Gb of space at time of writing so if you are in a rush or short of space, use the &lt;code&gt;--depth=1&lt;/code&gt; option to &lt;code&gt;git clone&lt;/code&gt;.&lt;/p&gt;
&lt;h2&gt;Building and Testing&lt;/h2&gt;
&lt;p&gt;The most important directory for workflow in WebKit is &lt;code&gt;Tools/Scripts&lt;/code&gt; &lt;sup class=&quot;footnote-ref&quot;&gt;&lt;a href=&quot;https://p.ocmatos.com/blog/a-brief-look-at-the-webkit-workflow.html#fn1&quot; id=&quot;fnref1&quot;&gt;[1]&lt;/a&gt;&lt;/sup&gt;. Here you will find all the scripts we will use in this post.&lt;/p&gt;
&lt;p&gt;There are several scripts to build different parts of WebKit in different modes -- called Ports. For JSC you would use &lt;code&gt;build-jsc&lt;/code&gt; script:&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ Tools/Scripts/build-jsc --jsc-only &lt;span class=&quot;token parameter variable&quot;&gt;--debug&lt;/span&gt;
&lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;.
&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;
 JavaScriptCore is now built &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;02m:44s&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;. 
&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The &lt;code&gt;--jsc-only&lt;/code&gt; flag specifies the WebKit port to build and you probably should use &lt;code&gt;--debug&lt;/code&gt; while developing since it makes debugging much easier. Once you reach a point where you are happy with how it&#39;s going, run some tests. I should also mention that WebKit follows a very specific coding style and has a script to check for style problems. Don&#39;t worry too much about this in the beginning, until you have a patch. After a few contributions, you&#39;ll just get the hang of how to style your code. Make sure, either during development or before creating a patch that you check for style issues with &lt;code&gt;Tools/Scripts/check-webkit-style&lt;/code&gt;:&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ Tools/Scripts/check-webkit-style
ERROR: Source/WTF/wtf/StackBounds.cpp:25:  Alphabetical sorting problem.  &lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;build/include_order&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;4&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt;
ERROR: Source/WTF/wtf/StackBounds.cpp:126:  Place brace on its own line &lt;span class=&quot;token keyword&quot;&gt;for&lt;/span&gt; &lt;span class=&quot;token keyword&quot;&gt;function&lt;/span&gt; definitions.  &lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;whitespace/braces&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;4&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt;
Total errors found: &lt;span class=&quot;token number&quot;&gt;2&lt;/span&gt; &lt;span class=&quot;token keyword&quot;&gt;in&lt;/span&gt; &lt;span class=&quot;token number&quot;&gt;2&lt;/span&gt; files&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The errors are self-explanatory and easy to fix. Go ahead and make the necessary changes.&lt;/p&gt;
&lt;p&gt;Time to run some tests - There are several test scripts and you should probably run the tests most appropriate to the changes you have made. In general, you can list the testing scripts with:&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ &lt;span class=&quot;token function&quot;&gt;ls&lt;/span&gt; &lt;span class=&quot;token parameter variable&quot;&gt;-1&lt;/span&gt; Tools/Scripts/run-*-tests
Tools/Scripts/run-api-tests
Tools/Scripts/run-bindings-tests
Tools/Scripts/run-builtins-generator-tests
Tools/Scripts/run-dashboard-tests
Tools/Scripts/run-gtk-tests
Tools/Scripts/run-iexploder-tests
Tools/Scripts/run-inspector-generator-tests
Tools/Scripts/run-javascriptcore-tests
Tools/Scripts/run-jsc-stress-tests
Tools/Scripts/run-mangleme-tests
Tools/Scripts/run-perf-tests
Tools/Scripts/run-regexp-tests
Tools/Scripts/run-webdriver-tests
Tools/Scripts/run-webkit-tests
Tools/Scripts/run-web-platform-tests
Tools/Scripts/run-wpe-tests&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Scripts tend to have a very good &lt;code&gt;--help&lt;/code&gt; menu so you should give those a try before you run them. For tests in JSC use: &lt;code&gt;run-javascriptcore-tests&lt;/code&gt;.&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ Tools/Scripts/run-javascriptcore-tests --no-build --no-fail-fast &lt;span class=&quot;token parameter variable&quot;&gt;--debug&lt;/span&gt; --jsc-only
&lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;.
    &lt;span class=&quot;token number&quot;&gt;0&lt;/span&gt; failures found.&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;If you run the tests and see some possibly unrelated failures be sure to run them again without your changes since they might be failures that made it into upstream and not necessarily a result of your changes.&lt;/p&gt;
&lt;h2&gt;Submitting a patch&lt;/h2&gt;
&lt;p&gt;Once you have the patch ready for upstream, you can submit it automatically with &lt;code&gt;webkit-patch&lt;/code&gt;, which will automate much of the workflow process. Before proceeding though, make sure you have an account in the &lt;a href=&quot;https://bugs.webkit.org/&quot;&gt;WebKit bug database&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;The process can be summarised as follows:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;Create a patch upstream with &lt;code&gt;webkit-patch upload --no-review&lt;/code&gt; (the script will create a new bug to hold your patch)&lt;/li&gt;
&lt;/ol&gt;
&lt;ul&gt;
&lt;li&gt;This will prompt you to fill at least one &lt;code&gt;ChangeLog&lt;/code&gt; in your favourite editor. If in doubt of the style, run a &lt;code&gt;git log&lt;/code&gt; to see how other logs have been written and follow the same style&lt;/li&gt;
&lt;li&gt;Remember not to change the line with the bug title and URL, these should remain unchanged from what &lt;code&gt;webkit-patch&lt;/code&gt; generated&lt;/li&gt;
&lt;/ul&gt;
&lt;ol start=&quot;2&quot;&gt;
&lt;li&gt;If you are confident about it, set the flag &lt;code&gt;review?&lt;/code&gt; to request a review of your code&lt;/li&gt;
&lt;/ol&gt;
&lt;ul&gt;
&lt;li&gt;Wait for a review... if you end up waiting a week without reply feel free to ping someone on the &lt;a href=&quot;http://lists.webkit.org/mailman/listinfo/webkit-help&quot;&gt;mailing list&lt;/a&gt; or the &lt;a href=&quot;irc://irc.freenode.net/#webkit&quot;&gt;irc channel&lt;/a&gt; about it&lt;/li&gt;
&lt;li&gt;If you get a &lt;code&gt;review-&lt;/code&gt; then, read the comments provided and fix the issues mentioned in the comments. Once the issues have been fixed, upload the new patch with &lt;code&gt;webkit-patch upload&lt;/code&gt;. Referring to the bug number in the command line is optional because the script can read the bug number from the already written &lt;code&gt;ChangeLog&lt;/code&gt;. But to be explicit about it, you can specify the bug number with &lt;code&gt;webkit-patch upload &amp;lt;BUG&amp;gt;&lt;/code&gt;, where &lt;code&gt;&amp;lt;BUG&amp;gt;&lt;/code&gt; was the bug number created in step 1.&lt;/li&gt;
&lt;/ul&gt;
&lt;ol start=&quot;3&quot;&gt;
&lt;li&gt;After the review loop in point 2., you have a &lt;code&gt;review+&lt;/code&gt;. Until you have the correct permissions, you&#39;ll need someone to &lt;code&gt;commit-queue+&lt;/code&gt; your code. In order to request your code to be put into the commit queue, mark it as &lt;code&gt;commit-queue?&lt;/code&gt;. As soon as it is marked &lt;code&gt;commit-queue+&lt;/code&gt;, the bots will check your code, run the necessary tests, commit the patch upstream and close the bug.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;That&#39;s it. You got your contribution upstream -- Congratulations!&lt;/p&gt;
&lt;h2&gt;Further notes on the review/commit-queue cycle&lt;/h2&gt;
&lt;p&gt;In order not to make the above three-point loop more complicated, I am using this section to drill down on the review and commit-queue flags. As first time contributors, all of our patches need to be reviewed and put into the commit-queue. These flags are known as &lt;code&gt;review&lt;/code&gt; and &lt;code&gt;commit-queue&lt;/code&gt; or by the shorthands &lt;code&gt;r&lt;/code&gt; and &lt;code&gt;cq&lt;/code&gt;. To these you can append the symbols &lt;code&gt;-&lt;/code&gt;, &lt;code&gt;?&lt;/code&gt; or &lt;code&gt;+&lt;/code&gt;. We can only set &lt;code&gt;r?&lt;/code&gt; and &lt;code&gt;cq?&lt;/code&gt; and wait for someone with more permissions to change these accordingly.&lt;/p&gt;
&lt;p&gt;There are two ways to change these flags. In the web interface of your bug, you can click on the &amp;quot;Details&amp;quot; link for your patch and then change the flags accordingly.&lt;/p&gt;
&lt;div style=&quot;text-align:center;&quot;&gt;
	&lt;img alt=&quot;Patch Details&quot; class=&quot;post-photo&quot; src=&quot;https://p.ocmatos.com/img/2019-10-02-WebKitWorkflow-PatchDetails.png&quot; /&gt;
&lt;/div&gt;
&lt;p&gt;Once in the patch details view, you can see the patch flags.&lt;/p&gt;
&lt;div style=&quot;text-align:center;&quot;&gt;
	&lt;img alt=&quot;Patch Flags&quot; class=&quot;post-photo&quot; src=&quot;https://p.ocmatos.com/img/2019-10-02-WebKitWorkflow-Flags.png&quot; /&gt;
&lt;/div&gt;
&lt;p&gt;When you submit a bug with &lt;code&gt;webkit-patch upload&lt;/code&gt;, if confident,  you can set the &lt;code&gt;r?&lt;/code&gt; flag requesting a review by &lt;em&gt;not&lt;/em&gt; using &lt;code&gt;--no-review&lt;/code&gt; in the command. However, if you want to &lt;em&gt;in addition&lt;/em&gt; request the patch to be sent to the commit queue after review, you can add to the &lt;code&gt;upload&lt;/code&gt; command &lt;code&gt;--request-commit&lt;/code&gt;. This will add the flags &lt;code&gt;r?&lt;/code&gt; and &lt;code&gt;cq?&lt;/code&gt;. If you don&#39;t use &lt;code&gt;cq?&lt;/code&gt;, then you can manually set it once you get an &lt;code&gt;r+&lt;/code&gt;.&lt;/p&gt;
&lt;h2&gt;The review process&lt;/h2&gt;
&lt;p&gt;There are a couple of situations that can happen during the review process (point 2. above) that are worth mentioning.&lt;/p&gt;
&lt;h3&gt;Wrong bug title&lt;/h3&gt;
&lt;p&gt;Imagine that you got your patch reviewed and you realised that the bug title, which you created when you first submitted your patch was based on incorrect knowledge and you wish to change it? Also, this needs to be reflected in the patch.&lt;/p&gt;
&lt;p&gt;When you modify your patch locally and keep the ChangeLog unmodified, &lt;code&gt;webkit-patch upload --no-review&lt;/code&gt; will automatically use the &lt;code&gt;ChangeLog&lt;/code&gt; that you have locally. The best way to change the title of your bug is to do the following:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;Update the bug title in your bug page using the web interface, and;&lt;/li&gt;
&lt;li&gt;Execute &lt;code&gt;webkit-patch upload&lt;/code&gt; whenever you are ready to re-submit your fixed patch but add &lt;code&gt;--update-changelog&lt;/code&gt;. This will automatically grab the new title and update the &lt;code&gt;ChangeLog&lt;/code&gt; for you.&lt;/li&gt;
&lt;/ol&gt;
&lt;h3&gt;Apply patch locally&lt;/h3&gt;
&lt;p&gt;Imagine a collaborator improved your patch, uploaded it to the bug but it is still not quite right. You need for some reason to apply the patch locally to keep working on it. You can download it manually and apply it, however you can also automatically do this by checking the patch id (also known as attachment id) on the web interface and running on your WebKit checkout &lt;code&gt;webkit-patch apply-attachment --force-clean &amp;lt;ATTACH&amp;gt;&lt;/code&gt;, where &lt;code&gt;&amp;lt;ATTACH&amp;gt;&lt;/code&gt; is your patch id. Do note that &lt;code&gt;--force-clean&lt;/code&gt; will remove all your local changes and commit.&lt;/p&gt;
&lt;p&gt;There is a simpler alternative, where you don&#39;t need to look up the attachment id. If all you need is the latest patch uploaded to a bug, you can use &lt;code&gt;webkit-patch apply-from-bug &amp;lt;BUG&amp;gt;&lt;/code&gt;. As with &lt;code&gt;apply-attachment&lt;/code&gt;, the command &lt;code&gt;apply-from-bug&lt;/code&gt; also accepts &lt;code&gt;--force-clean&lt;/code&gt;&lt;/p&gt;
&lt;h2&gt;Command summary&lt;/h2&gt;
&lt;h3&gt;Help&lt;/h3&gt;
&lt;h4&gt;Global Help&lt;/h4&gt;
&lt;p&gt;&lt;code&gt;webkit-patch help --all-commands&lt;/code&gt;&lt;/p&gt;
&lt;h4&gt;Command Help&lt;/h4&gt;
&lt;p&gt;Take &lt;code&gt;upload&lt;/code&gt; as an example:&lt;/p&gt;
&lt;p&gt;&lt;code&gt;webkit-patch help upload&lt;/code&gt;&lt;/p&gt;
&lt;h3&gt;Upload patch&lt;/h3&gt;
&lt;p&gt;&lt;code&gt;webkit-patch upload&lt;/code&gt;&lt;/p&gt;
&lt;p&gt;This will upload your current local changes as a patch upstream. It will show you the patch it is going to upload before it actually does so it&#39;s safe to do to so what&#39;s included in the patch before the actual upload takes place.&lt;/p&gt;
&lt;p&gt;Relevant flags:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;code&gt;--no-review&lt;/code&gt;: by default, the &lt;code&gt;upload&lt;/code&gt; command will set that patch to &lt;code&gt;review?&lt;/code&gt; (commonly shortedned to &lt;code&gt;r?&lt;/code&gt;). Use &lt;code&gt;--no-review&lt;/code&gt; if you think the patch is not ready to review, or if you want to confirm the patch is not breaking any tests by using the upstream build workers.&lt;/li&gt;
&lt;li&gt;&lt;code&gt;--request-review&lt;/code&gt;: this will set the &lt;code&gt;cq?&lt;/code&gt; flag on your patch so that it is added to the commit queue after review.&lt;/li&gt;
&lt;li&gt;&lt;code&gt;--update-changelog&lt;/code&gt;: by default if a &lt;code&gt;ChangeLog&lt;/code&gt; already exists it will use that, otherwise it will ask you to write one by opening your favourite editor. Sometimes, however you might have written a &lt;code&gt;ChangeLog&lt;/code&gt; but you want to modify it. Use &lt;code&gt;--update-changelog&lt;/code&gt; to request that.&lt;/li&gt;
&lt;li&gt;&lt;code&gt;--comment=&amp;lt;COMMENT&amp;gt;&lt;/code&gt;: you can specify a comment to add to the post along with the patch upload. If you are uploading a new patch with changes requested by the reviewer, you should post a comment with &lt;code&gt;--comment=&amp;quot;Here is a new patch that addresses the commands in the review&amp;quot;&lt;/code&gt;.&lt;/li&gt;
&lt;/ul&gt;
&lt;h2&gt;Apply upstream attachment&lt;/h2&gt;
&lt;p&gt;&lt;code&gt;webkit-patch apply-attachment&lt;/code&gt; or &lt;code&gt;webkit-patch apply-from-bug&lt;/code&gt;&lt;/p&gt;
&lt;p&gt;This will apply an upstream patch locally. The command &lt;code&gt;apply-attachment&lt;/code&gt; requires the patch (attachment) identifier so you might have to look that up beforehand. If you only need the latest patch to a bug, you can use &lt;code&gt;apply-from-bug&lt;/code&gt; with the bug identifier.&lt;/p&gt;
&lt;p&gt;Relevant flags:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;code&gt;--force-clean&lt;/code&gt;: if you have changes locally the patch might not apply cleanly. Use this flag to clean everything locally before attempting to apply patch. You might want to use &lt;code&gt;git stash&lt;/code&gt; to stash your changes, if they are important.&lt;/li&gt;
&lt;/ul&gt;
&lt;h2&gt;Bonus -- WebKit on GitHub&lt;/h2&gt;
&lt;p&gt;I prefer to use GitHub to keep my project forks however, the WebKit repository is so large that you won&#39;t be able to import it straight into GitHub (either through the UI or manually). If you try, you&#39;ll get an error about packs being too large.&lt;/p&gt;
&lt;p&gt;The easiest way to get WebKit into GitHub is by forking the &lt;a href=&quot;https://github.com/webkit/webkit&quot;&gt;unofficial mirror here&lt;/a&gt;.&lt;/p&gt;
&lt;h2&gt;Thanks&lt;/h2&gt;
&lt;p&gt;Thanks to my Igalian colleagues Caio Lima and Guillaume Emont for reading a draft of this post and providing valuable suggestions.&lt;/p&gt;
&lt;h2&gt;Corrections or Comments?&lt;/h2&gt;
&lt;p&gt;Feel free to send any corrections or comments &lt;a href=&quot;mailto:pmatos@igalia.com&quot;&gt;straight into my inbox&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;Edits since publication (02.10.2019):&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;02.10.2019: my colleague Guillaume Emont found a typo in the &amp;quot;Upload patch&amp;quot; subsection, where I wrote &lt;code&gt;r+&lt;/code&gt; instead of &lt;code&gt;r?&lt;/code&gt;;&lt;/li&gt;
&lt;/ul&gt;
&lt;hr class=&quot;footnotes-sep&quot; /&gt;
&lt;section class=&quot;footnotes&quot;&gt;
&lt;ol class=&quot;footnotes-list&quot;&gt;
&lt;li id=&quot;fn1&quot; class=&quot;footnote-item&quot;&gt;&lt;p&gt;These scripts require Python2, therefore if you are using Python3 by default, you&#39;ll run into troubles running them. Try creating a virtual environment with &lt;code&gt;$ python2 -m virtualenv venv &amp;amp;&amp;amp; source venv/bin/activate&lt;/code&gt; at the root directory of WebKit. &lt;a href=&quot;https://p.ocmatos.com/blog/a-brief-look-at-the-webkit-workflow.html#fnref1&quot; class=&quot;footnote-backref&quot;&gt;↩︎&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;/section&gt;
</content>
  </entry>
  <entry>
    <title>S10 Overview</title>
    <link href="https://p.ocmatos.com/blog/s10-overview.html"/>
    <updated>2018-05-21T00:00:00Z</updated>
    <id>https://p.ocmatos.com/blog/s10-overview.html</id>
    <content xml:lang="en" type="html">&lt;h2&gt;Superoptimization&lt;/h2&gt;
&lt;p&gt;Superoptimization was introduced by H. Massalin in his paper &lt;em&gt;Superoptimizer: a look at the smallest program&lt;/em&gt;&lt;sup class=&quot;footnote-ref&quot;&gt;&lt;a href=&quot;https://p.ocmatos.com/blog/s10-overview.html#fn1&quot; id=&quot;fnref1&quot;&gt;[1]&lt;/a&gt;&lt;/sup&gt; back in 1987. The idea is that for every sequence of instructions for a given computer architecture and a metric, there is an infinite number of sequences which perform the same task (are semantically equivalent to the original sequence), but one of them is the best one with respect to the metric.&lt;/p&gt;
&lt;p&gt;Take for example the instruction sequence in RV32I:&lt;/p&gt;
&lt;pre class=&quot;language-nasm&quot;&gt;&lt;code class=&quot;language-nasm&quot;&gt;addi x1, zero, &lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt;
mv   x2, zero
add  x2, x2, x1
add  x2, x2, x1&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This sets the register &lt;code&gt;x2&lt;/code&gt; to 2. If we choose a metric that orders programs by increasing order of size, then the best possible program is:&lt;/p&gt;
&lt;pre class=&quot;language-nasm&quot;&gt;&lt;code class=&quot;language-nasm&quot;&gt;addi x2, zero, &lt;span class=&quot;token number&quot;&gt;2&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Now take another example in RV32IM (which includes the multiplication extension):&lt;/p&gt;
&lt;pre class=&quot;language-nasm&quot;&gt;&lt;code class=&quot;language-nasm&quot;&gt;mul x1, x1, &lt;span class=&quot;token number&quot;&gt;2&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This doubles the value in register &lt;code&gt;x1&lt;/code&gt;. Given the same metric, it is not possible to find a better program. However, if we change our metric to the number of clock cycles taken by the whole sequence of instructions and our micro-architecture takes 8 cycles for multiplications and 2 cycles for shifting then there&#39;s a better program:&lt;/p&gt;
&lt;pre class=&quot;language-nasm&quot;&gt;&lt;code class=&quot;language-nasm&quot;&gt;slli x1, x1, &lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Which shifts register &lt;code&gt;x1&lt;/code&gt; by 1 bit to the left, in essence multiplying its value by two and taking only 2 cycles whereas the original example took 8 (under our micro-architectural assumptions).&lt;/p&gt;
&lt;p&gt;For long sequence of instructions the state space that requires searching for the best sequence is enormous, which makes superoptimization a CPU intensive program. Research on superoptimization slowed down after the initial paper but has seen a resurgence in the last decade.&lt;/p&gt;
&lt;h2&gt;The compiler&lt;/h2&gt;
&lt;p&gt;Will superoptimizers replace your compiler? No!&lt;/p&gt;
&lt;p&gt;Production compilers are highly complex pieces of software that generate very good code, under very tight time constraints. On the other hand, superoptimizers are very slow since they need to search very large state spaces and therefore can take minutes, hours or even days to find the best possible instruction sequence.&lt;/p&gt;
&lt;p&gt;The placement of the superoptimizer in the development toolchain is up to the user but, in general, it will be placed after the compilation step picking up where the compiler left off and trying to optimize the code it generated.&lt;/p&gt;
&lt;h2&gt;S10&lt;/h2&gt;
&lt;p&gt;S10 was created out of the resurgence in superoptimization research mentioned earlier. &lt;a href=&quot;https://dblp.org/pid/127/3128&quot;&gt;Mangpo Phothilimthana&lt;/a&gt; from UC Berkeley published in 2016 a paper&lt;sup class=&quot;footnote-ref&quot;&gt;&lt;a href=&quot;https://p.ocmatos.com/blog/s10-overview.html#fn2&quot; id=&quot;fnref2&quot;&gt;[2]&lt;/a&gt;&lt;/sup&gt; and technical report on Greenthumb. The initial prototype of S10 was built upon Greenthumb and once we verified the potential we decided to turn it into a commercial product -- where at this point about 15% of code comes from Mangpo&#39;s work.&lt;/p&gt;
&lt;p&gt;Since most of the code was rewritten, including the interface between the backend architectures and the main modules, the architectures supported by Greenthumb are not supported by S10, namely &lt;em&gt;arm&lt;/em&gt;, &lt;em&gt;llvm ir&lt;/em&gt;, and &lt;em&gt;greenarrays&lt;/em&gt;. We have therefore chosen to implement support initially for &lt;a href=&quot;https://riscv.org/&quot;&gt;RISC-V&lt;/a&gt;, with other architectures on the roadmap.&lt;/p&gt;
&lt;p&gt;S10 is written completely in &lt;a href=&quot;https://racket-lang.org/&quot;&gt;Racket&lt;/a&gt; and implements three distinct algorithms that cooperate in a distributed setting. A symbolic&lt;sup class=&quot;footnote-ref&quot;&gt;&lt;a href=&quot;https://p.ocmatos.com/blog/s10-overview.html#fn2&quot; id=&quot;fnref2:1&quot;&gt;[2:1]&lt;/a&gt;&lt;/sup&gt;, stochastic&lt;sup class=&quot;footnote-ref&quot;&gt;&lt;a href=&quot;https://p.ocmatos.com/blog/s10-overview.html#fn3&quot; id=&quot;fnref3&quot;&gt;[3]&lt;/a&gt;&lt;/sup&gt; and lens&lt;sup class=&quot;footnote-ref&quot;&gt;&lt;a href=&quot;https://p.ocmatos.com/blog/s10-overview.html#fn4&quot; id=&quot;fnref4&quot;&gt;[4]&lt;/a&gt;&lt;/sup&gt; algorithms are distributed over several cores (or different machines) and communicate with each other through message passing finding incrementally better instruction sequences until it&#39;s proved that the best sequence has already been found.&lt;/p&gt;
&lt;p&gt;S10 includes a statistical profiler and debugging aids to ensure that any error is found quickly and any hotspots are first to be optimized. Due to the statistical nature of some methods it is also possible to automatically tune these parameters on an architecture independent way to ensure the best possible performance from the framework.&lt;/p&gt;
&lt;p&gt;Another area we focused on, given the long times taken to find solutions to the optimization problem was partial caching. S10 is supported by a database which uses previous found solutions to the superoptimization problem to simplify future searches.&lt;/p&gt;
&lt;h2&gt;Superoptimization in practice&lt;/h2&gt;
&lt;p&gt;In practice waiting for the perfect solution is not always possible or realistic, that is why S10 will incrementally provide incrementally better instruction sequences until it proves there is none better.&lt;/p&gt;
&lt;p&gt;An interaction with S10 for the first example I mentioned in this post might look like this:&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ riscv-s10 &lt;span class=&quot;token parameter variable&quot;&gt;-j&lt;/span&gt; &lt;span class=&quot;token number&quot;&gt;8&lt;/span&gt; &lt;span class=&quot;token parameter variable&quot;&gt;--timeout&lt;/span&gt; &lt;span class=&quot;token number&quot;&gt;60&lt;/span&gt; &lt;span class=&quot;token parameter variable&quot;&gt;--verbose&lt;/span&gt; &lt;span class=&quot;token parameter variable&quot;&gt;--progress&lt;/span&gt; add2.s
S10 Superoptimization Framework, by Linki Tools UG
https://linki.tools/s10

Booting up &lt;span class=&quot;token number&quot;&gt;8&lt;/span&gt; cores &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;stochastic &lt;span class=&quot;token number&quot;&gt;3&lt;/span&gt;, lens &lt;span class=&quot;token number&quot;&gt;4&lt;/span&gt;, symbolic &lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;
Input score: &lt;span class=&quot;token number&quot;&gt;16&lt;/span&gt;

* stochastic found optimization &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;score: &lt;span class=&quot;token number&quot;&gt;12&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;

  &lt;span class=&quot;token function&quot;&gt;mv&lt;/span&gt; x2, zero
  addi x2, x2, &lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt;
  addi x2, x2, &lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt;

* lens found optimization &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;score: &lt;span class=&quot;token number&quot;&gt;4&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;

  addi x2, zero, &lt;span class=&quot;token number&quot;&gt;2&lt;/span&gt;
  
* symbolic found best

  addi x2, zero, &lt;span class=&quot;token number&quot;&gt;2&lt;/span&gt;
  
Shutting down cores

Caching solution:
  addi x2, zero, &lt;span class=&quot;token number&quot;&gt;2&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The superoptimizer will incrementally give information (due to the &lt;code&gt;--progress&lt;/code&gt; flag) about optimizations found until we can prove there&#39;s no better solution, or a timeout is reached (which in this case was set to 60 seconds with &lt;code&gt;--timeout&lt;/code&gt;).&lt;/p&gt;
&lt;p&gt;The metric for the previous example was instruction sequence size, but had our metric been performance the situation would have been similar with one exception. The metric has to be fast to execute because it is executed potentially millions of times. If a cycle accurate simulation takes a long time, then it is better to implement a fast heuristic and request the best few solutions found and run an accurate metric on those.&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ riscv-s10 &lt;span class=&quot;token parameter variable&quot;&gt;-j&lt;/span&gt; &lt;span class=&quot;token number&quot;&gt;8&lt;/span&gt; &lt;span class=&quot;token parameter variable&quot;&gt;--timeout&lt;/span&gt; &lt;span class=&quot;token number&quot;&gt;20&lt;/span&gt; &lt;span class=&quot;token parameter variable&quot;&gt;--verbose&lt;/span&gt; &lt;span class=&quot;token parameter variable&quot;&gt;--progress&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;&#92;&lt;/span&gt;
  &lt;span class=&quot;token parameter variable&quot;&gt;--metric&lt;/span&gt; speed --accurate-metric-script sim.sh --run-metric-on &lt;span class=&quot;token number&quot;&gt;2&lt;/span&gt; add2.s
S10 Superoptimization Framework, by Linki Tools UG
https://linki.tools/s10

Booting up &lt;span class=&quot;token number&quot;&gt;8&lt;/span&gt; cores &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;stochastic &lt;span class=&quot;token number&quot;&gt;3&lt;/span&gt;, lens &lt;span class=&quot;token number&quot;&gt;4&lt;/span&gt;, symbolic &lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;
Input score: &lt;span class=&quot;token number&quot;&gt;16&lt;/span&gt;

* stochastic found optimization &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;score: &lt;span class=&quot;token number&quot;&gt;12&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;

  &lt;span class=&quot;token function&quot;&gt;mv&lt;/span&gt; x2, zero
  addi x2, x2, &lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt;
  addi x2, x2, &lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt;

* lens found optimization &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;score: &lt;span class=&quot;token number&quot;&gt;4&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;

  addi x2, zero, &lt;span class=&quot;token number&quot;&gt;2&lt;/span&gt;

Timeout reached.
Shutting down cores

Best two solutions:

&lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;mv&lt;/span&gt; x2, zero
    addi x2, x2, &lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt;
    addi x2, x2, &lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt;

&lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;2&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt; addi x2, zero, &lt;span class=&quot;token number&quot;&gt;2&lt;/span&gt;

Running sim.sh on &lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. Done
Running sim.sh on &lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;2&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. Done

Best solution &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;sim.sh&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;:

&lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;2&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt; addi x2, zero, &lt;span class=&quot;token number&quot;&gt;2&lt;/span&gt;

Caching solution:
  addi x2, zero, &lt;span class=&quot;token number&quot;&gt;2&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;S10 in this case ran against a tight timeout and at that point since it was given a script to calculate an accurate metric (with &lt;code&gt;--accurate-metric-script&lt;/code&gt;) it called it on the best two optimizations found at that point (due to &lt;code&gt;--run-metric-on&lt;/code&gt;). The metric scored solution &lt;code&gt;[2]&lt;/code&gt; as better and therefore that was the one which was returned and cached.&lt;/p&gt;
&lt;p&gt;This is a simple example of what S10 can do. I have not mentioned how to extend the algorithms and internal structures, how to extend S10 to a new architecture or autotune it for better performance. I hope to touch those points on future posts.&lt;/p&gt;
&lt;p&gt;Linki Tools is the first company to offer a commercial product for Superoptimization. Join us in superoptimizing your software! &lt;a href=&quot;mailto:p@ocmatos.com&quot;&gt;Contact me&lt;/a&gt; for more information.&lt;/p&gt;
&lt;hr class=&quot;footnotes-sep&quot; /&gt;
&lt;section class=&quot;footnotes&quot;&gt;
&lt;ol class=&quot;footnotes-list&quot;&gt;
&lt;li id=&quot;fn1&quot; class=&quot;footnote-item&quot;&gt;&lt;p&gt;Henry Massalin, Superoptimizer: a look at the smallest program, 1987 &lt;a href=&quot;https://p.ocmatos.com/blog/s10-overview.html#fnref1&quot; class=&quot;footnote-backref&quot;&gt;↩︎&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li id=&quot;fn2&quot; class=&quot;footnote-item&quot;&gt;&lt;p&gt;Mangpo Phothilimthana, et al., GreenThumb: superoptimizer construction framework, 2016 &lt;a href=&quot;https://p.ocmatos.com/blog/s10-overview.html#fnref2&quot; class=&quot;footnote-backref&quot;&gt;↩︎&lt;/a&gt; &lt;a href=&quot;https://p.ocmatos.com/blog/s10-overview.html#fnref2:1&quot; class=&quot;footnote-backref&quot;&gt;↩︎&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li id=&quot;fn3&quot; class=&quot;footnote-item&quot;&gt;&lt;p&gt;Alex Aiken, et al., Stochastic superoptimization, 2013 &lt;a href=&quot;https://p.ocmatos.com/blog/s10-overview.html#fnref3&quot; class=&quot;footnote-backref&quot;&gt;↩︎&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li id=&quot;fn4&quot; class=&quot;footnote-item&quot;&gt;&lt;p&gt;Mangpo Phothilimthana, et al., Scaling up Superoptimization, 2016 &lt;a href=&quot;https://p.ocmatos.com/blog/s10-overview.html#fnref4&quot; class=&quot;footnote-backref&quot;&gt;↩︎&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;/section&gt;
</content>
  </entry>
</feed>