<?xml version="1.0" encoding="utf-8"?>
<feed xmlns="http://www.w3.org/2005/Atom" xml:base="https://p.ocmatos.com/">
  <title>pmatos rambles</title>
  <subtitle>Programming, sports and music - oh wow - enjoy!</subtitle>
  <link href="https://p.ocmatos.com/feed.xml" rel="self"/>
  <link href="https://p.ocmatos.com/"/>
  <updated>2026-04-16T14:10:19Z</updated>
  <id>https://p.ocmatos.com/</id>
  <author>
    <name>Paulo Matos</name>
    <email>p@ocmatos.com</email>
  </author>
  <entry>
    <title>Code Is Free Now. What&#39;s Left Is Us.</title>
    <link href="https://p.ocmatos.com/blog/code-is-free-now-whats-left-is-us.html"/>
    <updated>2026-04-16T00:00:00Z</updated>
    <id>https://p.ocmatos.com/blog/code-is-free-now-whats-left-is-us.html</id>
    <content xml:lang="en" type="html">&lt;p&gt;&lt;img src=&quot;https://p.ocmatos.com/img/2026/04/code-is-free-now.png&quot; alt=&quot;Humans in warm conversation while code dissolves into light around them&quot; /&gt;&lt;/p&gt;
&lt;p&gt;In the first months of agentic coding, I grieved.&lt;/p&gt;
&lt;p&gt;The joy of hunting a bug for weeks, the satisfaction of finally tracing it to a single misplaced condition, the meditative rhythm of writing code line by line. Gone. I had spent decades building a relationship with programming and suddenly the thing I loved doing was being done by something else. It felt like losing a craft I had spent my whole career perfecting.&lt;/p&gt;
&lt;p&gt;I wasn&#39;t alone. Nolan Lawson captured this feeling perfectly in &lt;a href=&quot;https://nolanlawson.com/2026/02/07/we-mourn-our-craft/&quot;&gt;We Mourn Our Craft&lt;/a&gt;. He gave words to what many of us felt but struggled to articulate. The mourning is real, and it&#39;s understandable.&lt;/p&gt;
&lt;p&gt;But that&#39;s not where the story ends.&lt;/p&gt;
&lt;h2&gt;Then: July 2025&lt;/h2&gt;
&lt;p&gt;In July 2025, a few months into using Claude Code daily, I recorded some voice notes trying to make sense of what was happening. The tools were already good but rough around the edges. Orchestrating multiple agents across worktrees was manual and painful. Getting notified when an agent finished a task meant checking your terminal every few minutes. Choosing the right model for the right job (planning vs. implementation vs. review) was guesswork.&lt;/p&gt;
&lt;p&gt;I knew things would improve. What I didn&#39;t know was how fast.&lt;/p&gt;
&lt;p&gt;At the time I was wrestling with basic questions: Can I trust an agent to implement a feature unsupervised? How do I review code I didn&#39;t write? What does my job even look like if the machine writes the code? I had more questions than answers, and the tooling wasn&#39;t helping me find them faster.&lt;/p&gt;
&lt;h2&gt;Now: April 2026&lt;/h2&gt;
&lt;p&gt;Fourteen months in, the landscape looks different.&lt;/p&gt;
&lt;p&gt;Last week I returned from AI Engineer and the shift was palpable. What was once a niche topic on a side track is now the default assumption. Every company represented there was using agents, developing agents, or deploying agents. The conversations didn&#39;t start with &amp;quot;are you using AI for coding?&amp;quot; but with &amp;quot;how are you orchestrating your agents?&amp;quot; The question is no longer whether. It&#39;s how.&lt;/p&gt;
&lt;p&gt;The tooling caught up too. We have Claude Code with Opus 4.6, skills, hooks, MCPs, and multi-agent orchestration. We have OpenClaw. We have GPT 5.4 and the lightning-fast 5.3-codex-spark. The harnesses are better, the workflows are more mature, and the rough edges I struggled with in July 2025 have been sanded down significantly.&lt;/p&gt;
&lt;p&gt;In January, I &lt;a href=&quot;https://p.ocmatos.com/blog/jsse-a-javascript-engine-built-by-an-agent.html&quot;&gt;built a JavaScript engine in Rust&lt;/a&gt; entirely through agentic coding. 170,000 lines of Rust. 100% of test262 non-staging tests passing. Zero lines written by me. My total hands-on-keyboard time was about four hours spread across six weeks. When I tell people this, they focus on the &amp;quot;zero lines&amp;quot; part. But the interesting part is what I was actually doing with my time: planning, reviewing architectural decisions, choosing what to work on next, and occasionally telling the agent to stop avoiding the hard problems. The work was strategic, not mechanical.&lt;/p&gt;
&lt;p&gt;That project changed how I think about my role.&lt;/p&gt;
&lt;h2&gt;The Role Shift&lt;/h2&gt;
&lt;p&gt;Here is what I&#39;ve come to believe after 14 months of practice: the developer&#39;s job is no longer to write code. It&#39;s to specify intent, verify behavior, and architect systems.&lt;/p&gt;
&lt;p&gt;This isn&#39;t a prediction about the future. It&#39;s a description of my present. I spend my days thinking about what should be built, how the pieces fit together, and whether the output meets the requirements. The act of translating intent into syntax, the thing we used to call &amp;quot;programming,&amp;quot; is increasingly handled by the machine.&lt;/p&gt;
&lt;p&gt;What remains is the hard part. The part that was always the hard part, honestly, but that we used to spend less time on because we were so busy typing.&lt;/p&gt;
&lt;h2&gt;More Human, Not Less&lt;/h2&gt;
&lt;p&gt;Here&#39;s the thing nobody seems to be saying: agentic coding makes software engineering more human, not less.&lt;/p&gt;
&lt;p&gt;Think about it. Machines can code. They&#39;re getting better at it every month. But it&#39;s the human who needs to tell the machine what to code, and in many instances, how to code it. Humans will do what humans do best: meet, discuss, and reach a consensus on what&#39;s needed.&lt;/p&gt;
&lt;p&gt;In a world where code is developed at near-zero cost and near-instant speed, what&#39;s left is the human side of software engineering. The discussion of what should be built. The negotiation of trade-offs. The alignment of a team around a shared vision. Our jobs will be more human.&lt;/p&gt;
&lt;p&gt;There&#39;s a paradox here too. Before, we didn&#39;t implement all the features because there was no time. Now there&#39;s time. So the difficult part shifts. It&#39;s no longer &amp;quot;can we build this?&amp;quot; but &amp;quot;should we build this?&amp;quot; Just because you can, it doesn&#39;t mean you should. The constraint is no longer engineering capacity. It&#39;s human judgment.&lt;/p&gt;
&lt;h2&gt;What Happens to Code Review?&lt;/h2&gt;
&lt;p&gt;Code review is one of the clearest examples of how practices are evolving in real time.&lt;/p&gt;
&lt;p&gt;When an agent generates thousands of lines in a single session, the traditional model of reading every diff line by line breaks down. It&#39;s not possible to do code review in the same way. If you&#39;re working alone, you can push code without a thorough review and fix things later. But when you&#39;re collaborating with others, review matters because human time is on the line, and human time is the scarce resource now.&lt;/p&gt;
&lt;p&gt;The strategy I&#39;ve found most effective: small, focused changes. Many of them, if needed. But each one small enough to review meaningfully, and small enough to revert cleanly without the rest of the system falling apart. This isn&#39;t new advice, but it takes on new urgency when an agent can generate a 2,000-line PR in twenty minutes if you let it.&lt;/p&gt;
&lt;p&gt;In the future, better models and better harnesses will integrate more tightly with verification and testing tools, which may make traditional review less essential. I&#39;ll have more to say about this in a future post on integrating formal verification into the agentic coding feedback loop. But for now, the practical answer is: keep changes small, keep them focused, and accept that the review process is evolving.&lt;/p&gt;
&lt;h2&gt;What Should You Study?&lt;/h2&gt;
&lt;p&gt;If you&#39;re at university studying software engineering right now, prepare yourself for agentic coding. The curriculum you&#39;re following was designed for a world where humans write all the code. That world is ending. The fundamentals still matter (architecture, systems thinking, design, verification) but the emphasis on syntax mastery and manual implementation will age poorly.&lt;/p&gt;
&lt;p&gt;If you&#39;re a teenager wondering whether software engineering is the right path: it&#39;s hard to know exactly what will be here in 10 years. The field is going to look radically different. What I&#39;d suggest instead is to be a generalist. Have a wide reach. Build connections across different areas. Be interested in many things. Extremely specific knowledge won&#39;t stay relevant when the tools are changing this fast.&lt;/p&gt;
&lt;p&gt;What will be essential: hard-working, smart people with great relationships and the ability to bridge between different areas when needed. People who can communicate, who can see the big picture, who can bring others along. And above all, people who are good human beings. The more our tools handle the mechanical work, the more the human qualities matter.&lt;/p&gt;
&lt;h2&gt;Where the Trend Points&lt;/h2&gt;
&lt;p&gt;Someone once said: predictions are hard, especially about the future. I won&#39;t try to pin dates on what&#39;s coming. But trends are easier to read than futures.&lt;/p&gt;
&lt;p&gt;The trend is clear: stronger models, better harnesses, smarter workflows, more targeted tools for agentic coding. If you follow that trend line, it&#39;s easy to see a world where everyone is doing agentic coding and humans are directing rather than implementing. We&#39;ll see more software, not less. Hyper-personalized software, because people can. It&#39;s possible we&#39;ll then see consolidation as a few solutions emerge as more successful than others. But right now we&#39;re in the experimentation phase, and it&#39;s exciting to see all sorts of ideas being tested.&lt;/p&gt;
&lt;p&gt;A few directions I&#39;m particularly interested in, and will write about separately:&lt;/p&gt;
&lt;p&gt;The first is integrating verification into the agentic coding feedback loop. If agents write the code, we need automated ways to ensure correctness that go beyond testing. I&#39;ve been working on the &lt;a href=&quot;https://p.ocmatos.com/blog/formal-verification-in-your-terminal-esbmc-meets-claude-code.html&quot;&gt;integration of ESBMC and Claude Code&lt;/a&gt; and there&#39;s a much larger story to tell there.&lt;/p&gt;
&lt;p&gt;The second is the emergence of programming languages designed for agents. Languages that are easier for machines to generate and that can produce proof certificates for humans to verify correctness. I&#39;m developing one such language (more on that in a later post), and projects like &lt;a href=&quot;https://moonbitlang.com/&quot;&gt;MoonBit&lt;/a&gt; are already exploring this space.&lt;/p&gt;
&lt;h2&gt;After the Mourning&lt;/h2&gt;
&lt;p&gt;I won&#39;t pretend the transition was smooth. For months the work felt like it had boiled down to reviewing broken agent output and fixing things the model got wrong. It was frustrating. It felt like the joy had been extracted from the work and replaced with a tedious supervisory role.&lt;/p&gt;
&lt;p&gt;But things improved. I understood the flow, the tools, the rhythm of working with agents. And something unexpected happened: I found a different passion. Not the old satisfaction of crafting code by hand, but a new kind of thrill. The thrill of seeing an idea materialize in hours instead of months. Of building things I never could have built alone. Of spending my time on the interesting problems, the design, the architecture, the &amp;quot;what should we build?&amp;quot; rather than the &amp;quot;how do I implement this loop?&amp;quot;&lt;/p&gt;
&lt;p&gt;If you identify yourself as someone who can only code and that&#39;s it, then this is going to be a painful ride. But if you can let go and look at what&#39;s on the other side, there&#39;s a world that is more exciting than the one we had before. A world where the work is more creative, more collaborative, and more fundamentally human.&lt;/p&gt;
&lt;p&gt;The code is free now. What&#39;s left is us.&lt;/p&gt;
&lt;p&gt;That&#39;s not a loss. That&#39;s a beginning.&lt;/p&gt;
</content>
  </entry>
  <entry>
    <title>JSSE: A JavaScript Engine Built by an Agent</title>
    <link href="https://p.ocmatos.com/blog/jsse-a-javascript-engine-built-by-an-agent.html"/>
    <updated>2026-03-17T00:00:00Z</updated>
    <id>https://p.ocmatos.com/blog/jsse-a-javascript-engine-built-by-an-agent.html</id>
    <content xml:lang="en" type="html">&lt;p&gt;&lt;img src=&quot;https://p.ocmatos.com/img/2026/03/jsse-agent-dreaming.png&quot; alt=&quot;An agent dreaming in JavaScript&quot; /&gt;&lt;/p&gt;
&lt;p&gt;In January, I came across a blog post by the author of &lt;a href=&quot;https://emsh.cat/one-human-one-agent-one-browser/&quot;&gt;one-agent-one-browser&lt;/a&gt;, a from-scratch browser built in Rust by a single person directing a single coding agent over a few days. No JavaScript support, though. I read it and thought: &amp;quot;How hard can it be to build a JavaScript engine the same way?&amp;quot;&lt;/p&gt;
&lt;p&gt;Six weeks later, &lt;a href=&quot;https://github.com/pmatos/jsse&quot;&gt;JSSE&lt;/a&gt; (JavaScript Simple Engine) became the first JavaScript engine I know of to pass 100% of &lt;a href=&quot;https://github.com/tc39/test262&quot;&gt;test262&lt;/a&gt; non-staging tests: all 98,426 scenarios across &lt;code&gt;language/&lt;/code&gt;, &lt;code&gt;built-ins/&lt;/code&gt;, &lt;code&gt;annexB/&lt;/code&gt;, and &lt;code&gt;intl402/&lt;/code&gt;. Not V8. Not SpiderMonkey. Not JavaScriptCore. A from-scratch engine in Rust, built entirely by &lt;a href=&quot;https://docs.anthropic.com/en/docs/claude-code/overview&quot;&gt;Claude Code&lt;/a&gt; running in YOLO mode.&lt;/p&gt;
&lt;p&gt;I didn&#39;t write a single line of Rust. Not one. The repository is a write-only data store from my perspective. I didn&#39;t even create the GitHub repo by hand; the agent did that too.&lt;/p&gt;
&lt;h2&gt;How It Started&lt;/h2&gt;
&lt;p&gt;It started with a conversation in a chat room about agentic coding and browsers. At 14:42 on January 27, I floated the idea: &amp;quot;OK, shall we embark on a JS engine from scratch to plug into that browser?&amp;quot; Seventeen minutes later I found the &lt;a href=&quot;https://tc39.es/ecma262/&quot;&gt;single-page spec&lt;/a&gt; and the key insight: &amp;quot;the great thing is we have test262, so we can create a feedback loop to create a JS engine that passes all the tests.&amp;quot;&lt;/p&gt;
&lt;p&gt;By 15:05 the repo was live. I set up the initial scaffolding (&lt;code&gt;CLAUDE.md&lt;/code&gt;, &lt;code&gt;PLAN.md&lt;/code&gt;, skills) and at 15:59 launched a &lt;a href=&quot;https://github.com/anthropics/claude-code/tree/main/plugins/ralph-wiggum&quot;&gt;Ralph Loop&lt;/a&gt; with a massive prompt to autonomously implement the engine task by task, run test262, review code, and commit.&lt;/p&gt;
&lt;p&gt;At 16:02 the first commit landed. I went into a meeting. An hour later at 17:09 it was already executing JavaScript:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;target/release/jsse -e &#39;function Foo(x) { this.x = x; } var f = new Foo(42); console.log(f.x);&#39;
42
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;By 19:10 that evening it had reached 17.63% test262 pass rate. I signed off for the night. From zero to a functioning JS engine in Rust in about four hours.&lt;/p&gt;
&lt;h2&gt;The Numbers&lt;/h2&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Metric&lt;/th&gt;
&lt;th&gt;Value&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Start date&lt;/td&gt;
&lt;td&gt;January 27, 2026&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;100% non-staging test262&lt;/td&gt;
&lt;td&gt;March 9, 2026 (42 days)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Total commits&lt;/td&gt;
&lt;td&gt;592&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Total Rust&lt;/td&gt;
&lt;td&gt;~170,000 lines&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Lines added (all time)&lt;/td&gt;
&lt;td&gt;929,475&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Lines removed (all time)&lt;/td&gt;
&lt;td&gt;448,317&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;My hands-on-keyboard time&lt;/td&gt;
&lt;td&gt;~4 hours total&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;That last number is the one that surprises people. Four hours spread across six weeks: 30 seconds here, two minutes there. The rest was the agent working autonomously.&lt;/p&gt;
&lt;h2&gt;How It Was Built&lt;/h2&gt;
&lt;p&gt;The setup was deliberately vanilla. Claude Code running in YOLO mode (auto-accept all tool use). No fancy orchestration. The plugins I used were:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;&lt;code&gt;/simplify&lt;/code&gt;&lt;/strong&gt;, used maybe once or twice when files got too large and needed refactoring&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;&lt;a href=&quot;https://github.com/anthropics/claude-code-ralph-plugin&quot;&gt;&lt;code&gt;ralph-wiggum-loop&lt;/code&gt;&lt;/a&gt;&lt;/strong&gt;, the built-in autonomous loop plugin. I started with this but found it would get lost after a while (more on this below)&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;&lt;a href=&quot;https://chiefloop.com/&quot;&gt;chiefloop.com&lt;/a&gt;&lt;/strong&gt;, an external tool for running Claude Code overnight. Useful for long unattended runs, though it was missing some things from my usual workflow&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;The core of the project was &lt;code&gt;PLAN.md&lt;/code&gt;, a task list that Claude generated itself from the ECMAScript spec and test262 submodules. My typical prompt was: &amp;quot;Read the first unfinished item in &lt;a href=&quot;http://plan.md/&quot;&gt;PLAN.md&lt;/a&gt;, plan and implement it, mark it as complete, commit and push.&amp;quot; That was it. I never reviewed the code, never reviewed the plan, never read a diff. &lt;code&gt;CLAUDE.md&lt;/code&gt; had the rules: no regressions allowed, always attempt to pass more tests than before. Sometimes I&#39;d push back when Claude tried to skip a task it deemed too hard: &amp;quot;Why are you skipping this? We have to implement everything, so might as well tackle it now.&amp;quot; But that was the extent of my guidance.&lt;/p&gt;
&lt;p&gt;The longest successful unattended loop was 16 hours for the Temporal API implementation. I kicked it off before bed and came back to a full implementation of &lt;code&gt;Instant&lt;/code&gt;, &lt;code&gt;ZonedDateTime&lt;/code&gt;, &lt;code&gt;PlainDateTime&lt;/code&gt;, &lt;code&gt;PlainDate&lt;/code&gt;, &lt;code&gt;PlainTime&lt;/code&gt;, &lt;code&gt;Duration&lt;/code&gt;, &lt;code&gt;Temporal.Now&lt;/code&gt;, IANA timezone support via ICU4X, and 12 calendar systems, with 4,482 test262 tests passing. Temporal is one of the largest and most complex proposals in recent ECMAScript history, and the agent handled it in a single overnight session.&lt;/p&gt;
&lt;h2&gt;What the Prompts Actually Looked Like&lt;/h2&gt;
&lt;p&gt;The previous section describes the workflow in the abstract: plan, implement, test, commit. But what did that actually look like day-to-day? Here are real prompts from the project, and they tell a story of their own.&lt;/p&gt;
&lt;p&gt;The single most repeated prompt — the heartbeat of the entire project, appearing 20+ times — was this:&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;Take a look at &lt;a href=&quot;http://plan.md/&quot;&gt;PLAN.md&lt;/a&gt; and related plan/ files and file the next feature that has most impact on test262 pass rate. Come up with 3 possible features to work on and their respective impact and once we know that we can choose one.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;That was the core loop. I didn&#39;t pick what to work on; I asked the agent to analyze test262 coverage gaps, propose options ranked by impact, and then I&#39;d pick one (or just say &amp;quot;go&amp;quot;). Most of my interaction was at this level: strategic, not tactical.&lt;/p&gt;
&lt;p&gt;When things went sideways, the prompts shifted to debugging:&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;What happened? What were you trying to do? Why is this bash running for 40 minutes?&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;Long-running sessions would sometimes stall on a single test or get stuck in a compilation loop. The fix was usually to kill the session and restart with a narrower scope.&lt;/p&gt;
&lt;p&gt;As the project matured, I got more ambitious with parallelism:&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;How about we delegate each of these tasks to a team of agents to plan and implement where each agent is working in its own worktree and then you merge it all together once it&#39;s complete? That would likely give us ~300 new passes?&lt;/p&gt;
&lt;/blockquote&gt;
&lt;blockquote&gt;
&lt;p&gt;Let&#39;s create 3 plans, then I will start 3 agents to work on each in separate worktrees.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;Claude Code supports &lt;a href=&quot;https://docs.anthropic.com/en/docs/claude-code/sub-agents#worktree-isolation&quot;&gt;worktrees&lt;/a&gt; — isolated git branches that multiple agents can work on simultaneously. This is how the hardest features got done: split the work into independent tracks, run them in parallel, merge. The Temporal API implementation used this pattern heavily.&lt;/p&gt;
&lt;p&gt;Some prompts were just green lights for marathon sessions:&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;Yes. Start and complete all phases, committing in between.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;blockquote&gt;
&lt;p&gt;We should probably tackle option A straight ahead and not shy away. Create a plan to deal with array holes, then implement.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;That last one is about &lt;a href=&quot;https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Indexed_collections#sparse_arrays&quot;&gt;array holes&lt;/a&gt; — one of JavaScript&#39;s gnarliest semantic corners. &lt;code&gt;[1, , 3]&lt;/code&gt; has a hole at index 1 that behaves differently from &lt;code&gt;undefined&lt;/code&gt; in subtle, spec-mandated ways. The agent handled it, but it took a dedicated session.&lt;/p&gt;
&lt;p&gt;The endgame prompts (98%+) were the most interesting, because the nature of the work changed completely. Instead of implementing features, we were hunting individual test failures:&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;We are so close to full compliance. What&#39;s missing? Let&#39;s just find the first category not passing 100% and deal with that.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;blockquote&gt;
&lt;p&gt;Which of these will close up 100% compliance on a specific category?&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;And the final push to 100%:&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;We have a @PLAN.md and a bunch of @plan/ files. Our ultimate goal is 100% test262 compliance. We are close but need to close a few gaps. Your task is to create a plan to take us to 100% compliance — forward progress, no regressions.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;The pattern across all of these is that I was never telling the agent &lt;em&gt;how&lt;/em&gt; to implement anything. I was setting goals, choosing priorities, and occasionally unblocking. The agent did the engineering.&lt;/p&gt;
&lt;h2&gt;The Pass Rate Curve&lt;/h2&gt;
&lt;p&gt;&lt;img src=&quot;https://p.ocmatos.com/img/2026/03/jsse_passrate.png&quot; alt=&quot;JSSE Test262 Pass Rate&quot; /&gt;&lt;/p&gt;
&lt;p&gt;The chart tells the story better than I can. A few milestones worth calling out:&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Date&lt;/th&gt;
&lt;th&gt;Rate&lt;/th&gt;
&lt;th&gt;What happened&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Jan 27&lt;/td&gt;
&lt;td&gt;26%&lt;/td&gt;
&lt;td&gt;Project kickoff: lexer, parser, basic interpreter&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Jan 31&lt;/td&gt;
&lt;td&gt;51%&lt;/td&gt;
&lt;td&gt;String.prototype wiring bug fix exposed ~11k tests (+23% in a single day)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Feb 3&lt;/td&gt;
&lt;td&gt;63%&lt;/td&gt;
&lt;td&gt;Generators via state machine approach&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Feb 10&lt;/td&gt;
&lt;td&gt;86%&lt;/td&gt;
&lt;td&gt;Temporal API phase 1&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Feb 13&lt;/td&gt;
&lt;td&gt;88%&lt;/td&gt;
&lt;td&gt;Full suite expansion: added intl402, test count nearly doubled (~48k → ~92k), pass rate still went &lt;em&gt;up&lt;/em&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Feb 22&lt;/td&gt;
&lt;td&gt;95%&lt;/td&gt;
&lt;td&gt;SharedArrayBuffer + Atomics (868 new passes in one day)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Mar 5&lt;/td&gt;
&lt;td&gt;99.6%&lt;/td&gt;
&lt;td&gt;Massive parser early-errors batch (274 fixes)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Mar 9&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;100%&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;Array.fromAsync&lt;/code&gt; was the final fix&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;The Jan 31 jump is my favorite. A single bug fix in how &lt;code&gt;String.prototype&lt;/code&gt; was wired up unblocked about 11,000 tests overnight. That&#39;s the kind of thing where having the full test262 suite as your feedback signal is invaluable; you find bugs by their blast radius.&lt;/p&gt;
&lt;h2&gt;Engine Comparison&lt;/h2&gt;
&lt;p&gt;I ran the same test262 suite against Node.js and Boa (another Rust-based JS engine) for comparison. Same checkout, same harness, same timeout, same machine (128-core):&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Engine&lt;/th&gt;
&lt;th&gt;Version&lt;/th&gt;
&lt;th&gt;Scenarios&lt;/th&gt;
&lt;th&gt;Pass&lt;/th&gt;
&lt;th&gt;Fail&lt;/th&gt;
&lt;th&gt;Rate&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;JSSE&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;latest&lt;/td&gt;
&lt;td&gt;101,234&lt;/td&gt;
&lt;td&gt;101,044&lt;/td&gt;
&lt;td&gt;190&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;99.81%&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Boa&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;v0.21&lt;/td&gt;
&lt;td&gt;91,986&lt;/td&gt;
&lt;td&gt;83,260&lt;/td&gt;
&lt;td&gt;8,726&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;90.51%&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Node&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;v25.8.0&lt;/td&gt;
&lt;td&gt;91,986&lt;/td&gt;
&lt;td&gt;79,201&lt;/td&gt;
&lt;td&gt;11,986&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;86.86%&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;Some important caveats: Node&#39;s failures are dominated by Temporal (not shipped in Node 25, that&#39;s 8,980 of its 11,986 failures). Module tests (799 scenarios) are skipped for Node because the test adapter can&#39;t run ES modules through it. Boa&#39;s main gaps are parser-level (assignment target validation, class destructuring, RegExp property escapes). These are mature, production-quality engines being compared on a metric they weren&#39;t specifically optimizing for, so take the comparison for what it is.&lt;/p&gt;
&lt;p&gt;JSSE&#39;s scenario count is higher (101,234 vs 91,986) because it includes the full staging test suite. On non-staging tests, JSSE is at a flat 100%.&lt;/p&gt;
&lt;p&gt;I also ran JSSE against the &lt;a href=&quot;https://github.com/acornjs/acorn&quot;&gt;acorn&lt;/a&gt; test suite as a real-world sanity check beyond test262. All 13,507 acorn tests pass.&lt;/p&gt;
&lt;h2&gt;A Note on Staging Tests&lt;/h2&gt;
&lt;p&gt;At one point I thought JSSE was passing staging tests too. It wasn&#39;t. &lt;code&gt;run-test262.py&lt;/code&gt; wasn&#39;t actually running them. When I finally ran the staging suite explicitly, things got interesting.&lt;/p&gt;
&lt;p&gt;The non-staging suite is well-maintained and internally consistent. Staging is where proposed tests live before promotion, and it shows. JSSE currently passes 2,762 of 2,808 staging scenarios (98.36%). The remaining failures fall into three categories: flaky timeouts, &lt;code&gt;libm&lt;/code&gt; precision gaps, and, most interestingly, what appear to be bugs in the test suite itself.&lt;/p&gt;
&lt;p&gt;For example, &lt;a href=&quot;https://github.com/pmatos/jsse/issues/31&quot;&gt;six staging tests&lt;/a&gt; use a shared harness function that relies on &lt;code&gt;eval&lt;/code&gt;-declared variables leaking into the enclosing scope, which strict mode explicitly prevents. These tests fail on both JSSE and Node. They need a &lt;code&gt;noStrict&lt;/code&gt; flag upstream. Another case: &lt;a href=&quot;https://github.com/pmatos/jsse/issues/36&quot;&gt;two staging tests&lt;/a&gt; expect AnnexB hoisting behavior for block-scoped &lt;code&gt;function arguments()&lt;/code&gt; declarations that directly contradicts a main-suite test. The main suite reflects a recent spec change that no major engine has implemented yet. JSSE follows the spec, so the main-suite test passes and the staging tests fail.&lt;/p&gt;
&lt;p&gt;When the test suite you&#39;re targeting starts testing &lt;em&gt;you&lt;/em&gt; back, that&#39;s a good sign.&lt;/p&gt;
&lt;h2&gt;What About Performance?&lt;/h2&gt;
&lt;p&gt;Bad. Intentionally so. Zero effort was spent on optimization. JSSE is a pure tree-walking interpreter with no bytecode compilation. Here are some microbenchmarks against Node (V8) and Boa:&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Benchmark&lt;/th&gt;
&lt;th&gt;Node v18&lt;/th&gt;
&lt;th&gt;Boa v0.21&lt;/th&gt;
&lt;th&gt;JSSE v0.1&lt;/th&gt;
&lt;th&gt;JSSE vs Node&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;loop&lt;/td&gt;
&lt;td&gt;0.18s&lt;/td&gt;
&lt;td&gt;2.02s&lt;/td&gt;
&lt;td&gt;2.90s&lt;/td&gt;
&lt;td&gt;16x&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;fib&lt;/td&gt;
&lt;td&gt;0.21s&lt;/td&gt;
&lt;td&gt;2.53s&lt;/td&gt;
&lt;td&gt;28.57s&lt;/td&gt;
&lt;td&gt;136x&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;string&lt;/td&gt;
&lt;td&gt;0.19s&lt;/td&gt;
&lt;td&gt;0.62s&lt;/td&gt;
&lt;td&gt;4.74s&lt;/td&gt;
&lt;td&gt;25x&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;array&lt;/td&gt;
&lt;td&gt;0.17s&lt;/td&gt;
&lt;td&gt;0.53s&lt;/td&gt;
&lt;td&gt;119.45s&lt;/td&gt;
&lt;td&gt;703x&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;object&lt;/td&gt;
&lt;td&gt;0.35s&lt;/td&gt;
&lt;td&gt;0.69s&lt;/td&gt;
&lt;td&gt;0.85s&lt;/td&gt;
&lt;td&gt;2.4x&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;regex&lt;/td&gt;
&lt;td&gt;0.16s&lt;/td&gt;
&lt;td&gt;0.24s&lt;/td&gt;
&lt;td&gt;5.62s&lt;/td&gt;
&lt;td&gt;35x&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;closures&lt;/td&gt;
&lt;td&gt;0.18s&lt;/td&gt;
&lt;td&gt;2.79s&lt;/td&gt;
&lt;td&gt;15.48s&lt;/td&gt;
&lt;td&gt;86x&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;json&lt;/td&gt;
&lt;td&gt;0.20s&lt;/td&gt;
&lt;td&gt;0.44s&lt;/td&gt;
&lt;td&gt;0.25s&lt;/td&gt;
&lt;td&gt;1.2x&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;The range is enormous: from 1.2x (JSON, where JSSE actually beats Boa) to 703x (array operations). The &lt;code&gt;fib&lt;/code&gt; benchmark is recursive Fibonacci, which hammers function call overhead, and 136x is exactly what you&#39;d expect from a tree-walker that re-traverses the AST on every call. The array benchmark is the worst case, likely hitting a pathological path in the array prototype implementation.&lt;/p&gt;
&lt;p&gt;Comparing against Boa is more fair since both are interpreters in Rust. JSSE is roughly 1.2x to 225x slower than Boa depending on the benchmark. Object operations and JSON are actually competitive.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://p.ocmatos.com/img/2026/03/benchmark_slowdown.png&quot; alt=&quot;Slowdown relative to Node.js&quot; /&gt;&lt;/p&gt;
&lt;p&gt;The test262 suite itself reveals where the pain is. All 30 of the slowest tests are RegExp Unicode property escape tests (&lt;code&gt;&#92;p{...}&lt;/code&gt;), each taking 64–84 seconds. The slowest, &lt;code&gt;Script_Extensions_-_Sharada.js&lt;/code&gt;, hits 84 seconds. These tests compile and run regexes that enumerate thousands of Unicode codepoints, and the bottleneck is byte-level WTF-8 regex matching for Unicode property classes. Out of 198,258 total test scenarios, 900 take over 10 seconds and 1,062 take over 1 second. Everything else finishes fast. The long tail is almost entirely regex.&lt;/p&gt;
&lt;p&gt;This is fine. Correctness was the only goal. Performance is the obvious next step.&lt;/p&gt;
&lt;h2&gt;The Cost&lt;/h2&gt;
&lt;p&gt;I used a Claude Code Max 20x subscription for this project, so I didn&#39;t pay per-token. But &lt;code&gt;ccusage&lt;/code&gt; tracks what the equivalent API cost would have been, and the number is interesting for context: &lt;strong&gt;$4,618.94&lt;/strong&gt; across 302 sessions over 47 days. The breakdown by model:&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Model&lt;/th&gt;
&lt;th&gt;Cost (API equivalent)&lt;/th&gt;
&lt;th&gt;Share&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Claude Opus 4.6&lt;/td&gt;
&lt;td&gt;$4,062.91&lt;/td&gt;
&lt;td&gt;88%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Claude Sonnet 4.6&lt;/td&gt;
&lt;td&gt;$345.54&lt;/td&gt;
&lt;td&gt;7.5%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Claude Haiku 4.5&lt;/td&gt;
&lt;td&gt;$124.83&lt;/td&gt;
&lt;td&gt;2.7%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Claude Sonnet 4.5&lt;/td&gt;
&lt;td&gt;$45.56&lt;/td&gt;
&lt;td&gt;1%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Claude Opus 4.5&lt;/td&gt;
&lt;td&gt;$40.11&lt;/td&gt;
&lt;td&gt;0.9%&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;Opus 4.6 did the vast majority of the heavy lifting. Haiku was used for background tasks like subagent searches. The total token count was about 8.9 billion, but aggressive prompt caching kept costs manageable; cache read tokens alone were 8.8 billion.&lt;/p&gt;
&lt;p&gt;To put it in perspective: $4,619 in API-equivalent cost for a 170,000-line Rust codebase that passes 100% of test262. That&#39;s roughly $0.03 per line of code, or about $47 per percentage point of test262 compliance.&lt;/p&gt;
&lt;h2&gt;What I Learned&lt;/h2&gt;
&lt;p&gt;&lt;strong&gt;The plan matters more than the code.&lt;/strong&gt; Claude generated the entire &lt;code&gt;PLAN.md&lt;/code&gt; itself from the spec and test262 submodules. I didn&#39;t write it, didn&#39;t review it, just accepted it. And it worked. But looking back, the plan&#39;s quality was the single biggest lever on the project&#39;s speed. Claude picked a reasonable feature order, but it made some architectural decisions early on (like how to handle generators, how to structure the GC, how to deal with WTF-8 strings) that caused expensive rework later. If I had invested time upfront researching the right architecture and feeding that into the plan, I&#39;m fairly confident this project would have taken half the time. The lesson isn&#39;t &amp;quot;you must write the plan yourself,&amp;quot; it&#39;s &amp;quot;make sure the plan is good, however it gets made.&amp;quot;&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;test262 is an incredible feedback signal.&lt;/strong&gt; Having a comprehensive, well-structured test suite that the agent can run autonomously is what makes this kind of project possible. The agent doesn&#39;t need to understand JavaScript semantics deeply; it needs to make the number go up while not making it go down. test262 provides exactly that signal.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Agents get lost in long contexts.&lt;/strong&gt; This needs some explanation. Claude Code automatically compacts the conversation when it approaches the context limit (which was 200k tokens at the time of this project). Compaction summarizes older messages to free up space, but it inevitably loses detail: which specific tests were failing, what approach was being tried, what the error messages said. After several rounds of compaction in a long-running task, Claude would visibly degrade. It would go in circles, attempting the same fix repeatedly. It would pass some tests but regress others, then try to fix the regressions and break something else. My fix was to stop the session, split the task into smaller pieces, and start fresh. A new session with a focused prompt (&amp;quot;implement SharedArrayBuffer&amp;quot;) worked far better than a long-running loop that had been through multiple compactions. The sweet spot was: one feature per session, plan it, implement it, run tests, stop.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Re-architecture is expensive but not catastrophic.&lt;/strong&gt; There was a point where Claude realized the architecture wasn&#39;t right for async functions and had to go back and restructure things. It took longer than it should have, but it worked. A human architect who understood the problem space would have avoided this, which circles back to the first point.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Rust is good for agentic coding.&lt;/strong&gt; I chose Rust explicitly because I believe it&#39;s currently the best language for agent-driven development. The type system and compiler catch a massive class of bugs before runtime, which means the agent spends less time debugging and more time making forward progress. The strict compiler is essentially a second feedback signal alongside test262.&lt;/p&gt;
&lt;h2&gt;What&#39;s Next&lt;/h2&gt;
&lt;p&gt;Performance. The engine is a pure tree-walker, and the lowest-hanging fruit is bytecode compilation and a simple VM. There&#39;s probably 10-100x to be gained just from eliminating repeated AST traversal. Beyond that, inline caches, hidden classes, and eventually JIT compilation if I want to get serious.&lt;/p&gt;
&lt;p&gt;But honestly, the point of JSSE was never to build a production JavaScript engine. It was to explore what&#39;s possible with current agentic coding tools, and to learn about agent workflows in the process. On both counts, I&#39;m satisfied with the result.&lt;/p&gt;
&lt;h2&gt;Conclusions&lt;/h2&gt;
&lt;p&gt;The tools are amazing and the models are incredible, but this is just the beginning. Agentic coding is already changing the way people work, and I believe it will revolutionize software production (I&#39;ve written about &lt;a href=&quot;https://p.ocmatos.com/blog/formal-verification-in-your-terminal-esbmc-meets-claude-code.html&quot;&gt;some of my ideas on this&lt;/a&gt;). In the near future, writing a JS engine from scratch and optimizing it to be faster than anything else will be a walk in the park, literally the duration of a walk in the park. I&#39;ve learned a lot from this experiment and I&#39;ll keep having fun with it.&lt;/p&gt;
&lt;p&gt;The code is at &lt;a href=&quot;https://github.com/pmatos/jsse&quot;&gt;github.com/pmatos/jsse&lt;/a&gt;. MIT licensed. If you want to run test262 yourself, the README has full reproduction instructions. Contributions to JSSE are welcome, but please keep them agentic.&lt;/p&gt;
</content>
  </entry>
  <entry>
    <title>Introducing Musiker: An Interactive Timeline of Classical Music</title>
    <link href="https://p.ocmatos.com/blog/introducing-musiker-an-interactive-timeline-of-classical-music.html"/>
    <updated>2026-02-23T00:00:00Z</updated>
    <id>https://p.ocmatos.com/blog/introducing-musiker-an-interactive-timeline-of-classical-music.html</id>
    <content xml:lang="en" type="html">&lt;p&gt;&lt;a href=&quot;https://musiker.page/&quot;&gt;Musiker&lt;/a&gt; is an interactive timeline that maps the lifetimes and connections of classical music composers and players — from William Byrd in the 1540s to Lang Lang today.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://p.ocmatos.com/img/2026/02/musiker-piano-timeline.png&quot; alt=&quot;Piano timeline on musiker.page&quot; /&gt;&lt;/p&gt;
&lt;h2&gt;What You Can Explore&lt;/h2&gt;
&lt;p&gt;The site currently covers three instruments: &lt;strong&gt;piano&lt;/strong&gt;, &lt;strong&gt;violin&lt;/strong&gt;, and &lt;strong&gt;trombone&lt;/strong&gt;. Each timeline lays out musicians as horizontal bars spanning their lifetimes, set against color-coded era backgrounds — Baroque, Classical, Romantic, Modern, and Contemporary.&lt;/p&gt;
&lt;p&gt;Click on any person to open a detail panel with their portrait, biography, Wikipedia link, and a list of related people. Connection lines between musicians show teacher-student relationships (dashed blue) and family ties (solid red), making it easy to trace how musical traditions passed from one generation to the next.&lt;/p&gt;
&lt;p&gt;You can zoom in with Ctrl+scroll to focus on a specific period, or scroll horizontally to explore the full span of centuries.&lt;/p&gt;
&lt;div style=&quot;position: relative; padding-bottom: 56.25%; height: 0; overflow: hidden; max-width: 100%; margin: 2rem 0;&quot;&gt;
&lt;iframe style=&quot;position: absolute; top: 0; left: 0; width: 100%; height: 100%;&quot; src=&quot;https://www.youtube.com/embed/Nyh2OJA5UxY&quot; title=&quot;Musiker promo video&quot; frameborder=&quot;0&quot; allow=&quot;accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture&quot; allowfullscreen=&quot;&quot;&gt;&lt;/iframe&gt;
&lt;/div&gt;
&lt;h2&gt;Under the Hood&lt;/h2&gt;
&lt;p&gt;Musiker is a React + TypeScript application that renders everything as SVG — no charting libraries involved. The main challenge was fitting dozens of overlapping lifetimes into a readable layout. A greedy lane-packing algorithm assigns each person to the first available vertical lane where they don&#39;t overlap with anyone else, keeping the timeline compact without losing clarity.&lt;/p&gt;
&lt;p&gt;The data is split into three concerns: a shared list of people with bios and metadata, a shared list of connections between them, and per-instrument configuration files that define which people appear and what eras to show. Adding a new instrument is just a matter of creating a new JSON file and listing the relevant people.&lt;/p&gt;
&lt;h2&gt;Part of Rightkey&lt;/h2&gt;
&lt;p&gt;Musiker is built in collaboration with &lt;a href=&quot;https://rightkey.app/&quot;&gt;Rightkey.app&lt;/a&gt;, a music education platform. It&#39;s one piece of a broader effort to make classical music more accessible and easier to explore — whether you&#39;re a student, a teacher, or just curious about how Beethoven connects to Czerny connects to Liszt.&lt;/p&gt;
&lt;h2&gt;Try It Out&lt;/h2&gt;
&lt;p&gt;Head over to &lt;a href=&quot;https://musiker.page/&quot;&gt;musiker.page&lt;/a&gt; and explore. If you notice a missing composer, a wrong connection, or an instrument you&#39;d like to see added, use the feedback links at the top of the page — suggestions and corrections are very welcome.&lt;/p&gt;
</content>
  </entry>
  <entry>
    <title>ESBMC Plugin Update: Python Support, a New Home, and Where This Is Going</title>
    <link href="https://p.ocmatos.com/blog/esbmc-plugin-update-python-support-a-new-home-and-where-this-is-going.html"/>
    <updated>2026-02-23T00:00:00Z</updated>
    <id>https://p.ocmatos.com/blog/esbmc-plugin-update-python-support-a-new-home-and-where-this-is-going.html</id>
    <content xml:lang="en" type="html">&lt;p&gt;A couple of weeks ago I &lt;a href=&quot;https://p.ocmatos.com/blog/formal-verification-in-your-terminal-esbmc-meets-claude-code.html&quot;&gt;wrote about&lt;/a&gt; the ESBMC plugin for Claude Code — a way to bring bounded model checking into your terminal through an AI coding assistant. The response was encouraging, and things have moved quickly since then. This is a short update on what changed, and a first sketch of where I think this is heading.&lt;/p&gt;
&lt;h2&gt;What&#39;s New&lt;/h2&gt;
&lt;h3&gt;Python Support&lt;/h3&gt;
&lt;p&gt;Professor Cordeiro, ESBMC&#39;s lead, picked up the plugin and improved Python verification support. ESBMC has had a Python frontend for a while, but the plugin wasn&#39;t tuned for it. Now it is — you can point it at Python code with type annotations and get the same kind of exhaustive property checking that C users have had. Professor Cordeiro &lt;a href=&quot;https://www.linkedin.com/feed/update/urn:li:activity:7431460883609989120/&quot;&gt;announced this on LinkedIn&lt;/a&gt; along with a demo of verifying Python code for common issues.&lt;/p&gt;
&lt;p&gt;This matters because Python is where a lot of AI-generated code lands. If your LLM writes a function with type annotations, you can now formally verify properties about it without switching tools or contexts.&lt;/p&gt;
&lt;h3&gt;The Agent Marketplace&lt;/h3&gt;
&lt;p&gt;The plugin has moved to its own repository: &lt;a href=&quot;https://github.com/esbmc/agent-marketplace&quot;&gt;esbmc/agent-marketplace&lt;/a&gt;. This is the new home for ESBMC&#39;s Claude Code integration, and I maintain it. The separation from the main ESBMC repo is deliberate — the plugin evolves on a different cadence than the verifier itself, and having a dedicated repo makes it easier to iterate on the agent-facing interface without coupling to ESBMC&#39;s release cycle.&lt;/p&gt;
&lt;p&gt;Installation now uses the new repository names:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;$ claude
&amp;gt; /plugin marketplace add esbmc/agent-marketplace
&amp;gt; /plugin install esbmc-plugin@esbmc-marketplace
&lt;/code&gt;&lt;/pre&gt;
&lt;h2&gt;The Idea I Keep Coming Back To&lt;/h2&gt;
&lt;p&gt;I want to share something that&#39;s been shaping how I think about this plugin, even if I&#39;m not ready to lay out the full argument yet.&lt;/p&gt;
&lt;p&gt;Most of the time, verification is something you do &lt;em&gt;after&lt;/em&gt; writing code. You finish a function, maybe write some tests, and if you&#39;re particularly disciplined, you might run a static analyzer or a model checker before merging. Verification is a gate at the end of a pipeline.&lt;/p&gt;
&lt;p&gt;But what if it wasn&#39;t? What if verification happened &lt;em&gt;while&lt;/em&gt; you write code, the same way &lt;code&gt;cargo check&lt;/code&gt; gives you type errors as you go in Rust? Not as a final gate, but as a co-temporal activity — verification and implementation advancing together, each informing the other.&lt;/p&gt;
&lt;p&gt;This is what excites me about having ESBMC inside a coding agent. The agent doesn&#39;t need to &amp;quot;finish&amp;quot; the code and then &amp;quot;run verification&amp;quot; as a separate step. It can write a function, immediately verify a property, discover a counterexample, and revise — all in one fluid loop. The verification isn&#39;t a post-hoc audit; it&#39;s part of the writing process. Think of it as the difference between a spell checker you run on a finished document and one that underlines words as you type.&lt;/p&gt;
&lt;p&gt;The ESBMC plugin is a first step toward this. Today it&#39;s mostly used reactively — you write code, then ask Claude to verify it. But the skill system means Claude can reach for ESBMC proactively during code generation, and the pieces are falling into place for a tighter loop.&lt;/p&gt;
&lt;p&gt;I have more to say about what this looks like concretely — strategy cascading, counterexample-driven refinement, how you&#39;d encode verification profiles for different domains — but I&#39;ll save that for a dedicated post. For now, I wanted to plant the flag: verification should be co-temporal with coding, not a separate phase, and AI coding agents are the mechanism that makes this practical.&lt;/p&gt;
&lt;h2&gt;Try It&lt;/h2&gt;
&lt;p&gt;If you have ESBMC installed and use Claude Code, give the plugin a spin. It now works with both C and Python, and the &lt;code&gt;/verify&lt;/code&gt; and &lt;code&gt;/audit&lt;/code&gt; commands are the fastest way to get started. File issues on the &lt;a href=&quot;https://github.com/esbmc/agent-marketplace&quot;&gt;agent-marketplace repo&lt;/a&gt; — I want to hear what works and what doesn&#39;t.&lt;/p&gt;
</content>
  </entry>
  <entry>
    <title>Formal Verification in Your Terminal: ESBMC meets Claude Code</title>
    <link href="https://p.ocmatos.com/blog/formal-verification-in-your-terminal-esbmc-meets-claude-code.html"/>
    <updated>2026-02-13T00:00:00Z</updated>
    <id>https://p.ocmatos.com/blog/formal-verification-in-your-terminal-esbmc-meets-claude-code.html</id>
    <content xml:lang="en" type="html">&lt;p&gt;&lt;em&gt;Edit (2026-02-23): Updated plugin installation instructions to use the new repository names.&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;&lt;a href=&quot;https://esbmc.org/&quot;&gt;ESBMC&lt;/a&gt; is an open-source bounded model checker for C, C++, Python, Solidity, and CUDA. Unlike traditional testing, which only explores the paths you think to test, bounded model checking exhaustively explores all execution paths up to a given bound and produces mathematically verified counterexamples when it finds a bug. It&#39;s a powerful technique that catches bugs no amount of unit testing would find.&lt;/p&gt;
&lt;p&gt;&lt;a href=&quot;https://docs.anthropic.com/en/docs/claude-code/overview&quot;&gt;Claude Code&lt;/a&gt; is Anthropic&#39;s agentic coding assistant that lives in your terminal. It supports a plugin system that lets third-party tools integrate deeply into the AI workflow through custom slash commands and skills.&lt;/p&gt;
&lt;p&gt;A new &lt;a href=&quot;https://github.com/esbmc/esbmc/commit/4b79ba4&quot;&gt;Claude Code plugin for ESBMC&lt;/a&gt;, now part of ESBMC upstream, bridges the two. Let me show you what that looks like.&lt;/p&gt;
&lt;h2&gt;Setup&lt;/h2&gt;
&lt;p&gt;You need ESBMC installed and available somewhere on your system (the plugin will find it). Then add the ESBMC marketplace and install the plugin:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;$ claude
&amp;gt; /plugin marketplace add esbmc/agent-marketplace
&amp;gt; /plugin install esbmc-plugin@esbmc-marketplace
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Restart Claude Code and you&#39;re ready to go.&lt;/p&gt;
&lt;h2&gt;How It Works&lt;/h2&gt;
&lt;p&gt;The plugin provides two slash commands -- &lt;code&gt;/verify&lt;/code&gt; for quick checks and &lt;code&gt;/audit&lt;/code&gt; for comprehensive multi-pass security audits -- but you don&#39;t actually need to use them. The plugin registers a &lt;em&gt;skill&lt;/em&gt; that Claude activates automatically when you mention anything related to verification: &amp;quot;check this code for bugs&amp;quot;, &amp;quot;find memory leaks&amp;quot;, &amp;quot;prove correctness&amp;quot;, &amp;quot;detect buffer overflows&amp;quot;, and so on. Just talk to Claude naturally and it will reach for ESBMC when appropriate.&lt;/p&gt;
&lt;p&gt;Under the hood, the skill carries extensive reference material: CLI options, verification strategies, language-specific features, and ESBMC&#39;s intrinsics API. This means Claude knows how to pick the right solver, set appropriate unwind bounds, use k-induction for unbounded proofs, and even add &lt;code&gt;__ESBMC_assume&lt;/code&gt; / &lt;code&gt;__ESBMC_assert&lt;/code&gt; annotations to your source code when needed. You get the full power of a formal verification tool without having to memorize its command-line interface.&lt;/p&gt;
&lt;h2&gt;A Quick Demo&lt;/h2&gt;
&lt;script src=&quot;https://asciinema.org/a/VO4lFSXM8pje8oB8.js&quot; id=&quot;asciicast-VO4lFSXM8pje8oB8&quot; async=&quot;true&quot;&gt;&lt;/script&gt;
&lt;p&gt;Consider this small C program (&lt;a href=&quot;https://gist.github.com/pmatos/a01a86770caf0252bfb490608a4ec2dd&quot;&gt;gist&lt;/a&gt;):&lt;/p&gt;
&lt;pre class=&quot;language-c&quot;&gt;&lt;code class=&quot;language-c&quot;&gt;&lt;span class=&quot;token macro property&quot;&gt;&lt;span class=&quot;token directive-hash&quot;&gt;#&lt;/span&gt;&lt;span class=&quot;token directive keyword&quot;&gt;include&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&amp;lt;stdlib.h&gt;&lt;/span&gt;&lt;/span&gt;

&lt;span class=&quot;token keyword&quot;&gt;int&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;main&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;
    &lt;span class=&quot;token keyword&quot;&gt;int&lt;/span&gt; buf&lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;10&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;

    &lt;span class=&quot;token keyword&quot;&gt;for&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token keyword&quot;&gt;int&lt;/span&gt; i &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;token number&quot;&gt;0&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt; i &lt;span class=&quot;token operator&quot;&gt;&amp;lt;=&lt;/span&gt; &lt;span class=&quot;token number&quot;&gt;10&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt; i&lt;span class=&quot;token operator&quot;&gt;++&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;
        buf&lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;i&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt; &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; i &lt;span class=&quot;token operator&quot;&gt;*&lt;/span&gt; &lt;span class=&quot;token number&quot;&gt;2&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;

    &lt;span class=&quot;token keyword&quot;&gt;int&lt;/span&gt; &lt;span class=&quot;token operator&quot;&gt;*&lt;/span&gt;p &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;malloc&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token keyword&quot;&gt;sizeof&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token keyword&quot;&gt;int&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
    &lt;span class=&quot;token operator&quot;&gt;*&lt;/span&gt;p &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; buf&lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;9&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt; &lt;span class=&quot;token operator&quot;&gt;+&lt;/span&gt; &lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;

    &lt;span class=&quot;token keyword&quot;&gt;int&lt;/span&gt; total &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;token number&quot;&gt;0&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
    &lt;span class=&quot;token keyword&quot;&gt;for&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token keyword&quot;&gt;int&lt;/span&gt; i &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;token number&quot;&gt;0&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt; i &lt;span class=&quot;token operator&quot;&gt;&amp;lt;&lt;/span&gt; &lt;span class=&quot;token number&quot;&gt;10&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt; i&lt;span class=&quot;token operator&quot;&gt;++&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;
        total &lt;span class=&quot;token operator&quot;&gt;+=&lt;/span&gt; buf&lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;i&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;

    &lt;span class=&quot;token function&quot;&gt;free&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;p&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
    &lt;span class=&quot;token keyword&quot;&gt;return&lt;/span&gt; total&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
&lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;There are two bugs here. Can you spot them? They&#39;re the kind that compilers won&#39;t warn about and that can easily slip through code review.&lt;/p&gt;
&lt;p&gt;I asked Claude to &amp;quot;formally verify the code in demo.c using state of the art techniques&amp;quot;. It recognized this as a verification task, activated the ESBMC skill, and ran the model checker with array bounds checking, overflow detection, memory leak analysis, and NULL pointer checks. Here&#39;s what it found.&lt;/p&gt;
&lt;h3&gt;Bug 1: Array out-of-bounds write&lt;/h3&gt;
&lt;p&gt;ESBMC immediately finds that the first loop&#39;s condition &lt;code&gt;i &amp;lt;= 10&lt;/code&gt; causes an off-by-one error. With an array of size 10, valid indices are 0-9, but the loop writes to &lt;code&gt;buf[10]&lt;/code&gt;:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;Violated property:
  file demo.c line 10 function main
  array bounds violated: array `buf&#39; upper bound
  (signed long int)i &amp;lt; 10
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This isn&#39;t a heuristic or a guess -- it&#39;s a formal proof that the property is violated, backed by a concrete counterexample trace showing exactly the state that triggers the bug.&lt;/p&gt;
&lt;h3&gt;Bug 2: NULL pointer dereference&lt;/h3&gt;
&lt;p&gt;The model checker stops at the first violation, so after fixing the off-by-one, a second run catches the &lt;code&gt;malloc&lt;/code&gt; without a NULL check:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;State 23 file demo.c line 12 function main
  p = (signed int *)0

Violated property:
  file demo.c line 13 function main
  dereference failure: NULL pointer
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;ESBMC models &lt;code&gt;malloc&lt;/code&gt; as potentially returning NULL (as the C standard permits), and proves that the dereference at line 13 is reachable in that case. Again, a concrete counterexample, not a warning.&lt;/p&gt;
&lt;h2&gt;Why This Matters&lt;/h2&gt;
&lt;p&gt;Static analyzers flag potential issues. Bounded model checkers &lt;em&gt;prove&lt;/em&gt; them. When ESBMC says &amp;quot;verification failed&amp;quot;, it means there is a real, reachable execution path that triggers the bug, and it shows you exactly what that path is.&lt;/p&gt;
&lt;p&gt;Having this integrated into Claude Code means you can ask it to verify your code as naturally as you&#39;d ask it to write a test. The AI handles constructing the right ESBMC invocation and interpreting the counterexample traces, while the mathematical rigor comes from the model checker itself -- a nice separation of concerns.&lt;/p&gt;
&lt;h2&gt;Formal Methods and the Age of Generated Code&lt;/h2&gt;
&lt;p&gt;I&#39;ve been a fan of formal methods for a long time, but I&#39;ll be honest: I rarely used them in practice. The barrier was always friction. Setting up the tooling, learning the command-line options, figuring out the right unwind bounds, interpreting the output -- it was a lot of overhead for finding corner-case bugs that might not surface for months or years. The payoff was real but the upfront cost was hard to justify in day-to-day work.&lt;/p&gt;
&lt;p&gt;I think that&#39;s about to change, and the reason is LLM-generated code. When an AI writes most of your code, you gain speed but you lose something: the deep familiarity with every line that comes from having typed it yourself. Subtle bugs -- off-by-one errors, missing NULL checks, integer overflows in edge cases -- are exactly the kind of thing that LLMs produce confidently and that humans skim past during review.&lt;/p&gt;
&lt;p&gt;Formal verification is a natural counterpart to code generation. The LLM generates code fast; the model checker proves properties about it exhaustively. Together they give you something neither can offer alone: rapid development with mathematical guarantees that entire classes of bugs are absent. The LLM lowers the barrier to writing code, and the model checker lowers the barrier to trusting it.&lt;/p&gt;
&lt;p&gt;This plugin is a small step in that direction -- making formal verification as easy as asking a question in your terminal. I hope it&#39;s the beginning of a much broader trend.&lt;/p&gt;
&lt;h2&gt;Beyond the Basics&lt;/h2&gt;
&lt;p&gt;The demo above only scratches the surface. The plugin also supports:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;Multi-language verification&lt;/strong&gt; -- C++, Python (with type annotations), Solidity smart contracts, and CUDA kernels.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Comprehensive audits&lt;/strong&gt; -- the &lt;code&gt;/audit&lt;/code&gt; command runs six verification passes: a quick scan, memory safety, integer safety, concurrency checks, deep verification with higher unwind bounds, and a k-induction proof attempt.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Verification annotations&lt;/strong&gt; -- Claude knows ESBMC&#39;s intrinsics API and can add &lt;code&gt;__ESBMC_nondet_*()&lt;/code&gt; for symbolic inputs, &lt;code&gt;__ESBMC_assume()&lt;/code&gt; for preconditions, and &lt;code&gt;__ESBMC_assert()&lt;/code&gt; for custom properties directly to your code.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Strategy selection&lt;/strong&gt; -- from incremental BMC for quick bug hunting to k-induction for unbounded proofs of correctness.&lt;/li&gt;
&lt;/ul&gt;
&lt;h2&gt;Current Status&lt;/h2&gt;
&lt;p&gt;The plugin has landed upstream in the ESBMC repository (&lt;a href=&quot;https://github.com/esbmc/esbmc/commit/4b79ba4&quot;&gt;commit 4b79ba4&lt;/a&gt;). Give it a try if you have ESBMC installed -- I&#39;d love to hear how it works for you.&lt;/p&gt;
</content>
  </entry>
  <entry>
    <title>Claude Code Usage in Waybar</title>
    <link href="https://p.ocmatos.com/blog/claude-code-usage-in-waybar.html"/>
    <updated>2026-02-11T00:00:00Z</updated>
    <id>https://p.ocmatos.com/blog/claude-code-usage-in-waybar.html</id>
    <content xml:lang="en" type="html">&lt;p&gt;If you&#39;re a heavy Claude Code user, you&#39;ve probably been caught off guard by hitting your session or weekly usage cap mid-flow. I wrote a small script that shows your current Claude Code API usage right in &lt;a href=&quot;https://github.com/Alexays/Waybar&quot;&gt;Waybar&lt;/a&gt;, so you can always see where you stand at a glance.&lt;/p&gt;
&lt;p&gt;Here&#39;s what it looks like:&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://p.ocmatos.com/img/2026/02/waybar-claude-usage.png&quot; alt=&quot;Waybar with Claude Code usage module&quot; /&gt;&lt;/p&gt;
&lt;p&gt;The module on the left shows session and weekly utilization percentages. It color-codes based on session utilization — neutral brown under 50%, orange at 50-79%, and red at 80%+. Hovering shows the reset times.&lt;/p&gt;
&lt;h2&gt;The Script&lt;/h2&gt;
&lt;p&gt;The full script is available as a &lt;a href=&quot;https://gist.github.com/pmatos/70f18d396f8e21804f8a437184de1292&quot;&gt;gist&lt;/a&gt;. It reads your Claude Code OAuth credentials, hits the Anthropic usage API, and outputs JSON that Waybar can consume directly.&lt;/p&gt;
&lt;p&gt;It requires &lt;code&gt;jq&lt;/code&gt; and &lt;code&gt;curl&lt;/code&gt; — both likely already on your system.&lt;/p&gt;
&lt;h2&gt;Waybar Configuration&lt;/h2&gt;
&lt;p&gt;Add a custom module to your Waybar config:&lt;/p&gt;
&lt;pre class=&quot;language-json&quot;&gt;&lt;code class=&quot;language-json&quot;&gt;&lt;span class=&quot;token property&quot;&gt;&quot;custom/claude&quot;&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;
    &lt;span class=&quot;token property&quot;&gt;&quot;exec&quot;&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&quot;~/.config/waybar/claude-usage.sh&quot;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt;
    &lt;span class=&quot;token property&quot;&gt;&quot;return-type&quot;&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&quot;json&quot;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt;
    &lt;span class=&quot;token property&quot;&gt;&quot;interval&quot;&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;token number&quot;&gt;120&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt;
    &lt;span class=&quot;token property&quot;&gt;&quot;format&quot;&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&quot;🤖 {}&quot;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt;
    &lt;span class=&quot;token property&quot;&gt;&quot;tooltip&quot;&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;token boolean&quot;&gt;true&lt;/span&gt;
&lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Then include &lt;code&gt;&amp;quot;custom/claude&amp;quot;&lt;/code&gt; in whichever module section you prefer (I have it in &lt;code&gt;modules-right&lt;/code&gt;).&lt;/p&gt;
&lt;p&gt;For styling, add something like this to your &lt;code&gt;style.css&lt;/code&gt;:&lt;/p&gt;
&lt;pre class=&quot;language-css&quot;&gt;&lt;code class=&quot;language-css&quot;&gt;&lt;span class=&quot;token selector&quot;&gt;#custom-claude&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;
    &lt;span class=&quot;token property&quot;&gt;background-color&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; #d4a574&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
    &lt;span class=&quot;token property&quot;&gt;color&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; #000000&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
    &lt;span class=&quot;token property&quot;&gt;font-weight&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; bold&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
&lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;

&lt;span class=&quot;token selector&quot;&gt;#custom-claude.claude-mid&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;
    &lt;span class=&quot;token property&quot;&gt;background-color&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; #f39c12&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
    &lt;span class=&quot;token property&quot;&gt;color&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; #000000&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
&lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;

&lt;span class=&quot;token selector&quot;&gt;#custom-claude.claude-high&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;
    &lt;span class=&quot;token property&quot;&gt;background-color&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; #e74c3c&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
    &lt;span class=&quot;token property&quot;&gt;color&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; #ffffff&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
&lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;

&lt;span class=&quot;token selector&quot;&gt;#custom-claude.claude-error&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;
    &lt;span class=&quot;token property&quot;&gt;background-color&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; #7f8c8d&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
    &lt;span class=&quot;token property&quot;&gt;color&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;:&lt;/span&gt; #ffffff&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
&lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The CSS classes are set by the script based on the session utilization percentage, so the module changes color as you approach your session limit.&lt;/p&gt;
&lt;p&gt;That&#39;s it — a simple way to keep an eye on your Claude Code budget without leaving your workflow.&lt;/p&gt;
</content>
  </entry>
  <entry>
    <title>Daring to imagine...</title>
    <link href="https://p.ocmatos.com/blog/daring-to-imagine.html"/>
    <updated>2026-01-19T00:00:00Z</updated>
    <id>https://p.ocmatos.com/blog/daring-to-imagine.html</id>
    <content xml:lang="en" type="html">&lt;p&gt;It&#39;s hard to even imagine what things will look like in 2-3 years time.&lt;/p&gt;
&lt;p&gt;I have been submerged in the day-to-day, pushing philosophical questions of the future aside. But when I do look up, I get the shivers. Here&#39;s what I see: A world where software materializes faster than we can read it — code blooming into existence like time-lapse flowers, flawless and strange. Where errors dissolve before we even notice them, perfection arriving faster than doubt.&lt;/p&gt;
&lt;p&gt;A world whose full implications I can barely grasp, like standing at the edge of something vast and inevitable, my stomach dropping, my mind racing to catch up.&lt;/p&gt;
</content>
  </entry>
  <entry>
    <title>Life on GenAI in 2025 (part 1 - Intro and Education)</title>
    <link href="https://p.ocmatos.com/blog/life-on-genai-in-2025-part-1-intro-and-education.html"/>
    <updated>2026-01-09T00:00:00Z</updated>
    <id>https://p.ocmatos.com/blog/life-on-genai-in-2025-part-1-intro-and-education.html</id>
    <content xml:lang="en" type="html">&lt;p&gt;I was already following closely the developments in GenAI, but watching my kids navigate these tools pushed me to keep exploring the landscape and following the trends.
Late 2024 and early 2025, I progressively moved from ChatGPT to Anthropic&#39;s Claude and Google&#39;s Gemini. Claude stood out for its conversational depth and ability to maintain context, while Gemini made it easy to experiment with both simultaneously and was already included with my Google One Subscription.&lt;/p&gt;
&lt;p&gt;Before I share specific use cases, though, it&#39;s important to talk about how I approach reliability when using these tools. LLMs are much better than they once were but still make mistakes. Different LLMs have different strengths and weaknesses—two models might perform equally well on certain benchmarks while struggling in completely different areas. For sensitive subjects like health or where I want to be more certain of something, I try to request science-backed responses that link to trusted websites and query multiple LLMs, so that I can compare the answers. Gemini itself will present its sources in an answer but you can also request a fact-check. When requested to do so Gemini will highlight the parts of the answer it&#39;s able to fact check, which is extremely useful.&lt;/p&gt;
&lt;p&gt;Let me show you what I mean with a quick example—even for straightforward historical facts. So, was D. Afonso Henriques the first king of Portugal? Gemini 3 Pro seems to think it was.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://p.ocmatos.com/img/2026/01/5beb0d68.png&quot; alt=&quot;image&quot; /&gt;&lt;/p&gt;
&lt;p&gt;But you&#39;re not convinced... so now you can actually ask it to double-check the answer.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://p.ocmatos.com/img/2026/01/badc1c74.png&quot; alt=&quot;image&quot; /&gt;&lt;/p&gt;
&lt;p&gt;Within moments, you get the answer highlighted and the source for each stated fact.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://p.ocmatos.com/img/2026/01/a4c066e3.png&quot; alt=&quot;image&quot; /&gt;&lt;/p&gt;
&lt;p&gt;Clearly there&#39;s already a concern to try to establish the facts in answers given by Gemini 3 Pro and this will continue to improve so that hopefully answers will be grounded on facts that you can manually check if you want.&lt;/p&gt;
&lt;p&gt;With these fact-checking tools in hand, I feel more confident using LLMs for learning new skills and helping my family. So with those caveats in mind, let me share how I&#39;ve been using these tools, starting with education.&lt;/p&gt;
&lt;h1&gt;Education&lt;/h1&gt;
&lt;p&gt;If I were studying full-time today, the experience would be dramatically more personal and exciting than when I was in school. Here are a few things that are possible now that simply weren&#39;t back in the day:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Scan a few pages of your textbook, feed them to &lt;a href=&quot;https://notebooklm.google.com/&quot;&gt;NotebookLM&lt;/a&gt;, generate a quiz, a podcast, a video presentation or a slide deck.&lt;/li&gt;
&lt;li&gt;Ask Claude using the Learning style to explain a topic to you.&lt;/li&gt;
&lt;li&gt;Use Suno to generate a song with lyrics which include the verb you want to learn to conjugate in French.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;The possibilities are pretty much endless depending on your learning preference. Kids these days will learn and want to use these technologies to get one step ahead. I have personally used these for my studies in Music Theory. Given that I am self-studying for the ABRSM exams, they have been extremely useful to help go through the curriculum without someone to ask.&lt;/p&gt;
&lt;p&gt;Beyond my own learning, what really excites me about LLMs is their potential to democratize education—helping those without access to a tutor, or complementing traditional tutoring with material presented from different perspectives.&lt;/p&gt;
&lt;p&gt;Let me get specific about how I organize these conversations, because the structure makes a huge difference.&lt;/p&gt;
&lt;p&gt;I use Claude Projects a lot as you&#39;ll see—these are workspaces where you can group related conversations and attach reference materials that persist across all chats. I tend to have projects to group a topic or set of chats. I have a project for &amp;quot;Piano Playing and Music Theory&amp;quot;. There I group all my chats about:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Goals - &amp;quot;are my goals for the next 6 months achievable given my progress in the last 6 months?&amp;quot;&lt;/li&gt;
&lt;li&gt;Doubts - &amp;quot;I really cannot understand compound time signatures. What&#39;s the point? How do I identify them?&amp;quot;&lt;/li&gt;
&lt;li&gt;Progress: &amp;quot;Given my current repertoire, what should I add next that takes me to the next level?&amp;quot;&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;As project knowledge I add all my practice sessions, my repertoire and the knowledge I started with. This means that at any time I can prompt Claude and discuss my practice methodology, my pieces, and repertoire and I will know that Claude is answering me with the full picture rather than with partial context.&lt;/p&gt;
&lt;p&gt;For example, when I asked about compound time signatures, Claude didn&#39;t just give me the textbook definition—it connected it to pieces I was already playing and explained how recognizing these patterns would help me with my current repertoire.&lt;/p&gt;
&lt;p&gt;Another exciting avenue for education is creation of material for studying. A recent example is the upcoming Physics exam of my daughter. With a few photo snaps of her Physics book, my wife &lt;a href=&quot;https://gemini.google.com/share/2e23260cb830&quot;&gt;queried Gemini for a learning plan&lt;/a&gt;. I then used this chat as a basis for Claude Code to create a &lt;a href=&quot;https://drive.google.com/file/d/1O8c8m3uOodzAFZDPMQ6fx1a0x7SEFIHP/view?usp=drive_link&quot;&gt;whole workbook&lt;/a&gt; with &lt;a href=&quot;https://drive.google.com/file/d/14Q8Pwv1sDvqu4aeuyIwLaDdfFT2QAft4/view?usp=drive_link&quot;&gt;solutions&lt;/a&gt; using Typst for my daughter.&lt;/p&gt;
&lt;p&gt;But education is just one piece of the puzzle—I&#39;ve found LLMs equally transformative for health and fitness goals, which I&#39;ll dive into next.&lt;/p&gt;
</content>
  </entry>
  <entry>
    <title>FEX x87 Stack Optimization</title>
    <link href="https://p.ocmatos.com/blog/fex-x87-stack-optimization.html"/>
    <updated>2024-09-25T00:00:00Z</updated>
    <id>https://p.ocmatos.com/blog/fex-x87-stack-optimization.html</id>
    <content xml:lang="en" type="html">&lt;p&gt;In this blog post, I will describe my recent work on improving and optimizing the handling of x87 code in FEX. This work landed on July 22, 2024, through commit &lt;a href=&quot;https://github.com/FEX-Emu/FEX/commit/a1378f94ce8e7843d5a3cd27bc72847973f8e7ec&quot;&gt;https://github.com/FEX-Emu/FEX/commit/a1378f94ce8e7843d5a3cd27bc72847973f8e7ec&lt;/a&gt; (tests and updates to size benchmarks landed separately).&lt;/p&gt;
&lt;p&gt;FEX is an ARM64 emulator of Intel assembly and, as such, needs to be able to handle things as old as x87, which was the way we did floating point pre-SSE.&lt;/p&gt;
&lt;p&gt;Consider a function to calculate the well-known Pythagorean Identity:&lt;/p&gt;
&lt;pre class=&quot;language-c&quot;&gt;&lt;code class=&quot;language-c&quot;&gt;&lt;span class=&quot;token keyword&quot;&gt;float&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;PytIdent&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token keyword&quot;&gt;float&lt;/span&gt; x&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;
  &lt;span class=&quot;token keyword&quot;&gt;return&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;sinf&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;x&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;*&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;sinf&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;x&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;token operator&quot;&gt;+&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;cosf&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;x&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;*&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;cosf&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;x&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
&lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Here is how this can be translated into assembly:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;  sub rsp, 4
  movss [rsp], xmm0
  fld [rsp]
  fsincos
  fmul st0, st0
  fxch
  fmul st0, st0
  faddp
  fstp [rsp]
  movss xmm0, [rsp]
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;These instructions will take &lt;code&gt;x&lt;/code&gt; from register &lt;code&gt;xmm0&lt;/code&gt; and return the result of the identity (the number 1, as we expect) to the same register.&lt;/p&gt;
&lt;h1&gt;A brief visit to the x87 FPU&lt;/h1&gt;
&lt;p&gt;The x87 FPU consists of 8 registers (&lt;code&gt;R0&lt;/code&gt; to &lt;code&gt;R7&lt;/code&gt;), which are aliased to the MMX registers - &lt;code&gt;mm0&lt;/code&gt; to &lt;code&gt;mm7&lt;/code&gt;. The FPU uses them as a stack and tracks the top of the stack through the &lt;code&gt;TOS&lt;/code&gt; (Top of Stack) or &lt;code&gt;TOP&lt;/code&gt; (my preferred way to refer to it), which is kept in the control word. To understand how it all fits together, I will describe the three most important components of this setup for the purpose of this post:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;FPU Stack-based operation&lt;/li&gt;
&lt;li&gt;Status register&lt;/li&gt;
&lt;li&gt;Tag register&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;From the &lt;a href=&quot;https://cdrdv2.intel.com/v1/dl/getContent/671200&quot;&gt;Intel spec&lt;/a&gt;, these components look like this (image labels are from the Intel in case you need to refer to it):&lt;/p&gt;
&lt;div style=&quot;text-align:center;&quot;&gt;
	&lt;img alt=&quot;Patch Details&quot; class=&quot;post-photo&quot; src=&quot;https://p.ocmatos.com/img/2024-09-23-fex-x87-stack-optimization/Screenshot%20From%202024-09-23%2010-54-21.png&quot; /&gt;
&lt;/div&gt;
&lt;p&gt;The 8 registers are labelled as &lt;code&gt;ST0&lt;/code&gt;-&lt;code&gt;ST7&lt;/code&gt; relative to the position of &lt;code&gt;TOP&lt;/code&gt;. In the case above &lt;code&gt;TOP&lt;/code&gt; is 3; therefore, the 3rd register is &lt;code&gt;ST0&lt;/code&gt;. Since the stack grows down, &lt;code&gt;ST1&lt;/code&gt; is the 4th register and so on until it wraps around. When the FPU is initialized &lt;code&gt;TOP&lt;/code&gt; is set to 0. Once a value is loaded, the &lt;code&gt;TOP&lt;/code&gt; is decreased, its value becomes 7 and the first value added to the stack is in the 7th register. Because the MMX registers are aliased to the bottom 64 bits of the stack registers, we can also refer to them as &lt;code&gt;MM0&lt;/code&gt; to &lt;code&gt;MM7&lt;/code&gt;. This is not strictly correct because the MMX registers are 64 bits and the FPU registers are 80 bits but if we squint, they are basically the same.&lt;/p&gt;
&lt;p&gt;The spec provides an example of how the FPU stack works. I copy the image here verbatim for your understanding. Further information can be found in the spec itself.&lt;/p&gt;
&lt;div style=&quot;text-align:center;&quot;&gt;
	&lt;img alt=&quot;Patch Details&quot; class=&quot;post-photo&quot; src=&quot;https://p.ocmatos.com/img/2024-09-23-fex-x87-stack-optimization/Screenshot%20From%202024-09-23%2010-54-43.png&quot; /&gt;
&lt;/div&gt;
&lt;p&gt;The &lt;code&gt;TOP&lt;/code&gt; value is stored along with other flags in the FPU Status Word, which we won&#39;t discuss in detail here.&lt;/p&gt;
&lt;div style=&quot;text-align:center;&quot;&gt;
	&lt;img alt=&quot;Patch Details&quot; class=&quot;post-photo&quot; src=&quot;https://p.ocmatos.com/img/2024-09-23-fex-x87-stack-optimization/Screenshot%20From%202024-09-23%2010-54-53.png&quot; /&gt;
&lt;/div&gt;
&lt;p&gt;And the tag word marks which registers are valid or empty. The Intel spec also allows marking the registers as zero or special. At the moment, tags in FEX are binary, marking the register as valid or invalid. Instead of two bits per register, we only use one bit of information per register.&lt;/p&gt;
&lt;div style=&quot;text-align:center;&quot;&gt;
	&lt;img alt=&quot;Patch Details&quot; class=&quot;post-photo&quot; src=&quot;https://p.ocmatos.com/img/2024-09-23-fex-x87-stack-optimization/Screenshot%20From%202024-09-23%2010-55-21.png&quot; /&gt;
&lt;/div&gt;
&lt;h1&gt;Current issues&lt;/h1&gt;
&lt;p&gt;The way things worked in FEX before this patch landed is that stack based operations would essentially always lower to perform three operations:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Load stack operation argument register values from special memory location;&lt;/li&gt;
&lt;li&gt;Call soft-float library to perform the operation;&lt;/li&gt;
&lt;li&gt;Store result in special memory location;&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;For example, let&#39;s consider the instruction &lt;code&gt;FADDP st4, st0&lt;/code&gt;. This instruction performs &lt;code&gt;st0 &amp;lt;- f80add(st4, st0) &amp;amp;&amp;amp; pop()&lt;/code&gt;. In &lt;code&gt;x87.cpp&lt;/code&gt;, the lowering of this instruction involved the following steps:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Load current value of top from memory;&lt;/li&gt;
&lt;li&gt;With the current value of top, we can load the values of the registers in the instruction, in this case st4 and st0 since they are at a memory offset from top;&lt;/li&gt;
&lt;li&gt;Then we call the soft-float library function to perform 80-bit adds;&lt;/li&gt;
&lt;li&gt;Finally we store the result into st4 (which is a location in memory);&lt;/li&gt;
&lt;li&gt;And pop the stack, which means marking the current top value invalid and incrementing top;&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;In code with a lot of x87 usage, these operations become cumbersome and data movement makes the whole code extremely slow. Therefore, we redesigned the x87 system to cope better with stack operations. FEX support a reduced precision mode that uses 64bit floating points rather than 80 bits, avoiding the soft-float library calls and this work applies equally to the code in reduced precision mode.&lt;/p&gt;
&lt;h1&gt;The new pass&lt;/h1&gt;
&lt;p&gt;The main observation to understand the operation of the new pass is that when compiling a block of instructions, where multiple instructions are x87 instructions, before code generation we have an idea of what the stack will look like, and if we have complete stack information we can generate much better code than the one generated on a per-instruction basis.&lt;/p&gt;
&lt;p&gt;Instead of lowering each x87 instruction in x87.cpp to its components as discussed earlier, we simply generate stack operation IR instructions (all of these added new since there were no stack operations in the IR). Then an optimization pass goes through the IR on a per block basis and optimizes the operations and lowers them.&lt;/p&gt;
&lt;p&gt;Here’s a step-by-step explanation of how this scheme works for the code above. When the above code gets to the OpcodeDispatcher, there’ll be one function a reasonable amount of opcodes, but generally one per functionality, meaning that although there are several opcodes for different versions of fadd, there is a single function in OpcodeDispatcher for x87 (in x87.cpp) implementing it.&lt;/p&gt;
&lt;p&gt;Each of these functions will just transform the incoming instruction into an IR node that deals with the stack. These stack IR operations have Stack in its name. See &lt;a href=&quot;https://github.com/FEX-Emu/FEX/blob/FEX-2409/FEXCore/Source/Interface/IR/IR.json&quot;&gt;IR.json&lt;/a&gt; for a list of these.&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;x86 Asm&lt;/th&gt;
&lt;th&gt;IR nodes&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;fld qword [rsp]&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;LoadMem&lt;/code&gt; + &lt;code&gt;PushStack&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;fsincos&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;F80SinCosStack&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;fmul st0, st0&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;F80MulStack&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;fxch&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;F80StackXchange&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;fmul st0, st0&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;F80MulStack&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;faddp&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;F80AddStack&lt;/code&gt; + &lt;code&gt;PopStackDestroy&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;fstp qword [rsp]&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;StoreStackMemory&lt;/code&gt; + &lt;code&gt;PopStackDestroy&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;The IR will then look like:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;	%9 i64 = LoadRegister #0x4, GPR
	%10 i128 = LoadMem FPR, %9 i64, %Invalid, #0x10, SXTX, #0x1
	(%11 i0) PushStack %10 i128, %10 i128, #0x10, #0x1
	(%12 i0) F80SINCOSStack
	%13 i64 = Constant #0x0
	(%14 i8) StoreContext GPR, %13 i64, #0x3fa
	(%15 i0) F80MulStack #0x0, #0x0
	(%16 i0) F80StackXchange #0x1
	%17 i64 = Constant #0x0
	(%18 i8) StoreContext GPR, %17 i64, #0x3f9
	(%19 i0) F80MulStack #0x0, #0x0
	(%20 i0) F80AddStack #0x1, #0x0
	(%21 i0) PopStackDestroy
	%23 i64 = LoadRegister #0x4, GPR
	(%24 i0) StoreStackMemory %23 i64, i128, #0x1, #0x8
	(%25 i0) PopStackDestroy
&lt;/code&gt;&lt;/pre&gt;
&lt;h2&gt;The Good Case&lt;/h2&gt;
&lt;p&gt;Remember that before this pass, we would have never generated this IR as is. There were no stack operations, therefore each of these &lt;code&gt;F80MulStack&lt;/code&gt; operations, would generate loads from the MMX registers that are mapped into memory for each of the operands, call to &lt;code&gt;F80Mul&lt;/code&gt; softfloat library function and then a write to the memory mapped MMX register to write the result.&lt;/p&gt;
&lt;p&gt;However, once we get to the optimization pass a see a straight line block line this, we can step through the IR nodes and create a virtual stack with pointers to the IR nodes. I have sketched the stacks for the first few loops in the pass.&lt;/p&gt;
&lt;p&gt;The pass will only process &lt;em&gt;x87&lt;/em&gt; instructions, these are the one whose IR nodes are marked with &lt;code&gt;&amp;quot;X87: true&lt;/code&gt; in &lt;code&gt;IR.json&lt;/code&gt;. Therefore the first two instructions above:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;	%9 i64 = LoadRegister #0x4, GPR
	%10 i128 = LoadMem FPR, %9 i64, %Invalid, #0x10, SXTX, #0x1
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;are ignored. But &lt;code&gt;PushStack&lt;/code&gt; is not. &lt;code&gt;PushStack&lt;/code&gt; will literally push the node into the virtual stack.&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;	(%11 i0) PushStack %10 i128, %10 i128, #0x10, #0x1
&lt;/code&gt;&lt;/pre&gt;
&lt;div style=&quot;display: flex; justify-content: flex-end;&quot;&gt;
	&lt;img alt=&quot;Patch Details&quot; class=&quot;post-photo&quot; src=&quot;https://p.ocmatos.com/img/2024-09-23-fex-x87-stack-optimization/X87-example1.svg&quot; width=&quot;35%&quot; /&gt;
&lt;/div&gt;
&lt;p&gt;Internally before we push the node into our internal stack, we update the necessary values like the top of the stack. It starts at zero and it&#39;s decremented to seven, so the value &lt;code&gt;%10 i128&lt;/code&gt; is inserted in what we&#39;ll call &lt;code&gt;R7&lt;/code&gt;, now &lt;code&gt;ST0&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;Next up is the computation of &lt;code&gt;SIN&lt;/code&gt; and &lt;code&gt;COS&lt;/code&gt;. This will result in &lt;code&gt;SIN ST0&lt;/code&gt; replacing &lt;code&gt;ST0&lt;/code&gt; and then pushing &lt;code&gt;COS ST0&lt;/code&gt;, where the value in &lt;code&gt;ST0&lt;/code&gt; here is &lt;code&gt;%10&lt;/code&gt;.&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;	(%12 i0) F80SINCOSStack
&lt;/code&gt;&lt;/pre&gt;
&lt;div style=&quot;display: flex; justify-content: flex-end;&quot;&gt;
	&lt;img alt=&quot;Patch Details&quot; class=&quot;post-photo&quot; src=&quot;https://p.ocmatos.com/img/2024-09-23-fex-x87-stack-optimization/X87-example2.svg&quot; width=&quot;35%&quot; /&gt;
&lt;/div&gt;
&lt;p&gt;Then we have two instructions that we just don&#39;t deal with in the pass, these set the &lt;code&gt;C2&lt;/code&gt; flag to zero - as required by &lt;code&gt;FSINCOS&lt;/code&gt;.&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;	%13 i64 = Constant #0x0
	(%14 i8) StoreContext GPR, %13 i64, #0x3fa
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Followed by a multiplication of the stack element to square them. So we square &lt;code&gt;ST0&lt;/code&gt;.&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;	(%15 i0) F80MulStack #0x0, #0x0
&lt;/code&gt;&lt;/pre&gt;
&lt;div style=&quot;display: flex; justify-content: flex-end;&quot;&gt;
	&lt;img alt=&quot;Patch Details&quot; class=&quot;post-photo&quot; src=&quot;https://p.ocmatos.com/img/2024-09-23-fex-x87-stack-optimization/X87-example3.svg&quot; width=&quot;35%&quot; /&gt;
&lt;/div&gt;
&lt;p&gt;The next instruction swaps &lt;code&gt;ST0&lt;/code&gt; with &lt;code&gt;ST1&lt;/code&gt; so that we can square the sine value.&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;	(%16 i0) F80StackXchange #0x1
&lt;/code&gt;&lt;/pre&gt;
&lt;div style=&quot;display: flex; justify-content: flex-end;&quot;&gt;
	&lt;img alt=&quot;Patch Details&quot; class=&quot;post-photo&quot; src=&quot;https://p.ocmatos.com/img/2024-09-23-fex-x87-stack-optimization/X87-example4.svg&quot; width=&quot;35%&quot; /&gt;
&lt;/div&gt;
&lt;p&gt;Again, similarly two instructions that are ignored by the pass, which set flag &lt;code&gt;C1&lt;/code&gt; to zero - as required by &lt;code&gt;FXCH&lt;/code&gt;.&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;	%17 i64 = Constant #0x0
	(%18 i8) StoreContext GPR, %17 i64, #0x3f9
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;And again we square the value at the top of the stack.&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;	(%19 i0) F80MulStack #0x0, #0x0
&lt;/code&gt;&lt;/pre&gt;
&lt;div style=&quot;display: flex; justify-content: flex-end;&quot;&gt;
	&lt;img alt=&quot;Patch Details&quot; class=&quot;post-photo&quot; src=&quot;https://p.ocmatos.com/img/2024-09-23-fex-x87-stack-optimization/X87-example5.svg&quot; width=&quot;35%&quot; /&gt;
&lt;/div&gt;
&lt;p&gt;We are almost there - to finish off the computation we need to add these two values together.&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;	(%20 i0) F80AddStack #0x1, #0x0
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;In this case we add &lt;code&gt;ST1&lt;/code&gt; to &lt;code&gt;ST0&lt;/code&gt; and store the result in &lt;code&gt;ST1&lt;/code&gt;.&lt;/p&gt;
&lt;div style=&quot;display: flex; justify-content: flex-end;&quot;&gt;
	&lt;img alt=&quot;Patch Details&quot; class=&quot;post-photo&quot; src=&quot;https://p.ocmatos.com/img/2024-09-23-fex-x87-stack-optimization/X87-example6.svg&quot; width=&quot;35%&quot; /&gt;
&lt;/div&gt;
&lt;p&gt;This is followed by a pop of the stack.&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;	(%21 i0) PopStackDestroy
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;As expected this removed the &lt;code&gt;F80Mul&lt;/code&gt; from the top of the stack, leaving us with the result of the computation at the top.&lt;/p&gt;
&lt;div style=&quot;display: flex; justify-content: flex-end;&quot;&gt;
	&lt;img alt=&quot;Patch Details&quot; class=&quot;post-photo&quot; src=&quot;https://p.ocmatos.com/img/2024-09-23-fex-x87-stack-optimization/X87-example7.svg&quot; width=&quot;35%&quot; /&gt;
&lt;/div&gt;
&lt;p&gt;OK - there are three more instructions.&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;	%23 i64 = LoadRegister #0x4, GPR
	(%24 i0) StoreStackMemory %23 i64, i128, #0x1, #0x8
	(%25 i0) PopStackDestroy
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The first two store the top of the stack to memory, which is the result of our computation and which maths assures us it&#39;s 1. And then we pop the stack just to clean it up, although this is not strictly necessary since we are finished  anyway.&lt;/p&gt;
&lt;p&gt;Note that we have finished analysing the block, and played it through in our virtual stack. Now we can generate the instructions that are necessary to calculate the stack values, &lt;strong&gt;without&lt;/strong&gt; always load/stor&#39;ing them back to memory. In addition to generating these instructions we also generate instructions to update the tag word, update the top of stack value and save whatever remaining values in the stack at the end of the block into memory - into their respective memory mapped MMX registers. This is the good case where all the values necessary for the computation were found in the block. However, this is not necessarily always the case.&lt;/p&gt;
&lt;h2&gt;The Bad Case&lt;/h2&gt;
&lt;p&gt;Let&#39;s say that FEX for some reason breaks the beautiful block we had before into two:&lt;/p&gt;
&lt;p&gt;Block 1:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;	%9 i64 = LoadRegister #0x4, GPR
	%10 i128 = LoadMem FPR, %9 i64, %Invalid, #0x10, SXTX, #0x1
	(%11 i0) PushStack %10 i128, %10 i128, #0x10, #0x1
	(%12 i0) F80SINCOSStack
	%13 i64 = Constant #0x0
	(%14 i8) StoreContext GPR, %13 i64, #0x3fa
	(%15 i0) F80MulStack #0x0, #0x0
	(%16 i0) F80StackXchange #0x1
	%17 i64 = Constant #0x0
	(%18 i8) StoreContext GPR, %17 i64, #0x3f9
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Block 2:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;	(%19 i0) F80MulStack #0x0, #0x0
	(%20 i0) F80AddStack #0x1, #0x0
	(%21 i0) PopStackDestroy
	%23 i64 = LoadRegister #0x4, GPR
	(%24 i0) StoreStackMemory %23 i64, i128, #0x1, #0x8
	(%25 i0) PopStackDestroy
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;When we generate code for the first block, there are absolutely no blockers and we run through it exactly as we did before, except that after &lt;code&gt;StoreContext&lt;/code&gt;, we reach the end of the block and our virtual stack is in this state:&lt;/p&gt;
&lt;div style=&quot;display: flex; justify-content: flex-end;&quot;&gt;
	&lt;img alt=&quot;Patch Details&quot; class=&quot;post-photo&quot; src=&quot;https://p.ocmatos.com/img/2024-09-23-fex-x87-stack-optimization/X87-example4.svg&quot; width=&quot;35%&quot; /&gt;
&lt;/div&gt;
&lt;p&gt;At this point, we&#39;ll do what we said we always do before at the end of the block. We save the values in the virtual stack to the respective MMX registers in memory mapped locations: &lt;code&gt;%36&lt;/code&gt; is saved to &lt;code&gt;MM7&lt;/code&gt; and &lt;code&gt;%34&lt;/code&gt; is saved to &lt;code&gt;MM6&lt;/code&gt;. The &lt;code&gt;TOP&lt;/code&gt; is recorded to be 6 and the tags for &lt;code&gt;MM7&lt;/code&gt; and &lt;code&gt;MM6&lt;/code&gt; are marked to be valid. Then we exit the block. And start analyzing the following block.&lt;/p&gt;
&lt;p&gt;When we start the pass for the following block, our virtual stack is empty. We have no knowledge of virtual stacks for previous JITTed blocks, and we see this instruction:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;	(%19 i0) F80MulStack #0x0, #0x0
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;We cannot play through this instruction in our virtual stack because it multiplies the value in &lt;code&gt;ST0&lt;/code&gt; with itself and our virtual stack &lt;strong&gt;does not&lt;/strong&gt; have anything in &lt;code&gt;ST0&lt;/code&gt;. It&#39;s empty after all.&lt;/p&gt;
&lt;p&gt;In this case, we switch to the slow path that does exactly what we used to do before this work. We load the value of &lt;code&gt;ST0&lt;/code&gt; from memory by finding the current value for &lt;code&gt;TOP&lt;/code&gt; and loading the respective register value, issue the &lt;code&gt;F80Mul&lt;/code&gt; on this value and storing it back to the &lt;code&gt;MMX&lt;/code&gt; register. This is then done for the remainder of the block.&lt;/p&gt;
&lt;h2&gt;The Ugly Case&lt;/h2&gt;
&lt;p&gt;The ugly case is when we actually force the slow path, not because we don&#39;t have all the values in the virtual stack but out of necessity.&lt;/p&gt;
&lt;p&gt;There are a few instructions that trigger a &lt;code&gt;_StackForceSlow()&lt;/code&gt; and forces us down the slow path. These are: &lt;code&gt;FLDENV&lt;/code&gt;, &lt;code&gt;FRSTOR&lt;/code&gt;, &lt;code&gt;FLDCW&lt;/code&gt;. All of these load some part or all of the FPU environment from memory and it triggers a switch to the slow path so that values are loaded properly and we can then use those new values in the block. It is not impossible to think we could at least in some cases, load those values from memory, try to rebuild our virtual stack for the current block and continue without switching to the slow path but that hasn&#39;t been yet done.&lt;/p&gt;
&lt;p&gt;There are a few instructions other instructions that trigger a &lt;code&gt;_SyncStackToSlow()&lt;/code&gt;, which doesn&#39;t force us down the slow path but instead just updates the in-memory data from the virtual stack state. These are: &lt;code&gt;FNSTENV&lt;/code&gt;, &lt;code&gt;FNSAVE&lt;/code&gt;, and &lt;code&gt;FNSTSW&lt;/code&gt;. All of these store some part or all of the FPU environment into memory and it ensures that the values in-memory are up-to-date so that the store doesn&#39;t use obsolete values.&lt;/p&gt;
&lt;h1&gt;Results&lt;/h1&gt;
&lt;p&gt;In addition to the huge savings in data movement from the loads and stores of register values, &lt;code&gt;TOP&lt;/code&gt; and tags for each stack instruction, we also optimize memory copies through the stack. So if there&#39;s a bunch of value loaded to the stack, which are then stored to memory, we can just memcopy these values without going through the stack.&lt;/p&gt;
&lt;p&gt;In our code size validation benchmarks from various Steam games, we saw significant reductions in code size. These size code reductions saw a similarly high jump in FPS numbers. For example:&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Game&lt;/th&gt;
&lt;th style=&quot;text-align:center&quot;&gt;Before&lt;/th&gt;
&lt;th&gt;After&lt;/th&gt;
&lt;th&gt;Reduction&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Half-Life (Block 1)&lt;/td&gt;
&lt;td style=&quot;text-align:center&quot;&gt;2034&lt;/td&gt;
&lt;td&gt;1368&lt;/td&gt;
&lt;td&gt;32.7%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Half-Life (Block 2)&lt;/td&gt;
&lt;td style=&quot;text-align:center&quot;&gt;838&lt;/td&gt;
&lt;td&gt;551&lt;/td&gt;
&lt;td&gt;34.2%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Oblivion (Block 1)&lt;/td&gt;
&lt;td style=&quot;text-align:center&quot;&gt;34065&lt;/td&gt;
&lt;td&gt;25782&lt;/td&gt;
&lt;td&gt;24.3%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Oblivion (Block 2)&lt;/td&gt;
&lt;td style=&quot;text-align:center&quot;&gt;22089&lt;/td&gt;
&lt;td&gt;16635&lt;/td&gt;
&lt;td&gt;24.6%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Psychonauts (Block 1)&lt;/td&gt;
&lt;td style=&quot;text-align:center&quot;&gt;20545&lt;/td&gt;
&lt;td&gt;16357&lt;/td&gt;
&lt;td&gt;20.3%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Psychonauts Hot Loop (Matrix Swizzle)&lt;/td&gt;
&lt;td style=&quot;text-align:center&quot;&gt;2340&lt;/td&gt;
&lt;td&gt;165&lt;/td&gt;
&lt;td&gt;92.9%&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;h1&gt;Future work&lt;/h1&gt;
&lt;p&gt;The bulk of this work is now complete, however there are a few details that still need fixing. There&#39;s still the mystery of why &lt;a href=&quot;https://github.com/FEX-Emu/FEX/issues/3685&quot;&gt;Touhou: Luna Nights works with reduced precision&lt;/a&gt; but not normal precision (if anything you would expect it to be the other way around). Then there&#39;s also some work to be done on the interface between the FPU state and the MMX state as described in &amp;quot;&lt;a href=&quot;https://github.com/FEX-Emu/FEX/issues/3850&quot;&gt;MMX/x87 interaction is subtly broken&lt;/a&gt;&amp;quot;.&lt;/p&gt;
&lt;p&gt;This is something I am already focusing on and will continue working on this until complete.&lt;/p&gt;
</content>
  </entry>
  <entry>
    <title>WebAssembly Reference Types in Clang/LLVM</title>
    <link href="https://p.ocmatos.com/blog/webassembly-reference-types-in-clang-llvm.html"/>
    <updated>2023-06-13T00:00:00Z</updated>
    <id>https://p.ocmatos.com/blog/webassembly-reference-types-in-clang-llvm.html</id>
    <content xml:lang="en" type="html">&lt;h1&gt;WebAssembly Reference Types in LLVM&lt;/h1&gt;
&lt;h2&gt;Introduction&lt;/h2&gt;
&lt;p&gt;This post is an overview of the implementation of the &lt;a href=&quot;https://github.com/WebAssembly/reference-types&quot;&gt;WebAssembly Reference Types proposal&lt;/a&gt; in Clang/LLVM. It is a follow-up to two presentations:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href=&quot;https://www.youtube.com/watch?v=UxnUht3uWzc&quot;&gt;&amp;quot;A walk on the weird side: Opaque WebAssembly values and LLVM&amp;quot;&lt;/a&gt; (2021 LLVM Dev Mtg) by Andy Wingo, and&lt;/li&gt;
&lt;li&gt;&lt;a href=&quot;https://www.youtube.com/watch?v=lwPg_Vjs7p4&quot;&gt;&amp;quot;Clang, Clang: Who&#39;s there? WebAssembly!&amp;quot;&lt;/a&gt; (2022 LLVM Dev Mtg) by myself.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;This is the culmination of a couple of years of work started by myself, Andy Wingo and Alex Bradbury at Igalia. It was sponsored in its entirety by Bloomberg.&lt;/p&gt;
&lt;p&gt;The implementation followed a bottom up approach. We worked on the linker first, followed by the WebAssembly backend, MC layer and LLVM IR. Support for reference types has been available in LLVM IR for quite some time, but it was not exposed in Clang. Extending Clang to support values of reference type and tables was the last piece of the puzzle.&lt;/p&gt;
&lt;p&gt;For context for the remainder of the post, I recommend watching the two presentations linked above. In summary, the work consisted of implementing support for the following reference types (read WebAssembly reference types unless stated, not C++ reference types) in LLVM:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;code&gt;externref&lt;/code&gt;, which is a garbage collected reference type, and&lt;/li&gt;
&lt;li&gt;&lt;code&gt;funcref&lt;/code&gt;, which is a non-nullable reference to a function.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;In addition, we implemented support for WebAssembly tables, which are arrays of references. Since values of reference types cannot be stored to linear memory, tables are storage containers for these.&lt;/p&gt;
&lt;h2&gt;Linker and MC Layer&lt;/h2&gt;
&lt;h2&gt;LLVM IR&lt;/h2&gt;
&lt;h2&gt;Clang&lt;/h2&gt;
&lt;h2&gt;Conclusions and Future&lt;/h2&gt;
&lt;p&gt;There was a lot of work done in the last few years to reach this point. Not just by Igalia but kind and patient reviewers and the WebAssembly Tools team at Google who offered a lot of advice on the matter. However, there is still work to be done to completely implement the WebAssembly Reference Types proposal. We don&#39;t support initialization of tables yet and the C++ language extension to deal with these types is not very developer friendly since it&#39;s based on builtins. A more ergonomic approach to solve table access, for example, would be to allow &lt;code&gt;table.get&lt;/code&gt; and &lt;code&gt;table.set&lt;/code&gt; to be deal as array access. For the moment the project is on hold, due to lack of funding. If you are interested in sponsoring this work, please get in touch with me.&lt;/p&gt;
</content>
  </entry>
  <entry>
    <title>From Middle-C to Concert</title>
    <link href="https://p.ocmatos.com/blog/from-middle-c-to-concert.html"/>
    <updated>2022-10-24T00:00:00Z</updated>
    <id>https://p.ocmatos.com/blog/from-middle-c-to-concert.html</id>
    <content xml:lang="en" type="html">&lt;p&gt;Last Saturday, on the 24th of September of 2022 I played piano publicly for the first time in a local concert organized for students of my piano teacher - &lt;a href=&quot;http://www.takako-ono.com/&quot;&gt;Takako Ono&lt;/a&gt;. My performance was far from perfect, but I am proud of it. This is a brief overview of my lessons and practice over the last 8 months.&lt;/p&gt;
&lt;p&gt;Earlier this year, on January 18, I started playing the piano with a 30-minute lesson. What I mean by &amp;quot;started playing&amp;quot;, is learning the location of the middle C on the piano. Or that when reading sheet music all of the following notes are the same - indeed, these are all middle C.&lt;/p&gt;
&lt;svg xmlns=&quot;http://www.w3.org/2000/svg&quot; xmlns:xlink=&quot;http://www.w3.org/1999/xlink&quot; version=&quot;1.2&quot; width=&quot;85.75mm&quot; height=&quot;26.16mm&quot; viewBox=&quot;-0.3115 -0.0210 48.7956 14.8867&quot;&gt;
&lt;style type=&quot;text/css&quot;&gt;
&lt;![CDATA[
tspan { white-space: pre; }
]]&gt;
&lt;/style&gt;
&lt;g transform=&quot;translate(47.8841, -0.0000)&quot;&gt;
&lt;rect x=&quot;0.0000&quot; y=&quot;5.8290&quot; width=&quot;0.6000&quot; height=&quot;4.9500&quot; ry=&quot;0.0000&quot; fill=&quot;currentColor&quot;&gt;&lt;/rect&gt;
&lt;/g&gt;
&lt;g transform=&quot;translate(47.3941, -0.0000)&quot;&gt;
&lt;rect x=&quot;0.0000&quot; y=&quot;5.8290&quot; width=&quot;0.1900&quot; height=&quot;4.9500&quot; ry=&quot;0.0000&quot; fill=&quot;currentColor&quot;&gt;&lt;/rect&gt;
&lt;/g&gt;
&lt;g transform=&quot;translate(28.4453, 3.7790)&quot;&gt;
&lt;rect x=&quot;0.0000&quot; y=&quot;-2.0000&quot; width=&quot;0.1900&quot; height=&quot;4.0500&quot; ry=&quot;0.0000&quot; fill=&quot;currentColor&quot;&gt;&lt;/rect&gt;
&lt;/g&gt;
&lt;g transform=&quot;translate(40.7449, 12.7790)&quot;&gt;
&lt;rect x=&quot;0.0000&quot; y=&quot;-2.0500&quot; width=&quot;0.1900&quot; height=&quot;4.0500&quot; ry=&quot;0.0000&quot; fill=&quot;currentColor&quot;&gt;&lt;/rect&gt;
&lt;/g&gt;
&lt;g transform=&quot;translate(47.8841, 3.7790)&quot;&gt;
&lt;rect x=&quot;0.0000&quot; y=&quot;-2.0000&quot; width=&quot;0.6000&quot; height=&quot;4.0500&quot; ry=&quot;0.0000&quot; fill=&quot;currentColor&quot;&gt;&lt;/rect&gt;
&lt;/g&gt;
&lt;g transform=&quot;translate(47.3941, 3.7790)&quot;&gt;
&lt;rect x=&quot;0.0000&quot; y=&quot;-2.0000&quot; width=&quot;0.1900&quot; height=&quot;4.0500&quot; ry=&quot;0.0000&quot; fill=&quot;currentColor&quot;&gt;&lt;/rect&gt;
&lt;/g&gt;
&lt;g transform=&quot;translate(47.8841, 12.7790)&quot;&gt;
&lt;rect x=&quot;0.0000&quot; y=&quot;-2.0500&quot; width=&quot;0.6000&quot; height=&quot;4.0500&quot; ry=&quot;0.0000&quot; fill=&quot;currentColor&quot;&gt;&lt;/rect&gt;
&lt;/g&gt;
&lt;g transform=&quot;translate(47.3941, 12.7790)&quot;&gt;
&lt;rect x=&quot;0.0000&quot; y=&quot;-2.0500&quot; width=&quot;0.1900&quot; height=&quot;4.0500&quot; ry=&quot;0.0000&quot; fill=&quot;currentColor&quot;&gt;&lt;/rect&gt;
&lt;/g&gt;
&lt;g transform=&quot;translate(40.7449, -0.0000)&quot;&gt;
&lt;rect x=&quot;0.0000&quot; y=&quot;5.8290&quot; width=&quot;0.1900&quot; height=&quot;4.9500&quot; ry=&quot;0.0000&quot; fill=&quot;currentColor&quot;&gt;&lt;/rect&gt;
&lt;/g&gt;
&lt;g transform=&quot;translate(28.4453, 12.7790)&quot;&gt;
&lt;rect x=&quot;0.0000&quot; y=&quot;-2.0500&quot; width=&quot;0.1900&quot; height=&quot;4.0500&quot; ry=&quot;0.0000&quot; fill=&quot;currentColor&quot;&gt;&lt;/rect&gt;
&lt;/g&gt;
&lt;g transform=&quot;translate(40.7449, 3.7790)&quot;&gt;
&lt;rect x=&quot;0.0000&quot; y=&quot;-2.0000&quot; width=&quot;0.1900&quot; height=&quot;4.0500&quot; ry=&quot;0.0000&quot; fill=&quot;currentColor&quot;&gt;&lt;/rect&gt;
&lt;/g&gt;
&lt;g transform=&quot;translate(28.4453, -0.0000)&quot;&gt;
&lt;rect x=&quot;0.0000&quot; y=&quot;5.8290&quot; width=&quot;0.1900&quot; height=&quot;4.9500&quot; ry=&quot;0.0000&quot; fill=&quot;currentColor&quot;&gt;&lt;/rect&gt;
&lt;/g&gt;
&lt;g transform=&quot;translate(8.5358, 14.7790)&quot;&gt;
&lt;line stroke-linejoin=&quot;round&quot; stroke-linecap=&quot;round&quot; stroke-width=&quot;0.1000&quot; stroke=&quot;currentColor&quot; x1=&quot;0.0500&quot; y1=&quot;0&quot; x2=&quot;39.8983&quot; y2=&quot;0&quot;&gt;&lt;/line&gt;
&lt;/g&gt;
&lt;g transform=&quot;translate(8.5358, 13.7790)&quot;&gt;
&lt;line stroke-linejoin=&quot;round&quot; stroke-linecap=&quot;round&quot; stroke-width=&quot;0.1000&quot; stroke=&quot;currentColor&quot; x1=&quot;0.0500&quot; y1=&quot;0&quot; x2=&quot;39.8983&quot; y2=&quot;0&quot;&gt;&lt;/line&gt;
&lt;/g&gt;
&lt;g transform=&quot;translate(8.5358, 12.7790)&quot;&gt;
&lt;line stroke-linejoin=&quot;round&quot; stroke-linecap=&quot;round&quot; stroke-width=&quot;0.1000&quot; stroke=&quot;currentColor&quot; x1=&quot;0.0500&quot; y1=&quot;0&quot; x2=&quot;39.8983&quot; y2=&quot;0&quot;&gt;&lt;/line&gt;
&lt;/g&gt;
&lt;g transform=&quot;translate(8.5358, 11.7790)&quot;&gt;
&lt;line stroke-linejoin=&quot;round&quot; stroke-linecap=&quot;round&quot; stroke-width=&quot;0.1000&quot; stroke=&quot;currentColor&quot; x1=&quot;0.0500&quot; y1=&quot;0&quot; x2=&quot;39.8983&quot; y2=&quot;0&quot;&gt;&lt;/line&gt;
&lt;/g&gt;
&lt;g transform=&quot;translate(8.5358, 10.7790)&quot;&gt;
&lt;line stroke-linejoin=&quot;round&quot; stroke-linecap=&quot;round&quot; stroke-width=&quot;0.1000&quot; stroke=&quot;currentColor&quot; x1=&quot;0.0500&quot; y1=&quot;0&quot; x2=&quot;39.8983&quot; y2=&quot;0&quot;&gt;&lt;/line&gt;
&lt;/g&gt;
&lt;g transform=&quot;translate(0.0000, 9.7790)&quot;&gt;
&lt;rect x=&quot;44.3340&quot; y=&quot;-0.1000&quot; width=&quot;1.9563&quot; height=&quot;0.2000&quot; ry=&quot;0.1000&quot; fill=&quot;currentColor&quot;&gt;&lt;/rect&gt;
&lt;/g&gt;
&lt;g transform=&quot;translate(0.0000, 9.7790)&quot;&gt;
&lt;rect x=&quot;37.6847&quot; y=&quot;-0.1000&quot; width=&quot;1.9563&quot; height=&quot;0.2000&quot; ry=&quot;0.1000&quot; fill=&quot;currentColor&quot;&gt;&lt;/rect&gt;
&lt;/g&gt;
&lt;g transform=&quot;translate(0.0000, 9.7790)&quot;&gt;
&lt;rect x=&quot;32.0344&quot; y=&quot;-0.1000&quot; width=&quot;1.9563&quot; height=&quot;0.2000&quot; ry=&quot;0.1000&quot; fill=&quot;currentColor&quot;&gt;&lt;/rect&gt;
&lt;/g&gt;
&lt;g transform=&quot;translate(0.0000, 9.7790)&quot;&gt;
&lt;rect x=&quot;25.3852&quot; y=&quot;-0.1000&quot; width=&quot;1.9563&quot; height=&quot;0.2000&quot; ry=&quot;0.1000&quot; fill=&quot;currentColor&quot;&gt;&lt;/rect&gt;
&lt;/g&gt;
&lt;g transform=&quot;translate(0.0000, 9.7790)&quot;&gt;
&lt;rect x=&quot;19.7349&quot; y=&quot;-0.1000&quot; width=&quot;1.9563&quot; height=&quot;0.2000&quot; ry=&quot;0.1000&quot; fill=&quot;currentColor&quot;&gt;&lt;/rect&gt;
&lt;/g&gt;
&lt;g transform=&quot;translate(0.0000, 6.7790)&quot;&gt;
&lt;rect x=&quot;41.5088&quot; y=&quot;-0.1000&quot; width=&quot;1.9563&quot; height=&quot;0.2000&quot; ry=&quot;0.1000&quot; fill=&quot;currentColor&quot;&gt;&lt;/rect&gt;
&lt;/g&gt;
&lt;g transform=&quot;translate(0.0000, 6.7790)&quot;&gt;
&lt;rect x=&quot;34.8596&quot; y=&quot;-0.1000&quot; width=&quot;1.9563&quot; height=&quot;0.2000&quot; ry=&quot;0.1000&quot; fill=&quot;currentColor&quot;&gt;&lt;/rect&gt;
&lt;/g&gt;
&lt;g transform=&quot;translate(0.0000, 6.7790)&quot;&gt;
&lt;rect x=&quot;29.2093&quot; y=&quot;-0.1000&quot; width=&quot;1.9563&quot; height=&quot;0.2000&quot; ry=&quot;0.1000&quot; fill=&quot;currentColor&quot;&gt;&lt;/rect&gt;
&lt;/g&gt;
&lt;g transform=&quot;translate(0.0000, 6.7790)&quot;&gt;
&lt;rect x=&quot;22.5601&quot; y=&quot;-0.1000&quot; width=&quot;1.9563&quot; height=&quot;0.2000&quot; ry=&quot;0.1000&quot; fill=&quot;currentColor&quot;&gt;&lt;/rect&gt;
&lt;/g&gt;
&lt;g transform=&quot;translate(0.0000, 6.7790)&quot;&gt;
&lt;rect x=&quot;16.9098&quot; y=&quot;-0.1000&quot; width=&quot;1.9563&quot; height=&quot;0.2000&quot; ry=&quot;0.1000&quot; fill=&quot;currentColor&quot;&gt;&lt;/rect&gt;
&lt;/g&gt;
&lt;g transform=&quot;translate(8.5358, 5.7790)&quot;&gt;
&lt;line stroke-linejoin=&quot;round&quot; stroke-linecap=&quot;round&quot; stroke-width=&quot;0.1000&quot; stroke=&quot;currentColor&quot; x1=&quot;0.0500&quot; y1=&quot;0&quot; x2=&quot;39.8983&quot; y2=&quot;0&quot;&gt;&lt;/line&gt;
&lt;/g&gt;
&lt;g transform=&quot;translate(8.5358, 4.7790)&quot;&gt;
&lt;line stroke-linejoin=&quot;round&quot; stroke-linecap=&quot;round&quot; stroke-width=&quot;0.1000&quot; stroke=&quot;currentColor&quot; x1=&quot;0.0500&quot; y1=&quot;0&quot; x2=&quot;39.8983&quot; y2=&quot;0&quot;&gt;&lt;/line&gt;
&lt;/g&gt;
&lt;g transform=&quot;translate(8.5358, 3.7790)&quot;&gt;
&lt;line stroke-linejoin=&quot;round&quot; stroke-linecap=&quot;round&quot; stroke-width=&quot;0.1000&quot; stroke=&quot;currentColor&quot; x1=&quot;0.0500&quot; y1=&quot;0&quot; x2=&quot;39.8983&quot; y2=&quot;0&quot;&gt;&lt;/line&gt;
&lt;/g&gt;
&lt;g transform=&quot;translate(8.5358, 2.7790)&quot;&gt;
&lt;line stroke-linejoin=&quot;round&quot; stroke-linecap=&quot;round&quot; stroke-width=&quot;0.1000&quot; stroke=&quot;currentColor&quot; x1=&quot;0.0500&quot; y1=&quot;0&quot; x2=&quot;39.8983&quot; y2=&quot;0&quot;&gt;&lt;/line&gt;
&lt;/g&gt;
&lt;g transform=&quot;translate(8.5358, 1.7790)&quot;&gt;
&lt;line stroke-linejoin=&quot;round&quot; stroke-linecap=&quot;round&quot; stroke-width=&quot;0.1000&quot; stroke=&quot;currentColor&quot; x1=&quot;0.0500&quot; y1=&quot;0&quot; x2=&quot;39.8983&quot; y2=&quot;0&quot;&gt;&lt;/line&gt;
&lt;/g&gt;
&lt;a style=&quot;color:inherit;&quot; xlink:href=&quot;textedit:///tmp/lyInsert-2026316-4293-s7ti37.ixglq.ly:12:73:74&quot;&gt;
&lt;g transform=&quot;translate(38.0108, 3.7790)&quot;&gt;
&lt;path transform=&quot;scale(0.0040, -0.0040)&quot; d=&quot;M-23 -116c0 28 11 42 40 42c33 0 78 -13 119 -31l-132 158c-7 9 -10 17 -10 25c0 34 50 66 87 99c25 22 37 52 37 83c0 24 -8 49 -25 69l-35 42c-3 3 -4 7 -4 10c0 9 9 15 17 15c4 0 8 -1 11 -5l151 -180c7 -9 10 -17 10 -25c0 -34 -50 -66 -87 -99
c-25 -22 -37 -52 -37 -83c0 -24 7 -49 24 -69l84 -99c3 -3 4 -7 4 -10c0 -9 -8 -16 -16 -16c-4 0 -9 2 -12 6c-18 21 -63 38 -97 38c-41 0 -53 -26 -53 -67c0 -35 11 -74 28 -94c7 -8 -5 -19 -12 -11c-45 54 -92 148 -92 202z&quot; fill=&quot;currentColor&quot;&gt;&lt;/path&gt;
&lt;/g&gt;
&lt;/a&gt;
&lt;a style=&quot;color:inherit;&quot; xlink:href=&quot;textedit:///tmp/lyInsert-2026316-4293-s7ti37.ixglq.ly:13:73:74&quot;&gt;
&lt;g transform=&quot;translate(38.0108, 9.7790)&quot;&gt;
&lt;path transform=&quot;scale(0.0040, -0.0040)&quot; d=&quot;M218 136c55 0 108 -28 108 -89c0 -71 -55 -121 -102 -149c-35 -21 -75 -34 -116 -34c-55 0 -108 28 -108 89c0 71 55 121 102 149c35 21 75 34 116 34z&quot; fill=&quot;currentColor&quot;&gt;&lt;/path&gt;
&lt;/g&gt;
&lt;/a&gt;
&lt;g transform=&quot;translate(38.0758, 12.7790)&quot;&gt;
&lt;rect x=&quot;-0.0650&quot; y=&quot;-2.8139&quot; width=&quot;0.1300&quot; height=&quot;3.3139&quot; ry=&quot;0.0400&quot; fill=&quot;currentColor&quot;&gt;&lt;/rect&gt;
&lt;/g&gt;
&lt;a style=&quot;color:inherit;&quot; xlink:href=&quot;textedit:///tmp/lyInsert-2026316-4293-s7ti37.ixglq.ly:12:75:76&quot;&gt;
&lt;g transform=&quot;translate(41.8349, 6.7790)&quot;&gt;
&lt;path transform=&quot;scale(0.0040, -0.0040)&quot; d=&quot;M218 136c55 0 108 -28 108 -89c0 -71 -55 -121 -102 -149c-35 -21 -75 -34 -116 -34c-55 0 -108 28 -108 89c0 71 55 121 102 149c35 21 75 34 116 34z&quot; fill=&quot;currentColor&quot;&gt;&lt;/path&gt;
&lt;/g&gt;
&lt;/a&gt;
&lt;a style=&quot;color:inherit;&quot; xlink:href=&quot;textedit:///tmp/lyInsert-2026316-4293-s7ti37.ixglq.ly:13:75:76&quot;&gt;
&lt;g transform=&quot;translate(41.8349, 12.7790)&quot;&gt;
&lt;path transform=&quot;scale(0.0040, -0.0040)&quot; d=&quot;M-23 -116c0 28 11 42 40 42c33 0 78 -13 119 -31l-132 158c-7 9 -10 17 -10 25c0 34 50 66 87 99c25 22 37 52 37 83c0 24 -8 49 -25 69l-35 42c-3 3 -4 7 -4 10c0 9 9 15 17 15c4 0 8 -1 11 -5l151 -180c7 -9 10 -17 10 -25c0 -34 -50 -66 -87 -99
c-25 -22 -37 -52 -37 -83c0 -24 7 -49 24 -69l84 -99c3 -3 4 -7 4 -10c0 -9 -8 -16 -16 -16c-4 0 -9 2 -12 6c-18 21 -63 38 -97 38c-41 0 -53 -26 -53 -67c0 -35 11 -74 28 -94c7 -8 -5 -19 -12 -11c-45 54 -92 148 -92 202z&quot; fill=&quot;currentColor&quot;&gt;&lt;/path&gt;
&lt;/g&gt;
&lt;/a&gt;
&lt;a style=&quot;color:inherit;&quot; xlink:href=&quot;textedit:///tmp/lyInsert-2026316-4293-s7ti37.ixglq.ly:12:69:70&quot;&gt;
&lt;g transform=&quot;translate(32.3605, 3.7790)&quot;&gt;
&lt;path transform=&quot;scale(0.0040, -0.0040)&quot; d=&quot;M-23 -116c0 28 11 42 40 42c33 0 78 -13 119 -31l-132 158c-7 9 -10 17 -10 25c0 34 50 66 87 99c25 22 37 52 37 83c0 24 -8 49 -25 69l-35 42c-3 3 -4 7 -4 10c0 9 9 15 17 15c4 0 8 -1 11 -5l151 -180c7 -9 10 -17 10 -25c0 -34 -50 -66 -87 -99
c-25 -22 -37 -52 -37 -83c0 -24 7 -49 24 -69l84 -99c3 -3 4 -7 4 -10c0 -9 -8 -16 -16 -16c-4 0 -9 2 -12 6c-18 21 -63 38 -97 38c-41 0 -53 -26 -53 -67c0 -35 11 -74 28 -94c7 -8 -5 -19 -12 -11c-45 54 -92 148 -92 202z&quot; fill=&quot;currentColor&quot;&gt;&lt;/path&gt;
&lt;/g&gt;
&lt;/a&gt;
&lt;a style=&quot;color:inherit;&quot; xlink:href=&quot;textedit:///tmp/lyInsert-2026316-4293-s7ti37.ixglq.ly:13:69:70&quot;&gt;
&lt;g transform=&quot;translate(32.3605, 9.7790)&quot;&gt;
&lt;path transform=&quot;scale(0.0040, -0.0040)&quot; d=&quot;M218 136c55 0 108 -28 108 -89c0 -71 -55 -121 -102 -149c-35 -21 -75 -34 -116 -34c-55 0 -108 28 -108 89c0 71 55 121 102 149c35 21 75 34 116 34z&quot; fill=&quot;currentColor&quot;&gt;&lt;/path&gt;
&lt;/g&gt;
&lt;/a&gt;
&lt;g transform=&quot;translate(32.4255, 12.7790)&quot;&gt;
&lt;rect x=&quot;-0.0650&quot; y=&quot;-2.8139&quot; width=&quot;0.1300&quot; height=&quot;3.3139&quot; ry=&quot;0.0400&quot; fill=&quot;currentColor&quot;&gt;&lt;/rect&gt;
&lt;/g&gt;
&lt;a style=&quot;color:inherit;&quot; xlink:href=&quot;textedit:///tmp/lyInsert-2026316-4293-s7ti37.ixglq.ly:12:71:72&quot;&gt;
&lt;g transform=&quot;translate(35.1856, 6.7790)&quot;&gt;
&lt;path transform=&quot;scale(0.0040, -0.0040)&quot; d=&quot;M218 136c55 0 108 -28 108 -89c0 -71 -55 -121 -102 -149c-35 -21 -75 -34 -116 -34c-55 0 -108 28 -108 89c0 71 55 121 102 149c35 21 75 34 116 34z&quot; fill=&quot;currentColor&quot;&gt;&lt;/path&gt;
&lt;/g&gt;
&lt;/a&gt;
&lt;a style=&quot;color:inherit;&quot; xlink:href=&quot;textedit:///tmp/lyInsert-2026316-4293-s7ti37.ixglq.ly:13:71:72&quot;&gt;
&lt;g transform=&quot;translate(35.1856, 12.7790)&quot;&gt;
&lt;path transform=&quot;scale(0.0040, -0.0040)&quot; d=&quot;M-23 -116c0 28 11 42 40 42c33 0 78 -13 119 -31l-132 158c-7 9 -10 17 -10 25c0 34 50 66 87 99c25 22 37 52 37 83c0 24 -8 49 -25 69l-35 42c-3 3 -4 7 -4 10c0 9 9 15 17 15c4 0 8 -1 11 -5l151 -180c7 -9 10 -17 10 -25c0 -34 -50 -66 -87 -99
c-25 -22 -37 -52 -37 -83c0 -24 7 -49 24 -69l84 -99c3 -3 4 -7 4 -10c0 -9 -8 -16 -16 -16c-4 0 -9 2 -12 6c-18 21 -63 38 -97 38c-41 0 -53 -26 -53 -67c0 -35 11 -74 28 -94c7 -8 -5 -19 -12 -11c-45 54 -92 148 -92 202z&quot; fill=&quot;currentColor&quot;&gt;&lt;/path&gt;
&lt;/g&gt;
&lt;/a&gt;
&lt;g transform=&quot;translate(36.4249, 3.7790)&quot;&gt;
&lt;rect x=&quot;-0.0650&quot; y=&quot;-0.5000&quot; width=&quot;0.1300&quot; height=&quot;3.3139&quot; ry=&quot;0.0400&quot; fill=&quot;currentColor&quot;&gt;&lt;/rect&gt;
&lt;/g&gt;
&lt;a style=&quot;color:inherit;&quot; xlink:href=&quot;textedit:///tmp/lyInsert-2026316-4293-s7ti37.ixglq.ly:13:77:78&quot;&gt;
&lt;g transform=&quot;translate(44.6600, 9.7790)&quot;&gt;
&lt;path transform=&quot;scale(0.0040, -0.0040)&quot; d=&quot;M218 136c55 0 108 -28 108 -89c0 -71 -55 -121 -102 -149c-35 -21 -75 -34 -116 -34c-55 0 -108 28 -108 89c0 71 55 121 102 149c35 21 75 34 116 34z&quot; fill=&quot;currentColor&quot;&gt;&lt;/path&gt;
&lt;/g&gt;
&lt;/a&gt;
&lt;g transform=&quot;translate(44.7250, 12.7790)&quot;&gt;
&lt;rect x=&quot;-0.0650&quot; y=&quot;-2.8139&quot; width=&quot;0.1300&quot; height=&quot;3.3139&quot; ry=&quot;0.0400&quot; fill=&quot;currentColor&quot;&gt;&lt;/rect&gt;
&lt;/g&gt;
&lt;a style=&quot;color:inherit;&quot; xlink:href=&quot;textedit:///tmp/lyInsert-2026316-4293-s7ti37.ixglq.ly:12:77:78&quot;&gt;
&lt;g transform=&quot;translate(44.6600, 3.7790)&quot;&gt;
&lt;path transform=&quot;scale(0.0040, -0.0040)&quot; d=&quot;M-23 -116c0 28 11 42 40 42c33 0 78 -13 119 -31l-132 158c-7 9 -10 17 -10 25c0 34 50 66 87 99c25 22 37 52 37 83c0 24 -8 49 -25 69l-35 42c-3 3 -4 7 -4 10c0 9 9 15 17 15c4 0 8 -1 11 -5l151 -180c7 -9 10 -17 10 -25c0 -34 -50 -66 -87 -99
c-25 -22 -37 -52 -37 -83c0 -24 7 -49 24 -69l84 -99c3 -3 4 -7 4 -10c0 -9 -8 -16 -16 -16c-4 0 -9 2 -12 6c-18 21 -63 38 -97 38c-41 0 -53 -26 -53 -67c0 -35 11 -74 28 -94c7 -8 -5 -19 -12 -11c-45 54 -92 148 -92 202z&quot; fill=&quot;currentColor&quot;&gt;&lt;/path&gt;
&lt;/g&gt;
&lt;/a&gt;
&lt;g transform=&quot;translate(43.0741, 3.7790)&quot;&gt;
&lt;rect x=&quot;-0.0650&quot; y=&quot;-0.5000&quot; width=&quot;0.1300&quot; height=&quot;3.3139&quot; ry=&quot;0.0400&quot; fill=&quot;currentColor&quot;&gt;&lt;/rect&gt;
&lt;/g&gt;
&lt;a style=&quot;color:inherit;&quot; xlink:href=&quot;textedit:///tmp/lyInsert-2026316-4293-s7ti37.ixglq.ly:12:61:62&quot;&gt;
&lt;g transform=&quot;translate(20.0610, 3.7790)&quot;&gt;
&lt;path transform=&quot;scale(0.0040, -0.0040)&quot; d=&quot;M-23 -116c0 28 11 42 40 42c33 0 78 -13 119 -31l-132 158c-7 9 -10 17 -10 25c0 34 50 66 87 99c25 22 37 52 37 83c0 24 -8 49 -25 69l-35 42c-3 3 -4 7 -4 10c0 9 9 15 17 15c4 0 8 -1 11 -5l151 -180c7 -9 10 -17 10 -25c0 -34 -50 -66 -87 -99
c-25 -22 -37 -52 -37 -83c0 -24 7 -49 24 -69l84 -99c3 -3 4 -7 4 -10c0 -9 -8 -16 -16 -16c-4 0 -9 2 -12 6c-18 21 -63 38 -97 38c-41 0 -53 -26 -53 -67c0 -35 11 -74 28 -94c7 -8 -5 -19 -12 -11c-45 54 -92 148 -92 202z&quot; fill=&quot;currentColor&quot;&gt;&lt;/path&gt;
&lt;/g&gt;
&lt;/a&gt;
&lt;a style=&quot;color:inherit;&quot; xlink:href=&quot;textedit:///tmp/lyInsert-2026316-4293-s7ti37.ixglq.ly:13:61:62&quot;&gt;
&lt;g transform=&quot;translate(20.0610, 9.7790)&quot;&gt;
&lt;path transform=&quot;scale(0.0040, -0.0040)&quot; d=&quot;M218 136c55 0 108 -28 108 -89c0 -71 -55 -121 -102 -149c-35 -21 -75 -34 -116 -34c-55 0 -108 28 -108 89c0 71 55 121 102 149c35 21 75 34 116 34z&quot; fill=&quot;currentColor&quot;&gt;&lt;/path&gt;
&lt;/g&gt;
&lt;/a&gt;
&lt;g transform=&quot;translate(20.1260, 12.7790)&quot;&gt;
&lt;rect x=&quot;-0.0650&quot; y=&quot;-2.8139&quot; width=&quot;0.1300&quot; height=&quot;3.3139&quot; ry=&quot;0.0400&quot; fill=&quot;currentColor&quot;&gt;&lt;/rect&gt;
&lt;/g&gt;
&lt;a style=&quot;color:inherit;&quot; xlink:href=&quot;textedit:///tmp/lyInsert-2026316-4293-s7ti37.ixglq.ly:13:63:64&quot;&gt;
&lt;g transform=&quot;translate(22.8861, 12.7790)&quot;&gt;
&lt;path transform=&quot;scale(0.0040, -0.0040)&quot; d=&quot;M-23 -116c0 28 11 42 40 42c33 0 78 -13 119 -31l-132 158c-7 9 -10 17 -10 25c0 34 50 66 87 99c25 22 37 52 37 83c0 24 -8 49 -25 69l-35 42c-3 3 -4 7 -4 10c0 9 9 15 17 15c4 0 8 -1 11 -5l151 -180c7 -9 10 -17 10 -25c0 -34 -50 -66 -87 -99
c-25 -22 -37 -52 -37 -83c0 -24 7 -49 24 -69l84 -99c3 -3 4 -7 4 -10c0 -9 -8 -16 -16 -16c-4 0 -9 2 -12 6c-18 21 -63 38 -97 38c-41 0 -53 -26 -53 -67c0 -35 11 -74 28 -94c7 -8 -5 -19 -12 -11c-45 54 -92 148 -92 202z&quot; fill=&quot;currentColor&quot;&gt;&lt;/path&gt;
&lt;/g&gt;
&lt;/a&gt;
&lt;g transform=&quot;translate(24.1253, 3.7790)&quot;&gt;
&lt;rect x=&quot;-0.0650&quot; y=&quot;-0.5000&quot; width=&quot;0.1300&quot; height=&quot;3.3139&quot; ry=&quot;0.0400&quot; fill=&quot;currentColor&quot;&gt;&lt;/rect&gt;
&lt;/g&gt;
&lt;g transform=&quot;translate(9.3358, 4.7790)&quot;&gt;
&lt;path transform=&quot;scale(0.0040, -0.0040)&quot; d=&quot;M266 -635h-6c-108 0 -195 88 -195 197c0 58 53 103 112 103c54 0 95 -47 95 -103c0 -52 -43 -95 -95 -95c-11 0 -21 2 -31 6c26 -39 68 -65 117 -65h4zM461 -203c68 24 113 95 113 164c0 90 -66 179 -173 190c24 -116 46 -231 60 -354zM74 28c0 -135 129 -247 264 -247
c28 0 55 2 82 6c-14 127 -37 245 -63 364c-79 -8 -124 -61 -124 -119c0 -44 25 -91 81 -123c5 -5 7 -10 7 -15c0 -11 -10 -22 -22 -22c-3 0 -6 1 -9 2c-80 43 -117 115 -117 185c0 88 58 174 160 197c-14 58 -29 117 -46 175c-107 -121 -213 -243 -213 -403zM335 -262
c-188 0 -333 172 -333 374c0 177 131 306 248 441c-19 62 -37 125 -45 190c-6 52 -7 104 -7 156c0 115 55 224 149 292c6 5 14 5 20 0c71 -84 133 -245 133 -358c0 -143 -86 -255 -180 -364c21 -68 39 -138 56 -207c4 0 9 1 13 1c155 0 256 -128 256 -261
c0 -76 -33 -154 -107 -210c-22 -17 -47 -28 -73 -36c3 -35 5 -70 5 -105c0 -19 -1 -39 -2 -58c-7 -119 -88 -225 -202 -228l1 43c93 2 153 92 159 191c1 18 2 37 2 55c0 31 -1 61 -4 92c-29 -5 -58 -8 -89 -8zM428 916c0 55 -4 79 -20 129c-99 -48 -162 -149 -162 -259
c0 -74 18 -133 36 -194c80 97 146 198 146 324z&quot; fill=&quot;currentColor&quot;&gt;&lt;/path&gt;
&lt;/g&gt;
&lt;g transform=&quot;translate(-0.3115, 9.0728)&quot;&gt;
&lt;text font-family=&quot;serif&quot; font-size=&quot;2.2001&quot; text-anchor=&quot;start&quot; fill=&quot;currentColor&quot;&gt;
&lt;tspan&gt;Piano&lt;/tspan&gt;
&lt;/text&gt;
&lt;/g&gt;
&lt;g transform=&quot;translate(8.1758, 8.2790)&quot;&gt;
&lt;path transform=&quot;scale(0.0040, -0.0040)&quot; d=&quot;M-189 -1146c0 267 108 528 108 780c0 135 -33 261 -129 360c-3 3 -3 3 -3 6s0 3 3 6c96 99 129 225 129 360c0 252 -108 513 -108 780c0 189 45 366 177 501c15 15 39 -9 24 -24c-99 -99 -129 -225 -129 -360c0 -249 102 -504 102 -765c0 -186 -45 -363 -174 -498
c129 -135 174 -312 174 -498c0 -261 -102 -516 -102 -765c0 -135 30 -261 129 -360c15 -15 -9 -39 -24 -24c-132 135 -177 312 -177 501z&quot; fill=&quot;currentColor&quot;&gt;&lt;/path&gt;
&lt;/g&gt;
&lt;g transform=&quot;translate(8.4758, 8.2790)&quot;&gt;
&lt;rect x=&quot;0.0000&quot; y=&quot;-6.5500&quot; width=&quot;0.1600&quot; height=&quot;13.1000&quot; ry=&quot;0.0500&quot; fill=&quot;currentColor&quot;&gt;&lt;/rect&gt;
&lt;/g&gt;
&lt;a style=&quot;color:inherit;&quot; xlink:href=&quot;textedit:///tmp/lyInsert-2026316-4293-s7ti37.ixglq.ly:12:59:60&quot;&gt;
&lt;g transform=&quot;translate(17.2358, 6.7790)&quot;&gt;
&lt;path transform=&quot;scale(0.0040, -0.0040)&quot; d=&quot;M218 136c55 0 108 -28 108 -89c0 -71 -55 -121 -102 -149c-35 -21 -75 -34 -116 -34c-55 0 -108 28 -108 89c0 71 55 121 102 149c35 21 75 34 116 34z&quot; fill=&quot;currentColor&quot;&gt;&lt;/path&gt;
&lt;/g&gt;
&lt;/a&gt;
&lt;a style=&quot;color:inherit;&quot; xlink:href=&quot;textedit:///tmp/lyInsert-2026316-4293-s7ti37.ixglq.ly:12:63:64&quot;&gt;
&lt;g transform=&quot;translate(22.8861, 6.7790)&quot;&gt;
&lt;path transform=&quot;scale(0.0040, -0.0040)&quot; d=&quot;M218 136c55 0 108 -28 108 -89c0 -71 -55 -121 -102 -149c-35 -21 -75 -34 -116 -34c-55 0 -108 28 -108 89c0 71 55 121 102 149c35 21 75 34 116 34z&quot; fill=&quot;currentColor&quot;&gt;&lt;/path&gt;
&lt;/g&gt;
&lt;/a&gt;
&lt;g transform=&quot;translate(13.5358, 3.7790)&quot;&gt;
&lt;path transform=&quot;scale(0.0040, -0.0040)&quot; d=&quot;M359 27c-49 0 -75 42 -75 75c0 38 27 77 72 77c4 0 9 0 14 -1c-28 37 -72 59 -120 59c-106 0 -113 -73 -113 -186v-51v-51c0 -113 7 -187 113 -187c80 0 139 70 158 151c2 7 7 10 12 10c6 0 13 -4 13 -12c0 -94 -105 -174 -183 -174c-68 0 -137 21 -184 70
c-49 51 -66 122 -66 193s17 142 66 193c47 49 116 69 184 69c87 0 160 -63 175 -149c1 -5 1 -10 1 -14c0 -40 -30 -72 -67 -72z&quot; fill=&quot;currentColor&quot;&gt;&lt;/path&gt;
&lt;/g&gt;
&lt;a style=&quot;color:inherit;&quot; xlink:href=&quot;textedit:///tmp/lyInsert-2026316-4293-s7ti37.ixglq.ly:13:59:60&quot;&gt;
&lt;g transform=&quot;translate(17.2358, 12.7790)&quot;&gt;
&lt;path transform=&quot;scale(0.0040, -0.0040)&quot; d=&quot;M-23 -116c0 28 11 42 40 42c33 0 78 -13 119 -31l-132 158c-7 9 -10 17 -10 25c0 34 50 66 87 99c25 22 37 52 37 83c0 24 -8 49 -25 69l-35 42c-3 3 -4 7 -4 10c0 9 9 15 17 15c4 0 8 -1 11 -5l151 -180c7 -9 10 -17 10 -25c0 -34 -50 -66 -87 -99
c-25 -22 -37 -52 -37 -83c0 -24 7 -49 24 -69l84 -99c3 -3 4 -7 4 -10c0 -9 -8 -16 -16 -16c-4 0 -9 2 -12 6c-18 21 -63 38 -97 38c-41 0 -53 -26 -53 -67c0 -35 11 -74 28 -94c7 -8 -5 -19 -12 -11c-45 54 -92 148 -92 202z&quot; fill=&quot;currentColor&quot;&gt;&lt;/path&gt;
&lt;/g&gt;
&lt;/a&gt;
&lt;g transform=&quot;translate(9.3358, 11.7790)&quot;&gt;
&lt;path transform=&quot;scale(0.0040, -0.0040)&quot; d=&quot;M557 -125c0 28 23 51 51 51s51 -23 51 -51s-23 -51 -51 -51s-51 23 -51 51zM557 125c0 28 23 51 51 51s51 -23 51 -51s-23 -51 -51 -51s-51 23 -51 51zM232 263c172 0 293 -88 293 -251c0 -263 -263 -414 -516 -521c-3 -3 -6 -4 -9 -4c-7 0 -13 6 -13 13c0 3 1 6 4 9
c202 118 412 265 412 493c0 120 -63 235 -171 235c-74 0 -129 -54 -154 -126c11 5 22 8 34 8c55 0 100 -45 100 -100c0 -58 -44 -106 -100 -106c-60 0 -112 47 -112 106c0 133 102 244 232 244z&quot; fill=&quot;currentColor&quot;&gt;&lt;/path&gt;
&lt;/g&gt;
&lt;g transform=&quot;translate(13.5358, 12.7790)&quot;&gt;
&lt;path transform=&quot;scale(0.0040, -0.0040)&quot; d=&quot;M359 27c-49 0 -75 42 -75 75c0 38 27 77 72 77c4 0 9 0 14 -1c-28 37 -72 59 -120 59c-106 0 -113 -73 -113 -186v-51v-51c0 -113 7 -187 113 -187c80 0 139 70 158 151c2 7 7 10 12 10c6 0 13 -4 13 -12c0 -94 -105 -174 -183 -174c-68 0 -137 21 -184 70
c-49 51 -66 122 -66 193s17 142 66 193c47 49 116 69 184 69c87 0 160 -63 175 -149c1 -5 1 -10 1 -14c0 -40 -30 -72 -67 -72z&quot; fill=&quot;currentColor&quot;&gt;&lt;/path&gt;
&lt;/g&gt;
&lt;g transform=&quot;translate(18.4750, 3.7790)&quot;&gt;
&lt;rect x=&quot;-0.0650&quot; y=&quot;-0.5000&quot; width=&quot;0.1300&quot; height=&quot;3.3139&quot; ry=&quot;0.0400&quot; fill=&quot;currentColor&quot;&gt;&lt;/rect&gt;
&lt;/g&gt;
&lt;a style=&quot;color:inherit;&quot; xlink:href=&quot;textedit:///tmp/lyInsert-2026316-4293-s7ti37.ixglq.ly:12:67:68&quot;&gt;
&lt;g transform=&quot;translate(29.5353, 6.7790)&quot;&gt;
&lt;path transform=&quot;scale(0.0040, -0.0040)&quot; d=&quot;M218 136c55 0 108 -28 108 -89c0 -71 -55 -121 -102 -149c-35 -21 -75 -34 -116 -34c-55 0 -108 28 -108 89c0 71 55 121 102 149c35 21 75 34 116 34z&quot; fill=&quot;currentColor&quot;&gt;&lt;/path&gt;
&lt;/g&gt;
&lt;/a&gt;
&lt;a style=&quot;color:inherit;&quot; xlink:href=&quot;textedit:///tmp/lyInsert-2026316-4293-s7ti37.ixglq.ly:13:67:68&quot;&gt;
&lt;g transform=&quot;translate(29.5353, 12.7790)&quot;&gt;
&lt;path transform=&quot;scale(0.0040, -0.0040)&quot; d=&quot;M-23 -116c0 28 11 42 40 42c33 0 78 -13 119 -31l-132 158c-7 9 -10 17 -10 25c0 34 50 66 87 99c25 22 37 52 37 83c0 24 -8 49 -25 69l-35 42c-3 3 -4 7 -4 10c0 9 9 15 17 15c4 0 8 -1 11 -5l151 -180c7 -9 10 -17 10 -25c0 -34 -50 -66 -87 -99
c-25 -22 -37 -52 -37 -83c0 -24 7 -49 24 -69l84 -99c3 -3 4 -7 4 -10c0 -9 -8 -16 -16 -16c-4 0 -9 2 -12 6c-18 21 -63 38 -97 38c-41 0 -53 -26 -53 -67c0 -35 11 -74 28 -94c7 -8 -5 -19 -12 -11c-45 54 -92 148 -92 202z&quot; fill=&quot;currentColor&quot;&gt;&lt;/path&gt;
&lt;/g&gt;
&lt;/a&gt;
&lt;g transform=&quot;translate(30.7746, 3.7790)&quot;&gt;
&lt;rect x=&quot;-0.0650&quot; y=&quot;-0.5000&quot; width=&quot;0.1300&quot; height=&quot;3.3139&quot; ry=&quot;0.0400&quot; fill=&quot;currentColor&quot;&gt;&lt;/rect&gt;
&lt;/g&gt;
&lt;a style=&quot;color:inherit;&quot; xlink:href=&quot;textedit:///tmp/lyInsert-2026316-4293-s7ti37.ixglq.ly:12:65:66&quot;&gt;
&lt;g transform=&quot;translate(25.7113, 3.7790)&quot;&gt;
&lt;path transform=&quot;scale(0.0040, -0.0040)&quot; d=&quot;M-23 -116c0 28 11 42 40 42c33 0 78 -13 119 -31l-132 158c-7 9 -10 17 -10 25c0 34 50 66 87 99c25 22 37 52 37 83c0 24 -8 49 -25 69l-35 42c-3 3 -4 7 -4 10c0 9 9 15 17 15c4 0 8 -1 11 -5l151 -180c7 -9 10 -17 10 -25c0 -34 -50 -66 -87 -99
c-25 -22 -37 -52 -37 -83c0 -24 7 -49 24 -69l84 -99c3 -3 4 -7 4 -10c0 -9 -8 -16 -16 -16c-4 0 -9 2 -12 6c-18 21 -63 38 -97 38c-41 0 -53 -26 -53 -67c0 -35 11 -74 28 -94c7 -8 -5 -19 -12 -11c-45 54 -92 148 -92 202z&quot; fill=&quot;currentColor&quot;&gt;&lt;/path&gt;
&lt;/g&gt;
&lt;/a&gt;
&lt;a style=&quot;color:inherit;&quot; xlink:href=&quot;textedit:///tmp/lyInsert-2026316-4293-s7ti37.ixglq.ly:13:65:66&quot;&gt;
&lt;g transform=&quot;translate(25.7113, 9.7790)&quot;&gt;
&lt;path transform=&quot;scale(0.0040, -0.0040)&quot; d=&quot;M218 136c55 0 108 -28 108 -89c0 -71 -55 -121 -102 -149c-35 -21 -75 -34 -116 -34c-55 0 -108 28 -108 89c0 71 55 121 102 149c35 21 75 34 116 34z&quot; fill=&quot;currentColor&quot;&gt;&lt;/path&gt;
&lt;/g&gt;
&lt;/a&gt;
&lt;g transform=&quot;translate(25.7763, 12.7790)&quot;&gt;
&lt;rect x=&quot;-0.0650&quot; y=&quot;-2.8139&quot; width=&quot;0.1300&quot; height=&quot;3.3139&quot; ry=&quot;0.0400&quot; fill=&quot;currentColor&quot;&gt;&lt;/rect&gt;
&lt;/g&gt;
&lt;/svg&gt;
&lt;p&gt;I knew what middle C was... I had been playing Ukulele for a while so I knew it was the note on the second string of the Ukulele, if it was tuned in standard tuning but was unaware of its relationship to the piano.&lt;/p&gt;
&lt;p&gt;I quickly learned to love the uniformity of the piano. If middle C is here, then the next note D is the white key next to it, and the E is the next one after. Also, there&#39;s only one place to play a given note.&lt;/p&gt;
&lt;p&gt;I was beyond excited to be learning the piano. For a long time, I had been wanting to learn the piano, which I always saw as the king of all instruments. Taking the plunge and showing up to piano lessons was a dream come true. We started using the book &lt;a href=&quot;https://www.alfred.com/alfreds-basic-adult-piano-course-lesson-book-1/p/00-2236/&quot;&gt;Alfred&#39;s Basic Adult Piano Course: Level 1&lt;/a&gt; and over the months that followed we made steady progress on that. We would practice a couple of pieces one hand at a time during the lesson and I would perfect those at home to perform them in the following lesson. If all was ok, great - we moved ahead. If not, we would practice more together and I would perfect them again at home. We quickly got to June and my teacher asked me if I would like to play publicly in a student concert. I immediately said &amp;quot;Yes&amp;quot;, which seemed to take her by surprise. Apparently, adult students and especially beginner students are not keen on joining concerts. At this point, we chose two pieces: Allegretto by Vanhal and Nächtliche Reise by Gurlitt. She played the pieces for me and I loved them so being able to play three months later seemed amazing. Of course, now I needed to be able to focus on them as they were slightly trickier than my weekly ones from Alfred&#39;s book. So for the following lessons, I focused on them and at home, I did the same. Every lesson was focused on perfecting some parts of each piece. And on Saturday it all came together.&lt;/p&gt;
&lt;p&gt;My wife, both of my children, and my parents-in-law came to the concert and although I was anxious the whole day, I got slightly less anxious before. I didn&#39;t get nervous until pretty much it was my turn to play, but I didn&#39;t have a meltdown. That felt like it came later. The Vanhal piece was always the one I had the most trouble with and Gurlitt was the one that seemed easier. However, I played Vanhal first after the tradition of playing older pieces first. Vanhal was going very well and the first repetition was flawless. During the second repetition I thought that &amp;quot;it&#39;s just a matter of finishing the second repetition and then the easy piece is coming.&amp;quot;. I deviated my look a bit from the keys as it occurred to me and as I looked into the keys, I forgot where in the scale I was and missed a beat. Damn! I continued as if nothing happened and I grew increasingly nervous wondering what else was going to happen. My brain went haywire as I started the easier piece and I missed a couple of beats. This was supposed to be the easy piece but I was murdering it. But like everything else, it came to an end. As I get up from the piano, everyone was clapping and afterward many people came to congratulate me but I was disappointed and couldn&#39;t shake the feeling I had screwed up.&lt;/p&gt;
&lt;p&gt;My wife filmed it and I later watched, half covering my face in embarrassment, but noticed that actually, it sounded quite nice. And although, yes, there were a couple of lapses, unless you know the piece you wouldn&#39;t necessarily notice them.&lt;/p&gt;
&lt;p&gt;I am just very excited I had the opportunity to join this concert and can&#39;t wait to do better, on harder pieces, next time.&lt;/p&gt;
&lt;p&gt;When I decide to take on a new hobby, I want to give it all and be the best amateur I can be. I believe that you need a strategy instead of going at it Adhoc given you&#39;re doing it in your spare time and you have to fit things in with your family life.
So I will finish off with a few points and tips for absolute beginners from my journey that might be more interesting to read than the above memory dump. These are things that worked very well for me and that I would advise you to consider but they are in no way gospel or remotely generalizable. But if you see you&#39;re in a position to try them out, please do.&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;Immerse yourself. If you are going to do a hobby make sure you stay up to date on developments and what&#39;s going on. Personally, I subscribed to Pianist, subscribed to &lt;a href=&quot;https://www.reddit.com/r/piano&quot;&gt;r/piano&lt;/a&gt; and &lt;a href=&quot;https://www.reddit.com/r/pianolearning&quot;&gt;r/pianolearning&lt;/a&gt;, follow several pianists on Twitter like &lt;a href=&quot;https://twitter.com/lang_lang&quot;&gt;@lang_lang&lt;/a&gt;, &lt;a href=&quot;https://twitter.com/lars_vogt&quot;&gt;@lars_vogt&lt;/a&gt;, or &lt;a href=&quot;https://twitter.com/daniil_trifonov&quot;&gt;@daniil_trifonov&lt;/a&gt;, and started listening to podcasts related to piano playing like the &lt;a href=&quot;https://open.spotify.com/show/72OQO63V0y3jZ0fh4tVlwj&quot;&gt;Josh Wright Piano Podcast&lt;/a&gt;. In addition to that my knowledge and feeling for classical music were zero. Since then, I have been regularly putting on some classical music, looking at the scores (even if all pieces are impossible for me to play), and trying to appreciate how hard it must be to play them. I don&#39;t read a huge amount except on holidays when I tend to disconnect and read quite a bit. So these last Summer holidays saw me going through 3 books related to Piano playing that I recommend: &lt;a href=&quot;https://www.goodreads.com/book/show/17332356-play-it-again&quot;&gt;Play it Again: An Amateur Against the Impossible&lt;/a&gt; by Alan Rusbridger, &lt;a href=&quot;https://www.goodreads.com/book/show/2727587-journey-of-a-thousand-miles&quot;&gt;Journey of a Thousand Miles: My Story&lt;/a&gt; by Lang Lang and David Ritz, and &lt;a href=&quot;https://www.goodreads.com/book/show/589577.Piano_Notes&quot;&gt;Piano Notes: The World of the Pianist&lt;/a&gt; by Charles Rosen.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Measure your progress. It&#39;s hard to know how much time you&#39;re actually putting in if you don&#39;t measure. You might guess but most likely you&#39;ll be wrong. I use a file where I write down all post-lesson notes about the lesson and track all my time in front of the piano at home using &lt;a href=&quot;https://instrumentive.com/&quot;&gt;Instrumentive&lt;/a&gt;.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Set a long-term goal to challenge yourself. Hard long-term goals are important in my opinion. They keep you focused with something to look forward to. If you were running, maybe a marathon could be a long-term goal. In piano, maybe playing a certain piece publicly (and this can mean to others or even just to your family who you invite for a concert). Some people recommend that you share that with others, or even post online for further motivation. I generally prefer to keep those (sometimes ridiculous goals) within a small circle of people. But I guess that&#39;s just a personal preference.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Research the best. Within your chosen hobby there are always some who are at the top of it. Follow them on social media, read about them, and see how they practice, train, etc. How they got to where they are, etc. It&#39;s likely that as a hobbyist you might not be able to emulate everything they do or how they do it. However, I think that just knowing what&#39;s possible is incredibly motivating. At the moment, I always keep an eye on Lang Lang, Daniil Trifonov, and Josh Wright.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Find support. Sometimes you have hobbies that are very social like playing football, poker, etc. Some other times it can be lonely like triathlon racing or piano playing. In any case, you should try to find some support... a little fan base. That can be your family or close circle of friends. Plan to play a piece for them, and ask them to criticize you. Keep playing, and improving. During the two weeks preceding my first public performance, I asked my wife and children to listen to me play every single evening before bedtime. They were and are my support, my fan base, and those who&#39;ll cheer for me even if I screw up. Find those people!&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Lastly: Prioritize, prioritize, prioritize! You&#39;ll need to prioritize a lot. If you need to take one or two hours of your day, you&#39;ll need to prioritize. Do I need to take that coffee break? Can I optimize my commute? Do I really need to watch that series on Netflix, etc? In the end, we all have 24 hours per day. What matters, is how we spend those 24 hours. There&#39;s no magic!&lt;/p&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;h2&gt;Book references:&lt;/h2&gt;
&lt;ul&gt;
&lt;li&gt;Alfred&#39;s Basic Adult Piano Course : Level 1 (ISBN: 978-0739082416).&lt;/li&gt;
&lt;li&gt;Play it Again: An Amateur Against the Impossible (ISBN: 978-1250875402)&lt;/li&gt;
&lt;li&gt;Journey of a Thousand Miles: My Story by Lang Land and David Ritz (ISBN: 978-0385524575)&lt;/li&gt;
&lt;li&gt;Piano Notes: The World of the Pianist by Charles Rosen (ISBN: 978-0140298635)&lt;/li&gt;
&lt;/ul&gt;
&lt;h2&gt;Music references:&lt;/h2&gt;
&lt;ul&gt;
&lt;li&gt;Allegretto by J. B. Vanhal&lt;/li&gt;
&lt;li&gt;Nächtliche Reise by C. Gurlitt&lt;/li&gt;
&lt;/ul&gt;
&lt;h2&gt;Online Forums:&lt;/h2&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href=&quot;https://www.reddit.com/r/piano&quot;&gt;&lt;code&gt;r/piano&lt;/code&gt;&lt;/a&gt;, take a look especially at the &lt;a href=&quot;https://www.reddit.com/r/piano/wiki/faq/&quot;&gt;FAQ&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href=&quot;https://www.reddit.com/r/pianolearning&quot;&gt;&lt;code&gt;r/pianolearning&lt;/code&gt;&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;h2&gt;Others:&lt;/h2&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href=&quot;https://www.pianistmagazine.com/&quot;&gt;Pianist Magazine&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href=&quot;https://open.spotify.com/show/72OQO63V0y3jZ0fh4tVlwj&quot;&gt;Josh Wright Piano Podcast&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href=&quot;https://instrumentive.com/&quot;&gt;Instrumentive Progress Tracking App&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
</content>
  </entry>
  <entry>
    <title>A tour of the `for..of` implementation for 32bits JSC</title>
    <link href="https://p.ocmatos.com/blog/a-tour-of-the-for-of-implementation-for-32bits-jsc.html"/>
    <updated>2020-11-20T00:00:00Z</updated>
    <id>https://p.ocmatos.com/blog/a-tour-of-the-for-of-implementation-for-32bits-jsc.html</id>
    <content xml:lang="en" type="html">&lt;h2&gt;Context&lt;/h2&gt;
&lt;p&gt;On April 18, 2020, Keith Miller from Apple committed a change titled &lt;a href=&quot;https://trac.webkit.org/changeset/260323/webkit&quot;&gt;Redesign how we do for-of iteration for JSArrays&lt;/a&gt;. This patch adds an intrinsic for the &lt;code&gt;for-of&lt;/code&gt; javascript construct when they are being iterated with the built-in &lt;code&gt;Symbol.iterator&lt;/code&gt;. However, the implementation is only for 64 bit architectures. At &lt;a href=&quot;https://www.igalia.com/&quot;&gt;Igalia&lt;/a&gt;, we maintain and support javascriptCore 32bits so on July 29, 2020, &lt;a href=&quot;https://trac.webkit.org/changeset/265036/webkit&quot;&gt;I committed support&lt;/a&gt; for the &lt;code&gt;for-of&lt;/code&gt; intrinsics work on 32 bit architectures as joint work with &lt;a href=&quot;https://caiolima.github.io/&quot;&gt;Caio Lima&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;In this blog post, I will attempt to give an overview of the implemented changes in revision &lt;a href=&quot;https://trac.webkit.org/changeset/265036/webkit&quot;&gt;r265036&lt;/a&gt; of WebKit which extends support of the &lt;code&gt;for-of&lt;/code&gt; intrinsics work to 32 bits. For the purposes of brevity I will focus on the iterator open intrinsic.&lt;/p&gt;
&lt;h2&gt;Overview&lt;/h2&gt;
&lt;p&gt;JSC consists of several tiers (compilation layers). These tiers start with LLInt (the Low-Level Interpreter), followed by the Baseline JIT, DFG (Data Flow Graph JIT), and FTL (Faster-Than-Light JIT). We will not discuss FTL here since we do not run the FTL tier for 32 bits.&lt;/p&gt;
&lt;p&gt;The patch for &lt;a href=&quot;https://trac.webkit.org/changeset/265036/webkit&quot;&gt;r265036&lt;/a&gt; touches architecture specific files, like &lt;a href=&quot;https://trac.webkit.org/browser/webkit/trunk/Source/javascriptCore/assembler/MacroAssemblerARMv7.h?rev=265036&quot;&gt;MacroAssemblerARMv7.h&lt;/a&gt; and &lt;a href=&quot;https://trac.webkit.org/browser/webkit/trunk/Source/javascriptCore/assembler/MacroAssemblerMIPS.h?rev=265036&quot;&gt;MacroAssemblerMIPS.h&lt;/a&gt;, however these changes are less relevant as we just add a couple of functions for emitting octet load/store instructions (&lt;code&gt;store8&lt;/code&gt;, &lt;code&gt;or8&lt;/code&gt;, etc). Instead, we will focus on changes to the tiers starting with the changes to LLInt.&lt;/p&gt;
&lt;h2&gt;javascript and for-of&lt;/h2&gt;
&lt;p&gt;I guess as Compiler Engineers we should try to understand what we have to implement first. Lets try to use an iterator implicitly through a &lt;a href=&quot;https://developer.mozilla.org/en-US/docs/Web/javascript/Reference/Statements/for...of&quot;&gt;&lt;code&gt;for-of&lt;/code&gt; loop&lt;/a&gt;.&lt;/p&gt;
&lt;pre class=&quot;language-javascript&quot;&gt;&lt;code class=&quot;language-javascript&quot;&gt;&lt;span class=&quot;token keyword&quot;&gt;const&lt;/span&gt; a &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;token string&quot;&gt;&quot;a&quot;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&quot;b&quot;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&quot;c&quot;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
&lt;span class=&quot;token keyword&quot;&gt;for&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token keyword&quot;&gt;const&lt;/span&gt; value &lt;span class=&quot;token keyword&quot;&gt;of&lt;/span&gt; a&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;
  console&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;log&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;value&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
&lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;
&lt;span class=&quot;token comment&quot;&gt;// =&gt;&lt;/span&gt;
&lt;span class=&quot;token comment&quot;&gt;// &quot;a&quot;&lt;/span&gt;
&lt;span class=&quot;token comment&quot;&gt;// &quot;b&quot;&lt;/span&gt;
&lt;span class=&quot;token comment&quot;&gt;// &quot;c&quot;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The &lt;code&gt;for-of&lt;/code&gt; statement gets an iterator from the array behind the scenes and loops through its values assigning them one by one to the variable &lt;code&gt;value&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;If we were to look at using the iterator explicitly, the above loop would be transformed to:&lt;/p&gt;
&lt;pre class=&quot;language-javascript&quot;&gt;&lt;code class=&quot;language-javascript&quot;&gt;&lt;span class=&quot;token keyword&quot;&gt;const&lt;/span&gt; a &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;token string&quot;&gt;&quot;a&quot;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&quot;b&quot;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&quot;c&quot;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
&lt;span class=&quot;token keyword&quot;&gt;const&lt;/span&gt; it &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; a&lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;Symbol&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;iterator&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
&lt;span class=&quot;token keyword&quot;&gt;let&lt;/span&gt; result &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; it&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;next&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
&lt;span class=&quot;token keyword&quot;&gt;while&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;!&lt;/span&gt;result&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;done&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;
  console&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;log&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;result&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;value&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
  result &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; it&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;next&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
&lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;So the symbol &lt;code&gt;Symbol.iterator&lt;/code&gt; returns a method that when called returns an iterator. This iterator has a &lt;code&gt;next&lt;/code&gt; function to get a result object. The result object has a &lt;code&gt;done&lt;/code&gt; property indicating if we are done iterating and a value which is the current value in the iteration. There&#39;s something to keep in mind though. The &lt;code&gt;done&lt;/code&gt; property is optional if its value would be &lt;code&gt;false&lt;/code&gt; and the &lt;code&gt;value&lt;/code&gt; property is optional if its value would be &lt;code&gt;undefined&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;Armed with this knowledge, &lt;a href=&quot;https://trac.webkit.org/changeset/260323/webkit&quot;&gt;revision 260323&lt;/a&gt; which implements &lt;code&gt;for-of&lt;/code&gt; intrinsics for 64 bits, makes a lot more sense. It implements two intrinsics &lt;code&gt;op_iterator_open&lt;/code&gt; and &lt;code&gt;op_iterator_next&lt;/code&gt;. The commit message explains the semantics of each operator as follows:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;The &lt;code&gt;op_iterator_open&lt;/code&gt; bytecode is semantically the same as:&lt;/li&gt;
&lt;/ul&gt;
&lt;pre&gt;&lt;code&gt;iterator = symbolIterator.@call(iterable);
next = iterator.next;
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;where iterable is the rhs of the &lt;code&gt;for-of&lt;/code&gt; and &lt;code&gt;symbolIterator&lt;/code&gt; is the result of running &lt;code&gt;iterable.symbolIterator&lt;/code&gt;;&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;The &lt;code&gt;op_iterator_next&lt;/code&gt; bytecode is semantically the same as:&lt;/li&gt;
&lt;/ul&gt;
&lt;pre&gt;&lt;code&gt;nextResult = next.@call(iterator);
done = nextResult.done;
value = done ? (undefined / bottom) : nextResult.value;
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;where &lt;code&gt;nextResult&lt;/code&gt; is a temporary (the value &lt;code&gt;VirtualRegister&lt;/code&gt; in the LLInt/Baseline and a &lt;code&gt;tmp&lt;/code&gt; in the DFG);&lt;/p&gt;
&lt;p&gt;We can now proceed to understand how to implement these on 32bits.&lt;/p&gt;
&lt;h2&gt;LLInt&lt;/h2&gt;
&lt;p&gt;LLInt is a bytecode interpreter implemented using a DSL (Domain Specific Language). The compiler for it is &lt;a href=&quot;https://github.com/WebKit/webkit/tree/master/Source/javascriptCore/offlineasm&quot;&gt;implemented in Ruby&lt;/a&gt;. The &lt;code&gt;offlineasm&lt;/code&gt; compiler reads this DSL and outputs a bytecode interpreter in native assembler, for a few supported architectures, or cpp. The cpp backend is known as Cloop and if you enable it, to for example compile JSC for an IBM System Z9, the JSC build system generates the cpp interpreter, compiles it and then links it with JSC. If you do enable Cloop, then none of the remaining JIT tiers are available. This is as much as we will discuss about Cloop as both ARMv7 and MIPS have LLInt support, so from here on out we will assume we are working with the native assembly backend for &lt;code&gt;offlineasm&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;The source code for LLInt (input to &lt;code&gt;offlineasm&lt;/code&gt;) lives in &lt;a href=&quot;https://github.com/WebKit/webkit/tree/master/Source/javascriptCore/llint&quot;&gt;&lt;code&gt;WebKit/Source/javascriptCore&lt;/code&gt;&lt;/a&gt;. The entry point is at &lt;a href=&quot;https://github.com/WebKit/webkit/blob/master/Source/javascriptCore/llint/LowLevelInterpreter.asm&quot;&gt;&lt;code&gt;LowLevelInterpreter.asm&lt;/code&gt;&lt;/a&gt; and it starts with a bunch of constant definitions followed by something you&#39;ll see often: a macro.&lt;/p&gt;
&lt;p&gt;At the time of writing, the first one is &lt;a href=&quot;https://github.com/WebKit/webkit/blob/ce2bff038035bac33352237652d33ff795a69959/Source/javascriptCore/llint/LowLevelInterpreter.asm#L319&quot;&gt;&lt;code&gt;nextInstruction&lt;/code&gt;&lt;/a&gt;:&lt;/p&gt;
&lt;pre class=&quot;language-ruby&quot;&gt;&lt;code class=&quot;language-ruby&quot;&gt;macro nextInstruction&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;
    loadb &lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;token constant&quot;&gt;PB&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;token constant&quot;&gt;PC&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; t0
    leap _g_opcodeMap&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; t1
    jmp &lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;t1&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; t0&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; PtrSize&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; BytecodePtrTag
&lt;span class=&quot;token keyword&quot;&gt;end&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Each time you call elsewhere &lt;code&gt;nextInstruction&lt;/code&gt; (which actually starts processing the next bytecode), the call will be replaced by the three (low-level) instructions in the body of the macro. These instructions &lt;code&gt;loadb&lt;/code&gt;, &lt;code&gt;leap&lt;/code&gt; and &lt;code&gt;jmp&lt;/code&gt; are defined on per-architecture in &lt;a href=&quot;https://github.com/WebKit/webkit/blob/master/Source/javascriptCore/offlineasm/arm.rb&quot;&gt;&lt;code&gt;arm.rb&lt;/code&gt;&lt;/a&gt;, &lt;a href=&quot;https://github.com/WebKit/webkit/blob/master/Source/javascriptCore/offlineasm/arm64.rb&quot;&gt;&lt;code&gt;arm64.rb&lt;/code&gt;&lt;/a&gt;, &lt;a href=&quot;https://github.com/WebKit/webkit/blob/master/Source/javascriptCore/offlineasm/arm64e.rb&quot;&gt;&lt;code&gt;arm64e.rb&lt;/code&gt;&lt;/a&gt;, &lt;a href=&quot;https://github.com/WebKit/webkit/blob/master/Source/javascriptCore/offlineasm/mips.rb&quot;&gt;&lt;code&gt;mips.rb&lt;/code&gt;&lt;/a&gt;, and &lt;a href=&quot;https://github.com/WebKit/webkit/blob/master/Source/javascriptCore/offlineasm/x86.rb&quot;&gt;&lt;code&gt;x86.rb&lt;/code&gt;&lt;/a&gt;. Listing of all available instructions are in &lt;a href=&quot;https://github.com/WebKit/webkit/blob/master/Source/javascriptCore/offlineasm/instructions.rb&quot;&gt;&lt;code&gt;instructions.rb&lt;/code&gt;&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;So for 32bits arm:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;the &lt;code&gt;loadb&lt;/code&gt; will generate a &lt;a href=&quot;https://github.com/WebKit/webkit/blob/b63199a0f04a5f7a435cb91e914419d2fd4f50d7/Source/javascriptCore/offlineasm/arm.rb#L447&quot;&gt;&lt;code&gt;ldrb&lt;/code&gt;&lt;/a&gt;,&lt;/li&gt;
&lt;li&gt;the &lt;code&gt;leap&lt;/code&gt; will generate a &lt;a href=&quot;https://github.com/WebKit/webkit/blob/b63199a0f04a5f7a435cb91e914419d2fd4f50d7/Source/javascriptCore/offlineasm/arm.rb#L659&quot;&gt;load effective address&lt;/a&gt; through an &lt;code&gt;add&lt;/code&gt; whose format depends on the first operand of &lt;code&gt;leap&lt;/code&gt;, and&lt;/li&gt;
&lt;li&gt;the &lt;code&gt;jmp&lt;/code&gt; will generate either a &lt;a href=&quot;https://github.com/WebKit/webkit/blob/b63199a0f04a5f7a435cb91e914419d2fd4f50d7/Source/javascriptCore/offlineasm/arm.rb#L597&quot;&gt;&lt;code&gt;b&lt;/code&gt; or &lt;code&gt;mov&lt;/code&gt; instruction&lt;/a&gt; which depends on if the first operand of &lt;code&gt;jmp&lt;/code&gt; is a label.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;The compilation of this DSL to assembler is done during JSC build-time which means that by the time the interpreter is called there&#39;s no execution of ruby code.&lt;/p&gt;
&lt;p&gt;Lets start by implementing the bytecode &lt;code&gt;op_iterator_open&lt;/code&gt;, which before looked like this in &lt;a href=&quot;https://github.com/WebKit/webkit/blob/99cee626c89e192a5fd211b6ba59be68b40c310b/Source/javascriptCore/llint/LowLevelInterpreter32_64.asm#L2645&quot;&gt;&lt;code&gt;LowLevelInterpreter32_64.asm&lt;/code&gt;&lt;/a&gt;:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;llintOp(op_iterator_open, OpIteratorOpen, macro (size, get, dispatch)
    defineOSRExitReturnLabel(op_iterator_open, size)
    break
    if C_LOOP or C_LOOP_WIN
        # Insert superflous call return labels for Cloop.
        cloopCallJSFunction a0 # symbolIterator
        cloopCallJSFunction a0 # get next
    end
end)
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Before proceeding a clarification of what this is. This is essentially a top-level call to the macro &lt;code&gt;llintOp&lt;/code&gt; with three arguments:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;&lt;code&gt;op_iterator_open&lt;/code&gt;,&lt;/li&gt;
&lt;li&gt;&lt;code&gt;OpIteratorOpen&lt;/code&gt;, and&lt;/li&gt;
&lt;li&gt;&lt;code&gt;macro(size, get, dispatch) ... end&lt;/code&gt;&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;Argument &lt;code&gt;3&lt;/code&gt; is an anonymous macro passed as a first-class citizen to &lt;code&gt;llintOp&lt;/code&gt; which can then call it. The macro &lt;code&gt;llintOp&lt;/code&gt; is defined in &lt;a href=&quot;https://github.com/WebKit/webkit/blob/master/Source/javascriptCore/llint/LowLevelInterpreter.asm&quot;&gt;&lt;code&gt;LowLevelInterpreter.asm&lt;/code&gt;&lt;/a&gt; and the declaration looks like:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;macro llintOp(opcodeName, opcodeStruct, fn)
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The anonymous macro defined in the &lt;code&gt;op_iterator_open&lt;/code&gt; is passed to the &lt;code&gt;llintOp&lt;/code&gt; macro and can be called as &lt;code&gt;fn(...)&lt;/code&gt;. Most of the code organization in LLInt is like this and you should get used to reading it.&lt;/p&gt;
&lt;p&gt;As we require access to the metadata for the implementation of &lt;code&gt;op_iterator_open&lt;/code&gt; we change the declaration slightly and add a few commonly defined macros:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;llintOpWithMetadata(op_iterator_open, OpIteratorOpen, macro (size, get, dispatch, metadata, return)
    macro fastNarrow()
        callSlowPath(_iterator_open_try_fast_narrow)
    end
    macro fastWide16()
        callSlowPath(_iterator_open_try_fast_wide16)
    end
    macro fastWide32()
        callSlowPath(_iterator_open_try_fast_wide32)
    end
    size(fastNarrow, fastWide16, fastWide32, macro (callOp) callOp() end)
    bbeq r1, constexpr IterationMode::Generic, .iteratorOpenGeneric
    dispatch()
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;There are two execution paths that can be taken at this point:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;Fast path: this path is currently implemented in cpp and the code resides in &lt;a href=&quot;https://github.com/WebKit/webkit/blob/89ece25c12467354e836b224948db00fc21edf11/Source/javascriptCore/runtime/CommonSlowPaths.cpp#L856&quot;&gt;&lt;code&gt;CommonSlowPaths.cpp:iterator_open_try_fast&lt;/code&gt;&lt;/a&gt;. If it succeeds then the branch &lt;code&gt;bbeq&lt;/code&gt; is not taken and the control flow dispatches to execute the next bytecode instruction.&lt;/li&gt;
&lt;li&gt;Slow path: the generic path is taken if the fast path fails. If it fails, then it &lt;a href=&quot;https://github.com/WebKit/webkit/blob/89ece25c12467354e836b224948db00fc21edf11/Source/javascriptCore/runtime/CommonSlowPaths.cpp#L898&quot;&gt;sets &lt;code&gt;r1&lt;/code&gt; with the value &lt;code&gt;constexpr IterationMode::Generic&lt;/code&gt;&lt;/a&gt;. In the 32bits ABI &lt;code&gt;r1&lt;/code&gt; is the tag of the return value.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;All the code that follows is part of the slow path and the next step is:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;.iteratorOpenGeneric:
    macro gotoGetByIdCheckpoint()
        jmp .getByIdStart
    end

    macro getCallee(dst)
        get(m_symbolIterator, dst)
    end

    macro getArgumentIncludingThisStart(dst)
        getu(size, OpIteratorOpen, m_stackOffset, dst)
    end

    macro getArgumentIncludingThisCount(dst)
        move 1, dst
    end

    callHelper(op_iterator_open,                    # opcodeName
               _llint_slow_path_iterator_open_call, # slowPath
               OpIteratorOpen,                      # opcodeStruct
               m_iteratorProfile,                   # valueProfileName
               m_iterator,                          # dstVirtualRegister
               prepareForRegularCall,               # prepareCall
               size,                                # size
               gotoGetByIdCheckpoint,               # dispatch
               metadata,                            # metadata
               getCallee,                           # getCallee
               getArgumentIncludingThisStart,       # getArgumentStart
               getArgumentIncludingThisCount)       # getArgumentCountIncludingThis
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Here we define a few helper macros required by &lt;code&gt;callHelper&lt;/code&gt;. I have commented on the &lt;code&gt;callHelper&lt;/code&gt; call with all the argument names to ease understanding what the function does. This is a helper function to simplify the call to the method stored in the &lt;code&gt;Symbol.Iterator&lt;/code&gt; property.&lt;/p&gt;
&lt;p&gt;The implementation of the &lt;code&gt;op_iterator_open&lt;/code&gt; in LLInt ends up with:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;.getByIdStart:
	macro storeNextAndDispatch(valueTag, valuePayload)
        move valueTag, t2
        move valuePayload, t3
        get(m_next, t1)
        storei t2, TagOffset[cfr, t1, 8]
        storei t3, PayloadOffset[cfr, t1, 8]
 	    dispatch()
 	end

    # We need to load m_iterator into t3 because that&#39;s where
 	# performGetByIDHelper expects the base object   
 	loadVariable(get, m_iterator, t3, t0, t3)
 	bineq t0, CellTag, .iteratorOpenGenericGetNextSlow
 	performGetByIDHelper(OpIteratorOpen,                  # opcodeStruct
                         m_modeMetadata,                  # modeMetadataName
                         m_nextProfile,                   # valueProfileName
                         .iteratorOpenGenericGetNextSlow, # slowLabel
                         size,                            # size
                         metadata,                        # metadata
                         storeNextAndDispatch)            # return

.iteratorOpenGenericGetNextSlow:
 	callSlowPath(_llint_slow_path_iterator_open_get_next)
 	dispatch()
end)
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The &lt;code&gt;callHelper&lt;/code&gt; above placed an iterator in register &lt;code&gt;t3&lt;/code&gt;, so we check what type of tag the iterator has. If the iterator is not a cell (tagged with &lt;code&gt;CellTag&lt;/code&gt; - a cell is essentially anything that&#39;s heap allocated) then we jump to the slow path. If, however, the iterator is an object we call &lt;code&gt;storeNextAndDispatch&lt;/code&gt; through &lt;code&gt;performGetByIDHelper&lt;/code&gt; which does a few useful checks. &lt;code&gt;storeNextAndDispatch&lt;/code&gt; initializes the iterator and dispatches to the next bytecode.&lt;/p&gt;
&lt;p&gt;The &lt;code&gt;op_iterator_next&lt;/code&gt; will follow the same pattern but instead implement the semantics of &lt;code&gt;iterator.next&lt;/code&gt;. Next we will see how a similar pattern is implemented in the Baseline JIT compiler.&lt;/p&gt;
&lt;h2&gt;Baseline&lt;/h2&gt;
&lt;p&gt;The Baseline JIT source code lives inside &lt;a href=&quot;https://github.com/WebKit/webkit/tree/master/Source/javascriptCore/jit&quot;&gt;Source/javascriptCore/jit&lt;/a&gt;. The part of the JIT compiler the deals with 32 bit platforms, mostly lives in &lt;a href=&quot;https://github.com/WebKit/webkit/blob/master/Source/javascriptCore/jit/JITCall32_64.cpp&quot;&gt;Source/javascriptCore/jit/JITCall32_64.cpp&lt;/a&gt;. It is responsible to generate assembly code for a given bytecode stream as input. This explains the resemblances we have between LLInt operations, since we need to emit code that has the same semantics as the LLInt operation. This compiler applies barely any compiler optimizations, since the goal is to generate binary code as quickly as it can and its major gain on performance is due to the removal of LLInt instruction dispatch.
The Baseline compiler is a heavy user of JSC’s AssemblerHelpers. This is an essential API that’s used to abstract the code generation for the multiple architectures JSC supports and also abstracts some operations that are quite common on LLInt like &lt;code&gt;branchIfNotCell&lt;/code&gt; or &lt;code&gt;branchIfNotEmpty&lt;/code&gt;.  It also implements a set of operations that are very helpful when we need to implement JIT support for a new opcode, such as &lt;code&gt;emitGetVirtualRegister&lt;/code&gt;, &lt;code&gt;emitPutVirtualRegister&lt;/code&gt;, and &lt;code&gt;callOperation&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;Lets look at how we implement the functions &lt;code&gt;emit_op_iterator_open&lt;/code&gt;, &lt;code&gt;emitSlow_op_iterator_open&lt;/code&gt; and the corresponding &lt;code&gt;next&lt;/code&gt; versions of those.&lt;/p&gt;
&lt;pre class=&quot;language-cpp&quot;&gt;&lt;code class=&quot;language-cpp&quot;&gt;&lt;span class=&quot;token keyword&quot;&gt;void&lt;/span&gt; &lt;span class=&quot;token class-name&quot;&gt;JIT&lt;/span&gt;&lt;span class=&quot;token double-colon punctuation&quot;&gt;::&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;emit_op_iterator_open&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token keyword&quot;&gt;const&lt;/span&gt; Instruction&lt;span class=&quot;token operator&quot;&gt;*&lt;/span&gt; instruction&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;
&lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;
    &lt;span class=&quot;token keyword&quot;&gt;auto&lt;/span&gt; bytecode &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; instruction&lt;span class=&quot;token operator&quot;&gt;-&gt;&lt;/span&gt;&lt;span class=&quot;token generic-function&quot;&gt;&lt;span class=&quot;token function&quot;&gt;as&lt;/span&gt;&lt;span class=&quot;token generic class-name&quot;&gt;&lt;span class=&quot;token operator&quot;&gt;&amp;lt;&lt;/span&gt;OpIteratorOpen&lt;span class=&quot;token operator&quot;&gt;&gt;&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
    &lt;span class=&quot;token keyword&quot;&gt;auto&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;*&lt;/span&gt; tryFastFunction &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;&amp;amp;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;
        &lt;span class=&quot;token keyword&quot;&gt;switch&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;instruction&lt;span class=&quot;token operator&quot;&gt;-&gt;&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;width&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;
        &lt;span class=&quot;token keyword&quot;&gt;case&lt;/span&gt; Narrow&lt;span class=&quot;token operator&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;token keyword&quot;&gt;return&lt;/span&gt; iterator_open_try_fast_narrow&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
        &lt;span class=&quot;token keyword&quot;&gt;case&lt;/span&gt; Wide16&lt;span class=&quot;token operator&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;token keyword&quot;&gt;return&lt;/span&gt; iterator_open_try_fast_wide16&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
        &lt;span class=&quot;token keyword&quot;&gt;case&lt;/span&gt; Wide32&lt;span class=&quot;token operator&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;token keyword&quot;&gt;return&lt;/span&gt; iterator_open_try_fast_wide32&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
        &lt;span class=&quot;token keyword&quot;&gt;default&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;RELEASE_ASSERT_NOT_REACHED&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
        &lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;
    &lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;

    JITSlowPathCall &lt;span class=&quot;token function&quot;&gt;slowPathCall&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token keyword&quot;&gt;this&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; instruction&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; tryFastFunction&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
    slowPathCall&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;call&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
    Jump fastCase &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;branch32&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;NotEqual&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; GPRInfo&lt;span class=&quot;token double-colon punctuation&quot;&gt;::&lt;/span&gt;returnValueGPR2&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;TrustedImm32&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token generic-function&quot;&gt;&lt;span class=&quot;token function&quot;&gt;static_cast&lt;/span&gt;&lt;span class=&quot;token generic class-name&quot;&gt;&lt;span class=&quot;token operator&quot;&gt;&amp;lt;&lt;/span&gt;&lt;span class=&quot;token keyword&quot;&gt;uint32_t&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;&gt;&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;IterationMode&lt;span class=&quot;token double-colon punctuation&quot;&gt;::&lt;/span&gt;Generic&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;It is not hard to see some resemblance between this code and the one shown for LLInt&#39;s &lt;code&gt;op_iterator_open&lt;/code&gt;. Once again we see if we can apply the fast case of this iterator and if not, we call the slow path.&lt;/p&gt;
&lt;pre class=&quot;language-cpp&quot;&gt;&lt;code class=&quot;language-cpp&quot;&gt;    &lt;span class=&quot;token generic-function&quot;&gt;&lt;span class=&quot;token function&quot;&gt;compileOpCall&lt;/span&gt;&lt;span class=&quot;token generic class-name&quot;&gt;&lt;span class=&quot;token operator&quot;&gt;&amp;lt;&lt;/span&gt;OpIteratorOpen&lt;span class=&quot;token operator&quot;&gt;&gt;&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;instruction&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; m_callLinkInfoIndex&lt;span class=&quot;token operator&quot;&gt;++&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;

    &lt;span class=&quot;token function&quot;&gt;advanceToNextCheckpoint&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
    
    &lt;span class=&quot;token comment&quot;&gt;// call result (iterator) is in regT1 (tag)/regT0 (payload)&lt;/span&gt;
    &lt;span class=&quot;token keyword&quot;&gt;const&lt;/span&gt; Identifier&lt;span class=&quot;token operator&quot;&gt;*&lt;/span&gt; ident &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;token operator&quot;&gt;&amp;amp;&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;vm&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;propertyNames&lt;span class=&quot;token operator&quot;&gt;-&gt;&lt;/span&gt;next&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
    
    &lt;span class=&quot;token function&quot;&gt;emitJumpSlowCaseIfNotJSCell&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;regT1&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The result of the call to the &lt;code&gt;Symbol.iterator&lt;/code&gt; method is again split, in 32bits machines, between a tag (in &lt;code&gt;regT1&lt;/code&gt;) and a payload (in &lt;code&gt;regT0&lt;/code&gt;). We jump to the slow case if the result is not a cell.&lt;/p&gt;
&lt;pre class=&quot;language-cpp&quot;&gt;&lt;code class=&quot;language-cpp&quot;&gt;    GPRReg tagIteratorGPR &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; regT1&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
    GPRReg payloadIteratorGPR &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; regT0&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;

    GPRReg tagNextGPR &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; tagIteratorGPR&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
    GPRReg payloadNextGPR &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; payloadIteratorGPR&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;

    JITGetByIdGenerator &lt;span class=&quot;token function&quot;&gt;gen&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;
        m_codeBlock&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt;
        &lt;span class=&quot;token function&quot;&gt;CodeOrigin&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;m_bytecodeIndex&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt;
        &lt;span class=&quot;token function&quot;&gt;CallSiteIndex&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;BytecodeIndex&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;m_bytecodeIndex&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;offset&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt;
        &lt;span class=&quot;token class-name&quot;&gt;RegisterSet&lt;/span&gt;&lt;span class=&quot;token double-colon punctuation&quot;&gt;::&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;stubUnavailableRegisters&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt;
        &lt;span class=&quot;token class-name&quot;&gt;CacheableIdentifier&lt;/span&gt;&lt;span class=&quot;token double-colon punctuation&quot;&gt;::&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;createFromImmortalIdentifier&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;ident&lt;span class=&quot;token operator&quot;&gt;-&gt;&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;impl&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt;
        &lt;span class=&quot;token function&quot;&gt;JSValueRegs&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;tagIteratorGPR&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; payloadIteratorGPR&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt;
        &lt;span class=&quot;token function&quot;&gt;JSValueRegs&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;tagNextGPR&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; payloadNextGPR&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt;
        AccessType&lt;span class=&quot;token double-colon punctuation&quot;&gt;::&lt;/span&gt;GetById&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;At this point we generate the access to the &lt;code&gt;next&lt;/code&gt; property of the iterator and ensure it is initialized.&lt;/p&gt;
&lt;pre class=&quot;language-cpp&quot;&gt;&lt;code class=&quot;language-cpp&quot;&gt;    gen&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;generateFastPath&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;*&lt;/span&gt;&lt;span class=&quot;token keyword&quot;&gt;this&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
    &lt;span class=&quot;token function&quot;&gt;addSlowCase&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;gen&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;slowPathJump&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
    m_getByIds&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;append&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;gen&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;

    &lt;span class=&quot;token function&quot;&gt;emitValueProfilingSite&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;bytecode&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;metadata&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;m_codeBlock&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
    &lt;span class=&quot;token function&quot;&gt;emitPutVirtualRegister&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;bytecode&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;m_next&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;JSValueRegs&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;tagNextGPR&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; payloadNextGPR&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;

    fastCase&lt;span class=&quot;token punctuation&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;token function&quot;&gt;link&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token keyword&quot;&gt;this&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
&lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The function finishes with some boilerplate code for linking and site profiling. However, most importantly it ensures that the values for the next bytecode are put into the correct registers in order for execution to continue. This function also shows a usage of another important kind of helper: branch instructions. Lets refocus just on the important pieces of &lt;code&gt;emit_op_iterator_open&lt;/code&gt; the branch helper usage.&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;void JIT::emit_op_iterator_open(const Instruction* instruction)
{
    // ...
    JITSlowPathCall slowPathCall(this, instruction, tryFastFunction);
    slowPathCall.call();
    Jump fastCase = branch32(NotEqual, GPRInfo::returnValueGPR2, TrustedImm32(static_cast&amp;lt;uint32_t&amp;gt;(IterationMode::Generic)));
    // ...
    fastCase.link(this);
}
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The code above will generate assembly code that branches to the end of &lt;code&gt;emit_op_iterator_open&lt;/code&gt; if &lt;code&gt;returnValueGPR2&lt;/code&gt; is not equal to &lt;code&gt;IterationMode::Generic&lt;/code&gt;. The branch jumps to where we are linking &lt;code&gt;fastCase&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;Another kind of very useful helpers are &lt;code&gt;JITSlowPathCall slowPathCall(this, instruction, tryFastFunction)&lt;/code&gt;, &lt;code&gt;callOperation&lt;/code&gt; and variants of these. That is because they abstract the ABI of each architecture to perform calls to cpp functions. On JSC we have a set of cpp functions that we call &lt;code&gt;JITOperations&lt;/code&gt; and are used by JIT when something is very complex to be done from assembly directly. For example, &lt;code&gt;emit_op_iterator_open&lt;/code&gt; uses these to call on &lt;code&gt;iterator_open_try_fast_narrow&lt;/code&gt;, &lt;code&gt;iterator_open_try_fast_wide16&lt;/code&gt;, or &lt;code&gt;iterator_open_try_fast_wide32&lt;/code&gt;. Also &lt;code&gt;emitSlow_op_iterator_open&lt;/code&gt; calls on &lt;code&gt;callOperation(operationThrowIteratorResultIsNotObject, TrustedImmPtr(m_codeBlock-&amp;gt;globalObject()));&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;When discussing &lt;code&gt;emit_op_iterator_open&lt;/code&gt; in baseline JIT, we saw above the usage of &lt;code&gt;JITGetByIdGenerator&lt;/code&gt; to generate assembly code to access the property &lt;code&gt;next&lt;/code&gt; from the iterator object. This class is a very useful as it encapsulates the Inline Cache (IC) mechanism for property access on JSC. It takes registers for base object where the &lt;code&gt;Get&lt;/code&gt; operation is going to be performed (&lt;code&gt;tagIteratorGPR&lt;/code&gt; and &lt;code&gt;payloadIteratorGPR&lt;/code&gt;) in addition to where the result of such access will be placed (&lt;code&gt;tagNextGPR&lt;/code&gt; and &lt;code&gt;payloadNextGPR&lt;/code&gt;). The code for fast path is generated by &lt;code&gt;gen.generateFastPath(*this);&lt;/code&gt; and since IC is only filled after its execution, the generation of fast path is a set of “noop” that eventually will  be repatched by fast code that accesses cached properties. We also configure &lt;code&gt;slowPathJump&lt;/code&gt; for this generator and this is important because when IC is empty, we always take the slow path to perform the &lt;code&gt;Get&lt;/code&gt; operation and configure the cache for it.&lt;/p&gt;
&lt;p&gt;The following code shows the &lt;code&gt;iterator_open&lt;/code&gt; slow path and where the &lt;code&gt;JITGetByIdGenerator&lt;/code&gt; is linked into when jumping to it.&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;void JIT::emitSlow_op_iterator_open(const Instruction* instruction, Vector&amp;lt;SlowCaseEntry&amp;gt;::iterator&amp;amp; iter)
{
    // …
    linkAllSlowCases(iter);

    GPRReg tagIteratorGPR = regT1;
    GPRReg payloadIteratorGPR = regT0;

    JumpList notObject;
    notObject.append(branchIfNotCell(tagIteratorGPR));
    notObject.append(branchIfNotObject(payloadIteratorGPR));

    auto bytecode = instruction-&amp;gt;as&amp;lt;OpIteratorOpen&amp;gt;();
    VirtualRegister nextVReg = bytecode.m_next;
    UniquedStringImpl* ident = vm().propertyNames-&amp;gt;next.impl();

    JITGetByIdGenerator&amp;amp; gen = m_getByIds[m_getByIdIndex++];
    
    Label coldPathBegin = label();

    Call call = callOperationWithProfile(
        bytecode.metadata(m_codeBlock),                  // metadata
        operationGetByIdOptimize,                        // operation
        nextVReg,                                        // result
        TrustedImmPtr(m_codeBlock-&amp;gt;globalObject()),      // arg1
        gen.stubInfo(),                                  // arg2
        JSValueRegs(tagIteratorGPR, payloadIteratorGPR), // arg3
        CacheableIdentifier::createFromImmortalIdentifier(ident).rawBits()); // arg4
    
    gen.reportSlowPathCall(coldPathBegin, call);
    auto done = jump();

    notObject.link(this);
    callOperation(operationThrowIteratorResultIsNotObject, TrustedImmPtr(m_codeBlock-&amp;gt;globalObject()));

    done.link(this);
    //…
}
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;During the execution the code above, we first perform a type check to guarantee that the iterator is an object - otherwise we need to throw a “Result is not object” exception. Assuming then that iterator is indeed an object, we perform a call to the JIT operation &lt;code&gt;operationGetByIdOptimize&lt;/code&gt;. This operation is responsible to check if the &lt;code&gt;Get&lt;/code&gt; we are performing is cacheable and then buffer those cases to repatch the fast path once we have enough evidence that this is a hot operation.&lt;/p&gt;
&lt;p&gt;The code for those generators lives on &lt;a href=&quot;https://github.com/WebKit/webkit/blob/master/Source/javascriptCore/jit/JITInlineCacheGenerator.h&quot;&gt;&lt;code&gt;JITInlineCacheGenerator.h&lt;/code&gt;&lt;/a&gt; and &lt;a href=&quot;https://github.com/WebKit/webkit/blob/master/Source/javascriptCore/jit/JITInlineCacheGenerator.cpp&quot;&gt;&lt;code&gt;JITInlineCacheGenerator.cpp&lt;/code&gt;&lt;/a&gt;. It includes generators for &lt;code&gt;op_in&lt;/code&gt;, &lt;code&gt;op_put_by_id&lt;/code&gt; and even for &lt;code&gt;op_get_by_val&lt;/code&gt;. These generators are also used by DFG and FTL code, mainly when there’s not enough type information for some opcodes during compile time.&lt;/p&gt;
&lt;p&gt;Lets look at &lt;code&gt;emit_op_iterator_next&lt;/code&gt; in order to understand &lt;code&gt;emitGetVirtualRegister&lt;/code&gt;.&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;void JIT::emit_op_iterator_next(const Instruction* instruction)
{
    //…
    JSValueRegs nextRegs(regT1, regT0);
    emitGetVirtualRegister(bytecode.m_next, nextRegs);
    //…
}
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This operation loads the &lt;code&gt;next&lt;/code&gt; operand into &lt;code&gt;regT1&lt;/code&gt; and &lt;code&gt;regT0&lt;/code&gt;. The call to &lt;code&gt;emitGetVirtualRegister&lt;/code&gt; generates assembly code that loads a &lt;code&gt;JSValue&lt;/code&gt; stored in a &lt;code&gt;VirtualRegister&lt;/code&gt; into the machine register(s) we pass as a parameter. For both LLInt and Baseline JIT, virtual registers are always assigned to stack slots, which means that &lt;code&gt;emitGetVirtualRegister&lt;/code&gt; will generate code that is going to load values from stack into machine registers.&lt;/p&gt;
&lt;p&gt;It is important to notice that &lt;code&gt;regT1&lt;/code&gt; and &lt;code&gt;regT0&lt;/code&gt; are aliases for registers of the architecture we are targetting. We also see the usage of &lt;code&gt;JSValueRegs&lt;/code&gt;, which is an abstraction created to reuse code among 64-bits and 32-bits code. It abstracts the fact that JSValues are 64-bits long, which means that we need 2 registers (tag and payload) to manage a JSValue on 32-bits code.
Following helpers used by &lt;code&gt;emit_op_iterator_open&lt;/code&gt;, we have &lt;code&gt;emitPutVirtualRegister&lt;/code&gt; that is very similar with &lt;code&gt;emitGetVirtualRegister&lt;/code&gt; but it stores a JSValue into a virtual register. A usage of this helper&#39;s counterpart &lt;code&gt;emitPutVirtualRegister&lt;/code&gt; can be seen in the code for &lt;code&gt;emit_op_iterator_open&lt;/code&gt;, which stores a &lt;code&gt;JSValue&lt;/code&gt; into a virtual register.&lt;/p&gt;
&lt;h2&gt;Conclusions&lt;/h2&gt;
&lt;p&gt;The important bits were the implementation of the intrinsics for LLInt and Baseline JIT. A few other code bits went into the patch but they are just ensuring everything works as expected instead of being large changes to the codebase.&lt;/p&gt;
&lt;p&gt;Although the implementation of the intrinsic is essentially the implementation of the same semantics, they require quite different knowledge since they operate at different levels of abstraction. On the one hand, LLInt is written in a DSL that compiles down to assembler and essentially implements a language interpreter. On the other, the Baseline JIT is written in cpp an implements a JIT Compiler. The API between both is completely different which requires the developer to know both in order to implement a single feature. The implementation of this feature for 64bits required further changes to other tiers of JSC, which we didn&#39;t need to touch for 32bits. In 32bits we do not have FTL enabled and DFG is essentially platform agnostic.&lt;/p&gt;
&lt;h2&gt;Thanks&lt;/h2&gt;
&lt;p&gt;My colleague &lt;a href=&quot;https://caiolima.github.io/&quot;&gt;Caio Lima&lt;/a&gt; has reviewed this post and spent a significant amount of time adding technical details where I initially had few. Any inaccuracies or lack of detail however, are still my own fault.&lt;/p&gt;
</content>
  </entry>
  <entry>
    <title>JSC - what are my options?</title>
    <link href="https://p.ocmatos.com/blog/jsc-what-are-my-options.html"/>
    <updated>2020-06-05T00:00:00Z</updated>
    <id>https://p.ocmatos.com/blog/jsc-what-are-my-options.html</id>
    <content xml:lang="en" type="html">&lt;p&gt;Compilers tend to be large pieces of software that provide an enormous amount of options. We take a quick look at how to find what JavaScriptCore (JSC) provides.&lt;/p&gt;
&lt;p&gt;If you grab a copy of the WebKit source, you can easily build &lt;code&gt;jsc&lt;/code&gt; with &lt;code&gt;Tools/Scripts/build-jsc --release --jsc-only&lt;/code&gt; (from the WebKit root directory). As usual, running the resulting binary will get you into a JavaScript REPL. For example (&lt;code&gt;$&lt;/code&gt; indicates the shell prompt):&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ WebKitBuild/Release/bin/jsc
&lt;span class=&quot;token operator&quot;&gt;&gt;&gt;&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;&gt;&lt;/span&gt; &lt;span class=&quot;token number&quot;&gt;2&lt;/span&gt;+2
&lt;span class=&quot;token number&quot;&gt;4&lt;/span&gt;
&lt;span class=&quot;token operator&quot;&gt;&gt;&gt;&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;&gt;&lt;/span&gt; &lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;That was simple enough, but which options does JSC accept? Using &lt;code&gt;--help&lt;/code&gt; will give you some information but if you&#39;re debuggin JSC, you might be interested in &lt;code&gt;--options&lt;/code&gt;.&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ WebKitBuild/Release/bin/jsc &lt;span class=&quot;token parameter variable&quot;&gt;--options&lt;/span&gt;
All JSC runtime options:
   &lt;span class=&quot;token assign-left variable&quot;&gt;useKernTCSM&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;true   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. Note: this needs to go before other options since they depend on this value.
   &lt;span class=&quot;token assign-left variable&quot;&gt;validateOptions&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. crashes &lt;span class=&quot;token keyword&quot;&gt;if&lt;/span&gt; mis-typed JSC options were passed to the VM
   &lt;span class=&quot;token assign-left variable&quot;&gt;dumpOptions&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;0&lt;/span&gt;   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. dumps JSC options &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;0&lt;/span&gt; &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; None, &lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt; &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; Overridden only, &lt;span class=&quot;token number&quot;&gt;2&lt;/span&gt; &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; All, &lt;span class=&quot;token number&quot;&gt;3&lt;/span&gt; &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; Verbose&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;

&lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;.&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;There is a huge amount of options and parameters to tune the execution of JSC, so here are some filtering hints.&lt;/p&gt;
&lt;p&gt;For verbosity related options:&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ WebKitBuild/Release/bin/jsc &lt;span class=&quot;token parameter variable&quot;&gt;--options&lt;/span&gt; &lt;span class=&quot;token operator&quot;&gt;&lt;span class=&quot;token file-descriptor important&quot;&gt;2&lt;/span&gt;&gt;&lt;/span&gt;&lt;span class=&quot;token file-descriptor important&quot;&gt;&amp;amp;1&lt;/span&gt; &lt;span class=&quot;token operator&quot;&gt;|&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;grep&lt;/span&gt; verbose
   &lt;span class=&quot;token assign-left variable&quot;&gt;verboseDFGBytecodeParsing&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false
   &lt;span class=&quot;token assign-left variable&quot;&gt;verboseCompilation&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false
   &lt;span class=&quot;token assign-left variable&quot;&gt;verboseFTLCompilation&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false
   &lt;span class=&quot;token assign-left variable&quot;&gt;verboseValidationFailure&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false
   &lt;span class=&quot;token assign-left variable&quot;&gt;verboseOSR&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false
   &lt;span class=&quot;token assign-left variable&quot;&gt;verboseDFGOSRExit&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false
   &lt;span class=&quot;token assign-left variable&quot;&gt;verboseFTLOSRExit&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false
   &lt;span class=&quot;token assign-left variable&quot;&gt;verboseCallLink&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false
   &lt;span class=&quot;token assign-left variable&quot;&gt;verboseCompilationQueue&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false
   &lt;span class=&quot;token assign-left variable&quot;&gt;verboseExitProfile&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false
   &lt;span class=&quot;token assign-left variable&quot;&gt;verboseCFA&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false
   &lt;span class=&quot;token assign-left variable&quot;&gt;verboseDFGFailure&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false
   &lt;span class=&quot;token assign-left variable&quot;&gt;verboseFTLToJSThunk&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false
   &lt;span class=&quot;token assign-left variable&quot;&gt;verboseFTLFailure&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false
   &lt;span class=&quot;token assign-left variable&quot;&gt;verboseSanitizeStack&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false
   &lt;span class=&quot;token assign-left variable&quot;&gt;verboseVisitRace&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false
   &lt;span class=&quot;token assign-left variable&quot;&gt;verboseExecutableAllocationFuzz&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;For JIT related options:&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ ~/dev/WebKit/WebKitBuild/Debug/bin/jsc &lt;span class=&quot;token parameter variable&quot;&gt;--options&lt;/span&gt; &lt;span class=&quot;token operator&quot;&gt;&lt;span class=&quot;token file-descriptor important&quot;&gt;2&lt;/span&gt;&gt;&lt;/span&gt;&lt;span class=&quot;token file-descriptor important&quot;&gt;&amp;amp;1&lt;/span&gt; &lt;span class=&quot;token operator&quot;&gt;|&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;grep&lt;/span&gt; JIT    
   &lt;span class=&quot;token assign-left variable&quot;&gt;useJIT&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;true   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. allows the executable pages to be allocated &lt;span class=&quot;token keyword&quot;&gt;for&lt;/span&gt; JIT and thunks &lt;span class=&quot;token keyword&quot;&gt;if&lt;/span&gt; &lt;span class=&quot;token boolean&quot;&gt;true&lt;/span&gt;
   &lt;span class=&quot;token assign-left variable&quot;&gt;useBaselineJIT&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;true   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. allows the baseline JIT to be used &lt;span class=&quot;token keyword&quot;&gt;if&lt;/span&gt; &lt;span class=&quot;token boolean&quot;&gt;true&lt;/span&gt;
   &lt;span class=&quot;token assign-left variable&quot;&gt;useDFGJIT&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;true   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. allows the DFG JIT to be used &lt;span class=&quot;token keyword&quot;&gt;if&lt;/span&gt; &lt;span class=&quot;token boolean&quot;&gt;true&lt;/span&gt;
   &lt;span class=&quot;token assign-left variable&quot;&gt;useRegExpJIT&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;true   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. allows the RegExp JIT to be used &lt;span class=&quot;token keyword&quot;&gt;if&lt;/span&gt; &lt;span class=&quot;token boolean&quot;&gt;true&lt;/span&gt;
   &lt;span class=&quot;token assign-left variable&quot;&gt;useDOMJIT&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;true   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. allows the DOMJIT to be used &lt;span class=&quot;token keyword&quot;&gt;if&lt;/span&gt; &lt;span class=&quot;token boolean&quot;&gt;true&lt;/span&gt;
   &lt;span class=&quot;token assign-left variable&quot;&gt;crashIfCantAllocateJITMemory&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false
   &lt;span class=&quot;token assign-left variable&quot;&gt;dumpDisassembly&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. dumps disassembly of all JIT compiled code upon compilation
   &lt;span class=&quot;token assign-left variable&quot;&gt;logJITCodeForPerf&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false
   &lt;span class=&quot;token assign-left variable&quot;&gt;bytecodeRangeToJITCompile&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;&amp;lt;&lt;/span&gt;null&lt;span class=&quot;token operator&quot;&gt;&gt;&lt;/span&gt;   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. bytecode size range to allow compilation on, e.g. &lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt;:100
   &lt;span class=&quot;token assign-left variable&quot;&gt;reportBaselineCompileTimes&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. dumps JS &lt;span class=&quot;token keyword&quot;&gt;function&lt;/span&gt; signature and the &lt;span class=&quot;token function&quot;&gt;time&lt;/span&gt; it took to BaselineJIT compile
   &lt;span class=&quot;token assign-left variable&quot;&gt;useFTLJIT&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;true   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. allows the FTL JIT to be used &lt;span class=&quot;token keyword&quot;&gt;if&lt;/span&gt; &lt;span class=&quot;token boolean&quot;&gt;true&lt;/span&gt;
   &lt;span class=&quot;token assign-left variable&quot;&gt;enableJITDebugAssertions&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;true
   &lt;span class=&quot;token assign-left variable&quot;&gt;useConcurrentJIT&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;true   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. allows the DFG / FTL compilation &lt;span class=&quot;token keyword&quot;&gt;in&lt;/span&gt; threads other than the executing JS thread
   &lt;span class=&quot;token assign-left variable&quot;&gt;jitPolicyScale&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt;   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. scale JIT thresholds to this specified ratio between &lt;span class=&quot;token number&quot;&gt;0.0&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;compile ASAP&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt; and &lt;span class=&quot;token number&quot;&gt;1.0&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;compile like normal&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;.
   &lt;span class=&quot;token assign-left variable&quot;&gt;thresholdForJITAfterWarmUp&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;500&lt;/span&gt;
   &lt;span class=&quot;token assign-left variable&quot;&gt;thresholdForJITSoon&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;100&lt;/span&gt;
   &lt;span class=&quot;token assign-left variable&quot;&gt;forceGCSlowPaths&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. If true, we will force all JIT fast allocations down their slow paths.
   &lt;span class=&quot;token assign-left variable&quot;&gt;alwaysGeneratePCToCodeOriginMap&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. This will &lt;span class=&quot;token function&quot;&gt;make&lt;/span&gt; sure we always generate a PCToCodeOriginMap &lt;span class=&quot;token keyword&quot;&gt;for&lt;/span&gt; JITed code.
   &lt;span class=&quot;token assign-left variable&quot;&gt;useBBQJIT&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;true   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. allows the BBQ JIT to be used &lt;span class=&quot;token keyword&quot;&gt;if&lt;/span&gt; &lt;span class=&quot;token boolean&quot;&gt;true&lt;/span&gt;
   &lt;span class=&quot;token assign-left variable&quot;&gt;useOMGJIT&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;true   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. allows the OMG JIT to be used &lt;span class=&quot;token keyword&quot;&gt;if&lt;/span&gt; &lt;span class=&quot;token boolean&quot;&gt;true&lt;/span&gt;
   &lt;span class=&quot;token assign-left variable&quot;&gt;traceBaselineJITExecution&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false
   &lt;span class=&quot;token assign-left variable&quot;&gt;dumpJITMemoryPath&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token string&quot;&gt;&quot;&quot;&lt;/span&gt;
   &lt;span class=&quot;token assign-left variable&quot;&gt;dumpJITMemoryFlushInterval&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;10&lt;/span&gt;   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. Maximum &lt;span class=&quot;token function&quot;&gt;time&lt;/span&gt; &lt;span class=&quot;token keyword&quot;&gt;in&lt;/span&gt; between flushes of the JIT memory dump &lt;span class=&quot;token keyword&quot;&gt;in&lt;/span&gt; seconds.&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;And lastly for GC related options:&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ ~/dev/WebKit/WebKitBuild/Debug/bin/jsc &lt;span class=&quot;token parameter variable&quot;&gt;--options&lt;/span&gt; &lt;span class=&quot;token operator&quot;&gt;&lt;span class=&quot;token file-descriptor important&quot;&gt;2&lt;/span&gt;&gt;&lt;/span&gt;&lt;span class=&quot;token file-descriptor important&quot;&gt;&amp;amp;1&lt;/span&gt; &lt;span class=&quot;token operator&quot;&gt;|&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;grep&lt;/span&gt; &lt;span class=&quot;token parameter variable&quot;&gt;-i&lt;/span&gt; gc
   &lt;span class=&quot;token assign-left variable&quot;&gt;repatchBufferingCountdown&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;8&lt;/span&gt;
   &lt;span class=&quot;token assign-left variable&quot;&gt;bytecodeRangeToDFGCompile&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;&amp;lt;&lt;/span&gt;null&lt;span class=&quot;token operator&quot;&gt;&gt;&lt;/span&gt;   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. bytecode size range to allow DFG compilation on, e.g. &lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt;:100
   &lt;span class=&quot;token assign-left variable&quot;&gt;logCompilationChanges&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false
   &lt;span class=&quot;token assign-left variable&quot;&gt;validateDoesGC&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;true
   &lt;span class=&quot;token assign-left variable&quot;&gt;reportDFGCompileTimes&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. dumps JS &lt;span class=&quot;token keyword&quot;&gt;function&lt;/span&gt; signature and the &lt;span class=&quot;token function&quot;&gt;time&lt;/span&gt; it took to DFG and FTL compile
   &lt;span class=&quot;token assign-left variable&quot;&gt;useGenerationalGC&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;true
   &lt;span class=&quot;token assign-left variable&quot;&gt;useConcurrentGC&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;true
   &lt;span class=&quot;token assign-left variable&quot;&gt;criticalGCMemoryThreshold&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;0.8&lt;/span&gt;   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. percent memory &lt;span class=&quot;token keyword&quot;&gt;in&lt;/span&gt; use the GC considers critical.  The collector is much &lt;span class=&quot;token function&quot;&gt;more&lt;/span&gt; aggressive above this threshold
   &lt;span class=&quot;token assign-left variable&quot;&gt;concurrentGCMaxHeadroom&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;1.5&lt;/span&gt;
   &lt;span class=&quot;token assign-left variable&quot;&gt;concurrentGCPeriodMS&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;2&lt;/span&gt;
   &lt;span class=&quot;token assign-left variable&quot;&gt;minimumGCPauseMS&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;0.3&lt;/span&gt;
   &lt;span class=&quot;token assign-left variable&quot;&gt;gcPauseScale&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;0.3&lt;/span&gt;
   &lt;span class=&quot;token assign-left variable&quot;&gt;gcIncrementBytes&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;10000&lt;/span&gt;
   &lt;span class=&quot;token assign-left variable&quot;&gt;gcIncrementMaxBytes&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;100000&lt;/span&gt;
   &lt;span class=&quot;token assign-left variable&quot;&gt;gcIncrementScale&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;0&lt;/span&gt;
   &lt;span class=&quot;token assign-left variable&quot;&gt;numberOfDFGCompilerThreads&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;2&lt;/span&gt;
   &lt;span class=&quot;token assign-left variable&quot;&gt;priorityDeltaOfDFGCompilerThreads&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;0&lt;/span&gt;
   &lt;span class=&quot;token assign-left variable&quot;&gt;maximumInliningCallerBytecodeCost&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;10000&lt;/span&gt;
   &lt;span class=&quot;token assign-left variable&quot;&gt;numberOfGCMarkers&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;8&lt;/span&gt;
   &lt;span class=&quot;token assign-left variable&quot;&gt;useParallelMarkingConstraintSolver&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;true
   &lt;span class=&quot;token assign-left variable&quot;&gt;slowPathAllocsBetweenGCs&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;0&lt;/span&gt;   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. force a GC on every Nth slow path alloc, where N is specified by this option
   &lt;span class=&quot;token assign-left variable&quot;&gt;sweepSynchronously&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. debugging option to sweep all dead objects synchronously at GC end before resuming mutator
   &lt;span class=&quot;token assign-left variable&quot;&gt;logGC&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;None   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. debugging option to log GC activity &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;0&lt;/span&gt; &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; None, &lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt; &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; Basic, &lt;span class=&quot;token number&quot;&gt;2&lt;/span&gt; &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; Verbose&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;
   &lt;span class=&quot;token assign-left variable&quot;&gt;useGC&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;true
   &lt;span class=&quot;token assign-left variable&quot;&gt;gcAtEnd&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. If true, the jsc CLI will &lt;span class=&quot;token keyword&quot;&gt;do&lt;/span&gt; a GC before exiting
   &lt;span class=&quot;token assign-left variable&quot;&gt;forceGCSlowPaths&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. If true, we will force all JIT fast allocations down their slow paths.
   &lt;span class=&quot;token assign-left variable&quot;&gt;forceDidDeferGCWork&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. If true, we will force all DeferGC destructions to perform a GC.
   &lt;span class=&quot;token assign-left variable&quot;&gt;gcMaxHeapSize&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;0&lt;/span&gt;
   &lt;span class=&quot;token assign-left variable&quot;&gt;recordGCPauseTimes&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false
   &lt;span class=&quot;token assign-left variable&quot;&gt;numberOfGCCyclesToRecordForVerification&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;3&lt;/span&gt;
   &lt;span class=&quot;token assign-left variable&quot;&gt;validateDFGClobberize&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;false   &lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. Emits extra validation code &lt;span class=&quot;token keyword&quot;&gt;in&lt;/span&gt; the DFG/FTL &lt;span class=&quot;token keyword&quot;&gt;for&lt;/span&gt; the Clobberize phase&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The options that are relevant to your use case depend on what you want to do, but my hope is that you have already found something useful here or that you now know how to find what you need. Good luck!&lt;/p&gt;
</content>
  </entry>
  <entry>
    <title>Announcing HTML examples (`html-examples`)</title>
    <link href="https://p.ocmatos.com/blog/announcing-html-examples-html-examples.html"/>
    <updated>2020-06-01T00:00:00Z</updated>
    <id>https://p.ocmatos.com/blog/announcing-html-examples-html-examples.html</id>
    <content xml:lang="en" type="html">&lt;p&gt;Announcing a new racket package &lt;a href=&quot;https://pkgs.racket-lang.org/package/html-examples&quot;&gt;&lt;code&gt;html-examples&lt;/code&gt;&lt;/a&gt; to convert an &lt;a href=&quot;https://docs.racket-lang.org/scribble/eval.html#(form._((lib._scribble%2Fexample..rkt)._examples))&quot;&gt;&lt;code&gt;examples&lt;/code&gt;&lt;/a&gt; block to HTML for inclusion in your own page.&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ raco pkg &lt;span class=&quot;token function&quot;&gt;install&lt;/span&gt; html-examples&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;a href=&quot;https://pkgs.racket-lang.org/package/html-examples&quot;&gt;&lt;code&gt;html-examples&lt;/code&gt;&lt;/a&gt; is a simple, one function, package to convert an &lt;a href=&quot;https://docs.racket-lang.org/scribble/eval.html&quot;&gt;&lt;code&gt;scribble/example&lt;/code&gt;&lt;/a&gt; block into HTML. I tried to find a pre-existing function to do this but all I could do was to get scribble to generate a whole page together with a bunch of assets for the page to render properly. Therefore I created this package which fishes out the correct &lt;code&gt;div&lt;/code&gt; from the page generated by &lt;code&gt;scribble&lt;/code&gt;, so you can typeset beautiful examples on your web page like:&lt;/p&gt;
&lt;p&gt;As usual, documentation can be &lt;a href=&quot;https://docs.racket-lang.org/html-examples/index.html&quot;&gt;found online&lt;/a&gt; and code can be found in the &lt;a href=&quot;https://github.com/pmatos/html-examples&quot;&gt;project&#39;s page on GitHub&lt;/a&gt; where I am happy to receive any bug reports and feature requests you might have.&lt;/p&gt;
</content>
  </entry>
  <entry>
    <title>Announcing Racket Bitvectors (`bv`)</title>
    <link href="https://p.ocmatos.com/blog/announcing-racket-bitvectors-bv.html"/>
    <updated>2020-05-31T00:00:00Z</updated>
    <id>https://p.ocmatos.com/blog/announcing-racket-bitvectors-bv.html</id>
    <content xml:lang="en" type="html">&lt;p&gt;Announcing a new racket package &lt;a href=&quot;https://pkgs.racket-lang.org/package/bv&quot;&gt;&lt;code&gt;bv&lt;/code&gt;&lt;/a&gt; for manipulating bitvectors with a &lt;code&gt;rosette&lt;/code&gt;-style API.&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ raco pkg &lt;span class=&quot;token function&quot;&gt;install&lt;/span&gt; bv&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Racket &lt;a href=&quot;https://pkgs.racket-lang.org/package/bv&quot;&gt;&lt;code&gt;bv&lt;/code&gt;&lt;/a&gt; (&lt;a href=&quot;https://github.com/pmatos/racket-bv&quot;&gt;source&lt;/a&gt; and &lt;a href=&quot;https://docs.racket-lang.org/bv/index.html&quot;&gt;documentation&lt;/a&gt;) is a package for bitvector manipulation. The interface follows the names and conventions defined by &lt;a href=&quot;https://pkgs.racket-lang.org/package/rosette&quot;&gt;&lt;code&gt;rosette&lt;/code&gt;&lt;/a&gt; and its long-term goal is to be a high-performance bitvector library.&lt;/p&gt;
&lt;p&gt;I am happy to receive any bug reports and feature requests you might have in the &lt;a href=&quot;https://github.com/pmatos/racket-bv/issues/new&quot;&gt;project&#39;s GitHub page&lt;/a&gt;.&lt;/p&gt;
&lt;h1&gt;Why?&lt;/h1&gt;
&lt;p&gt;This package release has been extremely overdue. It was first developed in 2017 and much improved through 2018 as part of &lt;a href=&quot;https://linki.tools/s10.html&quot;&gt;s10&lt;/a&gt;, it hasn&#39;t seen the light of open source until now. Partially because it was intertwined with a lot of other code and partially because I never bothered to sit down and write some documentation.&lt;/p&gt;
&lt;p&gt;I am slowly untangling &lt;a href=&quot;https://p.ocmatos.com/s10.html&quot;&gt;s10&lt;/a&gt; into libraries and documenting them and hope to release them as open source sooner or later. The reason the library has a &lt;code&gt;rosette&lt;/code&gt; API because I needed a fast bitvector library with the same API rosette uses due to constraints on how I engineered the equivalence solvers and search techniques within &lt;a href=&quot;https://p.ocmatos.com/s10.html&quot;&gt;s10&lt;/a&gt; however, I think it turned out to be a good decision in any case because the API is intuitive and it has great naming for functions operating on bitvectors.&lt;/p&gt;
</content>
  </entry>
  <entry>
    <title>CReduce - it&#39;s a kind of magic!</title>
    <link href="https://p.ocmatos.com/blog/creduce-its-a-kind-of-magic.html"/>
    <updated>2020-02-17T00:00:00Z</updated>
    <id>https://p.ocmatos.com/blog/creduce-its-a-kind-of-magic.html</id>
    <content xml:lang="en" type="html">&lt;p&gt;When I started compiler work &lt;a href=&quot;http://delta.tigris.org/&quot;&gt;delta&lt;/a&gt; was all the rage. It would turn a testcase with thousands of lines into a single small one with perhaps no more than 15 crashing your compiler in exactly the same way as the original testcase. I have used and abused this tool but it turns out it could be improved. John Regehr and his team did so with &lt;a href=&quot;https://embed.cs.utah.edu/creduce/&quot;&gt;C-Reduce&lt;/a&gt; (who by he way also developed &lt;a href=&quot;https://embed.cs.utah.edu/csmith/&quot;&gt;C-Smith&lt;/a&gt;, but that&#39;s another store entirely).&lt;/p&gt;
&lt;p&gt;C-Reduce takes things to another level squeezing that testcase ever farther until it&#39;s just a few bytes, and yet still crashing your compiler in the same interesting way it did before. In C and C++ this is a very important tool and its use is not &lt;a href=&quot;https://gcc.gnu.org/wiki/A_guide_to_testcase_reduction&quot;&gt;recommended&lt;/a&gt; often enough. It&#39;s very important and in languages like C and C++ because the preprocessor can a seemingly small example into a behemoth of a source file and therefore reducing the testcase appropriately is extremely important. The recommendation page for &lt;a href=&quot;https://gcc.gnu.org/bugs/&quot;&gt;reporting GCC bugs&lt;/a&gt; goes as far as saying:&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;However, providing a minimal testcase increases the chances of
getting your bug fixed. The only excuses to not send us the
preprocessed sources are (i) if you&#39;ve found a bug in the
preprocessor, (ii) if you&#39;ve reduced the testcase to a small file
that doesn&#39;t include any other file or (iii) if the bug appears only
when using precompiled headers.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;However, one advantage of &lt;code&gt;delta&lt;/code&gt; was that it worked on any text file as opposed to C-Reduce that due its reduction strategy required the source to be C or C++. That is, until I was told I was wrong and that C-Reduce would be able to eat a large &lt;a href=&quot;https://racket-lang.org/&quot;&gt;Racket&lt;/a&gt; and spit out a small one. This stayed in the back of my head until present day.&lt;/p&gt;
&lt;p&gt;Currently I am working on JavaScriptCore, which turns out to have bugs of its own as well. Due to the way these JIT compilers are designed you can go through several compiler tiers, code generations, on stack
replacements and back until you find your bug. How great would it be if you could launch your compiler and the bug would reveal itself straightaway without having to wait for a breakpoint to be hit 60 times, step through 20 machine instructions and two OSR calls?&lt;/p&gt;
&lt;p&gt;Well, that&#39;s what C-Reduce is here for. I remembered being told it worked with Racket and if it works with Racket it must work with JavaScript.&lt;/p&gt;
&lt;p&gt;I develop on a &lt;code&gt;x86_64&lt;/code&gt; but my crash is on &lt;code&gt;MIPS32el&lt;/code&gt; and I can reproduce it on &lt;code&gt;qemu-user&lt;/code&gt; like this:&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ /home/pmatos/roots/br-root.jsc32-mips/host/bin/qemu-mipsel &lt;span class=&quot;token parameter variable&quot;&gt;-L&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;&#92;&lt;/span&gt;
	/home/pmatos/roots/br-root.jsc32-mips/target ./WebKitBuild/Debug/bin/jsc &lt;span class=&quot;token punctuation&quot;&gt;&#92;&lt;/span&gt;
	load-varargs-then-inlined-call-and-exit.js 
qemu: uncaught target signal &lt;span class=&quot;token number&quot;&gt;4&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;Illegal instruction&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt; - core dumped&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The idea here is to reduce &lt;code&gt;load-varargs-then-inlined-call-and-exit.js&lt;/code&gt; but keep the &lt;code&gt;Illegal instruction&lt;/code&gt; error so I developed the simple reducing script:&lt;/p&gt;
&lt;pre class=&quot;language-bash&quot;&gt;&lt;code class=&quot;language-bash&quot;&gt;&lt;span class=&quot;token shebang important&quot;&gt;#!/bin/bash&lt;/span&gt;
/home/pmatos/roots/br-root.jsc32-mips/host/bin/qemu-mipsel &lt;span class=&quot;token parameter variable&quot;&gt;-L&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;&#92;&lt;/span&gt;
	/home/pmatos/roots/br-root.jsc32-mips/target /home/pmatos/dev/WebKit/WebKitBuild/Debug/bin/jsc &lt;span class=&quot;token punctuation&quot;&gt;&#92;&lt;/span&gt;
	load-varargs-then-inlined-call-and-exit.js &lt;span class=&quot;token operator&quot;&gt;&amp;amp;&gt;&lt;/span&gt; err.txt
&lt;span class=&quot;token function&quot;&gt;grep&lt;/span&gt; &lt;span class=&quot;token parameter variable&quot;&gt;-q&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&#39;Illegal instruction&#39;&lt;/span&gt; err.txt&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This simply redirects all output from &lt;code&gt;jsc&lt;/code&gt; into a file and checks that the file still contains the error &lt;code&gt;Illegal instruction&lt;/code&gt;. This file can be as complex as you want to constrain the error you wish to reproduce. For our present situation this is enough.&lt;/p&gt;
&lt;p&gt;Then, call C-Reduce:&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ creduce --not-c &lt;span class=&quot;token parameter variable&quot;&gt;--n&lt;/span&gt; &lt;span class=&quot;token number&quot;&gt;16&lt;/span&gt; ./red.sh load-varargs-then-inlined-call-and-exit.js
&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;&amp;lt;&lt;/span&gt; &lt;span class=&quot;token number&quot;&gt;26334&lt;/span&gt; &lt;span class=&quot;token operator&quot;&gt;&gt;=&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;
running &lt;span class=&quot;token number&quot;&gt;16&lt;/span&gt; interestingness tests &lt;span class=&quot;token keyword&quot;&gt;in&lt;/span&gt; parallel
&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;&amp;lt;&lt;/span&gt; pass_blank :: &lt;span class=&quot;token number&quot;&gt;0&lt;/span&gt; &lt;span class=&quot;token operator&quot;&gt;&gt;=&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;
&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;0.6&lt;/span&gt; %, &lt;span class=&quot;token number&quot;&gt;1046&lt;/span&gt; bytes&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;
&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;&amp;lt;&lt;/span&gt; pass_lines :: &lt;span class=&quot;token number&quot;&gt;0&lt;/span&gt; &lt;span class=&quot;token operator&quot;&gt;&gt;=&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;
&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;-0.3 %, &lt;span class=&quot;token number&quot;&gt;1055&lt;/span&gt; bytes&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;
&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;1.2&lt;/span&gt; %, &lt;span class=&quot;token number&quot;&gt;1039&lt;/span&gt; bytes&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;
&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;&amp;lt;&lt;/span&gt; pass_lines :: &lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt; &lt;span class=&quot;token operator&quot;&gt;&gt;=&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;
&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;-1.9 %, &lt;span class=&quot;token number&quot;&gt;1072&lt;/span&gt; bytes&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;
&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;17.6&lt;/span&gt; %, &lt;span class=&quot;token number&quot;&gt;867&lt;/span&gt; bytes&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;
&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;20.9&lt;/span&gt; %, &lt;span class=&quot;token number&quot;&gt;832&lt;/span&gt; bytes&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;
&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;26.5&lt;/span&gt; %, &lt;span class=&quot;token number&quot;&gt;773&lt;/span&gt; bytes&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;
&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;34.8&lt;/span&gt; %, &lt;span class=&quot;token number&quot;&gt;686&lt;/span&gt; bytes&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;
&lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;.&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;There are many arguments possible to creduce (which you can check with &lt;code&gt;creduce --help&lt;/code&gt;. Here I used:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;code&gt;--not-c&lt;/code&gt;: disable C/C++ specific passes, use for source languages that are, well, not C-like;&lt;/li&gt;
&lt;li&gt;&lt;code&gt;--n 16&lt;/code&gt;: use 16 cores to do the reduction, the more the merrier. If you interestingness script takes a long time to run, grab yourself a coffee;&lt;/li&gt;
&lt;li&gt;&lt;code&gt;./red.sh&lt;/code&gt;: path to the interestingness script (make sure it&#39;s executable);&lt;/li&gt;
&lt;li&gt;&lt;code&gt;load-varargs-then-inlined-call-and-exit.js&lt;/code&gt;: testcase to reduce (this is going to be modified in-place);&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;After a while you&#39;ll be greeted with:&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;token keyword&quot;&gt;done&lt;/span&gt; &lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;

pass statistics:
  method pass_balanced :: curly2 worked &lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt; and failed &lt;span class=&quot;token number&quot;&gt;18&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt;
  method pass_balanced :: curly-inside worked &lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt; and failed &lt;span class=&quot;token number&quot;&gt;12&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt;
  method pass_clex :: rm-toks-11 worked &lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt; and failed &lt;span class=&quot;token number&quot;&gt;105&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt;
  method pass_balanced :: parens-to-zero worked &lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt; and failed &lt;span class=&quot;token number&quot;&gt;25&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt;
  method pass_blank :: &lt;span class=&quot;token number&quot;&gt;0&lt;/span&gt; worked &lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt; and failed &lt;span class=&quot;token number&quot;&gt;0&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt;
  method pass_clex :: rename-toks worked &lt;span class=&quot;token number&quot;&gt;2&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt; and failed &lt;span class=&quot;token number&quot;&gt;4&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt;
  method pass_lines :: &lt;span class=&quot;token number&quot;&gt;8&lt;/span&gt; worked &lt;span class=&quot;token number&quot;&gt;2&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt; and failed &lt;span class=&quot;token number&quot;&gt;123&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt;
  method pass_lines :: &lt;span class=&quot;token number&quot;&gt;4&lt;/span&gt; worked &lt;span class=&quot;token number&quot;&gt;2&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt; and failed &lt;span class=&quot;token number&quot;&gt;123&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt;
  method pass_lines :: &lt;span class=&quot;token number&quot;&gt;10&lt;/span&gt; worked &lt;span class=&quot;token number&quot;&gt;2&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt; and failed &lt;span class=&quot;token number&quot;&gt;123&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt;
  method pass_lines :: &lt;span class=&quot;token number&quot;&gt;3&lt;/span&gt; worked &lt;span class=&quot;token number&quot;&gt;2&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt; and failed &lt;span class=&quot;token number&quot;&gt;123&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt;
  method pass_indent :: regular worked &lt;span class=&quot;token number&quot;&gt;2&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt; and failed &lt;span class=&quot;token number&quot;&gt;0&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt;
  method pass_lines :: &lt;span class=&quot;token number&quot;&gt;6&lt;/span&gt; worked &lt;span class=&quot;token number&quot;&gt;2&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt; and failed &lt;span class=&quot;token number&quot;&gt;123&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt;
  method pass_lines :: &lt;span class=&quot;token number&quot;&gt;2&lt;/span&gt; worked &lt;span class=&quot;token number&quot;&gt;5&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt; and failed &lt;span class=&quot;token number&quot;&gt;134&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt;
  method pass_clex :: rm-tok-pattern-4 worked &lt;span class=&quot;token number&quot;&gt;5&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt; and failed &lt;span class=&quot;token number&quot;&gt;784&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt;
  method pass_lines :: &lt;span class=&quot;token number&quot;&gt;0&lt;/span&gt; worked &lt;span class=&quot;token number&quot;&gt;5&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt; and failed &lt;span class=&quot;token number&quot;&gt;70&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt;
  method pass_balanced :: parens-inside worked &lt;span class=&quot;token number&quot;&gt;6&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt; and failed &lt;span class=&quot;token number&quot;&gt;7&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt;
  method pass_clex :: rm-toks-1 worked &lt;span class=&quot;token number&quot;&gt;7&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt; and failed &lt;span class=&quot;token number&quot;&gt;132&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt;
  method pass_lines :: &lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt; worked &lt;span class=&quot;token number&quot;&gt;7&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt; and failed &lt;span class=&quot;token number&quot;&gt;147&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt;
  method pass_clex :: rm-toks-2 worked &lt;span class=&quot;token number&quot;&gt;8&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt; and failed &lt;span class=&quot;token number&quot;&gt;116&lt;/span&gt; &lt;span class=&quot;token builtin class-name&quot;&gt;times&lt;/span&gt;

          ******** /home/pmatos/dev/WebKit/load-varargs-then-inlined-call-and-exit.js ********

&lt;span class=&quot;token keyword&quot;&gt;function&lt;/span&gt; b&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;a&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;
  &lt;span class=&quot;token builtin class-name&quot;&gt;return&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt; a: a + &lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;
&lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;
&lt;span class=&quot;token keyword&quot;&gt;function&lt;/span&gt; &lt;span class=&quot;token function-name function&quot;&gt;bar&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt; b.apply&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;this, array&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;
&lt;span class=&quot;token keyword&quot;&gt;function&lt;/span&gt; &lt;span class=&quot;token function-name function&quot;&gt;c&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;
  bar&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;
  c&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;
&lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;
array &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt; &lt;span class=&quot;token number&quot;&gt;2147483647&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt;
c&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Your test file now contains a much shorter testcase but will still crash your compiler in the same beautiful way.&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ /home/pmatos/roots/br-root.jsc32-mips/host/bin/qemu-mipsel &lt;span class=&quot;token parameter variable&quot;&gt;-L&lt;/span&gt; /home/pmatos/roots/br-root.jsc32-mips/target ./WebKitBuild/Debug/bin/jsc load-varargs-then-inlined-call-and-exit.js 
qemu: uncaught target signal &lt;span class=&quot;token number&quot;&gt;4&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;Illegal instruction&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt; - core dumped&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;For comparison the original file had 41 lines and 1052 bytes and the reduced file has 9 lines and 137 bytes which is an 87% reduction in size. But the big improvement here is not that the file is just smaller but that the number of paths taken by &lt;code&gt;jsc&lt;/code&gt; is much smaller taking me strange to the illegal instruction without JIT compiling functions I don&#39;t need or jumping through tiers I don&#39;t care.&lt;/p&gt;
&lt;p&gt;Of course, this would work just as well in a non cross-platform environment. I just used &lt;code&gt;qemu-user&lt;/code&gt; due to the nature of my bug. The important thing is to have a testcase, and a script to ensure that at each reduction step, you keep the interestingness of the testcase - in other words, a way to reproduce the failure.&lt;/p&gt;
&lt;p&gt;Here are a few examples I can think of as a script for reduction:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;code&gt;jsc&lt;/code&gt; crashes with a specific exit code so you create a script testing that specific exit code;&lt;/li&gt;
&lt;li&gt;&lt;code&gt;jsc&lt;/code&gt; crashes with a specific output (above was &lt;code&gt;Illegal instruction&lt;/code&gt;) but it could be anything else;&lt;/li&gt;
&lt;li&gt;&lt;code&gt;jsc&lt;/code&gt; does not crash but generates some strange JIT code which you noticed by using &lt;code&gt;--dumpDisassembly=true&lt;/code&gt;. Well, ensure in your interestingness script the exit code of &lt;code&gt;jsc&lt;/code&gt; is 0, redirect the output to a file and parse for that specific JIT code block;&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Most cases could probably be reduced (pun-intented) to the above three items. You reduce for a specific exit code, a specific error, or a specific block of text. Or a combination of these.&lt;/p&gt;
&lt;p&gt;In any case, if you implement your reducing script appropriately the result will be a testcase smaller than the original that passes your interestingness script - and C-Reduce will look like it is a kind of magic.&lt;/p&gt;
</content>
  </entry>
  <entry>
    <title>Cross-Arch Reproducibility using Containers</title>
    <link href="https://p.ocmatos.com/blog/cross-arch-reproducibility-using-containers.html"/>
    <updated>2020-01-16T00:00:00Z</updated>
    <id>https://p.ocmatos.com/blog/cross-arch-reproducibility-using-containers.html</id>
    <content xml:lang="en" type="html">&lt;h2&gt;Introduction&lt;/h2&gt;
&lt;p&gt;Part of my work for Igalia is on the 32bit support on MIPS (little endian) and ARM for JSC (JavaScriptCore - the JavaScript compiler in WebKit) and one of the problems we face is that of reproducibility of failures.&lt;/p&gt;
&lt;p&gt;We have boards to test these, namely Raspberry Pi 3 Model B+ boards, running a 32bits ARM kernel and Imagination CI20 boards running a mipsel kernel - all built with buildroot which provides images for these boards out-of-the-box. However, we don&#39;t work on these - most of us have higher performance x86_64 machines and whenever a failure occurs upstream it&#39;s generally time consuming to reproduce.&lt;/p&gt;
&lt;p&gt;I have therefore set out to create an environment where we can easily reproduce cross-architectural failures on JSC. I had worked on similar issues for &lt;a href=&quot;https://racket-lang.org/&quot;&gt;Racket&lt;/a&gt;, when building the cross-architectural Racket chroot environment on GitLab CI.&lt;/p&gt;
&lt;h2&gt;Starting with chroot&lt;/h2&gt;
&lt;p&gt;Let&#39;s start the discussion by talking about &lt;code&gt;chroot&lt;/code&gt;. In any linux system you&#39;ll find a &lt;code&gt;chroot&lt;/code&gt; binary.&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ &lt;span class=&quot;token function&quot;&gt;which&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;chroot&lt;/span&gt;
/usr/sbin/chroot
$ &lt;span class=&quot;token function&quot;&gt;chroot&lt;/span&gt; &lt;span class=&quot;token parameter variable&quot;&gt;--help&lt;/span&gt;
Usage: &lt;span class=&quot;token function&quot;&gt;chroot&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;OPTION&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt; NEWROOT &lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;COMMAND &lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;ARG&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;.&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt;
  or:  &lt;span class=&quot;token function&quot;&gt;chroot&lt;/span&gt; OPTION
Run COMMAND with root directory &lt;span class=&quot;token builtin class-name&quot;&gt;set&lt;/span&gt; to NEWROOT.

  &lt;span class=&quot;token parameter variable&quot;&gt;--groups&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;G_LIST        specify supplementary &lt;span class=&quot;token function&quot;&gt;groups&lt;/span&gt; as g1,g2,&lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;,gN
  &lt;span class=&quot;token parameter variable&quot;&gt;--userspec&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token environment constant&quot;&gt;USER&lt;/span&gt;:GROUP  specify user and group &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;ID or name&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt; to use
  --skip-chdir           &lt;span class=&quot;token keyword&quot;&gt;do&lt;/span&gt; not change working directory to &lt;span class=&quot;token string&quot;&gt;&#39;/&#39;&lt;/span&gt;
      &lt;span class=&quot;token parameter variable&quot;&gt;--help&lt;/span&gt;     display this &lt;span class=&quot;token builtin class-name&quot;&gt;help&lt;/span&gt; and &lt;span class=&quot;token builtin class-name&quot;&gt;exit&lt;/span&gt;
      &lt;span class=&quot;token parameter variable&quot;&gt;--version&lt;/span&gt;  output version information and &lt;span class=&quot;token builtin class-name&quot;&gt;exit&lt;/span&gt;

If no &lt;span class=&quot;token builtin class-name&quot;&gt;command&lt;/span&gt; is given, run &lt;span class=&quot;token string&quot;&gt;&#39;&quot;$SHELL&quot; -i&#39;&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;default: &lt;span class=&quot;token string&quot;&gt;&#39;/bin/sh -i&#39;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;.

GNU coreutils online help: &lt;span class=&quot;token operator&quot;&gt;&amp;lt;&lt;/span&gt;https://www.gnu.org/software/coreutils/&lt;span class=&quot;token operator&quot;&gt;&gt;&lt;/span&gt;
Full documentation at: &lt;span class=&quot;token operator&quot;&gt;&amp;lt;&lt;/span&gt;https://www.gnu.org/software/coreutils/chroot&lt;span class=&quot;token operator&quot;&gt;&gt;&lt;/span&gt;
or available locally via: info &lt;span class=&quot;token string&quot;&gt;&#39;(coreutils) chroot invocation&#39;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;However, at a slightly lower level, &lt;a href=&quot;http://man7.org/linux/man-pages/man2/chroot.2.html&quot;&gt;&lt;code&gt;chroot&lt;/code&gt;&lt;/a&gt; is a system call.&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ &lt;span class=&quot;token function&quot;&gt;man&lt;/span&gt; &lt;span class=&quot;token parameter variable&quot;&gt;-s2&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;chroot&lt;/span&gt;
CHROOT&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;2&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;

NAME
       &lt;span class=&quot;token function&quot;&gt;chroot&lt;/span&gt; - change root directory

SYNOPSIS
       &lt;span class=&quot;token comment&quot;&gt;#include &amp;lt;unistd.h&gt;&lt;/span&gt;

       int chroot&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;const char *path&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;

   Feature Test Macro Requirements &lt;span class=&quot;token keyword&quot;&gt;for&lt;/span&gt; glibc &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;see feature_test_macros&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;7&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;))&lt;/span&gt;:
&lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;.&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;As mentioned in the man page, it changes the root of the file system for a process to the path passed as argument. This new environment created for the process is known as a chroot jail, to which we will refer simply as a &lt;em&gt;jail&lt;/em&gt;.&lt;/p&gt;
&lt;p&gt;The jail allows us to trap a process inside a filesystem, i.e. the process cannot access anything outside the filesystem it is in. So, if it tried to access the root of the filesystem, inside the jail it will  only see the root of the new file system which is the path we passed on to &lt;code&gt;chroot&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;As an example throughout the post I will be using the factorial function. Whenever I refer to &lt;code&gt;factorial.c&lt;/code&gt;, I refer to a file that you might have to create as needed and consists of the following C source code.&lt;/p&gt;
&lt;pre class=&quot;language-c&quot;&gt;&lt;code class=&quot;language-c&quot;&gt;&lt;span class=&quot;token macro property&quot;&gt;&lt;span class=&quot;token directive-hash&quot;&gt;#&lt;/span&gt;&lt;span class=&quot;token directive keyword&quot;&gt;include&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&amp;lt;stdio.h&gt;&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;token macro property&quot;&gt;&lt;span class=&quot;token directive-hash&quot;&gt;#&lt;/span&gt;&lt;span class=&quot;token directive keyword&quot;&gt;include&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&amp;lt;stdint.h&gt;&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;token macro property&quot;&gt;&lt;span class=&quot;token directive-hash&quot;&gt;#&lt;/span&gt;&lt;span class=&quot;token directive keyword&quot;&gt;include&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&amp;lt;inttypes.h&gt;&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;token macro property&quot;&gt;&lt;span class=&quot;token directive-hash&quot;&gt;#&lt;/span&gt;&lt;span class=&quot;token directive keyword&quot;&gt;include&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&amp;lt;stdlib.h&gt;&lt;/span&gt;&lt;/span&gt;

&lt;span class=&quot;token keyword&quot;&gt;int&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;main&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token keyword&quot;&gt;int&lt;/span&gt; argc&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;token keyword&quot;&gt;char&lt;/span&gt; &lt;span class=&quot;token operator&quot;&gt;*&lt;/span&gt;argv&lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;{&lt;/span&gt;

  &lt;span class=&quot;token class-name&quot;&gt;uint64_t&lt;/span&gt; result &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
  &lt;span class=&quot;token class-name&quot;&gt;uint32_t&lt;/span&gt; arg&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;

  &lt;span class=&quot;token keyword&quot;&gt;if&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;argc &lt;span class=&quot;token operator&quot;&gt;!=&lt;/span&gt; &lt;span class=&quot;token number&quot;&gt;2&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;
    &lt;span class=&quot;token keyword&quot;&gt;return&lt;/span&gt; &lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;

  arg &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;strtoul&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;argv&lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;token constant&quot;&gt;NULL&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;token number&quot;&gt;0&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
  &lt;span class=&quot;token keyword&quot;&gt;while&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;arg&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt; result &lt;span class=&quot;token operator&quot;&gt;*=&lt;/span&gt; arg&lt;span class=&quot;token operator&quot;&gt;--&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;

  &lt;span class=&quot;token function&quot;&gt;printf&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;token string&quot;&gt;&quot;Result: %&quot;&lt;/span&gt; PRIu64 &lt;span class=&quot;token string&quot;&gt;&quot;&#92;n&quot;&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;,&lt;/span&gt; result&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
  &lt;span class=&quot;token keyword&quot;&gt;return&lt;/span&gt; &lt;span class=&quot;token number&quot;&gt;0&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;;&lt;/span&gt;
&lt;span class=&quot;token punctuation&quot;&gt;}&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Let&#39;s compile it and run an example.&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ gcc &lt;span class=&quot;token parameter variable&quot;&gt;-Wall&lt;/span&gt; &lt;span class=&quot;token parameter variable&quot;&gt;-Wextra&lt;/span&gt; &lt;span class=&quot;token parameter variable&quot;&gt;-o&lt;/span&gt; factorial factorial.c
$ ./factorial &lt;span class=&quot;token number&quot;&gt;20&lt;/span&gt;
Result: &lt;span class=&quot;token number&quot;&gt;2432902008176640000&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Let&#39;s create a jail for this binary using &lt;code&gt;chroot&lt;/code&gt; and run it.&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ &lt;span class=&quot;token function&quot;&gt;mkdir&lt;/span&gt; jail
$ &lt;span class=&quot;token function&quot;&gt;cp&lt;/span&gt; factorial jail/
$ &lt;span class=&quot;token function&quot;&gt;chroot&lt;/span&gt; jail/ /factorial
chroot: cannot change root directory to &lt;span class=&quot;token string&quot;&gt;&#39;jail/&#39;&lt;/span&gt;&lt;span class=&quot;token builtin class-name&quot;&gt;:&lt;/span&gt; Operation not permitted&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;First lesson here: only &lt;code&gt;root&lt;/code&gt; can &lt;code&gt;chroot&lt;/code&gt;. For security reasons, a normal user cannot &lt;code&gt;chroot&lt;/code&gt;. Being able to do so, would allow the user privilege escalation (&lt;a href=&quot;https://web.archive.org/web/20160127150916/http://www.bpfh.net/simes/computing/chroot-break.html&quot;&gt;further details&lt;/a&gt; on breaking out of the chroot jail and privilege escalation).&lt;/p&gt;
&lt;p&gt;So we &lt;code&gt;sudo&lt;/code&gt;:&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ &lt;span class=&quot;token function&quot;&gt;sudo&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;chroot&lt;/span&gt; jail/ /factorial
chroot: failed to run &lt;span class=&quot;token builtin class-name&quot;&gt;command&lt;/span&gt; ‘/factorial’: No such &lt;span class=&quot;token function&quot;&gt;file&lt;/span&gt; or directory&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Now this really starts to annoy you and you start wondering if the path to &lt;code&gt;factorial&lt;/code&gt; is correct. It is correct, the path is relative to the jail root. Once the root becomes &lt;code&gt;jail/&lt;/code&gt;, the &lt;code&gt;factorial&lt;/code&gt; binary is in the root of the filesystem so &lt;code&gt;/factorial&lt;/code&gt; is correct. The problem is subtle but will teach you an important lesson: dependencies. This is, after all, a dynamically linked executable 💡.&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ ldd factorial
        linux-vdso.so.1 &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;0x00007ffe1f7f9000&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;
        libc.so.6 &lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;&gt;&lt;/span&gt; /lib/x86_64-linux-gnu/libc.so.6 &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;0x00007f65bf62e000&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;
        /lib64/ld-linux-x86-64.so.2 &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;0x00007f65bf844000&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Those files exist in your file system, but not in the jail, so you get an error (although a pretty terrible error message at that). Let&#39;s try a static executable instead.&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ gcc &lt;span class=&quot;token parameter variable&quot;&gt;-Wall&lt;/span&gt; &lt;span class=&quot;token parameter variable&quot;&gt;-Wextra&lt;/span&gt; &lt;span class=&quot;token parameter variable&quot;&gt;-static&lt;/span&gt; &lt;span class=&quot;token parameter variable&quot;&gt;-o&lt;/span&gt; factorial factorial.c
$ ldd factorial
        not a dynamic executable
$ &lt;span class=&quot;token function&quot;&gt;sudo&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;chroot&lt;/span&gt; jail/ /factorial &lt;span class=&quot;token number&quot;&gt;20&lt;/span&gt;
Result: &lt;span class=&quot;token number&quot;&gt;2432902008176640000&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This is exactly what we wanted from the beginning - to show that the binary is now jailed in this root and cannot escape without explicitly trying to, something no benign binary is likely to do (see my comment above on escaping jail).&lt;/p&gt;
&lt;p&gt;In chroot jails, the filesystem root changes but it&#39;s still running on the same kernel. This allows us to create a new userspace inside this jail, separate from the host&#39;s, and possibly based on a different
linux distribution. For reproducibility, you can potentially tarball this jail and send it to someone else, who could themselves &lt;code&gt;chroot&lt;/code&gt; into it and reproduce a specific problem.&lt;/p&gt;
&lt;h2&gt;QEMU and binfmt&lt;/h2&gt;
&lt;p&gt;I will now introduce two other essential components to achieve our goal of cross architecture reproducibility: QEMU and binfmt.&lt;/p&gt;
&lt;p&gt;Up until now our jail has contained binaries compiled for our host architecture (in my case x86_64), however this need not be the case. &lt;a href=&quot;https://www.qemu.org/&quot;&gt;QEMU&lt;/a&gt; is an open-source hardware emulator and virtualizer. It implements two execution modes: system mode, and user mode;&lt;/p&gt;
&lt;p&gt;In system mode, QEMU works like &lt;a href=&quot;https://www.virtualbox.org/&quot;&gt;VirtualBox&lt;/a&gt; it emulates a whole system - from the applications, interrupts, and kernel, all the way to the hardware devices. The one I am interested in is user mode. In user mode, QEMU emulates a single binary leaving the rest of the system untouched.&lt;/p&gt;
&lt;p&gt;To demo how it works, lets use a cross-toolchain to compile a program to be run on a different architecture. Given I am on a &lt;code&gt;x86_64&lt;/code&gt;, I will use an &lt;code&gt;armhf&lt;/code&gt; toolchain. You can either download one provided from your distro or compile one with &lt;a href=&quot;https://crosstool-ng.github.io/&quot;&gt;crosstool-ng&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;For the sake of reproducibility, lets compile a pre-configured one with &lt;code&gt;crosstool-ng&lt;/code&gt;. Download it and install it in &lt;code&gt;$PATH&lt;/code&gt; - I used version 1.24.0. Then build the cross toolchain for &lt;code&gt;armv7-rpi2-linux-gnueabihf&lt;/code&gt;.&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ &lt;span class=&quot;token function&quot;&gt;mkdir&lt;/span&gt; build
$ &lt;span class=&quot;token builtin class-name&quot;&gt;cd&lt;/span&gt; build
build/ $ ct-ng armv7-rpi2-linux-gnueabihf
build/ $ ct-ng build&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Let&#39;s once again compile the &lt;code&gt;factorial.c&lt;/code&gt; example, but this time with our new toolchain. This toolchain will be in &lt;code&gt;$HOME/x-tools/armv7-rpi2-linux-gnueabihf&lt;/code&gt; by default.&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ &lt;span class=&quot;token builtin class-name&quot;&gt;export&lt;/span&gt; &lt;span class=&quot;token assign-left variable&quot;&gt;&lt;span class=&quot;token environment constant&quot;&gt;PATH&lt;/span&gt;&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token environment constant&quot;&gt;$HOME&lt;/span&gt;/x-tools/armv7-rpi2-linux-gnueabihf/bin:&lt;span class=&quot;token environment constant&quot;&gt;$PATH&lt;/span&gt;
$ armv7-rpi2-linux-gnueabihf-gcc &lt;span class=&quot;token parameter variable&quot;&gt;-static&lt;/span&gt; &lt;span class=&quot;token parameter variable&quot;&gt;-o&lt;/span&gt; factorial factorial.c
$ ./factorial
zsh: &lt;span class=&quot;token builtin class-name&quot;&gt;exec&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;format&lt;/span&gt; error: ./factorial
$ &lt;span class=&quot;token function&quot;&gt;file&lt;/span&gt; factorial
factorial: ELF &lt;span class=&quot;token number&quot;&gt;32&lt;/span&gt;-bit LSB executable, ARM, EABI5 version &lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;SYSV&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;, &lt;span class=&quot;token punctuation&quot;&gt;&#92;&lt;/span&gt;
dynamically linked, interpreter /lib/ld-linux-armhf.so.3, &lt;span class=&quot;token keyword&quot;&gt;for&lt;/span&gt; GNU/Linux &lt;span class=&quot;token punctuation&quot;&gt;&#92;&lt;/span&gt;
&lt;span class=&quot;token number&quot;&gt;4.19&lt;/span&gt;.21, with debug_info, not stripped&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;We were expecting this error, right? After all, we cannot just execute an arm binary in a x86_64 system. However, we can if we use QEMU in user mode which is what I want to show:&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ qemu-arm ./factorial &lt;span class=&quot;token number&quot;&gt;20&lt;/span&gt; 
Result: &lt;span class=&quot;token number&quot;&gt;2432902008176640000&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Now we have it working - however, there&#39;s a last tool we need to discuss and that&#39;s &lt;code&gt;binfmt&lt;/code&gt;. &lt;code&gt;binfmt_misc&lt;/code&gt; is a linux kernel capability that allows arbitrary executable file formats to be recognized and passed on to an interpreter. This means that we can transparently recognize arm (or any other architecture) executables and request them to be passed to QEMU transparently when we try to execute them.&lt;/p&gt;
&lt;p&gt;Before proceeding, if you wish to follow the examples, verify that your kernel has &lt;code&gt;binfmt&lt;/code&gt; enabled.&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ zcat /proc/config.gz&lt;span class=&quot;token operator&quot;&gt;|&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;grep&lt;/span&gt; BINFMT
&lt;span class=&quot;token assign-left variable&quot;&gt;CONFIG_BINFMT_ELF&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;y
&lt;span class=&quot;token assign-left variable&quot;&gt;CONFIG_COMPAT_BINFMT_ELF&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;y
&lt;span class=&quot;token assign-left variable&quot;&gt;CONFIG_BINFMT_SCRIPT&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;y
&lt;span class=&quot;token assign-left variable&quot;&gt;CONFIG_BINFMT_MISC&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;y&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;If you don&#39;t have &lt;code&gt;/proc/config.gz&lt;/code&gt; try instead &lt;code&gt;grep BINFMT /boot/config-$(uname -r)&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;Let&#39;s go back to our factorial on arm example, but this time install a &lt;code&gt;binfmt&lt;/code&gt; record to call &lt;code&gt;qemu-arm&lt;/code&gt; on the executable whenever we try to execute it directly. A &lt;code&gt;binfmt&lt;/code&gt; record looks like &lt;code&gt;:name:type:offset:magic:mask:interpreter:flags&lt;/code&gt; and needs to be installed by echoing the correct string to &lt;code&gt;/proc/sys/fs/binfmt_misc/register&lt;/code&gt;. For details on this consult the &lt;a href=&quot;https://www.kernel.org/doc/html/latest/admin-guide/binfmt-misc.html&quot;&gt;kernel
documentation&lt;/a&gt;. For &lt;code&gt;armv7&lt;/code&gt; we can register our interpreter and run our binary transparently like this:&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ &lt;span class=&quot;token function&quot;&gt;sudo&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;bash&lt;/span&gt; &lt;span class=&quot;token parameter variable&quot;&gt;-c&lt;/span&gt; &lt;span class=&quot;token string&quot;&gt;&#39;echo &quot;:qemu-arm:M:0:&#92;&#92;x7f&#92;&#92;x45&#92;&#92;x4c&#92;&#92;x46&#92;&#92;x01&#92;&#92;x01&#92;&#92;x01&#92;&#92;x00&#92;&#92;x00&#92;&#92;x00&#92;&#92;x00&#92;&#92;x00&#92;&#92;x00&#92;
&#92;&#92;x00&#92;&#92;x00&#92;&#92;x00&#92;&#92;x02&#92;&#92;x00&#92;&#92;x28&#92;&#92;x00:&#92;&#92;xff&#92;&#92;xff&#92;&#92;xff&#92;&#92;xff&#92;&#92;xff&#92;&#92;xff&#92;&#92;xff&#92;&#92;x00&#92;&#92;xff&#92;&#92;xff&#92;&#92;xff&#92;&#92;xff&#92;&#92;xff&#92;
&#92;&#92;xff&#92;&#92;xff&#92;&#92;xff&#92;&#92;xfe&#92;&#92;xff&#92;&#92;xff&#92;&#92;xff:/home/pmatos/installs/bin/qemu-arm:OCF&quot; &gt; &#92;
/proc/sys/fs/binfmt_misc/register&#39;&lt;/span&gt;
$ ./factorial &lt;span class=&quot;token number&quot;&gt;20&lt;/span&gt;
Result: &lt;span class=&quot;token number&quot;&gt;2432902008176640000&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The write to &lt;code&gt;/proc/sys/fs/binfmt_misc/register&lt;/code&gt; may fail if you already have a record for &lt;code&gt;qemu-arm&lt;/code&gt; setup. To remove the record run &lt;code&gt;echo -1 &amp;gt; /proc/sys/fs/binfmt_misc/qemu-arm&lt;/code&gt;. Note that the file, as shown above is an ARM binary and yet we transparently run it through QEMU thanks to the &lt;code&gt;binfmt&lt;/code&gt; magic we have initially set up.&lt;/p&gt;
&lt;p&gt;I should note, that while interesting to understand how this works behind the scenes, several people have created images to do the binfmt registration for you. One of those projects is &lt;a href=&quot;https://hub.docker.com/r/docker/binfmt/tags&quot;&gt;docker/binfmt&lt;/a&gt; which you can try to run using the latest tag: &lt;code&gt;docker run --rm --privileged docker/binfmt:66f9012c56a8316f9244ffd7622d7c21c1f6f28d&lt;/code&gt; (another example of a similar project is
&lt;a href=&quot;https://github.com/multiarch/qemu-user-static&quot;&gt;multiarch/qemu-user-static&lt;/a&gt;). If, for some reason, this does not work you should know enough by now to proceed with the registration manually.&lt;/p&gt;
&lt;h2&gt;Creating container base images&lt;/h2&gt;
&lt;p&gt;We have gone through creating chroot jails and transparently executing cross-architecture binaries with QEMU. A base image for a container is based pretty much on what we have just learned. To create a container
base image we will create a jail with a foreign root filesystem and we will use QEMU to execute binaries inside the jail. Once all is working, we import it into docker.&lt;/p&gt;
&lt;p&gt;To help us create a base system, we will use &lt;code&gt;debootstrap&lt;/code&gt; in two stages. We split this into two stages so we can setup QEMU in between.&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ &lt;span class=&quot;token function&quot;&gt;mkdir&lt;/span&gt; rootfs
$ &lt;span class=&quot;token function&quot;&gt;debootstrap&lt;/span&gt; &lt;span class=&quot;token parameter variable&quot;&gt;--foreign&lt;/span&gt; --no-check-gpg &lt;span class=&quot;token parameter variable&quot;&gt;--arch&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;arm buster ./rootfs http://httpredir.debian.org/debian/
$ &lt;span class=&quot;token function&quot;&gt;cp&lt;/span&gt; &lt;span class=&quot;token parameter variable&quot;&gt;-v&lt;/span&gt; /usr/bin/qemu-arm-static ./rootfs/usr/bin/
$ &lt;span class=&quot;token function&quot;&gt;chroot&lt;/span&gt; ./rootfs ./debootstrap/debootstrap --second-stage &lt;span class=&quot;token parameter variable&quot;&gt;--verbose&lt;/span&gt;
$ &lt;span class=&quot;token function&quot;&gt;mount&lt;/span&gt; &lt;span class=&quot;token parameter variable&quot;&gt;-t&lt;/span&gt; devpts devpts ./rootfs/dev/pts
$ &lt;span class=&quot;token function&quot;&gt;mount&lt;/span&gt; &lt;span class=&quot;token parameter variable&quot;&gt;-t&lt;/span&gt; proc proc ./rootfs/proc
$ &lt;span class=&quot;token function&quot;&gt;mount&lt;/span&gt; &lt;span class=&quot;token parameter variable&quot;&gt;-t&lt;/span&gt; sysfs sysfs ./rootfs/sys&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;At this point our system is a debian base system with root at &lt;code&gt;./rootfs&lt;/code&gt;. Note that the &lt;code&gt;qemu&lt;/code&gt; you need to copy into &lt;code&gt;./rootfs/usr/bin&lt;/code&gt; needs to be static in order to avoid dynamic loading issues inside the jail when it is invoked.&lt;/p&gt;
&lt;p&gt;Another important aspect to consider is that depending on your &lt;code&gt;binfmt&lt;/code&gt; setup, you need to put the binary in the proper place inside the jail. For the above to work, your &lt;code&gt;binfmt&lt;/code&gt; interpreter registration has to
point to an interpreter at &lt;code&gt;/usr/bin/qemu-arm-static&lt;/code&gt;, which is the absolute path as seen from inside the jail. Now we can install all dependencies in the system at will. All commands will be using &lt;code&gt;qemu&lt;/code&gt; transparently to execute if &lt;code&gt;binfmt&lt;/code&gt; setup worked correctly. Remember that in the case of chroot jails (and also of containers), the kernel in use is the kernel of your host. Therefore that is the only kernel that needs to be set up with &lt;code&gt;binfmt&lt;/code&gt;. You set it up outside your chroot jail and when the kernel needs to execute a foreign executable, it will look at the current &lt;code&gt;binfmt&lt;/code&gt; setup for the interpreter. However, the kernel cannot access a filesystem outside the jail, so that interpreter needs to exist inside the it.&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ &lt;span class=&quot;token function&quot;&gt;chroot&lt;/span&gt; ./rootfs &lt;span class=&quot;token function&quot;&gt;apt-get&lt;/span&gt; update
$ &lt;span class=&quot;token function&quot;&gt;chroot&lt;/span&gt; ./rootfs &lt;span class=&quot;token function&quot;&gt;apt-get&lt;/span&gt; &lt;span class=&quot;token parameter variable&quot;&gt;-y&lt;/span&gt; upgrade
$ &lt;span class=&quot;token function&quot;&gt;chroot&lt;/span&gt; ./rootfs &lt;span class=&quot;token function&quot;&gt;apt-get&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;install&lt;/span&gt; &lt;span class=&quot;token parameter variable&quot;&gt;-y&lt;/span&gt; g++ cmake libicu-dev &lt;span class=&quot;token function&quot;&gt;git&lt;/span&gt; ruby-highline ruby-json python
&lt;span class=&quot;token function&quot;&gt;chroot&lt;/span&gt; ./rootfs &lt;span class=&quot;token function&quot;&gt;apt-get&lt;/span&gt; &lt;span class=&quot;token parameter variable&quot;&gt;-y&lt;/span&gt; autoremove
&lt;span class=&quot;token function&quot;&gt;chroot&lt;/span&gt; ./rootfs &lt;span class=&quot;token function&quot;&gt;apt-get&lt;/span&gt; clean
&lt;span class=&quot;token function&quot;&gt;chroot&lt;/span&gt; ./rootfs &lt;span class=&quot;token function&quot;&gt;find&lt;/span&gt; /var/lib/apt/lists &lt;span class=&quot;token parameter variable&quot;&gt;-type&lt;/span&gt; f &lt;span class=&quot;token parameter variable&quot;&gt;-delete&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;These are the steps to install the dependencies to build and test JSC. Let&#39;s unmount the filesystems in order to create the docker base image.&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;&lt;span class=&quot;token function&quot;&gt;umount&lt;/span&gt; ./rootfs/dev/pts
&lt;span class=&quot;token function&quot;&gt;umount&lt;/span&gt; ./rootfs/proc
&lt;span class=&quot;token function&quot;&gt;umount&lt;/span&gt; ./rootfs/sys&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Let&#39;s &lt;code&gt;tar&lt;/code&gt; our jail and import it into a docker image.&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ &lt;span class=&quot;token function&quot;&gt;tar&lt;/span&gt; --numeric-owner &lt;span class=&quot;token parameter variable&quot;&gt;-cvf&lt;/span&gt; buster-arm.tar &lt;span class=&quot;token parameter variable&quot;&gt;-C&lt;/span&gt; ./rootfs &lt;span class=&quot;token builtin class-name&quot;&gt;.&lt;/span&gt;
$ &lt;span class=&quot;token function&quot;&gt;docker&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;import&lt;/span&gt; buster-arm.tar jsc-base:arm-raw&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The image &lt;code&gt;jsc-base:arm-raw&lt;/code&gt; now contains our raw system. To be able to add metadata or build on top of this image before releasing we can build a docker image that extends the raw image version.&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ &lt;span class=&quot;token function&quot;&gt;cat&lt;/span&gt; &lt;span class=&quot;token operator&quot;&gt;&amp;lt;&amp;lt;&lt;/span&gt;&lt;span class=&quot;token string&quot;&gt;EOF&lt;span class=&quot;token bash punctuation&quot;&gt; &lt;span class=&quot;token operator&quot;&gt;&gt;&lt;/span&gt; jsc32-base.Dockerfile&lt;/span&gt;
FROM jsc32-base:arm-raw

LABEL description=&quot;Minimal Debian image to reproduce JSC dev&quot;
LABEL maintainer=&quot;Paulo Matos &amp;lt;pmatos@igalia.com&gt;&quot;
 
CMD [&quot;/bin/bash&quot;]
EOF&lt;/span&gt;
$ &lt;span class=&quot;token function&quot;&gt;docker&lt;/span&gt; build &lt;span class=&quot;token parameter variable&quot;&gt;-t&lt;/span&gt; pmatos/jsc32-base:arm &lt;span class=&quot;token parameter variable&quot;&gt;-f&lt;/span&gt; jsc32-base.Dockerfile
$ &lt;span class=&quot;token function&quot;&gt;docker&lt;/span&gt; push pmatos/jsc32-base:arm&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Once we have pushed the image into the repository, we can run it from anywhere as long as &lt;code&gt;binfmt&lt;/code&gt; is setup properly.&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ &lt;span class=&quot;token function&quot;&gt;docker&lt;/span&gt; run pmatos/jsc32-base:arm &lt;span class=&quot;token function&quot;&gt;file&lt;/span&gt; /bin/bash
/bin/bash: ELF &lt;span class=&quot;token number&quot;&gt;32&lt;/span&gt;-bit LSB pie executable, ARM, EABI5 version &lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;SYSV&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;, dynamically linked, &lt;span class=&quot;token punctuation&quot;&gt;&#92;&lt;/span&gt;
interpreter /lib/ld-linux-armhf.so.3, &lt;span class=&quot;token keyword&quot;&gt;for&lt;/span&gt; GNU/Linux &lt;span class=&quot;token number&quot;&gt;3.2&lt;/span&gt;.0, &lt;span class=&quot;token punctuation&quot;&gt;&#92;&lt;/span&gt;
BuildID&lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;sha1&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;40e50d160a6c70d1a4e961200202cf853b4a2145, stripped&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Or to get an interactive shell inside the container, use &lt;code&gt;-it&lt;/code&gt;: &lt;code&gt;docker run -it pmatos/jsc32-base:arm /bin/bash&lt;/code&gt;.&lt;/p&gt;
&lt;h2&gt;Going rootless with podman&lt;/h2&gt;
&lt;p&gt;While &lt;a href=&quot;https://www.docker.com/&quot;&gt;&lt;code&gt;docker&lt;/code&gt;&lt;/a&gt; is a product of Docker Inc with various components and pricing tiers, &lt;a href=&quot;https://podman.io/&quot;&gt;&lt;code&gt;podman&lt;/code&gt;&lt;/a&gt; is a free and open source daemonless engine for developing, managing, and running containers on Linux. One of the main internal differences between &lt;code&gt;docker&lt;/code&gt; and &lt;code&gt;podman&lt;/code&gt; is that &lt;code&gt;podman&lt;/code&gt; uses &lt;a href=&quot;https://en.wikipedia.org/wiki/Cgroups&quot;&gt;cgroups&lt;/a&gt; v2, which &lt;code&gt;docker&lt;/code&gt; doesn&#39;t yet support. Since Fedora 31 ships with cgroups v2, for many users &lt;code&gt;podman&lt;/code&gt; is the only container engine available. Fortunately, &lt;code&gt;podman&lt;/code&gt; is largely &lt;code&gt;docker&lt;/code&gt; compatible meaning that you can &lt;code&gt;alias docker=podman&lt;/code&gt; without any issues.&lt;/p&gt;
&lt;p&gt;So you can do something like this.&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ &lt;span class=&quot;token function&quot;&gt;podman&lt;/span&gt; run pmatos/jsc32-base:arm &lt;span class=&quot;token function&quot;&gt;file&lt;/span&gt; /bin/bash
/bin/bash: ELF &lt;span class=&quot;token number&quot;&gt;32&lt;/span&gt;-bit LSB pie executable, ARM, EABI5 version &lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;SYSV&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;, dynamically linked, &lt;span class=&quot;token punctuation&quot;&gt;&#92;&lt;/span&gt;
interpreter /lib/ld-linux-armhf.so.3, &lt;span class=&quot;token keyword&quot;&gt;for&lt;/span&gt; GNU/Linux &lt;span class=&quot;token number&quot;&gt;3.2&lt;/span&gt;.0, &lt;span class=&quot;token punctuation&quot;&gt;&#92;&lt;/span&gt;
BuildID&lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;sha1&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;40e50d160a6c70d1a4e961200202cf853b4a2145, stripped&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;As long your &lt;code&gt;binfmt&lt;/code&gt; is properly setup, it will run as well as it did with docker.&lt;/p&gt;
&lt;h2&gt;Application to JSC&lt;/h2&gt;
&lt;p&gt;The initial code developed for reproducible JSC32 builds can be found in the &lt;a href=&quot;https://github.com/pmatos/WebKit-misc/&quot;&gt;WebKit-misc&lt;/a&gt; repository and was initially pushed under commit &lt;a href=&quot;https://github.com/pmatos/WebKit-misc/commit/94bf25ebe888ae1824113eeb269fe0c0fcb5f3d4&quot;&gt;94bf25e&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;The main script is in &lt;a href=&quot;https://github.com/pmatos/WebKit-misc/blob/94bf25ebe888ae1824113eeb269fe0c0fcb5f3d4/containers/reprojsc.sh&quot;&gt;&lt;code&gt;containers/reprojsc.sh&lt;/code&gt;&lt;/a&gt; and follows the plan laid out in this blog post.&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;An image is created using &lt;a href=&quot;https://github.com/pmatos/WebKit-misc/blob/94bf25ebe888ae1824113eeb269fe0c0fcb5f3d4/containers/jsc32-base/build-image.sh&quot;&gt;&lt;code&gt;containers/jsc32-base/build-image.sh&lt;/code&gt;&lt;/a&gt; and pushed into docker hub &lt;a href=&quot;https://hub.docker.com/r/pmatos/jsc32-base&quot;&gt;pmatos/jsc32-base&lt;/a&gt;. This only needs to be done when the image is changed.&lt;/li&gt;
&lt;li&gt;The &lt;a href=&quot;https://github.com/pmatos/WebKit-misc/blob/94bf25ebe888ae1824113eeb269fe0c0fcb5f3d4/containers/reprojsc.sh&quot;&gt;&lt;code&gt;containers/reprojsc.sh&lt;/code&gt;&lt;/a&gt; script is run each time one wants to trigger a build and/or test of JSC. This sets up QEMU for the desired architecture, starts the image and issues the necessary commands for the actions laid out in the command line.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;To run it yourself, checkout &lt;a href=&quot;https://github.com/pmatos/WebKit-misc/&quot;&gt;WebKit-misc&lt;/a&gt; and &lt;a href=&quot;https://webkit.org/&quot;&gt;WebKit&lt;/a&gt;, and run &lt;code&gt;reprojsc.sh&lt;/code&gt;.&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ &lt;span class=&quot;token function&quot;&gt;git&lt;/span&gt; clone &lt;span class=&quot;token parameter variable&quot;&gt;--depth&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt; git://git.webkit.org/WebKit.git
$ &lt;span class=&quot;token builtin class-name&quot;&gt;export&lt;/span&gt; &lt;span class=&quot;token assign-left variable&quot;&gt;WEBKIT_PATH&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token environment constant&quot;&gt;$PWD&lt;/span&gt;/WebKit
$ &lt;span class=&quot;token function&quot;&gt;git&lt;/span&gt; clone &lt;span class=&quot;token parameter variable&quot;&gt;--depth&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt; git@github.com:pmatos/WebKit-misc.git
$ &lt;span class=&quot;token builtin class-name&quot;&gt;cd&lt;/span&gt; WebKit-misc/containers
$ ./reprojsc.sh &lt;span class=&quot;token parameter variable&quot;&gt;-a&lt;/span&gt; arm &lt;span class=&quot;token parameter variable&quot;&gt;-b&lt;/span&gt; &lt;span class=&quot;token parameter variable&quot;&gt;-t&lt;/span&gt; &lt;span class=&quot;token parameter variable&quot;&gt;-i&lt;/span&gt; &lt;span class=&quot;token variable&quot;&gt;$WEBKIT_PATH&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This will build, test and send you into an interactive session so you can debug some possibly failed tests. Inside the container, you&#39;ll be in an ARM Cortex-A7 (32bits) environment. The other available architecture at the moment is MIPS which can be chosen with &lt;code&gt;-a mips&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;If you have any issues or requests please open an issue in &lt;a href=&quot;https://github.com/pmatos/WebKit-misc/issues&quot;&gt;GitHub&lt;/a&gt;.&lt;/p&gt;
&lt;h2&gt;What&#39;s missing?&lt;/h2&gt;
&lt;p&gt;There are many things that deserved discussion but I won&#39;t elaborate further. One of them is the difference between the container execution engine and the image builder. When using docker it&#39;s easy to think
that they are the same but it is not the case. For example, docker has a new image builder, called &lt;a href=&quot;https://docs.docker.com/buildx/working-with-buildx/&quot;&gt;&lt;code&gt;buildx&lt;/code&gt;&lt;/a&gt; whichhas in-built support for the creation of multi-architecture images. On the other hand, &lt;code&gt;podman&lt;/code&gt; is a container execution engine but an image creator is, for example, &lt;a href=&quot;https://buildah.io/&quot;&gt;&lt;code&gt;buildah&lt;/code&gt;&lt;/a&gt;, which doesn&#39;t have multi-architecture image support
&lt;a href=&quot;https://github.com/containers/buildah/issues/1590&quot;&gt;yet&lt;/a&gt;. Don&#39;t worry if you are confused - things are moving fast in this area and it&#39;s easy to lose track of all the tools out there.&lt;/p&gt;
&lt;p&gt;With the ability to transparently emulate other architectures QEMU user mode will see more usage patterns. Bugs in QEMU can look like application bugs and debugging is not straightforward, so bear in this
in mind when using this technology.&lt;/p&gt;
&lt;h2&gt;Acknowledgments&lt;/h2&gt;
&lt;p&gt;Thanks to my fellow Igalians Angelos Oikonomopoulos and Philip Chimento for providing suggestions and corrections for this blog post.&lt;/p&gt;
</content>
  </entry>
  <entry>
    <title>A Brief Look at the WebKit Workflow</title>
    <link href="https://p.ocmatos.com/blog/a-brief-look-at-the-webkit-workflow.html"/>
    <updated>2019-10-02T00:00:00Z</updated>
    <id>https://p.ocmatos.com/blog/a-brief-look-at-the-webkit-workflow.html</id>
    <content xml:lang="en" type="html">&lt;p&gt;Since I joined &lt;a href=&quot;https://www.igalia.com/&quot;&gt;Igalia&lt;/a&gt; back in May, I have been working on WebKit. After more than a decade working on GCC and LLVM, moving codebases, workflows, and compilation techniques and optimizations meant I was in very unfamiliar territory.&lt;/p&gt;
&lt;div style=&quot;text-align:center;&quot;&gt;
	&lt;img alt=&quot;Where am I? GIF&quot; class=&quot;post-photo&quot; src=&quot;https://p.ocmatos.com/img/where-am-i.gif&quot; /&gt;
&lt;/div&gt;
&lt;p&gt;One of the main things to understand is the contribution process and each project has their own rules and methodology, so lets start straight into how to contribute to WebKit. In a few places I will focus specifically on JSC -- for other parts of WebKit you might want to contact the &lt;a href=&quot;http://lists.webkit.org/mailman/listinfo/webkit-help&quot;&gt;mailing list&lt;/a&gt; or the &lt;a href=&quot;irc://irc.freenode.net/#webkit&quot;&gt;irc channel&lt;/a&gt; for specific advice.&lt;/p&gt;
&lt;p&gt;The information in this post should be used in addition to what is available in the official &lt;a href=&quot;https://webkit.org/getting-started/&quot;&gt;Getting Started&lt;/a&gt; and &lt;a href=&quot;https://webkit.org/contributing-code/&quot;&gt;Contributing Code&lt;/a&gt; documents.&lt;/p&gt;
&lt;h2&gt;Cloning&lt;/h2&gt;
&lt;p&gt;WebKit uses &lt;code&gt;svn&lt;/code&gt; as the default for version control but it has a &lt;code&gt;git&lt;/code&gt; mirror, therefore given my preference for using &lt;code&gt;git&lt;/code&gt;, we&#39;ll use that in the following examples.&lt;/p&gt;
&lt;p&gt;To clone WebKit we type:&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ &lt;span class=&quot;token function&quot;&gt;git&lt;/span&gt; clone git://git.webkit.org/WebKit.git
&lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;.
$ &lt;span class=&quot;token function&quot;&gt;du&lt;/span&gt; &lt;span class=&quot;token parameter variable&quot;&gt;-hs&lt;/span&gt; WebKit
&lt;span class=&quot;token number&quot;&gt;9&lt;/span&gt;.7G    WebKit&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This will take almost 10Gb of space at time of writing so if you are in a rush or short of space, use the &lt;code&gt;--depth=1&lt;/code&gt; option to &lt;code&gt;git clone&lt;/code&gt;.&lt;/p&gt;
&lt;h2&gt;Building and Testing&lt;/h2&gt;
&lt;p&gt;The most important directory for workflow in WebKit is &lt;code&gt;Tools/Scripts&lt;/code&gt; &lt;sup class=&quot;footnote-ref&quot;&gt;&lt;a href=&quot;https://p.ocmatos.com/blog/a-brief-look-at-the-webkit-workflow.html#fn1&quot; id=&quot;fnref1&quot;&gt;[1]&lt;/a&gt;&lt;/sup&gt;. Here you will find all the scripts we will use in this post.&lt;/p&gt;
&lt;p&gt;There are several scripts to build different parts of WebKit in different modes -- called Ports. For JSC you would use &lt;code&gt;build-jsc&lt;/code&gt; script:&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ Tools/Scripts/build-jsc --jsc-only &lt;span class=&quot;token parameter variable&quot;&gt;--debug&lt;/span&gt;
&lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;.
&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;
 JavaScriptCore is now built &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;02m:44s&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;. 
&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;span class=&quot;token operator&quot;&gt;==&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The &lt;code&gt;--jsc-only&lt;/code&gt; flag specifies the WebKit port to build and you probably should use &lt;code&gt;--debug&lt;/code&gt; while developing since it makes debugging much easier. Once you reach a point where you are happy with how it&#39;s going, run some tests. I should also mention that WebKit follows a very specific coding style and has a script to check for style problems. Don&#39;t worry too much about this in the beginning, until you have a patch. After a few contributions, you&#39;ll just get the hang of how to style your code. Make sure, either during development or before creating a patch that you check for style issues with &lt;code&gt;Tools/Scripts/check-webkit-style&lt;/code&gt;:&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ Tools/Scripts/check-webkit-style
ERROR: Source/WTF/wtf/StackBounds.cpp:25:  Alphabetical sorting problem.  &lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;build/include_order&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;4&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt;
ERROR: Source/WTF/wtf/StackBounds.cpp:126:  Place brace on its own line &lt;span class=&quot;token keyword&quot;&gt;for&lt;/span&gt; &lt;span class=&quot;token keyword&quot;&gt;function&lt;/span&gt; definitions.  &lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;whitespace/braces&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;4&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt;
Total errors found: &lt;span class=&quot;token number&quot;&gt;2&lt;/span&gt; &lt;span class=&quot;token keyword&quot;&gt;in&lt;/span&gt; &lt;span class=&quot;token number&quot;&gt;2&lt;/span&gt; files&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The errors are self-explanatory and easy to fix. Go ahead and make the necessary changes.&lt;/p&gt;
&lt;p&gt;Time to run some tests - There are several test scripts and you should probably run the tests most appropriate to the changes you have made. In general, you can list the testing scripts with:&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ &lt;span class=&quot;token function&quot;&gt;ls&lt;/span&gt; &lt;span class=&quot;token parameter variable&quot;&gt;-1&lt;/span&gt; Tools/Scripts/run-*-tests
Tools/Scripts/run-api-tests
Tools/Scripts/run-bindings-tests
Tools/Scripts/run-builtins-generator-tests
Tools/Scripts/run-dashboard-tests
Tools/Scripts/run-gtk-tests
Tools/Scripts/run-iexploder-tests
Tools/Scripts/run-inspector-generator-tests
Tools/Scripts/run-javascriptcore-tests
Tools/Scripts/run-jsc-stress-tests
Tools/Scripts/run-mangleme-tests
Tools/Scripts/run-perf-tests
Tools/Scripts/run-regexp-tests
Tools/Scripts/run-webdriver-tests
Tools/Scripts/run-webkit-tests
Tools/Scripts/run-web-platform-tests
Tools/Scripts/run-wpe-tests&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Scripts tend to have a very good &lt;code&gt;--help&lt;/code&gt; menu so you should give those a try before you run them. For tests in JSC use: &lt;code&gt;run-javascriptcore-tests&lt;/code&gt;.&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ Tools/Scripts/run-javascriptcore-tests --no-build --no-fail-fast &lt;span class=&quot;token parameter variable&quot;&gt;--debug&lt;/span&gt; --jsc-only
&lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;.
    &lt;span class=&quot;token number&quot;&gt;0&lt;/span&gt; failures found.&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;If you run the tests and see some possibly unrelated failures be sure to run them again without your changes since they might be failures that made it into upstream and not necessarily a result of your changes.&lt;/p&gt;
&lt;h2&gt;Submitting a patch&lt;/h2&gt;
&lt;p&gt;Once you have the patch ready for upstream, you can submit it automatically with &lt;code&gt;webkit-patch&lt;/code&gt;, which will automate much of the workflow process. Before proceeding though, make sure you have an account in the &lt;a href=&quot;https://bugs.webkit.org/&quot;&gt;WebKit bug database&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;The process can be summarised as follows:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;Create a patch upstream with &lt;code&gt;webkit-patch upload --no-review&lt;/code&gt; (the script will create a new bug to hold your patch)&lt;/li&gt;
&lt;/ol&gt;
&lt;ul&gt;
&lt;li&gt;This will prompt you to fill at least one &lt;code&gt;ChangeLog&lt;/code&gt; in your favourite editor. If in doubt of the style, run a &lt;code&gt;git log&lt;/code&gt; to see how other logs have been written and follow the same style&lt;/li&gt;
&lt;li&gt;Remember not to change the line with the bug title and URL, these should remain unchanged from what &lt;code&gt;webkit-patch&lt;/code&gt; generated&lt;/li&gt;
&lt;/ul&gt;
&lt;ol start=&quot;2&quot;&gt;
&lt;li&gt;If you are confident about it, set the flag &lt;code&gt;review?&lt;/code&gt; to request a review of your code&lt;/li&gt;
&lt;/ol&gt;
&lt;ul&gt;
&lt;li&gt;Wait for a review... if you end up waiting a week without reply feel free to ping someone on the &lt;a href=&quot;http://lists.webkit.org/mailman/listinfo/webkit-help&quot;&gt;mailing list&lt;/a&gt; or the &lt;a href=&quot;irc://irc.freenode.net/#webkit&quot;&gt;irc channel&lt;/a&gt; about it&lt;/li&gt;
&lt;li&gt;If you get a &lt;code&gt;review-&lt;/code&gt; then, read the comments provided and fix the issues mentioned in the comments. Once the issues have been fixed, upload the new patch with &lt;code&gt;webkit-patch upload&lt;/code&gt;. Referring to the bug number in the command line is optional because the script can read the bug number from the already written &lt;code&gt;ChangeLog&lt;/code&gt;. But to be explicit about it, you can specify the bug number with &lt;code&gt;webkit-patch upload &amp;lt;BUG&amp;gt;&lt;/code&gt;, where &lt;code&gt;&amp;lt;BUG&amp;gt;&lt;/code&gt; was the bug number created in step 1.&lt;/li&gt;
&lt;/ul&gt;
&lt;ol start=&quot;3&quot;&gt;
&lt;li&gt;After the review loop in point 2., you have a &lt;code&gt;review+&lt;/code&gt;. Until you have the correct permissions, you&#39;ll need someone to &lt;code&gt;commit-queue+&lt;/code&gt; your code. In order to request your code to be put into the commit queue, mark it as &lt;code&gt;commit-queue?&lt;/code&gt;. As soon as it is marked &lt;code&gt;commit-queue+&lt;/code&gt;, the bots will check your code, run the necessary tests, commit the patch upstream and close the bug.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;That&#39;s it. You got your contribution upstream -- Congratulations!&lt;/p&gt;
&lt;h2&gt;Further notes on the review/commit-queue cycle&lt;/h2&gt;
&lt;p&gt;In order not to make the above three-point loop more complicated, I am using this section to drill down on the review and commit-queue flags. As first time contributors, all of our patches need to be reviewed and put into the commit-queue. These flags are known as &lt;code&gt;review&lt;/code&gt; and &lt;code&gt;commit-queue&lt;/code&gt; or by the shorthands &lt;code&gt;r&lt;/code&gt; and &lt;code&gt;cq&lt;/code&gt;. To these you can append the symbols &lt;code&gt;-&lt;/code&gt;, &lt;code&gt;?&lt;/code&gt; or &lt;code&gt;+&lt;/code&gt;. We can only set &lt;code&gt;r?&lt;/code&gt; and &lt;code&gt;cq?&lt;/code&gt; and wait for someone with more permissions to change these accordingly.&lt;/p&gt;
&lt;p&gt;There are two ways to change these flags. In the web interface of your bug, you can click on the &amp;quot;Details&amp;quot; link for your patch and then change the flags accordingly.&lt;/p&gt;
&lt;div style=&quot;text-align:center;&quot;&gt;
	&lt;img alt=&quot;Patch Details&quot; class=&quot;post-photo&quot; src=&quot;https://p.ocmatos.com/img/2019-10-02-WebKitWorkflow-PatchDetails.png&quot; /&gt;
&lt;/div&gt;
&lt;p&gt;Once in the patch details view, you can see the patch flags.&lt;/p&gt;
&lt;div style=&quot;text-align:center;&quot;&gt;
	&lt;img alt=&quot;Patch Flags&quot; class=&quot;post-photo&quot; src=&quot;https://p.ocmatos.com/img/2019-10-02-WebKitWorkflow-Flags.png&quot; /&gt;
&lt;/div&gt;
&lt;p&gt;When you submit a bug with &lt;code&gt;webkit-patch upload&lt;/code&gt;, if confident,  you can set the &lt;code&gt;r?&lt;/code&gt; flag requesting a review by &lt;em&gt;not&lt;/em&gt; using &lt;code&gt;--no-review&lt;/code&gt; in the command. However, if you want to &lt;em&gt;in addition&lt;/em&gt; request the patch to be sent to the commit queue after review, you can add to the &lt;code&gt;upload&lt;/code&gt; command &lt;code&gt;--request-commit&lt;/code&gt;. This will add the flags &lt;code&gt;r?&lt;/code&gt; and &lt;code&gt;cq?&lt;/code&gt;. If you don&#39;t use &lt;code&gt;cq?&lt;/code&gt;, then you can manually set it once you get an &lt;code&gt;r+&lt;/code&gt;.&lt;/p&gt;
&lt;h2&gt;The review process&lt;/h2&gt;
&lt;p&gt;There are a couple of situations that can happen during the review process (point 2. above) that are worth mentioning.&lt;/p&gt;
&lt;h3&gt;Wrong bug title&lt;/h3&gt;
&lt;p&gt;Imagine that you got your patch reviewed and you realised that the bug title, which you created when you first submitted your patch was based on incorrect knowledge and you wish to change it? Also, this needs to be reflected in the patch.&lt;/p&gt;
&lt;p&gt;When you modify your patch locally and keep the ChangeLog unmodified, &lt;code&gt;webkit-patch upload --no-review&lt;/code&gt; will automatically use the &lt;code&gt;ChangeLog&lt;/code&gt; that you have locally. The best way to change the title of your bug is to do the following:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;Update the bug title in your bug page using the web interface, and;&lt;/li&gt;
&lt;li&gt;Execute &lt;code&gt;webkit-patch upload&lt;/code&gt; whenever you are ready to re-submit your fixed patch but add &lt;code&gt;--update-changelog&lt;/code&gt;. This will automatically grab the new title and update the &lt;code&gt;ChangeLog&lt;/code&gt; for you.&lt;/li&gt;
&lt;/ol&gt;
&lt;h3&gt;Apply patch locally&lt;/h3&gt;
&lt;p&gt;Imagine a collaborator improved your patch, uploaded it to the bug but it is still not quite right. You need for some reason to apply the patch locally to keep working on it. You can download it manually and apply it, however you can also automatically do this by checking the patch id (also known as attachment id) on the web interface and running on your WebKit checkout &lt;code&gt;webkit-patch apply-attachment --force-clean &amp;lt;ATTACH&amp;gt;&lt;/code&gt;, where &lt;code&gt;&amp;lt;ATTACH&amp;gt;&lt;/code&gt; is your patch id. Do note that &lt;code&gt;--force-clean&lt;/code&gt; will remove all your local changes and commit.&lt;/p&gt;
&lt;p&gt;There is a simpler alternative, where you don&#39;t need to look up the attachment id. If all you need is the latest patch uploaded to a bug, you can use &lt;code&gt;webkit-patch apply-from-bug &amp;lt;BUG&amp;gt;&lt;/code&gt;. As with &lt;code&gt;apply-attachment&lt;/code&gt;, the command &lt;code&gt;apply-from-bug&lt;/code&gt; also accepts &lt;code&gt;--force-clean&lt;/code&gt;&lt;/p&gt;
&lt;h2&gt;Command summary&lt;/h2&gt;
&lt;h3&gt;Help&lt;/h3&gt;
&lt;h4&gt;Global Help&lt;/h4&gt;
&lt;p&gt;&lt;code&gt;webkit-patch help --all-commands&lt;/code&gt;&lt;/p&gt;
&lt;h4&gt;Command Help&lt;/h4&gt;
&lt;p&gt;Take &lt;code&gt;upload&lt;/code&gt; as an example:&lt;/p&gt;
&lt;p&gt;&lt;code&gt;webkit-patch help upload&lt;/code&gt;&lt;/p&gt;
&lt;h3&gt;Upload patch&lt;/h3&gt;
&lt;p&gt;&lt;code&gt;webkit-patch upload&lt;/code&gt;&lt;/p&gt;
&lt;p&gt;This will upload your current local changes as a patch upstream. It will show you the patch it is going to upload before it actually does so it&#39;s safe to do to so what&#39;s included in the patch before the actual upload takes place.&lt;/p&gt;
&lt;p&gt;Relevant flags:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;code&gt;--no-review&lt;/code&gt;: by default, the &lt;code&gt;upload&lt;/code&gt; command will set that patch to &lt;code&gt;review?&lt;/code&gt; (commonly shortedned to &lt;code&gt;r?&lt;/code&gt;). Use &lt;code&gt;--no-review&lt;/code&gt; if you think the patch is not ready to review, or if you want to confirm the patch is not breaking any tests by using the upstream build workers.&lt;/li&gt;
&lt;li&gt;&lt;code&gt;--request-review&lt;/code&gt;: this will set the &lt;code&gt;cq?&lt;/code&gt; flag on your patch so that it is added to the commit queue after review.&lt;/li&gt;
&lt;li&gt;&lt;code&gt;--update-changelog&lt;/code&gt;: by default if a &lt;code&gt;ChangeLog&lt;/code&gt; already exists it will use that, otherwise it will ask you to write one by opening your favourite editor. Sometimes, however you might have written a &lt;code&gt;ChangeLog&lt;/code&gt; but you want to modify it. Use &lt;code&gt;--update-changelog&lt;/code&gt; to request that.&lt;/li&gt;
&lt;li&gt;&lt;code&gt;--comment=&amp;lt;COMMENT&amp;gt;&lt;/code&gt;: you can specify a comment to add to the post along with the patch upload. If you are uploading a new patch with changes requested by the reviewer, you should post a comment with &lt;code&gt;--comment=&amp;quot;Here is a new patch that addresses the commands in the review&amp;quot;&lt;/code&gt;.&lt;/li&gt;
&lt;/ul&gt;
&lt;h2&gt;Apply upstream attachment&lt;/h2&gt;
&lt;p&gt;&lt;code&gt;webkit-patch apply-attachment&lt;/code&gt; or &lt;code&gt;webkit-patch apply-from-bug&lt;/code&gt;&lt;/p&gt;
&lt;p&gt;This will apply an upstream patch locally. The command &lt;code&gt;apply-attachment&lt;/code&gt; requires the patch (attachment) identifier so you might have to look that up beforehand. If you only need the latest patch to a bug, you can use &lt;code&gt;apply-from-bug&lt;/code&gt; with the bug identifier.&lt;/p&gt;
&lt;p&gt;Relevant flags:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;code&gt;--force-clean&lt;/code&gt;: if you have changes locally the patch might not apply cleanly. Use this flag to clean everything locally before attempting to apply patch. You might want to use &lt;code&gt;git stash&lt;/code&gt; to stash your changes, if they are important.&lt;/li&gt;
&lt;/ul&gt;
&lt;h2&gt;Bonus -- WebKit on GitHub&lt;/h2&gt;
&lt;p&gt;I prefer to use GitHub to keep my project forks however, the WebKit repository is so large that you won&#39;t be able to import it straight into GitHub (either through the UI or manually). If you try, you&#39;ll get an error about packs being too large.&lt;/p&gt;
&lt;p&gt;The easiest way to get WebKit into GitHub is by forking the &lt;a href=&quot;https://github.com/webkit/webkit&quot;&gt;unofficial mirror here&lt;/a&gt;.&lt;/p&gt;
&lt;h2&gt;Thanks&lt;/h2&gt;
&lt;p&gt;Thanks to my Igalian colleagues Caio Lima and Guillaume Emont for reading a draft of this post and providing valuable suggestions.&lt;/p&gt;
&lt;h2&gt;Corrections or Comments?&lt;/h2&gt;
&lt;p&gt;Feel free to send any corrections or comments &lt;a href=&quot;mailto:pmatos@igalia.com&quot;&gt;straight into my inbox&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;Edits since publication (02.10.2019):&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;02.10.2019: my colleague Guillaume Emont found a typo in the &amp;quot;Upload patch&amp;quot; subsection, where I wrote &lt;code&gt;r+&lt;/code&gt; instead of &lt;code&gt;r?&lt;/code&gt;;&lt;/li&gt;
&lt;/ul&gt;
&lt;hr class=&quot;footnotes-sep&quot; /&gt;
&lt;section class=&quot;footnotes&quot;&gt;
&lt;ol class=&quot;footnotes-list&quot;&gt;
&lt;li id=&quot;fn1&quot; class=&quot;footnote-item&quot;&gt;&lt;p&gt;These scripts require Python2, therefore if you are using Python3 by default, you&#39;ll run into troubles running them. Try creating a virtual environment with &lt;code&gt;$ python2 -m virtualenv venv &amp;amp;&amp;amp; source venv/bin/activate&lt;/code&gt; at the root directory of WebKit. &lt;a href=&quot;https://p.ocmatos.com/blog/a-brief-look-at-the-webkit-workflow.html#fnref1&quot; class=&quot;footnote-backref&quot;&gt;↩︎&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;/section&gt;
</content>
  </entry>
  <entry>
    <title>S10 Overview</title>
    <link href="https://p.ocmatos.com/blog/s10-overview.html"/>
    <updated>2018-05-21T00:00:00Z</updated>
    <id>https://p.ocmatos.com/blog/s10-overview.html</id>
    <content xml:lang="en" type="html">&lt;h2&gt;Superoptimization&lt;/h2&gt;
&lt;p&gt;Superoptimization was introduced by H. Massalin in his paper &lt;em&gt;Superoptimizer: a look at the smallest program&lt;/em&gt;&lt;sup class=&quot;footnote-ref&quot;&gt;&lt;a href=&quot;https://p.ocmatos.com/blog/s10-overview.html#fn1&quot; id=&quot;fnref1&quot;&gt;[1]&lt;/a&gt;&lt;/sup&gt; back in 1987. The idea is that for every sequence of instructions for a given computer architecture and a metric, there is an infinite number of sequences which perform the same task (are semantically equivalent to the original sequence), but one of them is the best one with respect to the metric.&lt;/p&gt;
&lt;p&gt;Take for example the instruction sequence in RV32I:&lt;/p&gt;
&lt;pre class=&quot;language-nasm&quot;&gt;&lt;code class=&quot;language-nasm&quot;&gt;addi x1, zero, &lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt;
mv   x2, zero
add  x2, x2, x1
add  x2, x2, x1&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This sets the register &lt;code&gt;x2&lt;/code&gt; to 2. If we choose a metric that orders programs by increasing order of size, then the best possible program is:&lt;/p&gt;
&lt;pre class=&quot;language-nasm&quot;&gt;&lt;code class=&quot;language-nasm&quot;&gt;addi x2, zero, &lt;span class=&quot;token number&quot;&gt;2&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Now take another example in RV32IM (which includes the multiplication extension):&lt;/p&gt;
&lt;pre class=&quot;language-nasm&quot;&gt;&lt;code class=&quot;language-nasm&quot;&gt;mul x1, x1, &lt;span class=&quot;token number&quot;&gt;2&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This doubles the value in register &lt;code&gt;x1&lt;/code&gt;. Given the same metric, it is not possible to find a better program. However, if we change our metric to the number of clock cycles taken by the whole sequence of instructions and our micro-architecture takes 8 cycles for multiplications and 2 cycles for shifting then there&#39;s a better program:&lt;/p&gt;
&lt;pre class=&quot;language-nasm&quot;&gt;&lt;code class=&quot;language-nasm&quot;&gt;slli x1, x1, &lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Which shifts register &lt;code&gt;x1&lt;/code&gt; by 1 bit to the left, in essence multiplying its value by two and taking only 2 cycles whereas the original example took 8 (under our micro-architectural assumptions).&lt;/p&gt;
&lt;p&gt;For long sequence of instructions the state space that requires searching for the best sequence is enormous, which makes superoptimization a CPU intensive program. Research on superoptimization slowed down after the initial paper but has seen a resurgence in the last decade.&lt;/p&gt;
&lt;h2&gt;The compiler&lt;/h2&gt;
&lt;p&gt;Will superoptimizers replace your compiler? No!&lt;/p&gt;
&lt;p&gt;Production compilers are highly complex pieces of software that generate very good code, under very tight time constraints. On the other hand, superoptimizers are very slow since they need to search very large state spaces and therefore can take minutes, hours or even days to find the best possible instruction sequence.&lt;/p&gt;
&lt;p&gt;The placement of the superoptimizer in the development toolchain is up to the user but, in general, it will be placed after the compilation step picking up where the compiler left off and trying to optimize the code it generated.&lt;/p&gt;
&lt;h2&gt;S10&lt;/h2&gt;
&lt;p&gt;S10 was created out of the resurgence in superoptimization research mentioned earlier. &lt;a href=&quot;https://dblp.org/pid/127/3128&quot;&gt;Mangpo Phothilimthana&lt;/a&gt; from UC Berkeley published in 2016 a paper&lt;sup class=&quot;footnote-ref&quot;&gt;&lt;a href=&quot;https://p.ocmatos.com/blog/s10-overview.html#fn2&quot; id=&quot;fnref2&quot;&gt;[2]&lt;/a&gt;&lt;/sup&gt; and technical report on Greenthumb. The initial prototype of S10 was built upon Greenthumb and once we verified the potential we decided to turn it into a commercial product -- where at this point about 15% of code comes from Mangpo&#39;s work.&lt;/p&gt;
&lt;p&gt;Since most of the code was rewritten, including the interface between the backend architectures and the main modules, the architectures supported by Greenthumb are not supported by S10, namely &lt;em&gt;arm&lt;/em&gt;, &lt;em&gt;llvm ir&lt;/em&gt;, and &lt;em&gt;greenarrays&lt;/em&gt;. We have therefore chosen to implement support initially for &lt;a href=&quot;https://riscv.org/&quot;&gt;RISC-V&lt;/a&gt;, with other architectures on the roadmap.&lt;/p&gt;
&lt;p&gt;S10 is written completely in &lt;a href=&quot;https://racket-lang.org/&quot;&gt;Racket&lt;/a&gt; and implements three distinct algorithms that cooperate in a distributed setting. A symbolic&lt;sup class=&quot;footnote-ref&quot;&gt;&lt;a href=&quot;https://p.ocmatos.com/blog/s10-overview.html#fn2&quot; id=&quot;fnref2:1&quot;&gt;[2:1]&lt;/a&gt;&lt;/sup&gt;, stochastic&lt;sup class=&quot;footnote-ref&quot;&gt;&lt;a href=&quot;https://p.ocmatos.com/blog/s10-overview.html#fn3&quot; id=&quot;fnref3&quot;&gt;[3]&lt;/a&gt;&lt;/sup&gt; and lens&lt;sup class=&quot;footnote-ref&quot;&gt;&lt;a href=&quot;https://p.ocmatos.com/blog/s10-overview.html#fn4&quot; id=&quot;fnref4&quot;&gt;[4]&lt;/a&gt;&lt;/sup&gt; algorithms are distributed over several cores (or different machines) and communicate with each other through message passing finding incrementally better instruction sequences until it&#39;s proved that the best sequence has already been found.&lt;/p&gt;
&lt;p&gt;S10 includes a statistical profiler and debugging aids to ensure that any error is found quickly and any hotspots are first to be optimized. Due to the statistical nature of some methods it is also possible to automatically tune these parameters on an architecture independent way to ensure the best possible performance from the framework.&lt;/p&gt;
&lt;p&gt;Another area we focused on, given the long times taken to find solutions to the optimization problem was partial caching. S10 is supported by a database which uses previous found solutions to the superoptimization problem to simplify future searches.&lt;/p&gt;
&lt;h2&gt;Superoptimization in practice&lt;/h2&gt;
&lt;p&gt;In practice waiting for the perfect solution is not always possible or realistic, that is why S10 will incrementally provide incrementally better instruction sequences until it proves there is none better.&lt;/p&gt;
&lt;p&gt;An interaction with S10 for the first example I mentioned in this post might look like this:&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ riscv-s10 &lt;span class=&quot;token parameter variable&quot;&gt;-j&lt;/span&gt; &lt;span class=&quot;token number&quot;&gt;8&lt;/span&gt; &lt;span class=&quot;token parameter variable&quot;&gt;--timeout&lt;/span&gt; &lt;span class=&quot;token number&quot;&gt;60&lt;/span&gt; &lt;span class=&quot;token parameter variable&quot;&gt;--verbose&lt;/span&gt; &lt;span class=&quot;token parameter variable&quot;&gt;--progress&lt;/span&gt; add2.s
S10 Superoptimization Framework, by Linki Tools UG
https://linki.tools/s10

Booting up &lt;span class=&quot;token number&quot;&gt;8&lt;/span&gt; cores &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;stochastic &lt;span class=&quot;token number&quot;&gt;3&lt;/span&gt;, lens &lt;span class=&quot;token number&quot;&gt;4&lt;/span&gt;, symbolic &lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;
Input score: &lt;span class=&quot;token number&quot;&gt;16&lt;/span&gt;

* stochastic found optimization &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;score: &lt;span class=&quot;token number&quot;&gt;12&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;

  &lt;span class=&quot;token function&quot;&gt;mv&lt;/span&gt; x2, zero
  addi x2, x2, &lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt;
  addi x2, x2, &lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt;

* lens found optimization &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;score: &lt;span class=&quot;token number&quot;&gt;4&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;

  addi x2, zero, &lt;span class=&quot;token number&quot;&gt;2&lt;/span&gt;
  
* symbolic found best

  addi x2, zero, &lt;span class=&quot;token number&quot;&gt;2&lt;/span&gt;
  
Shutting down cores

Caching solution:
  addi x2, zero, &lt;span class=&quot;token number&quot;&gt;2&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The superoptimizer will incrementally give information (due to the &lt;code&gt;--progress&lt;/code&gt; flag) about optimizations found until we can prove there&#39;s no better solution, or a timeout is reached (which in this case was set to 60 seconds with &lt;code&gt;--timeout&lt;/code&gt;).&lt;/p&gt;
&lt;p&gt;The metric for the previous example was instruction sequence size, but had our metric been performance the situation would have been similar with one exception. The metric has to be fast to execute because it is executed potentially millions of times. If a cycle accurate simulation takes a long time, then it is better to implement a fast heuristic and request the best few solutions found and run an accurate metric on those.&lt;/p&gt;
&lt;pre class=&quot;language-shell&quot;&gt;&lt;code class=&quot;language-shell&quot;&gt;$ riscv-s10 &lt;span class=&quot;token parameter variable&quot;&gt;-j&lt;/span&gt; &lt;span class=&quot;token number&quot;&gt;8&lt;/span&gt; &lt;span class=&quot;token parameter variable&quot;&gt;--timeout&lt;/span&gt; &lt;span class=&quot;token number&quot;&gt;20&lt;/span&gt; &lt;span class=&quot;token parameter variable&quot;&gt;--verbose&lt;/span&gt; &lt;span class=&quot;token parameter variable&quot;&gt;--progress&lt;/span&gt; &lt;span class=&quot;token punctuation&quot;&gt;&#92;&lt;/span&gt;
  &lt;span class=&quot;token parameter variable&quot;&gt;--metric&lt;/span&gt; speed --accurate-metric-script sim.sh --run-metric-on &lt;span class=&quot;token number&quot;&gt;2&lt;/span&gt; add2.s
S10 Superoptimization Framework, by Linki Tools UG
https://linki.tools/s10

Booting up &lt;span class=&quot;token number&quot;&gt;8&lt;/span&gt; cores &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;stochastic &lt;span class=&quot;token number&quot;&gt;3&lt;/span&gt;, lens &lt;span class=&quot;token number&quot;&gt;4&lt;/span&gt;, symbolic &lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;
Input score: &lt;span class=&quot;token number&quot;&gt;16&lt;/span&gt;

* stochastic found optimization &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;score: &lt;span class=&quot;token number&quot;&gt;12&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;

  &lt;span class=&quot;token function&quot;&gt;mv&lt;/span&gt; x2, zero
  addi x2, x2, &lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt;
  addi x2, x2, &lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt;

* lens found optimization &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;score: &lt;span class=&quot;token number&quot;&gt;4&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;

  addi x2, zero, &lt;span class=&quot;token number&quot;&gt;2&lt;/span&gt;

Timeout reached.
Shutting down cores

Best two solutions:

&lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt; &lt;span class=&quot;token function&quot;&gt;mv&lt;/span&gt; x2, zero
    addi x2, x2, &lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt;
    addi x2, x2, &lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt;

&lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;2&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt; addi x2, zero, &lt;span class=&quot;token number&quot;&gt;2&lt;/span&gt;

Running sim.sh on &lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;1&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. Done
Running sim.sh on &lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;2&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;..&lt;/span&gt;. Done

Best solution &lt;span class=&quot;token punctuation&quot;&gt;(&lt;/span&gt;sim.sh&lt;span class=&quot;token punctuation&quot;&gt;)&lt;/span&gt;:

&lt;span class=&quot;token punctuation&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;token number&quot;&gt;2&lt;/span&gt;&lt;span class=&quot;token punctuation&quot;&gt;]&lt;/span&gt; addi x2, zero, &lt;span class=&quot;token number&quot;&gt;2&lt;/span&gt;

Caching solution:
  addi x2, zero, &lt;span class=&quot;token number&quot;&gt;2&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;S10 in this case ran against a tight timeout and at that point since it was given a script to calculate an accurate metric (with &lt;code&gt;--accurate-metric-script&lt;/code&gt;) it called it on the best two optimizations found at that point (due to &lt;code&gt;--run-metric-on&lt;/code&gt;). The metric scored solution &lt;code&gt;[2]&lt;/code&gt; as better and therefore that was the one which was returned and cached.&lt;/p&gt;
&lt;p&gt;This is a simple example of what S10 can do. I have not mentioned how to extend the algorithms and internal structures, how to extend S10 to a new architecture or autotune it for better performance. I hope to touch those points on future posts.&lt;/p&gt;
&lt;p&gt;Linki Tools is the first company to offer a commercial product for Superoptimization. Join us in superoptimizing your software! &lt;a href=&quot;mailto:p@ocmatos.com&quot;&gt;Contact me&lt;/a&gt; for more information.&lt;/p&gt;
&lt;hr class=&quot;footnotes-sep&quot; /&gt;
&lt;section class=&quot;footnotes&quot;&gt;
&lt;ol class=&quot;footnotes-list&quot;&gt;
&lt;li id=&quot;fn1&quot; class=&quot;footnote-item&quot;&gt;&lt;p&gt;Henry Massalin, Superoptimizer: a look at the smallest program, 1987 &lt;a href=&quot;https://p.ocmatos.com/blog/s10-overview.html#fnref1&quot; class=&quot;footnote-backref&quot;&gt;↩︎&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li id=&quot;fn2&quot; class=&quot;footnote-item&quot;&gt;&lt;p&gt;Mangpo Phothilimthana, et al., GreenThumb: superoptimizer construction framework, 2016 &lt;a href=&quot;https://p.ocmatos.com/blog/s10-overview.html#fnref2&quot; class=&quot;footnote-backref&quot;&gt;↩︎&lt;/a&gt; &lt;a href=&quot;https://p.ocmatos.com/blog/s10-overview.html#fnref2:1&quot; class=&quot;footnote-backref&quot;&gt;↩︎&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li id=&quot;fn3&quot; class=&quot;footnote-item&quot;&gt;&lt;p&gt;Alex Aiken, et al., Stochastic superoptimization, 2013 &lt;a href=&quot;https://p.ocmatos.com/blog/s10-overview.html#fnref3&quot; class=&quot;footnote-backref&quot;&gt;↩︎&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li id=&quot;fn4&quot; class=&quot;footnote-item&quot;&gt;&lt;p&gt;Mangpo Phothilimthana, et al., Scaling up Superoptimization, 2016 &lt;a href=&quot;https://p.ocmatos.com/blog/s10-overview.html#fnref4&quot; class=&quot;footnote-backref&quot;&gt;↩︎&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;/section&gt;
</content>
  </entry>
</feed>