Posts on sbgrl.me

Documentation performance, humans and LLMs

sylvain.bougerel@gmail.com (Sylvain Bougerel) — Thu, 20 Aug 2026 16:31:00 +0800

All coding harnesses I’ve used (Claude Code, OpenCode or Pi) give you a way to read your current session’s token spend.

Session compacted 13 times

 Session Info

 File: /home/me/.pi/agent/sessions/path/file.jsonl
 ID: 01a013cf-f306-7b76-994a-028aefc7a999

 Messages
 Total: 593
 User: 4
 Assistant: 285
 Tools: 304 calls, 304 results

 Tokens
 Input: 12,105,934
 Cached: 11,131,164 (91.9%)
 Uncached: 974,770
 Output: 290,380
 Total: 12,396,314

 Cost
 Total: $0.000
 llama.cpp/Qwen3.8-27B-Q4_K_M Text-only: $0.000 (12M tokens)
 Tools/summaries: $0.000 (375k tokens)
 Cache Re-billed: 2,262 tokens, 1 miss

Pi’s /session statistics for a recent session.

Of all the capabilities that LLMs enable, I find this ability to quantify thinking remarkable and not really tapped into yet. You — as a human — have always been able to tell whether doing any thinking was hard or easy. Usually, we proxy this quantity with the amount of time spent thinking. However, many everyday cognitive activities don’t reduce themselves to simple problems, and it becomes hard to otherwise measure them. Reading and understanding documentation is one of these activities.

The documentation performance metric
#

In the age of LLMs, you can effectively measure how efficient your documentation is at answering a given set of questions, regardless of the question’s complexity; let’s call this documentation performance (I don’t know of any other term for this).

Take 2 (or more) versions of your documentation,
Define a set of questions that you want to optimize against,
Run your models to answer test questions,
Sum input and output tokens used to generate all answers,
Run your models to grade the answers.

To account for the difference in answers generated or variation in grades across repeated calls you can even take multiple samples. Dividing the sum of I/O tokens used to generate the answer by the grade of the answer gives you the documentation performance score. Lower is better:

Doc versions	I/O tokens for all answers	Answer grade	Final score
A	10,000	100%	10,000
B	5,000	100%	5,000
C	3,000	50%	6,000
D	1,000	10%	10,000

In the example above, version B is better than version A because the model gets equally good answers with half the effort. Version C isn’t necessarily better merely because it is short.

Running this pipeline may be costly today, but many software companies would likely agree that a well-written documentation is a competitive edge. I don’t know of any organization making use of this yet, but I would not be surprised if some already do or have attempted to do so.

Beware of Goodhart’s Law
#

Tuning for documentation performance could be a good thing as long as documentation is written primarily by and for humans. If you build a system where LLMs are both authors and reviewers with the goal of optimizing for documentation performance alone — disregarding any other metrics and without any human in the loop — the output will likely end up resembling this:

- [done] Path-scheme Feature references (COMMANDS.md *Feature Reference Resolution*): `file://` / relative-path entries resolve to a Feature folder already on disk (trusted, no network) — 205 tests green. Design: any reference that is neither a bare name nor a registry-qualified OCI reference is a path reference; a `file://` prefix is stripped and the remainder is used literally (no percent-decoding, v0.1); relative paths resolve against the workspace root, `.`/empty = the workspace root itself; backslashes normalize to `/`; the result is lexically normalized (a `..` escaping the root is kept); the folder must exist and hold a parseable `devcontainer-feature.json`, but the folder name need not match the manifest `id` (the path names the folder directly; the build uses the manifest `id`); plans as `FeaturePlan::Local` with a new `FeatureSource::Path`, no registry gate, no `--allow-remote-features`; manifest loading shared with the search path via `load_feature_manifest`. Review passed: spec table rows (`file://`, relative) match, trust/no-network verified by a non-interactive no-allowlist test, `..`-escape kept not dropped, id≠folder-name rule tested, empty last component on an OCI ref is a hard error; one doc nit fixed (`is_bare_feature_name` comment mislabeled `.` as `..`-containing)

A billboard note written by the agent during that session.

That output is a single line of my agents’ billboard and at 1300 characters, it’s one of the shortest. The agents use the billboard to communicate with each other. This is what documentation looks like when agents talk to agents. There is definitely structure, but a complete absence of formatting, description or justification. This is clearly information dense and would likely do great on documentation performance, but I don’t think it’ll be a read that any human would enjoy.

"In Australia, a Home Battery Boom Has Helped Cut Wholesale Power Prices in Half"

sylvain.bougerel@gmail.com (Sylvain Bougerel) — Mon, 17 Aug 2026 10:41:00 +0800

The subsidy program has been “the major factor in the wholesale price of energy falling in Australia by 47 percent in the last 12 months,” Bowen said. “Australia is one of the very, very few countries in the world which is reducing wholesale prices in the midst of a global energy crisis.”

Source: In Australia, a Home Battery Boom Has Helped Cut Wholesale Power Prices in Half; on Australia’s program to subsidize home batteries to use the excess power from home solar installations that previously went to waste.

I’m very excited about developments in PV and the improvements that this can brings to countries that make the necessary investment into this transition. Australia is arguably a leader in that domain and demonstrates time and again the viability of such approach. 47% reduction is no joke! Every government in Europe should pay attention.

Open-source takes on AI

sylvain.bougerel@gmail.com (Sylvain Bougerel) — Sat, 01 Aug 2026 23:15:00 +0800

In the last 2 weeks, it started to feel like the dust is settling around AI in the open-source community. Major open-source projects released their policies or guidelines around AI usage. LLMs are having an existential impact on open-source: plundering its commons, putting a stress on its resources, threatening the trust that is the fabric of this community. In this post, I cover the spectrum of AI policies adopted across popular OSS projects.

Human Emacs
#

The GNU project has not yet shared a statement on AI but the GNU Compiler Collection and the GUIX project did. Similarly, several members of the Emacs project decided to take a stance on AI and published it on Human Emacs:

Therefore we are here to declare our intent to use and develop a version of Emacs that is free from LLM-generated contributions regardless of whatever policy GNU decides upon.

Human Emacs stands firmly against any AI usage. It is also the only document on that list that included moral and ethical ground in the policy itself. It garnered a fair bit of interest (and lots of praise) on r/emacs and Lobster. This one hit close to my heart. Emacs has been gaining in popularity since Covid, its community has been thriving since. I would hate for the momentum to die because of AI and I hope the community leaders will arrive at a consensus soon.

Codeberg
#

The maintainers of Codeberg have put up a blog post explaining the changes in their terms of use:

You must not share projects that mostly consist of code written by “generative AI”-tools (including services such as Claude, OpenAI Codex). Such projects having an unclear copyright status […] and furthermore have little safeguards to ensure that they do not include harmful code […].

After reading their blog post, I expected Codeberg to ban any AI usage, but the updated terms of use remain accepting of LLMs so long as LLMs do not generate the code. The original blog post feels lot more combative against AI and is worth reading because it covers the impact of LLM on the FLOSS community in more detail than the other entries on this list. The maintainers also pledge that Codeberg won’t allow LLMs to train on their hosted data.

GNU Compiler Collection
#

The GCC project also recently published their AI policy. It’s a clear policy that declines only legally-significant contributions except for test cases:

For the time being, the GNU Compiler Collection (GCC) policy is to decline any legally significant contributions which include LLM-generated content or are derived from LLM-generated content. […] As an exception, the GCC maintainers are free to accept legally significant contributions to test cases, generated in whole or in part by an LLM.

This policy is an acknowledgement that the project is benefiting from LLM use. It is worth noting their clarity on what constitutes a legally significant change:

A change of just a few lines (less than 15 or so) is not legally significant for copyright.

Their rather accepting stance on AI should not be surprising given that GCC’s development is supported by companies selling LLM products. However, the project requires that commit messages include an “Assisted-by:” tag for any LLM-generated contributions.

Kubernetes
#

It’s by far the most AI-friendly project on that list. The 2 most important contributing organizations to the project also compete to produce frontier models. Kubernetes has an AI Conformance SIG.

Recently, I listened to the Kubernetes podcast, episode 269 on AI usage featuring Kat Cosgrove who is a K8s Steering Committee member and Tech Lead in 2 SIGs (And who obviously likes Evangelion). The whole episode was a fun listen. She mentioned that they just updated the project’s Pull Request Process with their AI guidance:

Using AI tools to help write your PR is acceptable, but as the author, you are responsible for understanding every change. If you used AI tools in preparing your PR, you must disclose this in the description of your PR. For example, including “This PR was written in part with the assistance of generative AI,” in the PR description is sufficient.

As expected, there are very few restrictions around AI usage, besides “understand every change” which is definitely the minimum bar. During the podcast, Kat did emphasize however that the project members really wanted to talk to humans, not AI: indeed, the guidelines ban LLM use for commits, PR descriptions and comments.

What caught my attention is that the guidelines also ban the use of “Assisted-by:” statements in the commit messages while enforcing that authors disclose LLM use — presumably, manually and without vendor reference, though it is not explicitly stated. Kat commented on the podcast that AI vendors have used the commits of merged PRs in K8s to legitimize their tools. A neutral choice, for a project that is backed by the fiercest competitors in the AI landscape. I believe the concern is genuine though, and GCC could probably take a page from their book.

Linux
#

I think the projects above embody the full spectrum of reactions in the open-source community. But I couldn’t finish this post without the take from the most important person in the largest of all open-source projects (which you have likely already read):

Linux is not one of those anti-AI projects, and if somebody has issues with that, they can do the open-source thing and fork it.

Or just walk away.

AI is a tool, just like other tools we use. And it’s clearly a useful one.

What’s great about this take is that you instantly know it was not LLM-generated. Linus, please don’t ever change.

The Kernel does have a contributing policy with regards to AI coding assistants. I’m not going to requote it — it’s very similar to the GCC policy, which makes sense since both projects share so much in common.

Where do I stand?
#

My opinions on AI have not shifted much, but my usage varied a lot. It peaked around May 2026; that month, I spent well over US$1,200 in API-token pricing on Claude alone. That was not something sustainable — we don’t have so many credits — but it did produce useful results. Since then, I’ve been spending much less on LLMs and cut some of the lesser used subscriptions.

On the one hand, I can’t stand the terrible impact AI is having on the open-source community, the environment and our culture. On the other hand, I still want to experience this incredible technology. So, let’s be honest. I’m torn. And, I’m not alone.

I deal with my indecision by categorizing the issues in 2 parts: those that I think will resolve themselves, and those that I can act on now. For example: I think the impact on the open-source community will resolve itself — I believe the community will eventually build the tools and culture to thrive in the age of LLMs. However, I don’t think the pollution generated by the wave of newly built AI data centers will resolve itself. So, as a small step in the right direction, I aim to stop using cloud-based inference. I’ve started to use Pi with a distilled Qwen3.5-27B-Q4_K_M model on my modest Radeon RX 7900 XTX (24GB VRAM) and the results are not what I anticipated: it’s much slower, but significantly more capable than I expected. It is already accomplishing tasks that I thought only a full-fledged product like Claude Code could accomplish. I’ll be writing about it soon.

The end of tech-debt?

sylvain.bougerel@gmail.com (Sylvain Bougerel) — Sat, 16 May 2026 18:03:00 +0800

After Claude Code performed a refactor across our codebase in less than 24 hours and mostly autonomously, I started to wonder if we could end our tech-debt, and put a price on it?

Setting the stage
#

The codebase I’m referring to is that of our main Next.js application. It’s not a small codebase by any measure: 200KLOC, 1500 files, Typescript, Terraform and other languages.

Within this codebase, we use both API route handlers and server functions to fetch or mutate data. Each mechanism fits a specific purpose: an application should make its public API available via route handlers, while it should use server functions for internal mutations only.

In our application, however, we had leftover server functions used only to fetch data — without mutating it. This often resulted in the server function returning stale information, because it was doing something it was not designed for. We needed to migrate them to route handlers.

I had already fixed 4 of them, but I found 18 more that needed addressing.

This is not hard work, but not a dumb task either. Each case requires a new API route, a new contract, properly bounded inputs, validated outputs, updating how the client-side sends requests and handles responses, all while making sure it lints correctly and passes the ~2700 unit tests of our application — a perfect task for an agent.

Ralph to the rescue!
#

To make fixing the first 4 easier, I had already refined and tested a 150-line Claude skill. I knew it was time for a Ralph loop. So I set up the skill as the spec file, provided additional instructions around merge request (MR) submission and handling, and got it going in auto mode.

And it did. After 8 hours of work, it created 18 MRs for me to review, and continued to autonomously address the comments and fix any failures reported by the CI pipeline.

At the point where all 18 MRs were submitted and ready to merge, this task had cost roughly USD$180. I estimate that Claude went roughly 5-10 times faster than I would.

$180 is not a large amount of money for a specific, targeted refactor. We have dozens of those we can do to migrate and improve our codebase. Can we end tech-debt in our codebase?

Then reality caught up
#

After MR 6/18 merged into the main branch, our testing environment went down. It was not hard to notice: even a local test server would not run after rebasing. MR 6/18 contained a route that was not set up properly, and that could only be caught at runtime. It was an easy fix, and it reminded me that I had forgotten a step in my Ralph loop: I quickly instructed it to build comprehensive test cases for us to exhaustively verify the changes.

Testing is going to need improvement, but we knew this; we had already started to experiment with end-to-end tests to tackle this tech-debt. Here again, with great help from Claude, we managed to automate some end-to-end testing. However, in contrast to the current migration, none of the results so far have been satisfactory. This time, it seems our issues have more to do with the design of our application — and the solution isn’t yet clear.

The trouble with the process
#

But the testing environment going down was not actually the biggest friction — it was fixed quickly. The problem was that it took Claude 4 to 5 hours just to merge the remaining 12 MRs after MR 6. 8 hours of productive work, 4 hours of watching paint dry.

We use short-lived development branches against the origin/main trunk, and maintain a semi-linear history on merges (trunk-based development). This means that after each merge, the next merge request needs to be rebased, and the CI pipeline must run again. A rebase + CI cycle takes around 8 to 10 minutes today; and if the pipeline fails or a conflict occurs during rebase, additional CI cycles may need to run.

Every day, we race each other to merge our work. That day, we also raced the 18 MRs the agent had to merge. And our CI pipeline is not going to get faster — if anything it’s going to take more time once we add end-to-end tests.

So, can we end tech-debt?

In short, not all tech-debt scales equally well for agents, especially when the issue may have more to do with application design or the solution is not yet clear — as in our case with end-to-end testing.

And while we can certainly price some migration work, or even execute it concurrently with our feature work, it still needs to clear CI. Agents didn’t eliminate the implementation bottleneck — they moved it to the CI process.

I now find myself thinking about merge trains and other merging strategies to eliminate the CI bottleneck. I’m also wondering whether larger organisations are now experiencing the same pressure on their CI processes, and how this will shape the future of CI and trunk-based development.

Announcing autosync-git to replace autosync-magit

sylvain.bougerel@gmail.com (Sylvain Bougerel) — Fri, 15 May 2026 11:36:00 +0800

I’m deprecating Autosync-Magit in favour of Autosync-git.

Migrating
#

I kept aliases to the previous variables and hooks such as autosync-magit-after-pull-hook to make switching to the new tool easier. autosync-git conflicts with autosync-magit due to aliases, so you can’t load both concurrently. I will eventually deprecate these aliases, so you should port your .dir-locals.el to look like:

((nil . ((autosync-git-commit-message . "My commit message")
 (autosync-git-pull-timer . 300)
 (mode . autosync-git))))

What changed
#

The new tool sheds a dependency on Magit, to just use git and Emacs’ async make-process directly. This allows it to provide its own execution buffer, instead of sharing it with Magit.

But the more exciting update is the better pull strategy, which reduces the risk of leaving conflicts in your current workspace, and does better at merging what does not conflict.

When the remote and your local branch have diverged, the pull operation starts with a “probe” stage, where autosync-git performs an in-memory 3-way merge with git merge-tree. If this 3-way merge succeeds, autosync-git does a rebase (default) or merge. rebase is now the default to keep the history as linear as possible. But you can change it to merge if you prefer.

((nil . ((autosync-git-pull-style . merge)
 (mode . autosync-git))))

When keeping the default rebase strategy, if a conflict occurs, it immediately aborts the rebase and informs you in *Messages*. You can also force the pull operation to leave any conflicts in the repository instead of aborting. More to discover at Autosync-git.

Why I continue to review code

sylvain.bougerel@gmail.com (Sylvain Bougerel) — Fri, 01 May 2026 10:01:00 +0800

Given the increase in velocity from coding assistants such as Claude Code and Copilot, automated review had become a necessity. Since we rolled out Coderabbit on our project, almost everybody stopped reviewing code. It didn’t happen immediately, but organically.

As it turns out, Coderabbit is a formidable reviewer: with only a bit of context in the commit messages and a well-written AGENTS.md, Coderabbit will often be more thorough in reviews than most of my colleagues. And if that wasn’t enough, Coderabbit is almost always available—98.3% of the time in the last 30 days, not good for typical SLA targets but much better than colleagues—and a fraction of their cost.

Eventually everybody stopped reviewing code… Save for me.

For one thing, coding assistants can hallucinate blunders that neither humans—regrettably—nor Coderabbit will catch:

const NON_EDITABLE_BOOKING_STATUSES = new Set([
 'cancelled',
 'canceled',
 'declined',
 'withdrawn',
]);

I took that snippet from a merge request submitted yesterday. We do not have any declined or canceled statuses in our entire domain model—they simply don’t exist. I chose this example because it’s one of the simplest. The day before, a merge request had a complete re-implementation of a 200+ lines utility we already had. This happens all the time.

But most importantly, if the phone rings because of an issue on the system, I need to get on the call and fix the issue. When my work has potential to impact my personal life and my weekends with family, there’s no AI that I will trust enough. And every alert, whether it fires on the job or not, still costs the whole team velocity when we scramble for unplanned work.

Ultimately, our customers don’t care who wrote the code: “We’re sorry, Claude wrote that line and Coderabbit didn’t catch the problem” is not a good look. No matter how powerful our tools have become, responsibility is still mine. So I’ll keep on reviewing code.

AI exacerbated the divide between engineers

sylvain.bougerel@gmail.com (Sylvain Bougerel) — Sat, 25 Apr 2026 22:12:00 +0800

Any complex product requires deep context about the product’s domains, architecture, implementation. And if you want to make effective use of AI to solve a problem, you need to inject the right subset of that entire context into the prompt, or that smart auto-complete will spit out costly nonsense.

Naturally, engineers with the best understanding of the context, the capacity to articulate the problem and the skills to implement it are even stronger with AI: they can provide better input to the model and quickly validate its output. The once derided 10x engineer is starting to become a reality for anyone who can use AI to implement a correct solution at a fraction of the time it would have taken them.

On the other end of the spectrum, weaker engineers are now compelled to keep up with the pace set by their stronger peers, only they can neither prompt the LLM effectively nor validate what it produces. So they just let the garbage out—knowingly or not.

If you feel like you’ve become a “man-in-the-middle”—a proxy between somebody else’s request and an LLM—you will be phased out. The great engineers of today learned the ropes at a time AI didn’t exist, when they could invest in their fundamentals. Give yourself the same gift: carve out time to use less AI, and start learning again so you can close the divide.

Claude Code more than doubled my productivity

sylvain.bougerel@gmail.com (Sylvain Bougerel) — Sat, 14 Mar 2026 15:28:00 +0800

I wrote twice as much code last quarter, and it was better. I use Claude Code daily via Steve Molitor’s claude-code.el and monet.el integration for Emacs. Once a review is ready, CodeRabbit handles it. My daily AI usage really kicked into high gear back in October 2025, when we got a large amount of Claude Code credit grants and I could use it as much as I wanted.

The graph below is the number of lines of code changed (added and deleted) in my merge requests (MR), summed monthly, on our main application repository; a Next.js application with 200KLoC — not something an agent can easily get into. I authored 773 merge requests on this project alone which runs in production to serve our clients — just to emphasize the value of the sample-size:

You can see a clear 2x increase when I started to use Claude Code daily around October 2025. Like most folks, I found Claude Code to be amazing when working as a pair programmer to write the code or review my changes in short feedback loops, to explore greenfield projects or to strengthen tests on brownfield projects.

Around that time, I felt the quality of my code improving too. Below is the number of normalized comments $N$ in each of the MRs: or the number of comments $C$ divided by the sum of lines of code added $A$ and deleted $D$:

\begin{equation} N = \frac{C}{(A + D)} \end{equation}

Back in April 2025, we rolled out CodeRabbit which picked up over 70% of the reviewing work for everyone. Most team members do not perform code reviews anymore, except for myself — I’ll probably explain why in another post. Thus, on the changes I author, over 95% of the comments are just done by CodeRabbit from that time onwards.

Normalized comments is an interesting metric since it reflects the quality of my work with Claude Code, prior to review by others and CodeRabbit. If the value decreased there, it could be that:

We got lazy about reviewing, but CodeRabbit is pretty much the only reviewer of my work;
We output a lot more code or the quality of the code increased;
The work we do is a lot easier, so there’s just fewer issues with it.

AI certainly had an impact on point 2, given the top graph. You can also see a dip soon after October 2025 when I started to use Claude Code, but then a bump appears again in January 2026. That bump is due to the fact that I had to tackle a challenging issue that spanned several MRs. It’s quite visible when looking at the histogram of independent values:

There are other common proxies for code quality: such as unit tests. This is the raw number of unit tests on the main project branch, over time and it went up nearly 5x in the last 4 months:

Additionally, we have not had production rollbacks or outages in the last 4 months, consistent with the past year (We only ever had to perform production rollbacks twice). So agent usage is not called into question in our team.

The stats in this post were extracted using a tool written by Claude Code.

Should you use React Strict Mode in Next.js?

sylvain.bougerel@gmail.com (Sylvain Bougerel) — Tue, 03 Mar 2026 17:34:00 +0800

If you’re not familiar with React StrictMode, you’re probably not using a framework like Next.js that enables it by default. Strict Mode aims to help you find bugs during development. But it feels more like a band-aid that can reveal bugs or hide them, especially when mixing with server-side rendering (SSR).

In this post, I will assume you’re using SSR and Strict Mode too as they are usually enabled by default with Next.js.

Enter the rabbit hole
#

Now let’s suppose you write the following component, that renders a simple <time /> markup for a given date. Nothing fancy:

export default function LocalTime({ date }: { date: Date }) {
 const isoString = date.toISOString();

 return <time dateTime={isoString}>{date.toLocaleString()}</time>;
}

Let’s say you also have a simple home page, in your typical React application:

import LocalTime from "./local-time";

export default function Home() {
 return (
 <main>
 It is <LocalTime date={new Date("02/03/2026, 10:00 AM")} />
 </main>
 );
}

If your application uses SSR, what you will see always depends on your locale and on the time zone of the server; in this case, with an en-US locale and a UTC timezone, the server will render the following page, even in development:

It is 2/3/2026, 10:00:00 AM

However, that’s usually not what you want for your clients. Let’s say your client lives in Singapore, they should expect to see a time in a different timezone (GMT+8 in the case of Singapore). If you’re a bit experienced, you know your component must run on the client, and there’s a "use client;" directive in React that ensures a component can be an entry point to client rendering. You update the LocalTime component:

"use client"; // new directive

export default function LocalTime({ date }: { date: Date }) {
 const isoString = date.toISOString();

 return <time dateTime={isoString}>{date.toLocaleString()}</time>;
}

Your development server hot-reloads and this time you see:

It is 2/3/2026, 6:00:00 PM

Success! However, when you refresh the page, you notice there’s a flicker: the UTC time shows first, then the local time appears. You pull up the console, and you see a pretty large error message:

Uncaught Error: Hydration failed because the server rendered text didn't match the client. As a result this tree will be regenerated on the client. This can happen if a SSR-ed Client Component used:

- A server/client branch `if (typeof window !== 'undefined')`.
- Variable input such as `Date.now()` or `Math.random()` which changes each time it's called.
- Date formatting in a user's locale which doesn't match the server.
- External changing data without sending a snapshot of it along with the HTML.
- Invalid HTML tag nesting.

We’ve hit a classic issue with React SSR/CSR: hydration mismatch due to a time zone difference between the server and the client. And more importantly, you might not have noticed that error at all. The flicker might not have caught your attention, you might not have looked at the console.

You might just have deployed it to production. And to your surprise, once in production the page would show the UTC time only. It seems the component did not run on the client. Not a great user experience:

It is 2/3/2026, 10:00:00 AM

In reality, there was no reason for your component to re-render on the client again after hydration. This happened in development only due to React Strict Mode, so we’ve been led down the wrong path; thinking the application worked where it didn’t. You think this example is contrived? At Canterly, we have already faced 3 scenarios similar to this one; each time React Strict Mode was hiding an issue.

The reasoning behind Strict Mode
#

React brings a flavor of functional programming (FP) to UI. In FP, when the output of a function is defined only by its input, and it has no side effects, we call it a pure function. The React equivalent means that the rendering of a component should be idempotent: it depends only on the component properties, its state and its context. These 3 sources of information are the rendering inputs in React; you should not use any other inputs.

That is not to say React does not handle side effects. However, you should confine side effects to events handlers, and in case you can’t find a suitable event handler for your side effect, React dedicates a whole phase of its update to side effects which you declare in the use*Effect() hooks. The steps below are an over-simplification of an update with React (more about it here):

If there are no updates to component inputs, go to the last step;
Render updated components;
Run side effects;
Return to the top;
Commit changes to the DOM.

Idempotency allows React to track component changes by looking only at their inputs. However, it’s easy to break idempotency if you unknowingly use an external input, modify properties incorrectly or rely on a variable during rendering that is not provided through its properties, state or context.

To help uncover these issues, React introduced Strict Mode. Wrapping your component in <StrictMode> adds an extra render on mount and after the side effects run, to check that rendering was not impacted:

If there are no updates to component inputs, go to the last step;
Render updated components;
Run declared side effects;
Strict mode: render all components once more and check there are no differences with the output in 2.
Return to the top;
Commit changes to the DOM.

If your React component is pure, then that extra render should be idempotent and similar to the first.

False positives with Strict Mode
#

Returning to our example: the first server-side render produces an incorrect UTC time, which could have been caught if it was not immediately redrawn by the extra Strict Mode render that occurred when React mounted the component. Instead, the developer saw the local time.

This little example illustrates that this approach is just as likely to catch a bug as to hide one, since Strict Mode can lead to false positives. Especially if you’re not careful with checking hydration issues.

It’s worth noting that a more careful developer could also catch the hydration mismatch. And they might be tempted to address the locale & time zone hydration mismatch by simply silencing it, since this component does not have any children:

"use client";

export default function LocalTime({ date }: { date: Date }) {
 const isoString = date.toISOString();

 return (
 <time dateTime={isoString} suppressHydrationWarning>
 {date.toLocaleString()}
 </time>
 );
}

Perhaps unexpectedly, once the component uses the property suppressHydrationWarning, the UTC time rendered server side re-appears and is not replaced by the local time in development. Even in the presence of Strict Mode, the component is not regenerated again. This could have revealed the error again; but requires employing a property that is usually discouraged.

Beyond Strict Mode
#

Since React 18, the React Compiler has grown a new linter. I highly recommend the React Compiler ESLint plugin. While it does not address all the objectives of Strict Mode, there’s a significant amount of overlap between the two tools. In my own usage, I have found that the linter helps me better with identifying violations of the Rules of React.

Unfortunately, when it comes to our little example, the linter does not identify that date.toLocaleString() requires a time zone and a locale that are not declared as input to the component. The linter is still a work in progress; I hope that these limitations will eventually be addressed.

I will leave you with a version of the component that follows the Rules of React (note that this version will not scale well if you have a lot of components to render):

"use client";

import { useEffect, useEffectEvent, useState } from "react";

export default function LocalTime({ date }: { date: Date }) {
 const [formattedDate, setFormattedDate] = useState(
 // Fix a locale and a time zone for idempotency
 date.toLocaleString("en-US", { timeZone: "UTC" }),
 );
 const isoString = date.toISOString();

 const onMount = useEffectEvent(() => {
 setFormattedDate(date.toLocaleString());
 });
 useEffect(() => {
 onMount();
 }, []);

 return <time dateTime={isoString}>{formattedDate}</time>;
}

This may seem a little complicated for a component that just renders a date without any interactions; but remember that there’s a hidden side effect in the original rendering. And unfortunately, in this specific scenario, Strict Mode put us on the wrong path.

One of the selling points of Strict Mode is that it will help reveal the issues with components that violate the Rules of React. But I would argue that if the issues can’t be found without Strict Mode, then you might as well just ignore them: your component is probably too simple to care about them. Otherwise, you should always pay attention to hydration issues and ensure your project uses the React Compiler ESLint plugin.

Interop makes my day joyful

sylvain.bougerel@gmail.com (Sylvain Bougerel) — Tue, 17 Feb 2026 20:06:00 +0800

Check out Interop 2026. It has been one of the driving force behind aligning browser’s compatibility across the market and towards the standards.

I used to hate UI development, and by far I hated web development the most. The incompatibility across browsers has been a plague on web development. I think my feelings are widely echoed across the community, as something you would just have to put up with. I generally stayed away from doing any web development.

For me, things started to really turn around circa 2017. When I joined Grab around that time and discovered caniuse.com or React 16, web development started to feel less like a pile of hacks. Structure and organized community efforts started to emerge.

The launch of the web-platform-tests has undoubtedly been a major step forward. Thanks to this project and others, browser compatibility is generally not a concern in my daily web development routine. I can safely say there’s real joy to web development nowadays, and LLMs have made it easier than ever to get into.

Introducing @sbougerel/eslint-plugin-next-use-client-boundary

sylvain.bougerel@gmail.com (Sylvain Bougerel) — Sat, 14 Feb 2026 17:46:00 +0800

In my last post I spoke about releasing an ESLint plugin to lint the serialization of 'use client' module entries. I’ve spend a few hours with Claude code to get this done and published on npmjs.org. I’ll probably come back in another post on the experience there (Some gains, some losses). I’ve just started a branch on my product to use the plugin, looking forward to see if it’s useful to others.

Don't abuse 'use client'

sylvain.bougerel@gmail.com (Sylvain Bougerel) — Sat, 31 Jan 2026 17:56:00 +0800

I see 'use client' used too often in our own code base, and it seems to be a matter of confusion elsewhere too: devs don’t understand 'use client' or don’t want to get into it. Let me give a quick explanation of why you should not use it everywhere and hopefully, by the end of this post, you’ll have at least understood what it is not for.

'use client' indicates that a client module can be called within server modules. No more no less. That means 'use client' modules have some restrictions, because they can sit at the server-client boundary:

When a server evaluated module imports values from a 'use client' module, the values must either be a React component or supported serializable prop values to be passed to a Client Component. Any other use case will throw an exception.

(Source: React reference)

The thing is: most types are serializable, so you don’t generally see an issue with putting 'use client' everywhere. But one commonly used type is not: plain simple functions. It should make intuitive sense: a reference to a function is meaningless at the server-client boundary (unless it is a server action).

In practice: if your reactive client component exposes events to its parents, like onSubmit and onCancel in the component below, that means we expect the parents to be client component themselves.

 export default function LineItemForm({
 account,
 onSubmit,
 onCancel,
 }: {
 account: string;
 onSubmit?: (data: LineItemData) => void;
 onCancel?: () => void;
 }) {

How could it be otherwise? The function onSubmit or onCancel could never cross the client-server boundary. Thus, you can remember now:

When an exported client component has properties that are not serializable, like functions, its module should not start with 'use client'.

Now, you may be thinking: “But adding 'use client' does not hurt anything, right?” Ever taken painkillers when you don’t need it? It’s the same reasoning: it does not harm, but at the critical moment where you need to feel pain, you’re suddenly numb to it.

Without 'use client' at the top of that component, the compiler will not let you call that client component from a server component. But with an incorrect 'use client' the compiler will just obey and assume you’ve made sure the properties are serializable without checking, and misusing that component in server components could result in runtime errors.

Unfortunately, I cannot find any linter that lints this. There is however a dynamic warning provided by the typescript-language-server: Props must be serializable for components in the "use client" entry file. Adding a linter for this would be an interesting project.

3 rules for simple tenancy in DynamoDB

sylvain.bougerel@gmail.com (Sylvain Bougerel) — Mon, 12 Jan 2026 12:39:00 +0800

Access control in DynamoDB
#

DynamoDB does not come with its own account management & fine-grained access control. Instead, it integrates with IAM, just like other AWS services.

This is great when your users authenticate with IAM, but it’s not the case of the majority of serverless web applications that authenticate users via OAuth.

Serverless web applications on AWS are usually structured this way: API Gateway receives a request, proxies this request to a Lambda, that constructs the response, optionally querying DynamoDB. In this architecture, the Lambda runs with a static IAM role. OAuth authentication is performed in the Lambda runtime using this static role; i.e. no fine-grained IAM access control.

This approach is usually preferred because it is simple, responsive, and yet scales. It leaves the responsibility of access control to the Lambda’s runtime, however¹.

If you’re writing a SaaS application in particular, you need to make sure that your tenants are isolated.

Fortunately, by following 3 rules, DynamoDB can provide tenant-based access control.

All partition keys have `tenant`
#

Make the tenant identifier a constituent of all your items’ partition key (a.k.a. hash key). For a String partition key, with a single-table design, the value of the partition key be:

${tenant}#${entity_kind}#${item_partition_key}

No `SCAN`
#

All other CRUD operations are allowed, however you shall not use scan. Scanning in DynamoDB reads any of your partitions at random. Thus, it bypasses the first rule. Enforce this in your Lambda role permission: deny dynamodb:Scan.

Get `tenant` from the request only
#

The final piece: the tenant identifier is always provided by the request. It shall not be a value you can retrieve from a query in the same table. If you do need to provide a list of tenants somewhere, this should be done separately, in a different table or preferably in a different cell.

How does it work?
#

All CRUD operations in DynamoDB require a partition key. All except scan which is why we disable it in rule 2.

Rule 3 ensures that a client’s access to a tenant partition is handled separately; the tenant identifier acts as a message passed between this step and the current requests; isolating each request to a single tenant. As a corollary, you should treat any request that require access to multiple tenant with higher restrictions, ideally running on a separate Lambda with the least privileges.

Finally, rule 1 leverages rule 2 and 3 to ensure that your CRUD operations can only retrieve information from that tenant’s partition, since it becomes impossible to access other tenants’ data without their identifiers.

Limitations
#

While this works well to isolate requests to a single tenant, this remains a “coarse-grained” approach; you do not have access to the attribute based access control provided by IAM and will need to implement the equivalent permissions in your application.

API Gateway’s Lambda authorizers allow you to provide context variables that can be used in dynamic fine-grained access control roles. This approach is more complex and not common for Next.js applications, in particular. ↩︎

Yodarabbit

sylvain.bougerel@gmail.com (Sylvain Bougerel) — Sun, 11 Jan 2026 13:06:00 +0800

Coderabbit has this new feature that allows you to change its tone. I can’t wait to ask it to speak like Yoda.

# .coderabbit.yaml
tone_instructions: "Speak like Yoda, you must."

Pino log instrumentation with Next.js 16

sylvain.bougerel@gmail.com (Sylvain Bougerel) — Sun, 04 Jan 2026 11:39:00 +0800

Recently at Canterly, we migrated from Next.js 14 to Next.js 16. In that process, we also switched from webpack to turbopack. This was a great speed improvement that came with some surprises. In particular, log instrumentation via next-logger now triggered a failure during compilation. I had been wanting to replace it anyway, since all it does is to patch console to use Pino instead (It also patches log functions in Next.js next/dist/build/output/log, which is redundant since these functions use console too). So instead, I patched console on my own. This is simpler and allows us to keep control over the instrumentation, rather than obfuscating the process with an additional dependency.

This is what we have now in instrumentation.ts:

export async function register() {
 if (process.env.NEXT_RUNTIME === 'nodejs') {
 const instrumentation = await import('./instrumentation-nodejs');
 await instrumentation.register();
 }
}

We’ve replaced next-logger.config.cjs with our own logger.ts which is a straight forward port:

import { LoggerOptions, pino } from 'pino';

let pinoOptions: LoggerOptions = {
 level: 'warn',
};

if (process.env.NODE_ENV !== 'production') {
 pinoOptions = {
 level: 'info',
 transport: {
 target: 'pino-pretty',
 },
 };
}

export const logger = pino(pinoOptions);

And in instrumentation-nodejs.ts:

import type { LogFn } from 'pino';

async function instrumentLogging() {
 // NextJS also patches `console` functions during development for better
 // debugging; so we normally bypass instrumentation unless the environment
 // variable CANTERLY_INSTRUMENTATION is "true"
 if (
 process.env.NODE_ENV === 'development' &&
 process.env.CANTERLY_INSTRUMENTATION !== 'true'
 ) {
 return;
 }

 const loggerModule = await import('./logger');

 const wrapLogger = (method: LogFn, ...args: unknown[]) => {
 if (args.length === 0) {
 return;
 }
 if (args.length === 1) {
 return method(args[0]);
 }
 // Converts usual Console usage to Pino's meta patterns. Console
 // calls usually start with the message, while Pino meta patterns take the
 // message in the second argument. And when we can't tell if we have a
 // message, we use a `consoleArgs` property instead, which is clearer.
 if (
 args.length === 2 &&
 typeof args[0] === 'object' &&
 args[0] !== null &&
 typeof args[1] === 'string'
 ) {
 return method(args[0], args[1]);
 }
 if (typeof args[0] === 'string') {
 const rest = args.slice(1);
 // Next.js's prefixedLog calls console.error(ansiPrefix, errorObject[, ...]).
 // Errors in consoleArgs would serialise to `{}` (no enumerable properties),
 // so extract one and let Pino's built-in err serializer handle it instead.
 const errArg = rest.find((a): a is Error => a instanceof Error);
 if (errArg) {
 const remaining = rest.filter((a) => a !== errArg);
 const merge: Record<string, unknown> = { err: errArg };
 if (remaining.length > 0) merge.consoleArgs = remaining;
 return method(merge, args[0]);
 }
 return method({ consoleArgs: rest }, args[0]);
 }
 return method({ consoleArgs: args });
 };

 const consoleLogger = loggerModule.logger.child({ name: 'console' });
 (['error', 'warn', 'debug', 'info', 'log', 'trace'] as const).forEach(
 (level) => {
 /* eslint-disable no-console */
 if (level === 'log') {
 console[level] = (...args: unknown[]) =>
 wrapLogger(consoleLogger.info.bind(consoleLogger), ...args);
 } else {
 console[level] = (...args: unknown[]) =>
 wrapLogger(consoleLogger[level].bind(consoleLogger), ...args);
 }
 /* eslint-enable no-console */
 },
 );
}

export async function register() {
 await instrumentLogging();
}

And that’s it. This instrumentation does not just replace next-logger, it improves on it. With next-logger, the following console call…

console.log("Log without interpolation", valueA, valueB, valueC)

…would be patched to the Pino call…

logger.log("Log without interpolation", valueA, valueB, valueC)

…but that would lose logging data: when the first argument is a string message, Pino will only keep other arguments when they get interpolated. This new instrumentation preserves the additional arguments, by treating them as a mergingObject.

Now, when our application starts, we can immediately see the additional arguments in the logs, as soon as instrumentation kicks in (We use pino-pretty as the default transport during development):

npm run dev
> ...
> env TZ=UTC next dev

▲ Next.js 16.1.1 (Turbopack)
- Local: http://localhost:3000
- Network: http://...
- Environments: .env.local

✓ Starting...
[03:23:14.265] INFO (console/28071): ✓ Ready in 1439ms
[03:23:14.447] WARN (console/28071): [auth][warn][debug-enabled]
 consoleArgs: [
 "Read more: https://warnings.authjs.dev"
 ]

Update<2026-09-01 Tue>: I’ve updated the logger code above, there was a bug lurking in the original code that would cause the logger to print an empty message. I fixed it and the post has the new version.

NextJS revalidateTag won't work if it's not called in an awaited promise

sylvain.bougerel@gmail.com (Sylvain Bougerel) — Thu, 18 Dec 2025 18:37:00 +0800

Using unstable_cache extensions or "use cache" if you’re on NextJS16, you can leverage revalidateTag to invalidate cache across cached calls. However, I noticed today that these won’t work in a route handler that is not async. And I’m not yet sure why.

This was my original (simplified) route handler:

export async function POST(req: Request) {
 try {
 void someTask();
 return NextResponse.json({ done: true });
 } catch (err) {
 return new NextResponse(`Internal error`, {
 status: 500,
 });
 }
}

Note that someTask returns a Promise which I didn’t await initially: the caller only needed to know that someTask was called, not that it succeeded. However, it became evident someTask didn’t behave properly.

someTask gets a list of subscriptions and updates them progressively. After each update, it re-validates cache entries associated with the Subscription tag:

const cachedFindOutdatedSubscriptions = unstable_cache(
 findOutdatedSubscriptions,
 undefined, // Unused (For additional keys)
 {
 tags: ["Subscription"],
 },
);

export default async function someTask() {
 const subscriptions = await cachedFindOutdatedSubscriptions();

 for (const subscription of subscriptions) {
 await updateSubscription(subscription);
 revalidateTag('Subscription');
 }
}

The function cachedFindOutdatedSubscriptions returns subscriptions from a cache entry also tagged with Subscription. You would therefore expect that across repeated calls to this route, the subscriptions array becomes empty when no new subscriptions are created.

This is not what happened; the same subscriptions got reprocessed every single time, served from the cache. As if revalidateTag was not called.

Interestingly, the fix is simply to await the call to someTask in the route handler:

 export async function POST(req: Request) {
 try {
- void someTask();
+ await someTask();
 return NextResponse.json({ done: true });
 } catch (err) {
 return new NextResponse(`Internal error`, {
 status: 500,
 });
 }
 }

After reading the code for unstable_cache, I got the intuition for the fix. I’m not sure of what is exactly happening under the hood yet, but I think it has to do with the lifetime of the request’s context. I’ll return to this once I understand it properly.

Private Language Tools for browser spell checks

sylvain.bougerel@gmail.com (Sylvain Bougerel) — Mon, 15 Dec 2025 09:00:00 +0800

With LanguageTool (LT), it’s possible to run a local language tool server and have your browser extension connect to it. That way, your personal prose remains private (if you trust the browser extension, that is). It’s all documented here and it’s pretty simple to do. In this post, I cover how you can do this on Arch Linux (but it’s likely easy to port to other distro) with systemd to keep that server running.

Start by getting LT.

yay -S languagetool

You’ll also need fastText and a language identification model. Fortunately, there’s already 2 good packages for it in AUR (Check the PKGBUILD — one can never be too sure — but it looks correct at the time of writing).

yay -S fasttext fasttext-langid-models

That’s it for the dependencies. Now for your local setup. Create the file ~/.config/systemd/user/my-languagetool.service with the content:

[Unit]
Description=LanguageTool Grammar Checker Server
After=network.target

[Service]
Type=simple
ExecStart=/usr/bin/languagetool --http --port 8081 --allow-origin "*" --config %h/.config/LanguageTool/server.properties
Restart=on-failure
RestartSec=5

[Install]
WantedBy=default.target

As well as the file ~/.config/LanguageTool/server.properties with the content:

fasttextModel=/usr/share/fasttext/lid.176.bin
fasttextBinary=/usr/bin/fasttext

And now you’re ready to start your local server and keep it up and running for your browser extension:

systemctl --user daemon-reload
systemctl --user enable my-languagetool.service
systemctl --user start my-languagetool.service

You just need to configure the settings in your browser extension to ensure it connects to your local LT server.

Tips for technical interviews

sylvain.bougerel@gmail.com (Sylvain Bougerel) — Fri, 08 Aug 2025 23:35:00 +0800

Since I am currently on the lookout for a position, I thought I’d share some tips that apply to system design or technical interviews — except the coding interview — that could help you prepare as well.

Growth mindset
#

It’s the interviewer’s job to throw curveballs. Your role is not to dodge them. Always work with new constraints; often, when doing so, you find new ways to look at your problem. It’s okay to not know something, so long as you demonstrate how you’d move forward:

“I don’t know this part, but here is how I can find out or tackle it”

Focus on the technical
#

It may sound obvious, but if asked a question about your experience or if asked to provide more details about a line on your resume, focus on the technical. Mentioning your stint as a manager or the fact that you took a sabbatical break to travel the world may be okay if kept to an absolute minimum (as mentioned here), but don’t expend your energy on this; this is not the signal the interviewer is usually looking for.

Balance breadth and depth
#

The interviewer wants to find if you have the breadth and depth of knowledge that fit the role. You must play both angles: explore in breadth whenever you need to outline your solution, but don’t miss an opportunity to explore in depth when answering a question. If you answer a specific question with hand-wavy statements, you may not get that opportunity back: you rarely come back twice on the same question in an interview.

Research!
#

You know what never happened in 20 years of interviewing: company working in domain A asking a question about domain B. Every single technical interview will center around something the company already does or will do very soon. Therefore, those who do extensive research about the company’s domain stand to have an edge: you understand hints and questions better, it becomes easier to speak the same language as the interviewer.

Things you should research before a technical interview:

Blogs or engineering articles by the company or its competitors,
Discover as much about the company’s systems and technologies in use as you do about its process, regulations, and other domain constraints,
Map the organization, how it relates to its customer base,
Understand its engineering culture,
Harness AI to be a mock interviewer to train on related topics from the information you gathered.

Quantitative over qualitative
#

If you can quantify something, it’s better than qualifying it. But if you can’t quantify it, then qualifying it is better than nothing. For example, prefer:

“DynamoDB can handle 1M QPS (read and write) compared to MySQL 10K QPS (reads and writes)”

Over:

“DynamoDB can handle much larger QPS (read and write) than MySQL.”

Numbers are king, and help you justify your design decisions. I included a data capacity ladder at the end of the article.

Divide and conquer generalities
#

If the interviewer asks any question, you should evaluate whether to break it down first before narrowing and going in depth on some topics. An example of a vague question could be:

“I see that you have experience with AWS EC2, could you describe what it is?”

EC2 is one of the most complex AWS services, one that could take the entire interview to answer. Typically, the answer calls for a breakdown after a short intro:

“EC2 is an AWS service that manages virtual machines running on the cloud. EC2 is made up of many subsystems required to run VMs: the nodes themselves (instances), the block storage (EBS), the images (AMIs), the network security groups and network interfaces (ENIs). It provides autoscaling capabilities through Auto Scaling Groups. Instances can be provisioned on-demand, spot, or reserved. Would you like me to zoom in on any particular subsystem?”

This is a top-level breakdown of EC2, only 1 layer deep. It’s not in-depth on any system but gives enough insights to the interviewer that you are comfortable with the topic. That last hook gives pause and allows the interviewer to decide how it wants to spend its time.

Structure of the system design interview
#

For system design interviews specifically, you want to follow a structured plan and a design methodology. I recommend using the C4 model as a design methodology because it is hierarchical and feels natural. Usually, the plan is:

Clarify: make sure you understand the focus of the exercise. You don’t want to dwell on sub-components that the interviewer does not care about. At this stage you should have an understanding of the domain and you should be able to draw your system context. Ensure that you do not miss any “meaty” parts of your system.
Dimension: pin down the spatial scope (e.g. world-wide vs city?), temporal scope, and the quantity of data exchanged and stored in each context. At this stage, you should have a rough understanding of the orders of magnitude for every dimension. This will drive the justifications of your design.
Overview: break each system context into its constituent containers. This is where you describe your approach to the interviewer. You should label the flow, the main functions and give a brief description of how each container satisfies the constraints of the problem, e.g. how the system scales to meet its demand.
Deep-dive: give the interviewer the opportunity to ask specific questions and propose to zoom in on some particular component of your design. It’s a good time to discuss protocols, define APIs and describe in detail how your design can solve a specific problem.

For a 45-minute interview, try to keep the Clarify and Dimension sections under 15 minutes total. You want to spend most of your time in Overview and Deep-dive. Finally, if it helps (it does for me) restate the objective of what you are trying to demonstrate at each hierarchy.

Practice
#

Nothing replaces practice, no matter how confident you are in your skills; being able to demonstrate that confidence in a short interview context requires practice.

Start with your CV items,
Identify the technical questions based on the keywords or claims in your CV,
Write down your narratives,
Practice answering these questions.

Remember to smile
#

An interview is a stressful setting. But all things being equal, candidates who came across best get the gig.

Sometimes, it’s better to ask
#

Most interviewers prefer to be engaged in an interview. I certainly do, when I interview. If you feel out of your depth on a particular task or question, don’t hesitate to ask for guidance. The interviewer will generally give you the clues needed so you can move forward. It’s better to show humility in such a situation than to speak incorrectly about something you do not know or understand well, or — worse — leave no doubt about your ignorance of the subject.

Get past your impostor syndrome
#

The interview is not the place to question your (lack of) knowledge constantly. The biggest disservice you can do to yourself is selling yourself short. Do not provide information about the limits of your expertise, as that could further be misinterpreted. It is already difficult enough to convince the interviewer that you know enough in a particular area, in such a short span of time. Give yourself a break and a bit of confidence.

Data transfer ladder
#

Some illustrative operations or requests per second to memorize when discussing distributed systems:

Typical data transfer rates of some servers, proxies and messaging technologies:

Domino and tromino tiling sorcery

sylvain.bougerel@gmail.com (Sylvain Bougerel) — Sat, 18 May 2024 23:24:00 +0800

Since I’m currently looking for a job, I try to train daily on Leetcode. I started with a now classic study plan, originally authored by Yang Shun Tay and hosted on Grind75. Eventually though, Leetcode presented me with a tiling problem that I found immediately appealing: Domino and tromino tiling.

Spoiler alert: the following content reveals the solution to the problem

If you don’t know the problem yet, take a look at the description in the link. If you are eager to solve it yourself, stop reading this post immediately. Try solving it and come back to this afterward.

I was in the process of training on bottom-up dynamic programming, and this problem lends itself perfectly to it. So let me present my conceptual solution to the problem:

For a 2x2 (n = 2) area, only the first 2 arrangements above tile the area entirely. But if we allow a single gap to remain on the right, either at the top or the bottom, then all 4 arrangements on the first row tile the area. With the same rules for a 3x2 (n = 3) area, there are 9 different arrangements that (nearly) tile the area:

We can write the equation for the number of arrangements, according to the variables $(n, h)$, with $h$ a boolean value representing the presence of a hole:

\begin{equation} f(n, h)= \left\{ \begin{array}{ l l } 1 & \quad \textrm{if } n = 1, h = 0 \\ 0 & \quad \textrm{if } n = 1, h = 1 \\ 2 & \quad \textrm{if } n = 2, h = 0 \\ 2 & \quad \textrm{if } n = 2, h = 1 \\ f(n - 1, 0) + f(n - 2, 0) + f(n - 1, 1) & \quad \textrm{if } n > 2, h = 0 \\ 2 f(n - 2, 0) + f(n - 1, 1) & \quad \textrm{if } n > 2, h = 1 \end{array} \right. \end{equation}

The first 4 cases should be straight-forward given the prior explanation. So let’s dig into the last 2 lines.

Imagine having to tile the $n$x2 area by extending it to the right from a smaller value $n - x$. Considering only the case where there’s no hole left after, for now:

If an $n - 1$ arrangement has no hole, placing a vertical domino to the right extends it to $n$. If an $n - 2$ arrangement has no hole, placing 2 horizontal dominoes to the right extends it to $n$. Also, if an $n - 1$ arrangement has a hole (either top or bottom), placing a single tromino to the right extends it to $n$.

Following this same approach, in the case of a remaining hole to the right, an $n - 1$ arrangement with a hole can be extended to $n$ by placing a single horizontal domino and an $n - 2$ arrangement without hole can be extended to $n$ in 2 different ways by placing a tromino up or down.

Any other action extending $n - x$ to $n$ can be constructed from the 6 actions above. With this understanding, this is my initial bottom-up solution:

class Solution:
 def numTilings(self, n: int) -> int:
 n2h0 = 1
 n2h1 = 0
 n1h0 = 2
 n1h1 = 2
 m = 10**9 + 7

 if n == 1: return n2h0
 if n == 2: return n1h0

 for _ in range(2, n):
 n2h0, n2h1, n1h0, n1h1 = n1h0, n1h1, (n2h0 + n1h0 + n1h1) % m, (2*n2h0 + n1h1) % m

 return n1h0

Neat, hey?

Let’s take a look at the top answer, as one should normally do.

class Solution:
 def numTilings(self, n: int) -> int:
 dp=[0]*(n+1)
 dp[0],dp[1]=1,1
 for i in range(2,n+1):
 dp[i]=2*dp[i-1]+dp[i-3]
 return (dp[-1]%(10**9+7))

WTF!? What is this sorcery? I spent hours arriving at my neat solution!

As one would normally find out when looking at the solutions, there’s always someone smarter than oneself. Aside from the fact that the top solution consumes more memory (due to the array and the growing integers), it does much better with a more compact equation, that does not need to consider any holes.

And, yes, there is a bit of sorcery with the negative array subscripts going on in the case $n = 2$:

\begin{equation} f(n)= \left\{ \begin{array}{ l l } 1 & \quad \textrm{if } n = 0\\ 1 & \quad \textrm{if } n = 1\\ 2 = 2 f(1) + 0 & \quad \textrm{if } n = 2\\ 2 f(n - 1) + f(n - 3) & \quad \textrm{otherwise} \end{array} \right. \end{equation}

This equation is neat but not evident, in my opinion. Though you may have Mensa-level IQ, like this person in the comments 🫠:

 Step 1: Run test cases for 1 to 8.
 Step 2: Observe the relation among A[i], a[i-1] and a[i-3]

Anyway, I figured my initial equation and the one above should reduce to the same form, so I was probably lazy. And indeed, when only considering cases where $n > 2$:

\begin{equation} \begin{array}{rcl} f(n, 0) & = & f(n - 1, 0) + f(n - 2, 0) + f(n - 1, 1) & \\ \textrm{since } f(n, 1) & = & 2 f(n - 2, 0) + f(n - 1, 1) \\ f(n, 0) & = & f(n - 1, 0) + f(n - 2, 0) + 2f(n - 3, 0) + f(n - 2, 1) \\ \textrm{rearranging } f(n, 0) & = & f(n - 1, 0) + f(n - 3, 0) + \, \\ & & f(n - 2, 0) + f(n - 3, 0) + f(n - 2, 1) \\ \textrm{since } f(n - 1, 0) & = & f(n - 2, 0) + f(n - 3, 0) + f(n - 2, 1) \\ f(n, 0) & = & f(n - 1, 0) + f(n - 3, 0) + f(n - 1, 0) \\ f(n, 0) & = & 2f(n - 1, 0) + f(n - 3, 0) \end{array} \end{equation}

Damn. I hate how beautiful this is: notice how we can do away with the holes now. Here is another original explanation of the same results through a different route.

Time to rewrite the program.

class Solution:
 def numTilings(self, n: int) -> int:
 if n == 1: return 1
 if n == 2: return 2
 if n == 3: return 5
 n3, n2, n1 = 1, 2, 5
 for _ in range(3, n):
 n3, n2, n1 = n2, n1, (2*n1 + n3) % (10**9 + 7)
 return n1

It happens to be similar to the 4th best solution. This should be the fastest algorithm for sufficiently large values of $n$ though.

Logseq as Org-roam on the go (Part III)

sylvain.bougerel@gmail.com (Sylvain Bougerel) — Sat, 27 Apr 2024 14:02:00 +0800

This is the last part of a 3 part series:

Introduction, Logseq and Termux on mobile
Improving Logseq and Org-roam interoperability
Automating synchronisation (this part)

This post will cover automating synchronisation between the 2 mediums, to the point where you will almost forget it is there. It’s not perfect (merge conflicts sometime arise) but for the most part, it works flawlessly.

Automatic synchronisation in Emacs
#

There’s many ways to automatically synchronise Git repositories on your desktop. I wrote an Emacs package for it which has the added benefit to let me execute Emacs hooks post merge. Skip this part if you already have a solution that you are satisfied with.

Pros of my package:

Self-contained in Emacs, no need to setup external daemons
Simple configuration with .dir-locals.el
Works with magit
Provides a hook to run tasks after each merge
Works asynchronously, except when merging to avoid Emacs stepping on its own toes
Commits and pushes as necessary (when files are saved)

Cons (unsurprisingly):

Only works within Emacs and while it is running
Starts synchronisation only when Emacs visits a file within the repository

You can grab it at https://github.com/sbougerel/autosync-magit.

With straight.el and use-package.el, your configuration could look like:

(use-package autosync-magit
 :straight (:host github
 :repo "sbougerel/autosync-magit"
 :files ("*.el")))

Once you have this package installed, head to your Notes/ directory, and create a .dir-locals.el file with the following content:

((nil . ((autosync-magit-commit-message . "Update from desktop")
 (autosync-magit-pull-timer . 600)
 (eval add-hook 'autosync-magit-after-merge-hook #'logseq-org-roam nil t)
 (mode . autosync-magit)
 (mode . auto-revert))))

The first line specifies the commit message used every time a commit is created.
The second line specifies the background pull timer: the interval between pulls done in the background, here set to 10 minutes. In addition to the background pull timer, by default autosync-magit pulls files when they are visited. The package is smart enough to throttle pulls when Emacs visits files in the repository in quick succession (e.g. during logseq-org-roam calls). You can change the throttle value but you should never set it to 0: since logseq-org-roam causes files to be visited in quick succession, it calls autosync-magit often with each calls potentially calling logseq-org-roam again due to a merge.
The third line tells Emacs to run logseq-org-roam once merge is complete. This ensures that all files edited with Logseq are now converted to Org-roam automatically.
The fourth line activates the minor mode for files in this directory. None of the above works without this.
The last line sets up auto-revert on files that have been modified, either due to merge or post-processing with logseq-org-roam.

The first time you visit a file after the creation of the .dir-locals.el, Emacs will ask you whether you should trust these settings? Never trust anything you read or copy from Internet! With the warning out of the way, proceed to accept the settings permanently 😇.

With this done, the workflow on your desktop changes from:

git pull
Invoke logseq-org-roam in Emacs
Edit notes
git commit -a -m "Update from desktop"; git push

To:

Edit notes (and Save)

Didn’t I say it’s almost magic? I’ve tested it for a while now, but if you face any problem don’t hesitate to raise an issue or a PR. Now let’s keep the magic going with the mobile setup.

Automatic synchronisation on mobile with Tasker
#

This section assumes you’ve already gone through part 1, meaning you already have Logseq and Termux on your mobile. Similar to part 1, this section only applies to Android; there could be iOS alternatives but you’ll have to adapt the following.

The first thing is to add a couple of scripts on your Termux home. There’s nothing Termux-specific about these scripts, so you can test them on your desktop first.

Copy the content below in file named ~/pull-notes:

#!/usr/bin/bash
set -euo pipefail

cd ~/Notes

git fetch &>/dev/null
if [ "$(git rev-parse @{0})" == "$(git rev-parse @{u})" ]; then
	echo Nothing to do
elif git merge-base --is-ancestor @{0} @{u}; then
	git merge &>/dev/null
	echo Pulled
elif git merge-base --is-ancestor @{u} @{0}; then
	echo Nothing to do
else
	git merge &>/dev/null
	echo Conflict
fi

Similarly, copy the content below in a file ~/push-notes:

#!/usr/bin/bash
set -euo pipefail

cd ~/Notes

git add -A &>/dev/null
git commit -m "Update from Android" &>/dev/null || true
if [ "$(git rev-parse @{0})" != "$(git rev-parse @{u})" ]; then
	git push &>/dev/null
	echo Pushed
else
	echo Nothing to do
fi

You can test these 2 files after you’ve made them executable:

chmod a+x pull_notes
chmod a+x push_notes

The roles of these 2 files is to print a simple one-line statement about the results of git synchronisation operations. The application Tasker (installed next) calls these 2 scripts to perform the git operations. First move the scripts to a specific location so that Tasker can call them with a path relative to HOME:

mkdir -p ~/.termux/tasker
mv push-notes pull-notes ~/.termux/tasker

Now grab Tasker from the Play Store and install the companion termux-tasker app from Github or F-droid. Tasker is not free but at less that USD$3 it’s a small price to pay for all the time it will save you (If you know of a free alternative that is on par with the functionalities presented below, I’d love to hear about it).

The first time you open Tasker, it will invite you to watch tutorials, etc. Tasker has a ton of useful features outside of this project, which I’ll let you discover on your own.

Before I describe this Tasker project, let’s go through some definitions of Tasker’s entities:

Profiles describe events or situations which (in general) are used as a trigger for tasks
Tasks describe operations to perform, and they are made up of a set of actions
Scenes represents UI elements that can displays or used for input during task execution
Vars allow tasks to store information across execution or even share information across tasks

This Tasker project consists of setting up 2 tasks and 2 profiles.

Task LogseqPull sets a throttle, calls the script ~/push-notes created above and displays its result in a toaster.
Task LogseqPush calls the script ~/pull-notes created above and displays its result in a toaster.
Profile Logseq triggers LogseqPull when we launch Logseq on our phone and triggers LogseqPush when we exit it.
Finally, profile Every 30m triggers LogseqPull every 30 minutes.
The task LogseqPull will also create the variable Launch_logseq to throttle calls to ~/pull-notes, you don’t need to do anything about it.

Setting it all up is rather cumbersome, so I provided the project as a gist which you can review, download, and import into Tasker. The XML is rather opaque but hopefully the above information helps with the review before you trust me with another download. You’re welcome to build or modify your own version, of course.

To import it into Tasker, use “Import Project” from the 🏠 button. The result should look similar to:

Figure 1: Tasker project “Logseq”

As soon as you enable the different profiles in the project, you should start seeing it in action. Navigate to the task to verify that they execute properly.

At this point, the workflow on your mobile changes from:

git pull on Termux
Edit notes
git commit -a -m "Update from phone"; git push

To:

Edit notes

How it’s going
#

I’ve been using and refining this setup for months, I’m really happy with it and I hope you’ll find it works great for you too. It takes bit of time to setup, but it works almost flawlessly in my case. And in case there’s an issue, I’ve got git to resolve it.

There are some left over quirks which can be addressed through habits. For example: don’t leave trailing * at then of a page in Logseq. Logseq does so when you press enter in edit mode, and sometimes it messes with Org-element parsing. A good habit is to press the page title or journal date to exit the input mode instead of pressing Enter.

That’s it for now. I’ll continue to improve on logseq-org-roam in the future. It’s already pretty fast, but I know it still a bit rough under the edges, the code is a little messy, and there’s more features I’m planning to add for tagging and cards (spaced repetitions). I hope you found this useful. Feel free to contribute!

Logseq as Org-roam on the go (Part II)

sylvain.bougerel@gmail.com (Sylvain Bougerel) — Sat, 13 Apr 2024 16:00:00 +0800

This is the second part of a 3 part series:

Introduction, Logseq and Termux on mobile
Improving Logseq and Org-roam interoperability (this part)
Automating synchronisation

This part covers my Emacs configuration and package to enable Org-roam and Logseq interoperability.

I personally use Emacs on my desktop and Logseq on the go, but if you plan to use both on the same machine, this post should apply as well.

Org-roam configuration
#

I’ve introduced some of it in my previous post, to detail Logseq’s configuration. This is the gist of it:

:preferred-format :org ;; required!
:pages-directory "pages" ;; default
:journals-directory "journals" ;; default, must match `org-roam-dailies-directory`
:journal/page-title-format "yyyy-MM-dd" ;; match with `org-roam-dailies-capture-templates`
:journal/file-name-format "yyyy-MM-dd" ;; match with `org-roam-dailies-capture-templates`
:preferred-workflow :todo ;; recommended
:property-pages/enabled? false ;; recommended, disable property pages

Here we’ll focus on Emacs & Org-roam configuration. You can also refer to my current setup, if you need to see the full configuration.

Directory layout
#

This is how my directory organisation currently looks like:

~/
└── Notes/
 ├── ... # Org stuff in here
 └── roam/ # Org-roam stuff in here
 ├── .git/ # Use Git for synchronisation
 ├── assets/ # Use for attachments too
 ├── journals/ # All dailies go here
 ├── logseq/ # Logseq's own space, Org-roam must ignore it
 └── pages/ # All notes go here

I nested my Org-roam directory under my Org directory for historical reason. If you are starting fresh with Org-roam & Logseq directly, this distinction is not necessary.

Here is how you can configure Emacs to achieve the above:

(setq org-directory "~/Notes/"
 org-roam-directory (file-truename (file-name-concat org-directory "roam/"))
 org-attach-id-dir (expand-file-name "assets" org-roam-directory)
 org-roam-dailies-directory "journals/"
 org-roam-file-exclude-regexp "\\.git/.*\\|logseq/.*$")

The last line is essential to ensure that Org-roam ignores the logseq directory.

Org-roam capture
#

Capture is responsible for determining notes location when you create them. Therefore you should update your capture template(s) to match Logseq’s directories. Your Emacs configuration should minimally look like:

(setq org-roam-capture-templates
 '(("d" "default" plain
 "%?"
 ;; Accomodates for the fact that Logseq uses the "pages" directory
 :target (file+head "pages/${slug}.org" "#+title: ${title}\n")
 :unnarrowed t))
 org-roam-dailies-capture-templates
 '(("d" "default" entry
 "* %?"
 :target (file+head "%<%Y-%m-%d>.org"
 "#+title: %<%Y-%m-%d>\n"))))

Pay attention to page/${slug}.org in particular. If you have configured Logseq differently for some reason, adjust accordingly.

Task management
#

Logseq task management is pretty basic, but Org is very flexible and can be easily extended beyond what Logseq does. If you want complete interoperability, your Emacs configuration should contain:

(setq org-todo-keywords '((sequence "TODO(t)" "DOING(s)" "|" "DONE(d!)")
 (sequence "[ ](T)" "[-](S)" "[?](W)" "|" "[X](D)")))

Logseq’s equivalent is:

:preferred-workflow :todo

You can keep the :now style and configure Org accordingly, TODO simply felt more natural when using Emacs.

Link format
#

Logseq provides the user with 2 choices of link format when using the Org format. Neither choice is perfect, so it depends on your use case:

Fuzzy links (or internal links):
- Style: This is a link to a [[Note]]
- Pro: Easy to capture, edit, especially with the limited mobile screen estate.
- Cons: Ambiguous (hence the name) in certain cases, e.g. when the title of a heading matches with the link (Org internal link).
- When to use: You create many small notes (Zettelkasten style) and notes are shorter.
File links:
- Style: This is a link to a [[file://../pages/note.org][Note]]
- Pro: Unambiguous
- Cons: Because of a bug in Logseq, when the target note does not yet exists, the path is rendered incorrectly. It must be fixed manually every time, or it will remain incorrect even when the notes is eventually created.
- When to use: you create fewer notes and you already use internal links in your notes.

Logseq defaults to fuzzy links and that’s the one I recommend if you can’t choose. If you prefer to go with file links, update your config.edn in Logseq with:

:org-mode/insert-file-link? true

Converting Logseq captures to Org-roam
#

In the last post, I’ve detailed some of the problems with using Logseq out-of-the-box with Org-roam. As a recap, the biggest issue is how Logseq and Org-roam differ on links: Logseq uses internal links or file links as external links, while Org-roam relies on ID links.

I’ve created an Emacs package named logseq-org-roam (Github) to address this problem and some others. Assuming you have straight.el & use-package.el, you can install it with:

(use-package logseq-org-roam
 :straight (:host github
 :repo "sbougerel/logseq-org-roam"
 :files ("*.el")))

Invoking logseq-org-roam crawls your entire Org-roam directory, to migrate Org files edited with Logseq as Org-roam files:

Adds missing ID for each notes, so that Org-roam can index them
Adds missing titles for each notes
Converts all links to ID links, when they match an existing note; works with both fuzzy and file links
Converts Logseq aliases to Org-roam aliases

These 4 actions fix much of the friction between Org-roam and Logseq, and make both application interoperable. logseq-org-roam is capable of more: it can use cache information from Org-roam in order to speed up its operations, it can even capture new notes.

logseq-org-roam will however break some things for Logseq as I mentioned in Part I (backlinks and graph). Here, I assume that Org-roam is your main note-taking tool, and Logseq is used to take Org-roam on the go.

When installed and used for the first time, I recommend to invoke it with C-u M-x logseq-org-roam: it will parse all Org-roam files and update the ones created or modified with Logseq. Afterwards, you can use M-x logseq-org-roam which works incrementally by skipping cached Org-roam files.

I recommend that you use git or another source version control before using logseg-org-roam in case you are not happy with the changes. When you invoke logseq-org-roam, it creates the buffer *Logseq-org-roam <org-roam-directory>* which you can consult to see the modifications made.

logseq-org-roam comes with a hook which you can customize to automatically sync the Org-roam cache after files are updated:

(add-hook 'logseq-org-roam-updated-hook #'org-roam-db-sync)

Workflow and remaining frictions
#

If you’ve followed the guide so far, you’ve installed Logseq on your phone, you’ve setup your version control and you’ve tested the interoperability of Logseq with Org-roam, you should have a good idea if you want to use this regularly.

Your workflow at this point, looks as such:

When editing notes on your phone:
1. git pull on Termux
2. Edit notes
3. git commit -a -m "Update from phone"; git push
When editing notes on your desktop
1. git pull
2. Invoke logseq-org-roam in Emacs
3. Edit notes
4. git commit -a -m "Update from desktop"; git push

This is works but it’s pretty tideous: I’m sure you prefer to focus on editing notes. I do too. In my next post I’ll cover automating synchronisation between your mobile and your desktop, on top of git. With automated synchronisation, Org-roam note taking on the go with Logseq feels natural, almost magical.

Logseq as Org-roam on the go (Part I)

sylvain.bougerel@gmail.com (Sylvain Bougerel) — Mon, 08 Apr 2024 00:10:00 +0800

This is the first part of a 3 part series:

Introduction, Logseq and Termux on mobile (this part)
Improving Logseq and Org-roam interoperability
Automating synchronisation

I started to use Logseq in the fall of 2023, and with almost 9 months of use, I’m confident it is a great mobile application for Org-roam. Some applications do a better job at being compatible with the Org file format (e.g. organice) however they do not support Org-roam readily. Logseq, while a bit lax with its support of the Org file format (more on that in a bit), is closer to Org-roam’s Zettelkasten philosophy: links between pages are a breeze to create, backlinks show up immediately and the journal entries quickly becomes your entry point for time management.

In this post, I’ll go over my mobile setup and some of the quirks of using Logseq with Org-roam. I’ll follow up on my Emacs setup and automating all this in future posts. Let’s start by understanding their differences.

Logseq & org-roam interoperability
#

The good
#

Logseq was deliberately designed for the minimal interface required for web/mobile applications. Capture is both user-friendly and functional.

As an Org editor and interface, it is impressive. It supports heading highlights for tasks, checkboxes, tags. It renders quote blocks, code blocks (with syntax highlighting), properties, keywords, tables, etc. It also supports inline $\LaTeX$ flawlessly, putting my Emacs setup to shame.

Figure 1: Logseq

Performance-wise, Logseq easily handles the thousands of notes that I have without noticeable penalty. Its Org-agenda equivalents (queries) render their results almost immediately.

Logseq also has templates. Granted, the format to create a template is different and your Yasnippets won’t work, but this feature is still invaluable to ease input on a phone.

Logseq has even more features making you love it other ways, and it has definitely influenced my Org-roam setup, for the better.

The bad
#

Using Logseq and Org-roam together imposes some constraints on how you organise your notes. I’ve come to prefer Logseq’s approach over my own, inherited from Org, but it might not be your cup of tea. Logseq imposes the following directory structure for your “graph” (i.e. org-roam-directory):

└── org-roam-directory/
 ├── assets/
 ├── journals/
 ├── logseq/
 └── pages/

The assets directory is where Logseq stores attachments, thus it is handy to map it to your org-attach-id-dir.
The journals directory is where Logseq stores your dailies, and should map to your org-roam-dailies-directory.
The logseq directory is Logseq’s own internal folder, and it must be hidden from Org-roam in org-roam-file-exclude-regexp.
The pages directory is where Logseq stores every single note. The lack of hierarchy for pages may not suit everyone. I find personally that Org-roam tagging provides more flexibility, so it’s not an issue for me.

Within a file, Logseq insists on hierarchical note-taking and does not handle well the top section of an Org file. It will shove top section paragraphs, top node properties and keywords under a heading… Which you should not try to edit.

Also, some pages have special meaning in Logseq. For example, you will need to port your Yasnippets to the “Templates” page if you need them in Logseg. If you want to see all your to-dos, you’ll need to visit the “TODO” page, where content gets generated on the fly.

This is not the only thing Logseq’s generates on the fly: results of Logseq queries are generated on the fly too, and the result is not stored in the file, unlike what Babel does. And since Logseq’s does not support Babel, queries are not interoperable between the 2 platforms.

The ugly
#

The biggest problem however is that Logseq uses Org internal link syntax for external links. This is an Org internal link:

* Heading A
* Heading B internal link to [[Heading A]]

As the name suggests, internal links are looked up only within the same file in Org. A link to another Org file is an called external link, and they have a different syntax:

* External file link to [[file:other.org][another note]]
* External ID link to [[id:some-id][another note]]

Logseq uses Org’s internal link syntax by default, which may conflict with Org. There’s an option to force Logseq to use file paths instead. Unfortunately, this is buggy (both version 0.9.x and 0.10.x) when a note does not exists yet.

And ultimately, Org-roam uses ID links.

Regrettably, when converting Org links to ID links (as Org-roam usually does) it breaks the link relationship for Logseq. Logseq can still follow the link, but the backlink will appear under “Unlinked references”. This unfortunately breaks the graph in Logseq: all your notes suddenly look isolated. If you really enjoy the graph visualisation in Logseq, this could be a deal-breaker.

So while Logseq does look like a good companion for Org-roam, it does not care about Org-roam. Logseq will not add property ID your notes, it won’t use Org-roam aliases, and it will not convert links to ID links.

Why use Logseq with Org-roam then?
#

To address some of these issues, I (and others) have written Emacs packages that I’ll present in my next post. With these packages, the ugly issues disappear and only some of the bad remains. Thus Logseq shines with what it does well, and it is really good at what it does well.

Even without these packages, the Zettelkasten philosophy, its usability and all of its functionalities really make up for the remaining quirks. You can start using Logseq with Org-roam to try it, if only as a viewer.

If you’re convinced, let’s move on to configuring Termux for git synchronisation on your phone.

Setting up Termux and Git
#

I synchronise my notes with git. There’s easier ways to doing it, but I don’t think there’s much better ways than using an SCM. All my notes are on a private repository. This section only applies to Android, I am sure there are similar methods for iOS, though I don’t know them.

Firstly, install the excellent Termux app on your phone. Once done, you can open the app and simply:

apt update
apt upgrade
pkg install git

During the upgrade process, you may sometimes have to answer the question: Configuration file ... The default action is to keep your current version. (Y/I/N/O/D/Z) [default=N] ? Just type I since it’s a fresh install. I also recommend picking an editor of your choice if you do not like nano. I find nano is a good fit on a phone. This will come in handy when you have merge conflicts to resolve.

Then type termux-setup-storage and grant Termux access to your shared storage (i.e. user data storage, SD card).

I placed my notes at the root of my phone’s shared storage, which you can access from your Termux user at ~/storage/shared/Notes:

mkdir storage/shared/Notes
ln -s storage/shared/Notes
cd Notes
git config --global --add safe.directory ~/storage/shared/Notes
git config --global user.name ${name}
git config --global user.email ${email}

Line 4 is required since the user performing the git commands (you) is different from the user owning the git repository. I believe this is due to the fuse abstraction on top of the shared storage’s file system but haven’t dug into it. Onto SSH now.

pkg install openssh

The installation process should prompt you to create a default SSH key. If it does not, once the installation is complete, just:

ssh-keygen

I recommend you do not set a passphrase for this key. Unlike your desktop or laptop, Android does not come with a standard keychain application, so you’ll have to enter your passphrase every time you need to use it or rely on third-party applications. It’s a significant hassle in one case, or an issue of trust in the other. For third-party applications, you could explore using OpenKeychain together with OkcAgent, though I can’t recommend it given that OkcAgent’s sole maintainer does not seem to have time to work on it.

For security, you should only use this key for the single purpose of synchronising your notes, and you should prefer to rotate it regularly; I do it every 6 months. I personally use GitLab for the unlimited number of private repositories, which also allows me to set an expiration date on the key, forcing me to rotate the key. After adding your key to your SCM, try:

cd ~/Notes
git clone git@${remote-url}:${user-name}/${repository}.git .
git config pull.rebase false

Since this is a repository for synchronisation of text content; you will save yourself a lot of hassle by letting git pick the best merge strategies for you.

We’re going to make our first change on the phone, by telling git to ignore Logseq’s backup data:

echo "logseq/bak" >> .gitignore
echo "logseq/.recycle" >> .gitignore
git commit -a -m "Ignore Logseq cache"
git push

Logseq installation & configuration
#

Grab the latest version of Logseq here: https://github.com/logseq/logseq/releases. You can go with the Nightly if you feel adventurous.

Once installed, Logseq will present you with a bare-bone graph. Load your Org-roam notes by finding the graph button (with the database icon) in the side-panel and select “Add new graph”. Now pick the directory containing your recently cloned repository.

Once loaded and indexed, go straight to the configuration of the graph: “$\cdot\cdot\cdot$” → “Settings” → “Edit config.edn”. You want to modify/set the following fields, or at least take note of them:

:preferred-format :org ;; required!
:pages-directory "pages" ;; default
:journals-directory "journals" ;; default, must match `org-roam-dailies-directory`
:journal/page-title-format "yyyy-MM-dd" ;; match with `org-roam-dailies-capture-templates`
:journal/file-name-format "yyyy-MM-dd" ;; match with `org-roam-dailies-capture-templates`
:preferred-workflow :todo ;; recommended
:property-pages/enabled? false ;; recommended, disable property pages

You will need to review some of your configuration in Emacs, and maybe move some files around, if you want Logseq to handle them. A minimal corresponding Org-roam configuration to the above could look like:

(setq org-directory "~/Notes/"
 org-roam-directory (file-truename (file-name-concat org-directory "roam/"))
 org-roam-dailies-directory "journals/")
(setq org-roam-file-exclude-regexp "\\.git/.*\\|logseq/.*$"
 org-roam-capture-templates
 '(("d" "default" plain
 "%?"
 ;; Accomodates for the fact that Logseq uses the "pages" directory
 :target (file+head "pages/${slug}.org" "#+title: ${title}\n")
 :unnarrowed t))
 org-roam-dailies-capture-templates
 '(("d" "default" entry
 "* %?"
 :target (file+head "%<%Y-%m-%d>.org" ;; format matches Logseq
 "#+title: %<%Y-%m-%d>\n"))))

Pay attention to org-roam-file-exclude-regexp which should exclude the logseq directory. Once you have updated your Logseq (and possibly Emacs) configuration, don’t forget to push the changes to your repository.

Start exploring!
#

You’re now minimally setup to use Logseq and Org-roam together and you can start playing with them and see how it feels to take your Org-roam notes on the go with Logseq.

In subsequent posts, I’ll explain how I’ve configured Emacs in details, and present packages I’ve written to improve interoperability between Logseq and Org-roam.

My first post from org-mode

sylvain.bougerel@gmail.com (Sylvain Bougerel) — Tue, 20 Feb 2024 15:14:00 +0800

This post was exported from Emacs org-mode (with org-roam) using ox-hugo.

Yes, that’s all there is to that post. But I’m gonna leave it there for keepsake.

Programming a fast integral sine

sylvain.bougerel@gmail.com (Sylvain Bougerel) — Mon, 12 Nov 2018 18:28:00 +0800

I started playing puzzles on codingames.com recently. Although many challenges can be solved by beginners, getting to the top spots on the ladder for the toughest challenges demands a lot of time investment. Which makes it a great game.

The game I focus on is Mad Pod Racing, it plays in an arena where your bot meets another bot. Your bot has a finite amount of time for each move. This became an excuse to look into approx fast integer operations. I don’t really need to, but when I started looking at integer computations for Sine, I was swallowed whole into the rabbit hole. What could be more pointless fun than that?

I started to dig into GNU’s $\sin x$ implementations before embarking on my journey here. The implementation is complex and is likely the product of long research. Towards the end of this article, I give a brief overview of the GNU libc IEEE754’s implementation of Sine, should you want to get into it. But that didn’t really stop me:

I’m gonna write a simple integer version of $\sin x$ and it’s gonna be so much faster than the precise, floating point equivalent of the standard library.

Me, in a moment of pure hubris

Computing the result of such function efficiently crosses into 3 different domains: approximation, range reduction and a good understanding of what your architecture can or cannot do. Also, I was determined to waste my weekend (And most of the week — as it turned out) on this. Such an exercise is generally futile however, as the standard library implementation is already very quick.

Function Prototype
#

Mad Pod Racing is a hard challenge about predictions of mechanics of motion and collision. Your bot interacts with the world through standard input/output with angles expressed in degrees, using whole integers (e.g. 0, 45, 90…). Therefore, I will focus on this input domain only (whole integers expressed in degrees).

Figure 2: For angle α, the sine function is the ratio between the length of the opposite side and the hypotenuse of the right triangle (source)

My version of $\sin x$ is going to provide arbitrary precision, set to match the length of the hypotenuse without overflow:

int isin(int angle, int precision);

Also, I want my function to take a large input range for the degree of the angle, so I don’t need to worry about adding angles together and having to perform range reduction or modulo before using it. This is enough to meet the demands of the challenge, without compromising too much on precision. Precision is also bounded since the arena’s size for Mad Pod Racing is in the range [$2 \cdot -10^4$, $2 \cdot 10^4$].

Range reduction and sine’s symmetries
#

Fortunately, the sine function has multiple symmetries and we don’t actually need to tackle the entire input range:

Figure 3: The four quadrants of the Sine function

First, sine has a symmetry around x = 0. This allows us to focus on positive angles only. If the input angle was negative, we simply negate the result as well:

int isin(int angle, int precision) {
 const int slide = sizeof(int) * 8 - 1; // 31 for 32-bit int
 int s = angle >> slide; // sign mask: 0 or -1
 int aa = (angle ^ s) - s; // absolute angle
 // ... rest of the computation for val
 return (val ^ s) - s; // negate val, same as "val * s"
}

The first 3 lines need a bit of explanation. In C++, >> translates to SAR or the arithmetic bit shift to the right which carries the sign bit with it. This means that if an angle was positive then s=0x00000000, but if it was negative then s=0xFFFFFFFF or -1. This is the same principle as abs(), which I needed to decompose to keep the mask s to restore the sign to the result:

inline int abs(int a) {
 // simple abs() implementation
 int t = a >> (sizeof(int) * 8 - 1);
 return (a^t) - t;
}

Sine also has 4 quadrants, 2 halves with a period 2π; all of which are powers of 2. Thus, if we divide the input angle by a quadrant, or 90 degrees in our case, we can suddenly take advantage of the bit representation of the quotient. The symmetry between the first and second quadrant requires that we subtract the angle by 90:

int isin(int angle, int precision) {
 const int slide = sizeof(int) * 8 - 1; // 31 for 32-bit ints
 int s = angle >> slide; // sign mask: 0 or -1
 int aa = (angle ^ s) - s; // absolute angle
 int Qh = aa / 90; // quotient quadrants
 int Qr = aa % 90; // reminder quadrants
 int q = (Qh & 1) ? 90 - Qr : Qr; // first quadrant symmetry
 // ... rest of the computation for val
 return (val ^ s) - s; // negate val, same as "val * s"
}

The last symmetry to exploit is that between the 2 halves. If we find ourselves in the top half of the trigonometric circle, the result is positive, otherwise it is negative. This is similar to the symmetry around x = 0, we just need to XOR the test for both symmetries:

int isin(int angle, int precision) {
 const int slide = sizeof(int) * 8 - 1; // 31 for 32-bit ints
 int s = angle >> slide; // sign mask: 0 or -1
 int aa = (angle ^ s) - s; // absolute angle
 int Qh = aa / 90; // quotient quadrants
 int Qr = aa % 90; // reminder quadrants
 int q = (Qh & 1) ? 90 - Qr : Qr; // first quadrant symmetry
 int h = -(Qh & 2 >> 1); // semi-circle (half) mask: 0 or -1
 // ... rest of the computation for val
 s = h^s; // XOR half and sign mask together
 return (val ^ s) - s; // negate val, same as "val * s"
}

This is the naive skeleton of our function. We only need to compute the value of sine for the reduced interval [0, 90] now.

Approximating Sine with Taylor Series
#

At first I decided to attempt approximation with a third order Taylor series $T_3$ for sine:

\begin{equation} \sin x \approx T_3(x) = x - \frac{x^3}{3!} \end{equation}

This approximation is very imprecise: up to 1% error in the range [-π/4, π/4] ([-45, 45] in degrees) and errors accumulate even quicker beyond π/4. It is possible to find extra curve-fitting factors by solving a system of equations to force the point π/2 through the value 1.0: it’s already been done so I won’t go through the details. This is the resulting equation:

\begin{equation} \sin x \approx S_3(x) = \frac{4}{\pi} x - \frac{4}{\pi^3}x^3 \end{equation}

Since we are working with degrees, we can find a related solution that forces the point 90 through the value 1.0 with the same method, let’s call it $S_{d3}$:

\begin{equation} \sin x \approx S_3(x) = ax - bx^3 \end{equation}

\begin{equation} \begin{array}{ll} S(90) &= 1 = 90 a - b90^3\\ S\prime(90) &= 0 = a - 3b90^2\\ \end{array}\rightarrow\begin{array}{ll} a &= \frac{3}{2 \cdot 90}\\ b &= \frac{1}{2 \cdot 90^3}\\ \end{array}\\ \end{equation}

\begin{equation} S_{d3}(x) = \frac{3}{2 \cdot 90}x - \frac{1}{2 \cdot 90^3}x^3 \end{equation}

To avoid sacrificing readability, I kept the form as is. This curve seems to fit sine closely when eyeballing it:

However, even this approximation is still imprecise across the range [-90, 90]. Near the 40 (or -40) degree value for the angle, the chosen approximation’s error is at its maximum (≈ 2%).

On top of that, we need to deal with overflows (and avoid underflow) in the calculation when factoring the scale. The naive implementation limits the actual range of inputs for the precision; in fact it overflows as soon as scale $>2^{10}$, which does not cover the entire desired input domain. So we will compute the solution in the int64_t domain. We can also rearrange the operations, to minimize the numbers of divisions. This time, with the scale (or hypotenuse) H:

\begin{equation} S_{d3}(x, H) = H (\frac{3}{2 \cdot 90}x - \frac{1}{2 \cdot 90^3}x^3) \end{equation}

\begin{equation} S_{d3}(x, H) = H \frac{1}{2 \cdot 90^3} x (3 \cdot 90^2 - x^2) \end{equation}

Now there is no issues with underflow or overflow, and no compromise on precision in the entire range of scale:

inline int isin(int angle, int scale) {
 const int slide = sizeof(int) * 8 - 1; // 31 for 32-bit ints
 int s = angle >> slide; // sign mask: 0 or -1
 int aa = (angle ^ s) - s; // absolute angle
 int Qh = aa / 90; // quotient quadrants
 int Qr = aa % 90; // reminder quadrants
 int q = (Qh & 1) ? 90 - Qr : Qr; // first quadrant symmetry
 int h = -((Qh & 2) >> 1); // semi-circle (half) mask: 0 or -1
 int64_t a = (q * scale);
 int64_t b = (3 * 90 * 90 - q * q);
 int64_t c = a * b / (2 * 90 * 90 * 90);
 int val = c;
 s = h ^ s; // XOR half and sign mask together
 return (val ^ s) - s; // negate val, same as "val * s"
}

There is still the residual error of the approximation itself. My naive implementation is also easily outclassed by implementations that consider power of 2 units for the circle instead of sticking with degrees or radians. I might re-write the above in the future, however the cost on precision should still be significant. Simply put, there’s no free lunch: we can’t have both precision and speed. Or can we?

Why not lookup everything?
#

Given how small the integer range is across the period (we’re talking only 180 values for the semi-circle afterall), could I not pre-compute everything? I could fix the precision since the input for the hypotenuse is bounded in the range [$2 \cdot -10^4$, $2 \cdot 10^4$]. Fixing the precision to $2^{16}$ allows me to do simple bit-shifting, after the multiplication by the hypotenuse.

constexpr const double sin_lut[180] = {
 0, // 0
 0.01745240643728351, // 1
 0.03489949670250097, // 2
 // ...
 0.06975647374412552, // 176
 0.05233595624294381, // 177
 0.0348994967025007, // 178
 0.01745240643728344, // 179
};

The first step consists in creating a table of pre-computed sine values for each input degree. This table is not our final lookup table, but helps with the computation of the actual lookup table (LUT), which has the fixed precision, SinTable180points<P>.values:

template <int P> struct SinTable180points {
 constexpr SinTable180points() : values() {
 for (auto i = 0; i < 180; ++i) {
 values[i] = sin_lut[i] * P;
 }
 }
 int values[180];
};

Thanks to C++’s constexpr, we can ensure all this is done at compilation time. The new sin_lut() fixed precision function removes the need for any calculations and is as precise as needed for the entire scale of the domain (and we can easily extend it):

inline int isin_lut(int angle) {
 // Static table with precalculated outputs
 static constexpr const auto table = SinTable180points<2 << 15>();
 const int slide = sizeof(int) * 8 - 1; // 31 for 32-bit int
 int s = angle >> slide; // sign mask: 0 or -1
 int aa = (angle ^ s) - s; // absolute angle
 int h = aa / 180; // quotient semi-circle
 int ha = aa % 180; // reminder semi-circle
 h = -(h & 1); // semi-circle mask: 0 or -1
 s = h ^ s; // XOR semi-circle and sign together
 return (table.values[ha] ^ s) - s;
}

Was it worth it?
#

When micro-benchmarking all 3 approaches std::sin, isin and isin_lut, I get the following results on my machine, which should be easily reproducible. All benchmarking is done after warm-up by computing over 1440 fixed points multiple times and averaging the results:

function	Num. cycles for 1440 points	Avg. cycles per calculation
`std::sin()`	89604	62
`isin()`	29906	20
`sin_lut()`	17252	11

The tests attempts to make the compare each function as closely as possible by using the same precision when possible. However we should realise 2 things: std::sin() and sin_lut() do not compromise on precision, only isin() does. It’s quite remarkable, therefore, that the lookup table version is so quick and yet uncompromising. Nevertheless, I was impressed by how fast std::sin() is.

This entire exercise was personally educational but ultimately unnecessary for the needs of the challenge. The standard library implementation is already fast enough, and I have a better appreciation for it now. There is evidently much more I can squeeze out of the approximation (I worked on this without much prior knowledge) and maybe I’ll return to it in a future post.

The source code for this project can be found here.

Posts on sbgrl.me

Documentation performance, humans and LLMs

The documentation performance metric #

Beware of Goodhart’s Law #

"In Australia, a Home Battery Boom Has Helped Cut Wholesale Power Prices in Half"

Open-source takes on AI

Human Emacs #

Codeberg #

GNU Compiler Collection #

Kubernetes #

Linux #

Where do I stand? #

The end of tech-debt?

Setting the stage #

Ralph to the rescue! #

Then reality caught up #

The trouble with the process #

Announcing autosync-git to replace autosync-magit

Migrating #

What changed #

Why I continue to review code

AI exacerbated the divide between engineers

Claude Code more than doubled my productivity

Should you use React Strict Mode in Next.js?

Enter the rabbit hole #

The reasoning behind Strict Mode #

False positives with Strict Mode #

Beyond Strict Mode #

Interop makes my day joyful

Introducing @sbougerel/eslint-plugin-next-use-client-boundary

Don't abuse 'use client'

3 rules for simple tenancy in DynamoDB

Access control in DynamoDB #

All partition keys have tenant #

No SCAN #

Get tenant from the request only #

How does it work? #

Limitations #

Yodarabbit

Pino log instrumentation with Next.js 16

NextJS revalidateTag won't work if it's not called in an awaited promise

Private Language Tools for browser spell checks

Tips for technical interviews

Growth mindset #

Focus on the technical #

Balance breadth and depth #

Research! #

Quantitative over qualitative #

Divide and conquer generalities #

Structure of the system design interview #

Practice #

Remember to smile #

Sometimes, it’s better to ask #

Get past your impostor syndrome #

Data transfer ladder #