Gary Moore's Website

__ __ ___ ___ .-----.---.-.----.--.--.| |.-----. .-----.| |_.--.--.' _|.' _| | _ | _ | _| | | |_||__ --| |__ --|| _| | | _|| _| |___ |___._|__| |___ | |_____| |_____||____|_____|__| |__| |_____| |_____| --- Cursed Kata Idea ---------------- So there's these programming languages with popular programming exercises that are modelled after fixing the broken program by changing a few lines, like Ruby Koans, Ziglings, Rustlings etc. What if there was a cursed version of this? The langauge would have to be something like C or Zig. The idea would be to "fix" programs like the one in the "Cursed Code Snippet" article I wrote earlier. Could remove the mprotect stuff and then be like, "why does this not work?", "That's right, you can't execute code in a page unless it's read only." There would be hints of course, maybe just a link to some syscall docs. What other "cursed" programs are there that kind of provide some insight into the OS/userland boundary? Worth a think about I think. Tugging on this rope a bit more... yeah maybe it doesn't have to be "cursed" necessarily. Just examples that inform the user about something they wouldn't expect with regards to programming. --- BSides Belfast 2026 ------------------- Managed to get a ticket at the last minute for BSides 2026, the refreshing thing about this conference, it has a much, much lower density of talks oriented around AI compared to conferences focused on programming specifically. It's a bit of a breath of fresh air. Here's the talks I'm hoping to have a look at: 1. Reverse Engineering an OAuth Supply Chain Attack 2. No CVV, No 3DS, No Problem 3. Bedtime Bug Hunting: A Friendly Android Auth Bypass Story 4. The Silent Risk in Your Browser: Detecting Malicious Extensions 5. Communicating with QR Codes The last keynote of the day is called "AI is Here to Guide Not Decide", and I'm very much on the fence wrt whether to attend that one. I am completely at a wits end with AI talks. On the oauth one at the start there, it's the big Vercel one that happened recently. It's a bad sign when you think "which one?" This particular talk is about the OAuth one. I think it would be cool to hear a breakdown of the React Server Component one though, it's like we didn't learn the lesson from Java Serialization. The CVV one, I don't know that much about it, so a good candidate for learning something new. It was competing with a talk on a Kubernetes honeypot, which honestly doesn't sound that interesting to me. Malicious browser extensions, yes I'm interested in this one. If you've ever written a browser extension, you'd realise what a clown show all that is. Especially the manifest v2 stuff, you're trusting an extension vendor with everything you view on the web, everything you type in, pretty much it all. Communicating with QR codes, there's some interesting things here. For example using the Game Controller API to bypass browser permissions dialogs. Seems neat, and covers more than just QR codes. I don't understand how you could use them to exfiltrate data across an airgap, I assume you need physical access to a machine with a screen or something? Yeah that interested me, so let's see how that works. The AI one. I don't think it'll tell me anything I don't already know. Think I might skip it. So those are my picks, let's see how it goes :) I'll try to write about what I found particularly interesting. Talk Thoughts ~~~~~~~~~~~~~ Reverse Engineering an OAuth Supply Chain Attack Art gave a great talk on the recent-ish breach caused by a third party app integration at Vercel. Specifically involved an employee giving access to their google account with full permissions to sort sort of AI company, and the attacker using that token to read the users email, and thus the magic link sent to their email to log into vercel. The really neat thing, Art produced a sort of parallel copy of the infrastructure to demonstrate the likely approach used. Claude would refuse to produce the app unless it was spelled out in very bold letters that the website is backdoored or something. Kind of funny, as if this couldn't be changed after the fact manually. No CVV, No 3DS, No Problem Really interesting talk about a catastrophic chain of bad engineering with some airline, probably a US airline this be real. The endpoint was not checking the validity of the signatures on the JWT token. You could just delete the signature, replace the payload, and it just works. 200. There's four steps after the chain with pretty catastrophic failures at each step resulting in the ability to book airline tickets with someone elses card, with practically zero information required beforehand. Kind of scares me off putting my credit card information into the ropier airlines, or websites in general. Glad I use disposable virtual cards. The CVV was not checked at all, and 3DS was bypassed because the account was considered trusted. I'm not a cybersecurity professional, I can't perfectly recall the details of this. --- Cursed Code Snippet ------------------- Wrote this shitty little nugget a day or two ago. It's kind of interesting in that it illustrates basically what a JIT compiler does with virtual memory. Here, we define a function using an array, and call the array: #+feature dynamic-literals package main import "core:fmt" import "core:mem" import vmem "core:mem/virtual" Collatz_Step :: proc "c" (x: i64) -> i64 main :: proc() { arena: vmem.Arena _ = vmem.arena_init_growing(&arena) context.allocator = vmem.arena_allocator(&arena) instructions := [dynamic]u32 { 0b1_00_100110_1_000001_111111_00000_01000, 0b1_0_0_01011_00_0_00000_000001_00000_01001, 0b1_11_100100_1_000000_000000_00000_11111, 0b1_0_0_11010100_01001_0000_01_01000_00000, 0b1101011_0010_11111_000000_11110_00000, } vmem.protect( rawptr(arena.curr_block), uint(mem.PAGE_SIZE), {.Read, .Execute} ) collatz_step := transmute(Collatz_Step)raw_data(instructions[:]) fmt.println(collatz_step(12)) // 6 } The underscores in the instructions are field boundaries. Makes it a bit easier to see the structure in the instructions. The magic trick here, all arm64 instructions are 4 bytes wide so we can just wing them into an array. We use a dynamic array here which will use whatever allocator is defined in context. We use an arena allocator backed by virtual memory. After implicitly writing to the first page of the arena by defining the array, we set that page read only and executable. On macOS where I'm running this, it has a W^X scheme for executable memory. Fun fact, MIT/GNU Scheme doesn't support apple silicon for this reason[0]. Then we just cast the array and call it lol. Really this is just what JIT compilers like V8 do all the time, but I thought it was neat to define the function using array syntax. vmem.protect() is basically just a wrapper around the equivalent mprotect[1] procedure in the C stdlib. Well syscall really. [0]: https://www.gnu.org/software/mit-scheme/ [1]: https://man7.org/linux/man-pages/man2/mprotect.2.html --- ASM of the Week: csel ---------------------- It's Saturday already?! Ok, I'll give this a go before I forget. I'll attempt to write about a single assembly instruction every week. I will probably fail at this, but hopefully at least get a few ones I find interesting out there. I think I'm probably using this one up too soon because it's so good :-) ~| csel Xd, Xm, Xn, cond |~ - Xd is the destination register - Xm and Xn are the left and right registers. cond is one of the following flags binary cond condition ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 0000 EQ equals 0001 NE not equals 0010 CS carry set 0011 CC carry clear 0100 MI minus / negative 0101 PL plus / positive 0110 VS signed overflow 0111 VC no signed overflow 1000 HI unsigned higher 1001 LS unsigned lower or same 1010 GE signed greater than or equal 1011 LT signed less than 1100 GT signed greater than 1101 LE signed less than or equal 1110 AL always 1111 NV never (historical name) What does it do then? It's basically a branchless comparison, if you write a ternary in C, there's a good chance the compiler will turn it into this instruction, or a helpful related one like csinc (conditional select increment). On modern computers it's often faster to compute either side of a branch and just use an instruction like csel to save either the left or right side based on the condition. You avoid rolling the dice with the branch predictor that way, which can be handy in a tight loop as you don't risk flushing the instruction pipeline. For an example, let's look at an alternative to the naive implementation of a collatz step from a previous post here: collatz_step :: asm(n: int) -> (r: int) [ n -> r, even_result: int, odd_result: int, odd_mul_by: int, odd_add_by: int, #clobber flags, // tst still sets flags, even if branchless ]{ mov even_div_by, 2 mov odd_mul_by, 3 mov odd_add_by, 1 asr even_result, n, 1 // shift right, dividing by two madd odd_result, n, odd_mul_by, odd_add_by tst n, 1 csel r, even_result, odd_result, eq } > gary(todo): improve this explanation, it kind of sucks lol We set up our registers, then compute each side of the branch, the result you'd get if n was either even or odd. If it's even, we divide by two using a signed aware right shift. In a base 2 number system, reducing the power of the number is equivalent to a divide by two, in the same way going from 100 to 10 is a divide by ten. For the odd result, we need to get 3n + 1, so we use the madd instruction. This combines a multiply and an add into a single instruction. We then test if n is even or odd with tst, and use the csel instruction. This is basically your ternary, with the operand at the end being the comparison. That selects what result to save into the left most operand, and it doesn't involve the branch predictor. --- Web Dev Mini Rant ----------------- I don't think design is the greatest source of problems in a lot of web applications. Making everything CRUD via REST API where you have these serial dependency chains that cause a tree of loading spinners to kick off is the biggest problem. In Jira, why does my dropdown have a loading spinner? It's mad. I wish more companies were producing web apps that worked more like Linear, where you literally load everything relevant to the user in one go. Send down updates via server sent events or something. Use a sync engine, do what you need to do, but make all the data available to the user so they never have to wait on a server round trip. I wish there was a bit more creativity wrt where and how applications represent their data. If you think your app is unavoidably UI->API->DB and back, that might not be the case at all. There is often a more specialised shape for data in your domain that is more amenable to local processing with diff based syncing. --- Small Note on C# Discriminated Unions ------------------------------------- Had a look at the new discriminated union stuff C# and they're kind of disappointing. You don't get a key benefit you'd normally associate with tagged unions in a lang like C or Rust. Even if you create a union of structs, it's all by ref and an array of them is sprayed all over the heap. The only benefit I can see is explicit polymorphism, which is nice, but I'd have expected a union of structs in an array to have good cache locality. --- Odin's New Assembler -------------------- So I don't really know assembly, but Odin introduced their own take on inline assembly a few days ago and I've been farting about with it. They do some interesting things that I believe you don't really see in C or whatever. I have an ARM64 machine, so examples will use ARM64 assembly. Here's probably the most basic example of the syntax: double :: proc(x: int) -> int { return x * 2 } double :: asm(x: int) -> (r: int) [ x -> r, ]{ add r, x, x } I've also provided the equivalent odin procedure. So what's happening here? First thing, it kind of looks like a proc, but you declare some additional metadata between the square brackets. double :: asm(x: int) -> (r: int) [ Only thing notable about the signature vs a proc, if you're not familiar with Odin it would be the named return value. Naming it is important for the next line. x -> r, This is called a tie. here you're telling odin that it should use the same register for both x and r. In this case that would be x0 on arm64. ]{ add r, x, x } If you're used to x86_64 assembly, this might look odd. Arithmetic on ARM64 takes three registers rather than two. Another thing worth noting, arithmetic on ARM does not support immediates in the instruction, only registers. So it's: add Rd, Rn, Rm Where Rd is the destination register, Rn and Rm are the operands. Here's Odin's sort of magic trick for it's inline asm implementation. SIMD is made really trivial with it's SIMD specific types, here's an add example again, where instead we double a vector of four unsigned 32bit ints. double_4u32 :: asm(nums: #simd[4]u32) -> (r: #simd[4]u32) [ nums -> r, ]{ add r, nums, nums } Ok but how do you do register allocation and branching? Here's a pretty naive implementation of a collatz step: collatz_step :: asm(n: int) -> (r: int) [ n -> r, mul_by: int, add_by: int, div_by: int, #clobber flags, ]{ mov mul_by, 3 mov add_by, 1 mov div_by, 2 tst n, 1 b_eq .even // odd madd r, n, mul_by, add_by b .done .even: sdiv r, n, div_by .done: } This is a bit more involved, but still a pretty trivial example. Here's what's new: [ n -> r, mul_by: int, add_by: int, div_by: int, #clobber flags, ] Here's that prelude again, but there's more than just a tie now. We declare what variables we will be using, and Odin automatically does the register allocation for us now. You can also use this syntax in order to explicitly assign a register yourself: mul_by: int = %x1, We're going to be using a branch in this routine, so we use #clobber flags which tells odin we're modifying flags. That's opaque to the compiler, and this is information that's useful to it, I'm not exactly sure the mechanics to be quite honest. Will have to learn more about it. mov mul_by, 3 mov add_by, 1 mov div_by, 2 Here we just set up the registers for the arithmetic operations that we will be doing later, because like I said arm64 arithmetic doesn't support ops with immediates (that being sliterals like 1, 3 etc. tst n, 1 b_eq .even This is our branch, first we test to see if the number is even, and jump to the even label if that's the case. The "tst Rn, op2" instruction is actually an alias to "ands ZR, Rn, op2" instruction, which is a bitwise AND that also sets the Z flag (result is zero). We're basically checking the least significant bit to check if the number is even rather than whipping out a mod. Gotta love base 2 :-) The rest is just arithmetic, with a jump to done in the odd case. ~So my review~ I can't give a review really, but this getting introduced to the language I use daily for side projects, it's gotten rid of the mental roadblock for me to learn an assembly language, so for that alone I am a b i g fan ( ͡° ͜ʖ ͡°) Plenty of things I have not touched on at all here, this is just a progress report of a beginner. The ties, automatic register allocation, and experiencing assembly without the ugly [sp, displacement] stuff everywhere, it's been nice. I'll try to keep writing about my progress here, haven't really touched on memory access from inline asm and probably other topics I've missed. Maybe an instruction of the week mini post? Hmm. Thanks for reading! --- Migration from SourceHut ------------------------ I've migrated from SourceHut to my own personal Forgejo at git.mooremoore.net. There's a few reasons for this, the primary one is the SourceHut seem to want to tell you how you should do your work, even though you're paying for a service. I'm going to miss the mailing list stuff, so might try and whip up something with GNU mailman or something eventually. I had to update the links on two articles here, so the atom feed may bounce about a bit, not actually sure how that will go. Apologies! --- Atom Feed Support ----------------- I already consistently use the same title and article separator in this plain text file I'm writing, so just split along them, construct atom feed entries, slap in XML and throw onto bunny.net. This post is a test to see if it works. :) :( update ): Yes those are fowny faces. I have been rightly called out for bad timestamps. Fixing it required being a bit smarter about parsing, so I now parse out the articles from the format you see on the website, get a hash of the content, and store that in a wee state file so we can see if articles are clean or have been dirtied. Dirty articles get a new timestamp, non-modified, clean ones stay with the same one. In theory, anyway. Just realising today being sloppy about timestamps will mess up others feeds, I don't want that 😬 Let's see if this works, only this article should get a new timestamp. :(( update 2 )): Sorry, that caused every article timestamp to update again. That was because the state file didn't exist the first run. Should be fine now (I hope). Checked there, yes that did it. Sorry about disturbing the force, shouldn't happen again. --- toe update ---------- Just did a final bit of plank sanding, replace malloc/free usage with scratch space, and just general tidying up. Removed curl callbacks that really didn't need to be there, probably the most consequential change. --- Unusual Youtube Content ----------------------- This channel is so niche and out of the ordinary that I just felt I should comment on it lol. There's a guy in Czechia who waits until there's severe storms then goes to explore various storm drains around the country. This is the sort of niche content I can get behind! xD Example: https://www.youtube.com/watch?v=GQtzYgH8buc --- Website Next Steps ------------------ Just a bit of tidy up left to do on this site. For the contact card at the top right here, I want to just use a css grid to give the key/value pairs some nice alignment. The <pre> I'm using for this plan file, that's gotta go. It can cause some issues when using things like angle brackets. We should just slap this in some other tag and set it to display a monospace font with css. Should work grand. ~ update ~ That should be those changes in there now :) I also added HTML escaping so writing <pre> doesn't actually render that as a real <pre>. Should hopefully be nice and firm and solid. --- Site Performance Improvements ----------------------------- I threw together this site very quickly, and the templating for it is very inefficient. Here's basically what happens currently: - download plan from storage - stream to a string - template and string interpolate - return response This happens for every response, and the resulting latency is ~400ms from where I am. I shouldn't really be doing SSR for that templating step, I should just just render an index.html from the file watcher and serve that static asset directly from a bucket. Will have a look into this tomorrow, should hopefully bring down latency significantly from nearly half a second to hopefully ~30ms on my connection. ~ the next day ~ I've implemented the changes. We now just template the index.html file in the file watcher and push the fully formed thing to bunny.net. We can just expose that storage bucket directly, it's brought latency down from typically over 400ms to ~40ms, so that's a 10x improvement. Nice :) --- A Million Tracks with Instant Startup ------------------------------------- Recently I've been working on a music player app for myself. For now it only really plays .flac files using miniaudio, and the user interface is just a giant list of tracks. I've been using it as a test bed for different ideas around how the data for a music player should be structured though, and that's a focus on completely avoiding the use of pointers. Without pointers, that means any reference to a piece of data is actually a stable offset into a buffer. That makes the story around persistence really, really easy. Application state is just a file memory mapped into your applications virtual address space. This is essentially the core data structure of the app: tracks: #soa[]Track Any strings in a column of a track is a handle into a string interners []u8 buffer, so we can persist those too no problem. With all state represented using stable indices/handles into buffers, there is absolutely zero serialisation and deserialisation cost, and no impedance mismatch between persisted state and runtime state. That means your application is really just a window into application state on disk, and the OS only pages what's on screen into real memory. If your application is just a window that hangs on state that's already in it's final form on disk, opening the application is really just initing the UI and hooking into that state, and that means you can open a 1,000,000 track library in a few 10s of milliseconds. Very happy with the result, doing interesting things with search too. Will write about that at some point soon hopefully :) --- WASM SIMD in C Compiled With Zig -------------------------------- So I've been writing a bit of code for my JS string interner, and I've set a hard rule for myself to only use JS. Really bumping into some sort of limits there though. I'm talking about prefix search. The naive way to find strings that match a prefix is to do a byte by byte scan. A better way is to pack multiple bytes into one number and use bitmasks to match. That's effectively what I went with, this is called SIMD Within a Register (SWAR). You kinda have your hands tied behind your back from the start doing this in JS though, as JS engines only support floats at 64 bit width. You're stuck with 32 bit integers. We scan 4 characters in a single instruction, that still gets us 4x the performance over the simple byte scan in V8. Still, I was curious what you can do these days with WASM, which now supports 128 bit wide SIMD. That maps pretty much perfectly to my Mac's NEON instruction set for SIMD. ~|update|~ WASM SIMD does in fact not map pretty much perfectly to NEON SIMD instructions. Many operations require a few instructions and may involve things like swizzling things about. You pretty much need a spreadsheet open to see what does and doesn't work well for your ISA, and there's no fused multiply adds without what amounts to an unsafe flag. I'm glad this exists, but it's not fabulous. ~|------|~ Initially I tried just writing the weird Sexpr .wat stuff directly. It's relatively straight forward to do so. Still, wasn't the most pleasent thing in the world to write. The reason I avoided writing it in Zig is because it's not at all stable. I didn't write it in C because I didn't want the emscripten headache. Then it dawned on me, Zig is a really good C cross compiler. So I ended up just writing the code in C with WASM SIMD intrinsics, and cross compiled with Zig. It worked out well! And Clang wrote better WASM than I could. I'd definitely recommend trying this combo, WASM + C + Zig. Absolute power trio. --- Allocators in C --------------- Just finished my arena allocator, it works by using mmap to overcommit 4GB of virtual memory and when you reset the allocator, the pointer just gets set to 0 and we madvise MADV_DONTNEED. Seems to work well so far! :) It also can return an Allocator struct which defines an interface, so we can have functions just take that interface. We also set the last page to PROT_NONE so it acts as a guard page. If anything in the application tries to write into that page, the app just crashes. Gives us some basic protection for free, the CPU mem unit handles it. Here's how the memory is initialised, and the guard page set: Arena *arena_init() { void *mem = mmap(NULL, FOUR_GB, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); if (mem == MAP_FAILED) { return NULL; } long page_size = sysconf(_SC_PAGESIZE); void *guard_page = (char *)mem + FOUR_GB - page_size; mprotect(guard_page, page_size, PROT_NONE); Arena *arena = malloc(sizeof(Arena)); arena->mem = mem; arena->cap = FOUR_GB; arena->offset = 0; return arena; } Just point to the start of the last page, pass in the page size of your platform, PROT_NONE, and it's off to the races! Important to programmatically get the page size rather than hard code it as some popular platforms these days (Apple Silicon) use a 16K page size rather than 4K. Also, here's how the memory is cleared, and it's probably the main thing that makes memory arenas so powerful: void arena_reset(Arena *arena) { if (arena == NULL) { return; } madvise(arena->mem, arena->offset, MADV_DONTNEED); arena->offset = 0; } The important thing here, we just reset the offset to 0, and we can reuse all that mem. A sort of interesting thing here is the use of madvise to tell the kernel that we don't need this memory anymore, so the kernel can go ahead and reclaim it. Kind of on the fence about the madvise in there. It sort of defeats the point of the arena, adding a syscall in there is kinda heavy, and where arenas are used, memory usage before a free tends to reach a similar high water mark. Hmm, now I'm thinking about removing it. Update: I would not recommend using madvise in the arena reset function. --- Gabe Deck --------- I've been playing games a good bit on the Gabe Deck[0], and it's got a really unique control scheme. I've been having a mess with Raylib recently in order to get a bit better at C programming[1], I might have a look at integating Steam Input to see what cool stuff you can do with the track pads and gyro. [0]: https://store.steampowered.com/steamdeck [1]: https://git.mooremoore.net/gmoore/game-playground-raylib --- Automatic .plan Upload ---------------------- This is my first proper automatic upload of my .plan file using my kqueue based watcher[0]. Should use zero CPU while not in use, and I can write these plan files by just editing the file in neovim or whatever editor, and on save it'll automatically get uploaded to my website. I'm not sure how long the upload takes, but it's surely < 500ms. By the time I save and alt+tab refresh my browser the content is there already. We're using bunny.net and completely moved away from cloudflare. This website is just a template on the bunny.net equivalent of workers, and this plan just gets uploaded to their object storage. On request, the template gets filled in from object storage, so the way we update this plan is actually rather simple. 1. Edit .plan 2. Save 3. Watcher picks up the change 4. Watcher uploads to bunny.net object storage 5. Future requests to the worker with my site automatically picks up my changes Lovely! Personal micro-blogging doesn't need to be so complicated :) Just had another thought there, this is a pretty neat thing for covering live events. Might want to somehow send updates via server sent-events or something, but on the editor side, seems great for that. ~ update ~ I've cut out the bunny.net worker step entirely and I just template in the worker and upload the fully formed HTML to bunny.net storage. The final result is a much faster experience when visiting the site. Down from over 400ms to ~40ms, so that's a good result :) [0]: https://git.mooremoore.net/gmoore/toe --- Blog Publishing Platform ------------------------ So I have an idea for this, something similar to what I've done already for publishing to my gemlog on the gemini protocol. That is, why not just send the blog post to myself as a signed email? Already got a decent bit of the code, just need to flesh out HTML templating vs the much simpler gem format. That's what I think I'll do eventually in order to have a platform for the below article series. There's one important deviation compared to how I did it before, though. Instead of the silly stuff with the git repo and SourceHut build step as before, I'll just push to a bucket, and do something with a bunny.net script. --- "Low-Level" JS article series idea ---------------------------------- So this is something that's been floating about in my head for a while now. There's some interesting performance topics that I haven't really seen covered much with regards to JavaScript, and I suppose that's manual memory management with typed arrays. There's some specific probably niche topics here: 1. Packing low cardinality information into bitfields 2. Discussion on deopt pitfalls wrt polymorphic func parameters 3. SIMD within a register, specifically in u32s 4. Buffer backed string interning for cache locality 5. Eliminating GC churn with fixed slots and freelists 6. Improving instruction level parallelism with loop unrolling And more topics as I think of them I suppose. Before doing a wee series on JS perf, I suppose I should build out some way to publish blogs on my personal site. I already publish to the gemini protocol because the LLM crawlers don't go looking there, but I've got this .plan thing here now, and I think I can borrow the kqueue stuff for a really low friction CMS-ish thing.