twocentstudios

Print Design Mindset to App Design Mindset

Thu, 29 Jan 2026 14:26:08 -0600

The print design era

There was a time when iOS app design wasn’t all that far off from print design.

For the first couple years of the iPhone SDK (2008-2011):

All iPhones had 320 width by 480 height screens.
All iPhone screens were @1x non-retina.
The platform design language was highly skeuomorphic and rewarded delicately crafted Photoshop assets.
iOS did support localization but not for RTL languages and the launch country list was relatively English-speaking (and it was never required for App Store listing).
Accessibility features that dynamically altered app content visually did not exist.

iPhone, iPhone 3G, iPhone 4, and iPhone 5 — all 320 points wide

Sure, there were dynamic elements like buttons with highlighted, disabled, and selected states. Sure, there were scrolling screens. But the overall workflow of app design to app development was taking a Photoshop mockup and recreating it piece by piece in the SDK.

Until Sketch gained some prominence in the 2013-2015 era, Photoshop was still the most popular tool for less technical designers to work in before artifact hand off.

Sketch app showing iOS device artboard presets including iPhone 6, iPhone 5, and iPad sizes

The first cracks in the print design workflow surprisingly started with the sudden differentiation between points and pixels with the iPhone 4 in 2010. Even then, that didn’t necessitate a mindset change away from print design, just a few configuration changes in Photoshop and some extra work come asset export time.

It wasn’t until the iPhone 6 in 2014 that designers (and developers) could no longer assume devices had the same 320 point width. As an iOS developer, you could get away with hard-coding dimensions for your UIViews for a very long time! (Assuming you supported only landscape OR portrait orientation).

iPhone 6 (375pt), iPhone X (375pt), iPhone 13 (390pt), and iPad Air (768pt)

And if varying screen widths weren’t enough, the iPhone X in 2017 broke the assumption that the screen was even a simple rectangle. The notch, rounded corners, and the home indicator meant designers could no longer treat the canvas as a predictable bordered frame.

The complexity catches up

I’ll skip to present day to say that we’re long past the point that most app designers can, with a good conscience, keep a print design mindset when doing visual design for mobile.

The complexity grew, and luckily for iOS developers, the tooling provided by Apple to rein in that complexity has slowly caught up:

For view layout, we have stacks, content margins, size classes, line limits, leading/trailing instead of left/right.
For Dynamic Type (user adjustable font sizes), we have semantic fonts like .body, .headline, .largeTitle. We have the @Scaled property wrapper in SwiftUI. We have UIFontMetrics for scaling non-system fonts.
For Dark Mode, we have semantic colors like .label, .background, .secondary, .blue, .tint, and asset catalogs.
For localization, we have auto-updating xcstrings files, Double-Length pseudolanguage, RTL pseudolanguage, asset catalog localization, plural strings, and localization aware formatters.

HIG semantic label colors adapt automatically across appearances

HIG system colors vary across light mode, dark mode, and increased contrast

HIG Dynamic Type defines seven non-accessibility size categories with semantic text styles

However, there’s a similar time lag (measured in years) between SDK support and true developer tooling support that exists between developer tooling support and designer tooling support for all these features. There probably always will be. Users will adopt them and demand them faster than our tools will support them.

The mindset shift

But going back to print design.

I love (well, maybe love/hate) designing apps, but what I love more is working with talented designers. There’s something about the design field that draws individuals with a tenacious eye for detail and an equally fierce demand for seeing their work reflected with pixel perfection. And most of the time, I would interpret this desire as driven by empathy for the user and not selfishness. They know users recognize and deserve beauty. In other words, their intentions are good.

The trap I’ve seen some of these designers fall into is assuming it’s enough to have empathy for a user and not all users of their designs. In the early iPhone era, 100% of users would see the same screen pixel-for-pixel that the designer created in their Photoshop document. In 2026, the percent of users that see a screen exactly as it was designed is ballpark closer to single digit percentages, if that.

There are simply too many variables that affect the final visual design a user sees when they open your app. Too many to optimize for pixel accuracy with a print design mindset. Dozens of screen sizes. Dozens of languages. Dark Mode. Near a dozen Dynamic Type settings. Bold text. Button shapes. Virtual home buttons. A virtual cornucopia of more esoteric accessibility options. And on iPad, not even fixed screen sizes thanks to Slide Over, Split View, and Stage Manager.

And that’s not even digging into the impedance mismatches baked into the design tools themselves: system fonts displaying a little bit smaller on device than in Figma; color spaces not matching; static assets losing fidelity during export.

I’ve worked alongside many incredibly talented designers over the years. Almost all of them sooner or later made the jump from print design mindset to app design mindset. It took a lot of frustrating (on both sides!) conversations.

At the end of the day, as a developer, I knew the device was the final arbiter of truth. The painstakingly crafted mockups only reflected reality for the brief period of time before the app went live on App Store, and after that they were vestigial colored rectangles, decaying with a half life in the order of days not weeks. The same way that multi-page product spec lovingly crafted by the product manager is lost to the abyss of Google Docs at the end of the sprint.

I knew the signs of the mindset shift, and they all came like manna from heaven. I’d start to receive mockups with loading states leading to the happy path. I’d get error messages with HIG-respectful copywriting. I’d get sidecar notes in the margins pointing out fixed and flexible dimensions. I’d get line limits for text. I’d see table cells with long and short user names. With empty and populated avatar images. The apps were starting to show signs of being apps before I’d actually turned them into living & breathing apps.

More importantly, I would stop receiving design review feedback about marketing copy not line breaking at the exact same words as the mockup. About using a custom alert view instead of the system one.

The designers were now channeling their intrinsic motivation and eye for detail that previously went into pixel perfection into ensuring the maximum number of users with any combination of device settings would still be able to use the damn app to get their task accomplished. And hopefully be able to appreciate the fact the app looked and felt nice while they were getting things done.

The inter-department friction would melt away. We’d ship faster even though we were supporting more design variables than ever before. After all, designers were taking on more of the work I’d previously been doing turning screen after lifeless screen into constraints and states and conditionals. I had more brain bandwidth to double check their assumptions, cover more edge cases, explore architectures that would keep the code from rotting, experiment with animations, and still have time to pair with them on polish. And even more-so, develop abstractions to make this work faster and easier in the future.

When you make the foundational work easy, you have more time left for whimsy. Which is more fun for both the app makers and the app users.

Making the shift

So how can a designer with a print mindset adopt an app design mindset?

Half the battle is simply recognizing the differences between the two. Giving each a name. Recognizing the signs and fighting your instincts for pixel perfection.

From there, I can offer a few suggestions:

Pairing with developers who care about accessibility, design systems, the human interface guidelines, and most of all, users.
Immersing in Apple-sanctioned resources like the HIG, WWDC videos, and learning how to search, read, and understand the developer documentation.
Immediately learning the avant-garde features of design tools that lag behind the native developer tools.
Following the unicorn designer/developer hybrid folks that work in and share wisdom from both sides of the aisle.
Gaining enough proficiency with developer (and AI?) tools to become dangerous (or just be able to update a color value or knock out a quick interactive prototype).
Visualizing apps not just as a collection of snapshots of happy path screens, but a rich journey of pushes, pops, loads, errors, mistaps, drags, interruptions, and everything in between.
Do the painful work of watching over a user’s shoulder as they tap through your app on their 8 year old iPhone with their settings and their cracked screen.

Cheers to many more happy and healthy designer 🤝 developer partnerships in the years to come.

Introducing Eki Bright 2.0

Sun, 18 Jan 2026 16:16:39 -0600

Eki Bright 2.0 is available today.

Eki Bright app icon

Eki Bright is the fastest way to access station timetables for the Tokyo metropolitan area.

Download Eki Bright on the App Store

The original design for the Eki Bright app evolved out of the initial use case of configuring Widgets, but steadily grew to support searching, nearby stations, DIY routing, and Live Activities.

For Eki Bright 2.0, the goal was to optimize the app for how I’d actually been using it and how its users had told me they use it.

Home screen evolution from v1.0 to v2.0

The main focus is now nearby stations, which I found to be 95% of my usage over the year since its first release. In many cases, you’ll see the exact station timetable you want in less than a second after opening the app. Other nearby stations are a quick scroll at your resting thumb position, and order based on view history and distance from your current location.

Nearby stations screen with scrolling

From here, it’s super easy to start a new DIY route by tapping your departure train time, then tapping the route icon of your arrival station.

Creating a DIY route by tapping departure time and arrival station

The route bar from v1.x has moved to the native tab bar accessory, which automatically expands and collapses.

Route bar in the tab bar accessory

DIY routes now have a full screen view in Eki Bright 2.0. You can see the full route and transfers, and view all stops along your route.

Full screen route view with expanded stops list on the right

It’s easy to compare adjacent departure times. For example, to check whether the next departing local train will actually reach your destination after a later departing express train.

Comparing local and express departure times, the express arrives 7 minutes earlier than the local that departs 3 minutes earlier

Eki Bright remembers your most used route patterns and suggests them on the route screen based on your current location. After a few trips, you don’t need to manually create DIY routes anymore.

Suggested routes based on location and history, departure times updating live

The Live Activity now updates more reliably in the background throughout your trip.

Live Activity on the lock screen

My personal favorite new addition to Eki Bright 2.0 is the station relative direction arrow indicators on the station detail screens. This makes it much easier to orient yourself with the correct direction of a railway at a glance.

Station detail with direction arrow indicators in relation to the center station

The station detail screen now shows train frequencies during different periods of the day.

Train frequencies by time period

The bookmarks and search screens have been separated out into their own tabs. I doubled down on native iOS 26 UI and UX (for better or worse).

iOS 26 liquid glass tab bar with native accessory for the route view

In the search tab, you can now search railways.

Railway search in the search tab

Speaking of railways, the railway detail screen shows the full polyline of the railway.

Railway detail screen before and after with full polyline

I think Eki Bright is the fastest and best way to get around Tokyo for everything but route planning. Local timetables give a serious edge to how fast you can find your next train.

Closing the Loop on iOS with Claude Code

Sat, 27 Dec 2025 15:37:01 -0600

Closing the loop means giving Claude Code a way to view the output of its work. I’ll be focusing on iOS app development workflows.

Step 1 of closing the loop: building a target so that Claude Code can see the errors and warnings. And doing so in a way that preserves the build cache (clean builds take a long time). This allows Claude Code to see its syntax errors and fix them before you review its work.

Step 2 of closing the loop: installing & launching on the simulator. This saves you the step of opening Xcode and hitting build & run, letting you test each proposed code change right away.

Step 3 of closing the loop: reading the console & log output. This allows Claude Code to proactively verify codepaths and reactively do debugging.

Step 4 of closing the loop: controlling & viewing the iOS simulator. This allows Claude Code to step through entire flows, evaluate visual designs, and generate its own logs.

Step 5 of closing the loop: building, installing, launching, and logging on device. This allows you and Claude Code to test Apple Frameworks that are absent or broken on the simulator.

Building, installing, launching on the simulator from Claude Code

Disclaimers before we start

Agentic tooling is changing rapidly with model and agent versions. I’ll cover each step as thoroughly as I can. The strategies in this post cover about a month of work in December 2025 with Claude Opus 4.5 inside Claude Code v2.0.76 (and several versions below). I used Xcode 26.1 and 26.2 on macOS 15.7.3 mostly developing for iOS 26.

This post is written for humans but can easily be adapted to a Skill or added to your CLAUDE.md file. The command structure will change based on how your project and schemes are set up. I outline a few different strategies that are useful in different situations, but you may only want to use one workflow as your default, or completely ignore certain steps altogether.

If you’ve always used a manual Xcode-based flow, trying to both understand and incorporate these steps into your workflow can be intimidating. If you’ve never working with Xcode via the command line before, start with just the first step for a while. The best part about this workflow is you can seamlessly dip in and out of using Xcode and there’s no switching cost (not even needing to do clean builds).

The below CLI commands also share a lot of coverage with XcodeBuildMCP, a more full-service MCP-based solution. I won’t get into the pros and cons of MCPs vs CLIs (its author has already written about that).

I’m specifically targeting this post to Claude Code and Opus 4.5 based on my first-hand knowledge of their combined capabilities. Other harness and model pairs will work with most of the commands in this post. The command backgrounding feature in step 3 is the only potential snag for some harnesses.

Step 1: Building

Allowing Claude Code to build after every proposed change is a requirement for agentic workflows. Like it does for human developers, the compiler catches dumb syntax errors and, with Swift concurrency, even data races. The alternative is tabbing back over to Xcode, hitting cmd+b, waiting, copying and pasting error messages into the terminal; a massive waste of human time.

Prerequisites

Move DerivedData location to your project folder (optional)

Moving DerivedData to a location inside your project folder is perhaps an unusual suggestion, but it has several benefits for an agentic workflow:

Permissions: you’ll encounter fewer permissions dialogs when Claude is reading inside the project folder that you’re presumably running it in. Most devs expect DerivedData to be cleared regularly so it’s safe.
Git worktrees: An advanced technique is to use Git worktrees to have independent copies of your repo. Colocating DerivedData ensures the separate repos don’t interfere with each others build artifacts.
Docs: The DerivedData has a full copy of your Swift Packages, including any documentation. Claude Code can do fast greps to verify syntax or find examples. In my CLAUDE.md I have a direct link for each important package:

- `DerivedData/train-timetable/SourcePackages/checkouts/swift-composable-architecture/Sources/ComposableArchitecture/Documentation.docc`
- When using Search/Grep/etc. tools, ignore anything in the /DerivedData folder by default unless specifically looking for build artifacts or code/docs for Swift Packages used by this project

Before making this change, add DerivedData/ as a line in your .gitignore file if it’s not already there.

Find the setting in Xcode Settings -> Locations. Set Derived Data to “Relative” and Build Location to “Unique”. It will report /DerivedData as the location.

DerivedData settings in Xcode

Document project file & scheme

Build commands use your project/workspace file location and scheme name. Claude Code can find these pretty easily with tools but it’s faster to document them in CLAUDE.md.

Finding scheme names from Xcode

Get simulators

I’ll assume you’ve already downloaded the iOS simulators and iOS runtime versions you’d like to use in the Xcode interface.

The usual command you’ll use to find the simulator you want produces a very long list, so it’s reasonable to cache your favorite simulator’s UDID so each new Claude Code session doesn’t need do this from scratch each time. “Cache” meaning note it in your CLAUDE.md file, add it as an environment variable, etc.

# Get all available simulators
xcrun simctl list devices available

     -- iOS 26.1 --
         iPhone 17 Pro (89F6D0BC-E855-4BF7-A400-9C19ED7A7350) (Shutdown)
         iPhone 17 Pro Max (F1FA81FA-ED32-40C4-BD78-753254D685AC) (Shutdown)
         iPhone Air (77702E5F-85F5-4997-BA14-BC8D8F639B84) (Shutdown)
		 ...
     -- iOS 26.2 --
         iPhone 17 Pro (DB0531E0-B47E-42AC-9AAB-FEB76D3D563A) (Booted)
         iPhone 17 Pro Max (0C54CF4B-8A45-450E-AB93-B800B97BD4DA) (Shutdown)
         iPhone Air (83BECA5F-7894-4705-B198-3DCAE0C4778E) (Shutdown)
         ...
     -- ...

At first, you’ll probably start with a single threaded workflow, having one preferred simulator booted and in use at a time. The below command will output just the UDID for the latest iPhone Pro with the latest installed iOS version.

# Get the UDID of the latest iPhone Pro (non-max) model with the latest available iOS version 
xcrun simctl list devices available | grep "iPhone.*Pro (" | tail -1 | grep -Eo '[A-F0-9]{8}-[A-F0-9]{4}-[A-F0-9]{4}-[A-F0-9]{4}-[A-F0-9]{12}'

Once you have more Claudes running in parallel, you’ll can ask each to find its own UDID by looking for a non-booted simulator before it starts building.

Note that even though some commands can be run with more vague identifiers like name, os, or “booted”, it’s much more reliable to select a UDID and use it across commands. For example: specifying "platform=iOS Simulator,name=iPhone 17 Pro" in the build command will pick any simulator that matches, which could be any iOS version. For the build command it’s not as big of an issue, but to ensure predictable runs I recommend using UDIDs only.

Install xcsift

xcsift is a companion parsing library for build output. You’ll be building a lot, and you don’t want to fill up your context with hundreds of lines of “file.swift built”. xcsift solves this by producing just the actionable errors and warnings in json.

brew install xcsift

I add the -w flag to also include warnings in the output. I recommend browsing the xcsift docs to find other flags you might find useful for your project.

Building

OK, after all that setup, we should have all the info we need to assemble the actual build command.

Claude Code will be able to derive and customize the exact build command you need. I recommend doing a quick session with Claude - the goal being to produce a single, always-working command you can document somewhere and use automatically in each future session.

# Example: working build command for a specific simulator
xcodebuild -project train-timetable.xcodeproj -scheme "train-timetable" -destination "platform=iphonesimulator,id=DB0531E0-B47E-42AC-9AAB-FEB76D3D563A" -derivedDataPath DerivedData -configuration Debug build 2>&1 | xcsift -w

-project: path to your xcodeproj file. Use -workspace if you use an xcworkspace file.
-scheme: scheme name we found above.
-destination: for simulator, we use "platform=iphonesimulator,id=$UDID" where id is the UDID of our favorite simulator instance. Note platform=iOS Simulator has the same meaning and also works.
-derivedDataPath: this is super important if you’ve moved the DerivedData to the project folder. Without this, the Xcode instance will be using a different directory and you’ll have super slow (clean) builds each time.
-configuration: Debug is the default, so you don’t usually need this flag. It’s better to be explicit though because this affects the folder where your app binary will be copied to (see step 2).
build: the actual build command
2>&1 | xcsift -w: combines stdout and stderr and pipes them both into xcsift so it has access to all output. -w tells xcsift to also show build warnings, not just errors.

It’s important to test your ideal build command to confirm:

It doesn’t force a clean build each time.
It produces a concise set of errors and warnings.
It doesn’t interfere with builds via Xcode; you should be able to build/install/run from Xcode, use other SourceKit features, etc. and not clear the build cache.

In my understanding for builds, the simulator UDID (or at least simulator name) does not affect the build artifacts or app binary. However, there’s a lot going on behind the scenes so for simplicity I recommend using the same simulator UDID across all build, install, & launch steps. You need to be careful if running multiple simulators from the same project folder (i.e. without git worktrees) because each build command will overwrite the app binary regardless of whether you run the build command with different simulator UDIDs.

Clearing DerivedData

When left to its own devices (literally), sometimes Claude will get frustrated when a build is failing continuously and it can’t figure out how to fix things. It will sometimes try to remove the entire DerivedData folder. This is a bad idea because 1. clearing DerivedData usually doesn’t fix the underlying problem and 2. it will temporarily break Xcode’s ability to read your Swift packages and you’ll need to restart Xcode to get everything working again.

After I got my build commands more streamlined, Claude stopped doing this as much. But I still have decently strict permissions, so when it does happen, the session will usually block on any rm command and I’ll get a chance to step in and reprimand it. If you run into this problem, you can dig deeper into a configuration-based solution and modify your permissions, add hooks, or add more to your CLAUDE.md. Just something to look out for.

Step 2: Installing & Launching

Building should streamline a lot of your Claude Code workflow. But I slept on the automated install & launch step for too long.

When properly set up with step 1, you should be able to wait for Claude Code to build its changes and return control to you. Then you can tab over to Xcode and hit “run without building” to handle the install & launch.

But when you’re doing build->install->launch dozens of times a day, it’s way more streamlined to check Claude’s session output then tab over to the simulator and tap through screens to test out the changes.

Prerequisites

Document the app binary location for your scheme

Ask Claude to find the location of the app binary produced by the build command.

Since my setup has DerivedData in the project folder and a build directory inside it, that’s where my app binary is: DerivedData/Build/Products/Debug-iphonesimulator/Eki Bright.app.

You’ll notice the folder: Debug-iphonesimulator, which corresponds to our build configuration of Debug from earlier and the platform iphonesimulator. If you go off the beaten path and want to try out a Release build for example, make sure to understand this relationship.

Also be cautious because you want to make sure the most recent build is what you’re installing and launching and looking at. And there’s nothing in the file name that will indicate that.

Find your bundle identifier

This will be in your Target’s general settings pane beside Bundle Identifier: com.twocentstudios.train-timetable

Bundle identifier in Xcode target settings

Installing the app on the simulator

Installing is copying over the app binary into a specific simulator’s storage. This step depends on the build step having produced an app binary.

The parameters in the install command are:

UDID: for least headaches this should be the same simulator UDID you specified in the build command.
path/to/My App.app: the app binary location specified by your scheme.

# Example: install a previously built app binary on the simulator by UDID
xcrun simctl install DB0531E0-B47E-42AC-9AAB-FEB76D3D563A "DerivedData/Build/Products/Debug-iphonesimulator/Eki Bright.app"

Launching the app on the simulator

Launching the equivalent of tapping your app’s icon in Springboard. It of course depends on the install step having copied over the app binary.

The parameters in the install command are:

UDID: the simulator UDID you specified in the build & install commands.
bundle id: apps are uniquely identified by bundle id after installation.

# Example: launch a previously installed app binary by bundle id
xcrun simctl launch DB0531E0-B47E-42AC-9AAB-FEB76D3D563A com.twocentstudios.train-timetable

Navigating by URL (bonus)

Depending on your app’s navigation structure and pre-existing support for universal links or App Intents, you can save yourself even more time by having Claude automatically navigate the app to the tab, sheet, or navigation destination you’re currently testing.

The full set of caveats is beyond the scope of this post. In my experience, adding Universal Links support without some caution can lead to giving Claude access to data or flows that are impossible for normal app users to see. It may also add maintenance burden for initializers that are only used during debug. Regardless, jumping through a dozen screens automatically can save you hours of unnecessary manual screen-clicking labor.

The parameters for the openurl command are:

UDID: the simulator UDID you specified in the build & install commands.
URL: the deep link URL your app knows how to process.

xcrun simctl openurl DB0531E0-B47E-42AC-9AAB-FEB76D3D563A "train-timetable://tab?name=search"

In my testing, it’s safe to have Claude to run the openurl command immediately after the install command (or even in the same line) without needing a sleep or otherwise waiting for the launch to complete.

Step 3: Reading Console & Log Output

With Step 1, Claude has access to the compiler’s evaluation of its code changes. We can also give Claude access to the console and log outputs so it can evaluate the runtime results.

There are two strategies: console output via print statements and log output via OSLog/Logger. I use both depending on the situation.

Depending on the strategy, we’ll either amend the launch command from step 2 or prepend a CLI command.

Blocking vs. non-blocking

Claude can do blocking and non-blocking for the console variant, and non-blocking-only for the log output.

Blocking means that the prompt input and Claude’s thinking will be suspended until you explicitly stop it or the default timeout (currently 10 minutes) triggers. It will print output from the command inline, but usually truncate portions.

Non-blocking means Claude will use its background capability to keep the command alive and retain access to its output but immediately move the command to the background so that the prompt input is available.

I recommend the blocking flow for when you want to add a few quick print statements to verify a limited (maybe less than 15 seconds) code execution flow that you, the human, are driving in the simulator and have Claude immediately evaluate the results inline. The amount of lines generated should be small, within 10s of lines.

I recommend non-blocking for all other scenarios, including:

when you want Claude to drive the simulator (discussed in step 4) while monitoring the output.
when you want to use Logger instead of print logging, probably for more permanent logging code in your codebase.
when you’re expecting to generate dozens or hundreds of lines of logs in a single run. In order to be smart about preserving the session context, you’ll want to write to a file and allow either a subagent to extract meaning from it, or have the primary agent use parsing tools to read only the relevant portions.

`--terminate-running-process`

Adding the --terminate-running-process flag to launch ensures idempotency by terminating any existing instance of your app and ensuring the app is always cold launched with the console output available.

Adding the --terminate-running-process is super important to the logging flow since you may not be rebuilding and reinstalling between launches.

When you don’t terminate an existing process, the app instance will stay in memory on the simulator. By default, the launch command will not relaunch the app if it’s already launched. This will happen silently. Critically, it will also not read any console output and Claude will get very confused about why nothing is being logged and it will start thrashing and making very dumb changes, ranging from adding more print commands to clearing DerivedData.

(This was my biggest roadblock in getting a reliable and robust debugging flow with Claude; please learn from my mistakes).

Launching the app on simulator and reading the output

Replace the launch commands from step 2 with any of the below variants depending on your use case.

Blocking console/print direct (when you know output volume is reasonable)

Relevant flags and parameters:

--console-pty: produce console print output.
--terminate-running-process: as discussed above, ensure the command actually runs.
UDID - the simulator UDID you specified in the build & install commands.
bundle id - bundle id of the target that produces the app binary.

# Example with simulator UDID and bundle id
xcrun simctl launch --console-pty --terminate-running-process DB0531E0-B47E-42AC-9AAB-FEB76D3D563A com.twocentstudios.train-timetable

Note: the flags --stdout --stderr do not work. Don’t use them. Use --console-pty instead.

Blocking console/print to file (safer for unknown or expected heavy output)

Relevant flags and parameters:

--console-pty: produce console print output.
--terminate-running-process: as discussed above, ensure the command actually runs.
UDID: the simulator UDID you specified in the build & install commands.
bundle id: bundle id of the target that produces the app binary.
output file path: the plain text file console output will be written to. Note: I write to a tmp folder within DerivedData to ensure Claude has access to the result without triggering unnecessary permissions dialogs.
2>&1: ensure stdout & stderr both end up in the file.

xcrun simctl launch --console-pty --terminate-running-process DB0531E0-B47E-42AC-9AAB-FEB76D3D563A com.twocentstudios.train-timetable > DerivedData/tmp/console.log 2>&1

Non-blocking console/print

Non-blocking requires using Claude Code’s run_in_background parameter on the Bash tool. This will produce a task_id that Claude can later use to get the output (from an implicitly created text file) and kill the task.

After running the Bash tool, the prompt will be unblocked and you can ask Claude to monitor the output or ask it do anything else you want.

The non-blocking flow requires a bit more ceremony; you’ll need to tell Claude when you’re done working with the simulator and it should analyze the results. It usually leaves the background task running (potentially writing log data to the output), so you’ll need to specifically tell it to stop.

The command itself is the same as the one from Blocking console/print direct.

Bash(
  command: "xcrun simctl launch --console-pty --terminate-running-process DB0531E0-B47E-42AC-9AAB-FEB76D3D563A com.twocentstudios.train-timetable",
  run_in_background: true
)

Command running in background with ID: b8e2ca5.

# *wait for next user prompt*

TaskOutput(task_id: "b8e2ca5")
KillShell(shell_id: "b8e2ca5")

Non-blocking Logger/OSLog

With only its training data, Claude knows how to use Logging by importing the OSLog framework. OSLog has strengths and weaknesses compared to console/print logging. You may already be using it in your app. I consider it more of a long term solution you’d add to your codebase alongside each feature and keep it up to date with any changes.

Giving Claude access to these logs is different from the print/console flow we just discussed.

The root command is xcrun simctl spawn.

spawn log stream only captures logs emitted while it’s running (not before). If you want logs starting from launch, always run it before the launch command.

Blocking on spawn log stream doesn’t make sense because you still need to launch the app. You should dispatch it directly to the background as non-blocking.

Relevant flags and parameters (Claude knows how to adjust these freely):

UDID - the simulator UDID you specified in the build & install commands.
--level: matches the log level in your code; debug, info, warning, error, etc.
--predicate: filters the firehose output the messages you’re interested in. Lots of options here depending on how you’ve defined Loggers and added log statements in your codebase.

After spawn log stream is dispatched to the background, you’ll need to launch the app with the launch command. You can choose a blocking or non-blocking launch command.

Note that the raw spawn log stream command is not actually monitoring the specific app process. You can start this early in your session, cast a wide net, and keep this running through your whole session, asking Claude to filter the relevant time periods from the output. I personally haven’t needed this flow though.

Bash(
  command: "xcrun simctl spawn DB0531E0-B47E-42AC-9AAB-FEB76D3D563A log stream --level=debug --predicate 'subsystem == "com.twocentstudios.train-timetable"')"
  run_in_background: true
)

Command running in background with ID: b8e2ca5.

Bash(xcrun simctl launch --terminate-running-process DB0531E0-B47E-42AC-9AAB-FEB76D3D563A com.twocentstudios.train-timetable)

# *blocking prompt until user escapes*

TaskOutput(task_id: "b8e2ca5")
KillShell(shell_id: "b8e2ca5")

Non-blocking console/print & Logger/OSLog

You can combine everything above and give Claude access to both console/print output and Logger/OSLog output. The launch command can be blocking or non-blocking, but the below example is non-blocking.

Bash(
  command: "xcrun simctl spawn DB0531E0-B47E-42AC-9AAB-FEB76D3D563A log stream --level=debug --predicate 'subsystem == "com.twocentstudios.train-timetable"')"
  run_in_background: true
) 
Command running in background with ID: b8e2ca5.

Bash(
  command: "xcrun simctl launch --console-pty --terminate-running-process DB0531E0-B47E-42AC-9AAB-FEB76D3D563A com.twocentstudios.train-timetable",
  run_in_background: true
)
Command running in background with ID: a792db1.

# *wait for next user prompt*

Step 4: Controlling & Viewing the iOS simulator

Giving Claude eyes and virtual fingers to see and control the iOS simulator is where we start to reach the avant-garde. At the current (end of 2025) model & harness capabilities things start to go off the rails pretty quickly. I wouldn’t expect great results from Claude at tasks related to manipulating the simulator like a human, but in certain scenarios, the benefits outweigh the costs.

Prerequisites

AXe

AXe is a comprehensive CLI tool for interacting with iOS Simulators using Apple’s Accessibility APIs and HID (Human Interface Device) functionality.

Claude can use AXe to manipulate the simulator through taps, swipes, button presses, and keyboard typing.

Under the hood, AXe uses Facebook’s idb CLI.

Install AXe with Homebrew.

Image Magick (optional)

ImageMagick® is a free and open-source software suite, used for editing and manipulating digital images.

Claude can use ImageMagick to do some post-processing on screenshots from the simulator.

Install ImageMagick with Homebrew.

FFmpeg (optional)

Claude can use the venerable FFmpeg CLI for advanced video manipulation use cases. You may not need it but there’s a good chance you already have it.

Reading from the simulator

In order to navigate the simulator beyond the universal links openurl use case we detailed above, Claude needs to be able to see the current state of the simulator.

There are 3 options for this:

Accessibility info - Claude can read a hierarchical text description of the current screen using accessibility info.
Screenshots - Claude can take a screenshot of the simulator and use the Read tool to access its multimodal capabilities.
Video - Claude can record a short video capture of the simulator, slice it up into frames, and read a few to assess an animation.

Accessibility info via `describe-ui`

The AXe command for getting the accessibility trace is:

axe describe-ui --udid SIMULATOR_UDID

  ...
      {
        "frame": {"y": 82, "x": 346, "width": 36, "height": 36},
        "AXLabel": "閉じる",
        "type": "Button"
      }
  ...

The output is a big JSON array.

I thought Claude would be better at understanding and navigation with text information than image information, but in practice it almost always ignored my instructions in CLAUDE.md to use describe-ui before the screenshot flow. Perhaps there’s something in the system prompt or it’s less efficient to hunt through all the text.

I also immediately ran into a reported issue in AXe and idb where describe-ui does not print tab or toolbar info, perhaps only from iOS 26. This makes it very difficult to deterministically do any sort of navigation from the root in many apps.

All this is to say that, at the moment, it’s more reliable to use screenshots.

Screenshots

Claude can use simctl to get screenshots.

Like the other commands, I prefer to write to a tmp folder within DerivedData.

Screenshots for most simulators are taken at 3x scale, but input taps and swipes are at 1x. For the dual purposes of 1. reducing the amount of calculation required to translate screen position to next tap position and 2. reducing the amount of image data that needs to be sent to and processed by Claude, I automatically resize all screenshots to 1x via magick.

xcrun simctl io DB0531E0-B47E-42AC-9AAB-FEB76D3D563A screenshot DerivedData/tmp/screen.png && magick DerivedData/tmp/screen.png -resize 33.333% DerivedData/tmp/screen_1x.png

Video

Reading live or even recorded video is currently beyond Opus 4.5’s capabilities. I’m guessing this will be a supported flow sometime in 2026, but until then analyzing video output of the simulator is still at proof-of-concept maturity.

While I was debugging a tricky animation, I gave Claude some leash to test whether it could:

start recording a short clip immediately before a tap.
stop the recording after 2 seconds.
use FFmpeg to grab 5 or 6 frames spaced out across the video.
read the frames and analyze the motion.

It sort of worked? Not really? If you have a use case, you can try experimenting more with this flow. For now I’d consider the actual animation analysis a human-only endeavor. But Claude can still help get the simulator staged up to the start screen.

I believe I used this AXe command as a Claude Code background task:

axe record-video --udid DB0531E0-B47E-42AC-9AAB-FEB76D3D563A --fps 30 --output DerivedData/tmp/recording.mp4

Manipulating the simulator with taps and swipes

AXe has a variety of tap and gesture commands. Claude can tap on points or accessibility labels.

# Tap at logical coordinates (use frame center from describe-ui)
axe tap -x 201 -y 297 --udid DB0531E0-B47E-42AC-9AAB-FEB76D3D563A --post-delay 0.5

Without additional guidance, Claude gets confused about which scroll command maps to what logical direction.

# Scroll (named by finger direction, not content direction)
# scroll-up = finger UP = content UP = see content BELOW = triggers .onScrollDown
# scroll-down = finger DOWN = content DOWN = see content ABOVE
axe gesture scroll-down --udid DB0531E0-B47E-42AC-9AAB-FEB76D3D563A --post-delay 0.5
axe gesture scroll-up --udid DB0531E0-B47E-42AC-9AAB-FEB76D3D563A --post-delay 0.5

It’s useful to note the swipe-from-left-edge gesture because it’s the quickest way for Claude to pop back a level in a NavigationStack.

# Edge swipes (for back navigation, etc.)
axe gesture swipe-from-left-edge --udid DB0531E0-B47E-42AC-9AAB-FEB76D3D563A --post-delay 0.5

Strategies for increasing tap accuracy

The most significant source of indeterministic behavior is in Claude’s ability to accurately measure of coordinates on screen. In other words, it can’t read an image and always find the center point of a button. This means there is plenty of opportunity for situations like:

Claude reads a screen and wants to tap a button.
Claude makes a bad guess and taps above the button.
Claude reads the screen again. There was no change.
Claude makes another bad guess and taps below the button.

Claude’s only feedback about whether its tap was successful is based on its next screenshot. This can lead to situations where it gets irrecoverably lost while navigating your app:

Claude reads a screen and wants to tap a button.
Claude makes a bad guess and taps above the button, hitting a completely different button.
Claude reads the screen again.
Claude sees it’s on a different screen than expected and becomes confused.

I came up with an experimental flow to try to improve Claude’s accuracy, but:

It slows down the entire process by 2x.
By the time Claude realizes it needs to use the experimental flow, it’s already too far lost to recover.

Regardless, my flow, also using ImageMagick, is to make Claude draw a red circle on a screenshot in its targeted tap location before actually performing the tap:

Take screenshot and resize to 1x (so pixels = points):

xcrun simctl io DB0531E0-B47E-42AC-9AAB-FEB76D3D563A screenshot DerivedData/tmp/screen.png && magick DerivedData/tmp/screen.png -resize 33.333% DerivedData/tmp/screen_1x.png

Read the 1x image and estimate target element center in points

Verify guess by drawing a red box at those coordinates:

magick DerivedData/tmp/screen_1x.png -fill none -stroke red -strokewidth 2 -draw "rectangle $((X-30)),$((Y-30)) $((X+30)),$((Y+30))" DerivedData/tmp/screen_marked.png

Read marked image to check if box is on target
If missed, adjust coordinates and repeat from step 3
If correct, tap at the verified coordinates

Tap accuracy verification flow - original screenshot, marked target, and result after tap

Putting it all together

So far in practice, I’ve used this capability alongside universal links to fix and verify simple visual bugs. I have Claude craft input to reproduce the error and find the screen where it occurs, take a “before” screenshot, implement the fix, build/install/launch, then find the same screen, verify the fix, and take an “after” screenshot.

It takes way longer for Claude to do than me, but it’s mostly tedious work, and I’m usually in another tab working on a plan with another Claude. When I come back and see the before and after screenshots alongside the code change, I can feel confident in Claude’s work.

Step 5: Building, Installing, Launching, Reading Output on a Physical Device

Finally, for those Apple SDKs that only work on device, for ensuring observed buggy behavior isn’t just a simulator quirk, or just to get a more realistic look at our apps in context, we can implement steps 1, 2, and 3 on a physical device. Unfortunately, as far as I can tell, there’s no way to control a physical device via CLI tool, so step 4 is out reach for now.

However, building, installing, launching, and logging can still save some time and annoyance during iterative debugging sessions. It’s especially useful to have Claude help analyze logs for (underdocumented) frameworks like Core Location that behave wildly different on a real device than on the simulator.

Below is a collection of tested CLI commands for doing all the above tasks on a physical device.

Note that in my testing all the relevant commands below work equally for devices on the same network and devices connected directly to your Mac via USB.

Prerequisites

Get devices

The device Name and Identifier are both important for on-device debugging. State will be available when Wi-Fi debugging is available, and connected when directly connect via USB.

# Get all connected devices
xcrun devicectl list devices

Name          Hostname                      Identifier                             State                Model                                
-----------   ---------------------------   ------------------------------------   ------------------   -------------------------------------
CT's iPhone   CTs-iPhone.coredevice.local   ABCDEF01-1111-5555-AAAA-F7D81A900001   connected (no DDI)   iPhone 14 Pro (iPhone15,2)           
CT’s iPad     CTs-iPad.coredevice.local     ABCDEF01-2222-6666-BBBB-F44A19F00002   available            iPad Pro (11-inch) (iPad8,1)         
CT’s iPad     CTs-iPad-1.coredevice.local   ABCDEF01-3333-7777-CCCC-D33889000003   unavailable          iPad mini (5th generation) (iPad11,1)

Some example use cases for parsing out values in one go:

# Get the Identifier of the first accessible iPhone (WiFi or USB)
xcrun devicectl list devices | grep "iPhone" | grep -E "(available|connected)" | head -1 | grep -Eo '[A-F0-9]{8}-[A-F0-9]{4}-[A-F0-9]{4}-[A-F0-9]{4}-[A-F0-9]{12}'

ABCDEF01-1111-5555-AAAA-F7D81A900001

# Get the name of the first accessible iPhone (WiFi or USB)
xcrun devicectl list devices | grep "iPhone" | grep -E "(available|connected)" | head -1 | awk -F'  +' '{print $1}'

CT's iPhone

Build for device

Build commands are the same as those for the simulator, except platform=iOS instead of platform=iphonesimulator/platform=iOS Simulator.

Use the name of your target device from the list devices command.

# Using device name
xcodebuild -project train-timetable.xcodeproj -scheme "train-timetable" -destination "platform=iOS,name=CT's iPhone" -derivedDataPath DerivedData build 2>&1 | xcsift -w

Note: platform=iphoneos does not work. id instead of name does not work.

Install on device

Install commands use devicectl but are the similar to those for the simulator, except --device should use the device ID, and the build product directory should use Debug-iphoneos instead of Debug-iphonesimulator.

xcrun devicectl device install app --device E7E3E660-9E7A-5814-8BBB-F7D81A965CEB "DerivedData/Build/Products/Debug-iphoneos/Eki Bright.app"

Launch on device

The vanilla launch command is below. I again recommend using --terminate-existing, the device equivalent of the simulator’s --terminate-running-process.

xcrun devicectl device process launch --device E7E3E660-9E7A-5814-8BBB-F7D81A965CEB --console --terminate-existing com.twocentstudios.train-timetable

Blocking console/print capture on device

For console/print capture, use the launch command above with the --console flag. It works over USB and Wi-Fi.

xcrun devicectl device process launch --device E7E3E660-9E7A-5814-8BBB-F7D81A965CEB --console --terminate-existing com.twocentstudios.train-timetable

Non-blocking console/print capture on device

# Use run_in_background: true on Bash tool
# Works over USB and WiFi
Bash(
  command: "xcrun devicectl device process launch --device E7E3E660-9E7A-5814-8BBB-F7D81A965CEB --console --terminate-existing com.twocentstudios.train-timetable",
  run_in_background: true
)

Command running in background with ID: b8e2ca5.

# *wait for next user prompt*

TaskOutput(task_id: "b8e2ca5")

Logger/OSLog capture on device (requires manual sudo)

A downside of OSLog is that it requires sudo and Claude Code can’t use sudo commands directly. There are presumably some ways to give Claude this capability in more a dangerous fashion. But a safer workaround for now is for you, the human, to run the below commands in another terminal tab. Claude can give you the full command to copy/paste into the other terminal.

Another downside is that the process is slow and produces lots of logs.

These commands will produce groups of files that Claude can read.

Note that the log collect is of everything on the device in the past, and can quickly balloon to gigabytes of storage. The command to collect even the last 2 minutes of logs can take about 30 seconds to complete on Wi-Fi.

Content filtering with --predicate is not supported: Warning: --predicate is ignored when collecting from attached device.

Claude will handle filtering while reading/analyzing with log show --predicate ....

The most logical way to use this is to:

have Claude build, install, and launch the app on your device (ensure it’s unlocked), and have it note the start time.
tap around and do the testing you need in order to generate the logs you want.
have Claude give you the sudo log collect command with a start time a little before the launch time.
run the sudo log collect command in a separate terminal window, enter your password, wait ~1m for it to finish.
ask Claude to analyze the log archive.

# Human user must run these commands in another terminal tab (Claude Code can't provide sudo password)

# `--last` collects from N minutes before the command was run
sudo log collect --device-name "CT's iPhone" --last 2m --output DerivedData/tmp/device-logs.logarchive

# `--start` collects from the specified start time until the command was run
sudo log collect --device-name "CT's iPhone" --start "2025-12-30 16:11:00" --output DerivedData/tmp/device-logs.logarchive

# Then analyze with log show (Claude can do this)
log show DerivedData/tmp/device-logs.logarchive --predicate 'subsystem == "com.twocentstudios.train-timetable"'

Note --device-udid does not work, use --device-name instead - log: failed to create archive: Device not configured (6).

Note --size does not (seem to) work either.

Final thoughts

How to parameterize names, ids, etc. for these commands

So far, I’ve just been hardcoding these commands with my favorite simulator UDID and project path into my CLAUDE.md. When a new version of Xcode comes out I ask Claude to update all mentions of the UDID to the most recent simulator version and it only takes a minute. Hardcoding these values leaves the least room for hallucination. When running these commands dozens of times a day, you really want consistency.

Other ways to handle this would be:

set environment variables at some level.
add a start hook to have Claude fill in the environment variables fresh for each session.
set up another layer of orchestration that handles the pool of simulators and dispatches an ID to each new Claude instance that requests one.

These are all beyond the scope of this post. If you work primarily on one project with others, you probably already have some tooling for specifying the Xcode version, etc.

Why not include traditional testing?

Arguably, TDD was the original “closing the loop” in software development. TDD has never caught on in the iOS world.

I dabbled with an actual Swift Testing-based testing flow for another recent project, and even wrote about another experimental system a few months ago in Giving Claude Code Eyes to See Your SwiftUI Views that used snapshot testing. What I found was although tests are great for verifying correct behavior over the long term, in the short term they are super slow on iOS:

Installing and launching requires instantiating a brand new simulator for each run (e.g. Clone 1), which takes a long time.
All builds are clean builds (this could have just been a fluke in my setup at the time though).
Swift Testing does not output failures in a way that Claude can read and iterate on (again, potentially solvable).

Admittedly, I didn’t spend as much time debugging these flows. Hopefully someone else will fill in the blanks for testing and write this guide.

Don’t sleep on `simctl`

The simctl CLI we’ve used throughout this post has a ton of other abilities that Claude can use to make our lives easier. This includes adding images to Photos, changing the system time, changing the system language, resetting privacy, resetting the keychain, and many more.

Ask Claude to configure your simulator on the fly instead of clicking through settings with your mouse.

What does the future hold?

I’m honestly not sure how long the hard-won knowledge in this post will be relevant, given the pace of model & harness capabilities. Peter Steinberger already says Codex is good enough and doesn’t need any additional guidance about build commands or working with the simulator.

I can definitely see a world where Claude Code has a live feed of the simulator output it can process and react to at 60 fps, tapping and swiping with full accuracy. This is probably what’s missing in fully closing the development loop on iOS. Doing the same for a real device hopefully isn’t close behind.

At that point though, I’m not sure what else about development will have changed.

Going forward

Most of the material in this guide has been slowly compiled over the month in my various CLAUDE.md files. It was great getting a chance to formalize it even if I can’t make a quickly installable Plugin or Skill to share (hopefully you understand why after reading the post). I’m looking forward to seeing how far I can take each of these steps in the near future.

Corrections

Please reach out if you find any corrections or can contribute any additional knowledge or edge cases.

Shinkansen Live - Developing the App for iOS

Thu, 25 Dec 2025 07:41:42 -0600

In my last post I introduced the motivation and feature set of Shinkansen Live, my latest iOS app. I encourage you to read that one first to learn about what the app does.

Shinkansen Live app icon

In this post, I’ll discuss a few of the interesting development challenges I faced during its week of development from concept to App Store release.

Overall development strategy
OCR and parsing the ticket image
Live Activities
Animations
AlarmKit
Localization
Dynamic Type
VisionKit Camera

Overall development strategy

I created an Xcode project myself with Xcode 26.1 (later switching to Xcode 26.2), then added the TCA package.

Then, I set off to work using Claude Code with Opus 4.5. I started by having it lay out the SwiftUI View and TCA Feature without any logic. Then I built out the rest of the infrastructure around getting the input image, doing OCR, parsing the output, and displaying the results. I’ll go through more of the history later on in the post.

Early view layouts during initial development

OCR and parsing the ticket image

The most difficult part of getting this app to production was the ticket OCR & parsing system. This system went through the most churn over the week, partially due to expanding scope and partially due to my expanding understanding of the problem space.

My initial thought during the prototyping stage was to target only the “ticket” screenshot from Eki-net you get after purchase. It looks something like this:

Example Eki-net ticket screenshot

In theory, it’d be reasonable to limit the app’s input space to app screenshots from Eki-net (JR-East) and SmartEX (JR-central). Even including web browser screenshots wouldn’t be that much more burden on an OCR-based system. But later on in the project when I’d decided I was happy enough with the prototype that I wanted to target a production release on the App Store, I started thinking about how it would make marketing much harder to say “only works on screenshots” and not physical tickets.

OCR via VisionKit alongside manual parsing has a lot of upsides:

Free for user & developer
Fast
Multilingual
Privacy baked in
No network usage
Relatively mature: less risk of accuracy churn

In theory, multimodal LLMs can handle OCR and understanding more variations of tickets layouts and bad lighting. In fact, while I was writing the parser, I used Opus 4.5 to read the test ticket images in order to create the ground truth test expectation data.

My issue with prototyping with LLMs further was that they had essentially the opposite pros and cons as the VisionKit system:

Unknown, but some ongoing cost (meaning I’d need to come up with a monetization strategy before release)
Unknown which level of model would correctly balance accuracy, cost, and speed over the short term.
Requires network access
Requires sending photo data off device (not hugely private, but still)
Different outputs for the same input

What I didn’t explore directly was using the Apple Foundation model at the parsing layer. The current Apple Foundation model in iOS 26 has no image input API, but I could perhaps prompt it to take the raw output of the VisionKit model and try to make sense of it. My instinct is that this would be a waste of time, but still worth keeping on the table.

Overall strategy

While working on this, I honestly wasn’t thinking strictly in terms of prototype & production. From the first spark of idea I had an understanding of what the overall UX flow of the app would be. It was mostly getting to an answer of “is this feasible to productionize in a couple days?” while still being flexible on the scope of what production-ready meant.

That meant that I started by adding my ticket screenshot to the project, giving Claude my overall strategy, and having Claude create the OCR & parser system that output results directly into the UI.

OCR prototype

The first prototype parser supported just the two screenshots I had of Eki-net tickets (one from the morning of my trip; another from a similar trip a few months ago).

The implementation set up a VNRecognizeTextRequest in Japanese language mode, read out the highest ranking results into several lines of text ([String]), then fed that to Claude’s homegrown parser that pulled the ticket attributes out of that glob of text mostly using regex.

Since the input was from a perfectly legible screenshot, there was no issues with the VisionKit part nor the parser.

OCR for real Shinkansen ticket images

As soon as I tested the system on a real Shinkansen ticket in a photo, the system fell apart.

I searched through my personal Photo library for as many Shinkansen tickets as I could find. I googled for more. I started with 4 images (and later in development I ended up with about 10 images).

At first I was simply trying to naively patch out the parser with Claude. To do this, I set up a unit test system where I’d do the VisionKit request for each image once and write the resulting [String] data structure to disk. Then each unit test would read that data in, run the parser, and compare the expected ticket structure to the test result.

Unintuitively, the standard Swift Testing unit test setup was actually the less efficient way to iterate on this. Claude was having a lot of trouble reading the detailed test failure information after it ran xcodebuild in the command line. Each build & run & test iteration needed to boot up a fresh simulator, install the app, run the test, then tear down the simulator.

Instead, a built a @main App-based test harness that:

temporarily disabled the real UI.
ran the test code on onAppear.
printed the test results via print statements.
called exit(0) when finished.

For quickly prototyping, iterating, and understanding the scope of the problem space, this solved all the issues with the Swift Testing setup. The same booted simulator was reused on each run. DerivedData and build artifacts were reused, so builds were fast. Claude had no trouble reading print statements from the console output.

I let Claude run in its own loop for a while to see what it could and couldn’t improve with the parser based on the limitations of our system.

Claude found several underlying limitations with the VisionKit setup that were unsolvable at the parser level.

For example, concatenating all the text recognition objects into [String] was done somewhat naively by comparing y-coordinates. If the y-coordinates of objects were within a certain range, they were assumed to be on the same line. When the ticket was tilted, this strategy was interleaving text.

Additionally, some numbers and letters were just flat out being interpreted incorrectly. A 37 was read as a 30. CAR was read as CDR.

Off running on its own, Claude was trying to special case as much of these failure cases it could to get the tests passing.

Example test ticket image that produced the Japanese-only VisionKit parsing output shown below

CC制
東
3月18日（
はくたから5り考
¥3,380
新幹線特急券
京
→
8:41発）
軽井
（9:4着）
7号車
3番B席
R001
2025.-3.18東京北乗FN7（2－）
50159-01
沢
0

Comparing some of the fields, you can see the OCR output taken naively is not great:

The departure time is there but the arrival time is 9:4 instead of 9:43.
The arrival station’s 軽井 and 沢 are split up and should be 軽井沢.
The departure station’s 東 and 京 are split up and should be 東京.
The train name and number are a mess: はくたから5り考 instead of はくたか 555号

Using spatial data

The OCR part of the system was trying to abstract away the spatial parts from the parser. Looking at the raw data, my intuition was that the spacial data could be useful within the parsing layer. Instead of passing [String] between layers, I was now passing:

struct TextObservation: Equatable, Sendable {
    let text: String
    let x: Double
    let y: Double
    let width: Double
    let height: Double
}

Each field parser within the parsing system could decide for itself how to use the spacial data. This is especially useful considering how important anchor values are. For example “発” (indicating a departure time) and “→” (indicating the station to the left is the departure station).

This improved things a bit for 5 ticket images, but after adding another 5 and going all in on English ticket support, there were plenty more edge cases to consider.

Transforms

Only after seeing some wonky positioning of the text boxes in my loading animation (see below), I realized that I wasn’t accounting for the image transform properly when converting the input UIImage to CGImage as input to VNRecognizeTextRequest.

extension CGImagePropertyOrientation {
    init(_ uiOrientation: UIImage.Orientation) {
        switch uiOrientation {
        case .up: self = .up
        case .upMirrored: self = .upMirrored
        case .down: self = .down
        case .downMirrored: self = .downMirrored
        case .left: self = .left
        case .leftMirrored: self = .leftMirrored
        case .right: self = .right
        case .rightMirrored: self = .rightMirrored
        @unknown default: self = .up
        }
    }
}

Levenshtein distance

Like our previous example of CDR instead of CAR, VisionKit was reading strings like TOKIO instead of TOKYO. For these parts, I figured calculating Levenshtein distance from known strings was the right strategy. The Shinkansen system is large but not so large that I couldn’t ingest the station name values and the other known strings like JAN, FEB, etc.

One strategy I haven’t tested yet is using VNRecognizeTextRequest.customWords with the full dictionary of station names, etc. to see if that eliminates the need for using Levenshtein distance at all.

English and Japanese mode OCR

For some background, tickets can be printed in an “English” variant that includes a mix of English and Japanese text. If you buy from a ticket vending machine and complete the purchase in English mode, it’ll print an English ticket. Similarly, if you buy from a human ticket vendor at the counter, they will print your ticket in English variant if you speak English to them.

Example of an English variant Shinkansen ticket

At the VisionKit layer, I was first using VNRecognizeTextRequest in Japanese-mode only.

I tried expanding a single instance to include both English and Japanese text. But checking the raw results, a dual language setup severely impaired its abilities.

For a while, I had a dual parsing system that would check for a few English strings, and if any were found, it would assume the ticket was an “English ticket” and run a separate English VNRecognizeTextRequest and return those results. This didn’t work well for a few reasons:

The Japanese VNRecognizeTextRequest was surprisingly bad at reading numbers compared to the English one.
Deciding at the VisionKit layer which text results should come from the English request didn’t make a lot of sense conceptually if I wanted to keep the majority of the logic in the parser layer.

Therefore, I decided to run both the English and Japanese VNRecognizeTextRequests on every input and provide all the results to the parser.

struct BilingualOCRResult: Sendable, Equatable, Codable {
    let jp: [TextObservation]
    let en: [TextObservation]
}

I was also previously discarding the confidence score (0.0...1.0) that VisionKit provides with each observation. This score was actually different in the parallel observations for English and Japanese in some cases, especially number recognition. I added the confidence score to the output so the parser could use it.

struct TextObservation: Equatable, Sendable, Codable {
    let text: String
    let x: Double
    let y: Double
    let width: Double
    let height: Double
    var confidence: Float
}

Unparseable fields fallback UX

At this point in the development of the parser, I was pretty certain reading photos of tickets was never going to be reach 100% accuracy for every field.

I took a break from working on the parser to implement editing for every field in the ticket UI. This meant that users could manually recover from parsing errors and omissions.

Editing screen with editable ticket fields

Not needing to reach 100% accuracy in the parser while still ensuring the user gets value out of the system as a whole opened up a more reasonable strategy for the parser.

Defense-in-depth parsing system

After tweaking more and more of the VisionKit layer, I had to regenerate the VisionKit output test data for each test ticket image, and then essentially rewrite the parser layer.

This time, the strategy was to:

Pass in the full English and Japanese results including relative x and y coordinates to each field.
Create a sub-parser dedicated to each field of the ticket that needed to be parsed.

Within each sub-parser, my strategy was to start with the best case scenario of input data quality, then step-by-step keep loosening the guidelines to account for more unideal cases that had come up in the test data, then finally falling back to returning nil for that field.

OCR testing system

Claude iterated on the field parser implementations for an hour or two, one at a time, checking the test output to ensure there were no regressions along the way.

At a certain point all the tests were passing and as much as I wanted to keep finding test data and tweaking the parser, I knew I had to move on.

Live Activities

During development I had to keep reminding myself that the whole point of this endeavor was to have a slick (read: useful) Live Activity.

Live Activity in Dynamic Island compact, expanded, and lock screen views

My design process was quick and to-the-point:

list out all the Live Activity contexts: lock screen, dynamic island compact leading, compact trailing, minimal, and expanded.
consider all the ticket info I had available from the parser output.
consider all of the above for the “before” and “during” trip phases (if I knew I had more accurate control over Live Activity update timing, I might have divided these phases up even further).

In lock screen and expanded contexts, you can mostly display everything you want. The challenge is in aesthetics and visual hierarchy like in any design.

Lock screen Live Activity showing trip details

When dealing with the compact and minimal contexts, you really do only have the equivalent of about 6 very small characters to work with, and 2 lines if you want to push your luck. If you try to fill the entire available space of the Dynamic Island on either side, you’ll lose the system clock which is no go for my use case.

Compact Live Activity in Dynamic Island

When I thought about it, the most important of all the contexts was the compact leading and trailing in the before trip phase. This is when the information is most needed at a glance.

I stacked in departure time and train number in the compact leading. This information is used to decide when to go to the platform and which platform to go to.

I stacked the car and seat number in the compact trailing. These are used to decide where to line up on the platform and of course where you’ll sit.

View guidelines and tips

Designing for Live Activities is painful. There are significantly more constraints to the SwiftUI View system than in normal app contexts. Most are undocumented. Some quirks can be teased out in the SwiftUI Preview if you’re lucky. Others only appear on the simulator or a real device.

I have a couple guidelines and tips I follow for Live Activities.

Use non-semantic font sizes for compact and minimal

The system ignores Dynamic Type settings in the compact and minimal Dynamic Island contexts. Using point sizes directly gives more flexibility while designing in the very limited space.

In the lock screen and expanded contexts, there’s limited Dynamic Type support (4-levels total), so it’s still worth using semantic fonts as usual (e.g. .headline, .title3).

For dynamically updating times, prepare to spend a lot of time in trial and error

Maybe someday I’ll write a full explainer post on which countdown-style Text fields are supported. In short, if you want dynamically updated fields in any part of your Live Activity, your formatting options are limited and underdocumented.

A couple configurations I used:

// `44 min, 23 sec` or `1 hr, 52 min`
Text(attributes.departureTime, style: .relative)

// Centers `Departing in N min, M, sec` due to `.relative`'s implicit `maxWidth: .infinity`
HStack(spacing: 4) {
    Text(String(localized: "widget.departing-in", comment: "Footer label shown before departure"))
        .frame(maxWidth: .infinity, alignment: .trailing)
    Text(attributes.departureTime, style: .relative)
}

// Linear progress view with no label
ProgressView(timerInterval: attributes.departureTime ... attributes.arrivalTime, countsDown: false)
    .progressViewStyle(.linear)
    .labelsHidden()
   
// `60:00`
Text(
     timerInterval: (Date.now)...(Date(timeIntervalSinceNow: 60*60)),
     pauseTime: Date(timeIntervalSinceNow: 60*60),
     countsDown: true,
     showsHours: false
)

As noted above, any Text using: Text(Date(), style: .relative) will expand to fill its full width.

It’s frustrating just thinking about this again. I basically just banged my head against the wall until I landed on a design I felt embarrassed but comfortable shipping.

Clamped width custom Layout

I use this custom Layout judiciously in the Dynamic Island.

It makes the underlying view’s frame collapse to fit its ideal width, but clamped to a maximum value.

I want the compact or minimal context to be as narrow as possible. I want short input text to result in a very narrow Dynamic Island layout. I want longer input not to expand beyond a certain width; and even more, I want to use the minimumScaleFactor modifier to further shrink the text size once that maximum width is reached.

private struct SingleViewClampedWidthLayout: Layout {
    var maxWidth: CGFloat

    func sizeThatFits(proposal: ProposedViewSize, subviews: Subviews, cache: inout Void) -> CGSize {
        guard let subview = subviews.first else { return .zero }
        let idealWidth = subview.sizeThatFits(.init(width: nil, height: proposal.height)).width
        let width = min(idealWidth, maxWidth)
        return subview.sizeThatFits(.init(width: width, height: proposal.height))
    }

    func placeSubviews(in bounds: CGRect, proposal: ProposedViewSize, subviews: Subviews, cache: inout Void) {
        guard let subview = subviews.first else { return }
        subview.place(at: bounds.origin, proposal: .init(bounds.size))
    }
}

private struct ClampedWidth: ViewModifier {
    let maxWidth: CGFloat

    func body(content: Content) -> some View {
        SingleViewClampedWidthLayout(maxWidth: maxWidth) {
            content
        }
    }
}

extension View {
    @ViewBuilder func clamped(maxWidth: CGFloat) -> some View {
        modifier(ClampedWidth(maxWidth: maxWidth))
    }
}

Updating in background

Live Activities can usually only be updated via remote Push Notification. But if your app gets a chance to wake up and run in the background, it can also issue updates to the Live Activity.

One of the few reliable ways to have your app woken up in the background regularly is to use significant location updates from Core Location. In order to have your app get background time you need to:

Be approved by the user for Always Location Services permission.
Be approved by the user for WhenInUse Location Services permission AND one of the following
- Have an active Live Activity OR
- Start a CLBackgroundActivitySession (that essentially creates a default Live Activity)

In the case of Shinkansen Live, I have two phases for the Live Activity:

before the train departs
after the train departs

The time of that change over is pretty reliably scheduled. But Live Activities have no update schedule like Widgets do (for some reason).

To update the Live Activity after the train departs I could:

Have the user tap a button in the lock screen or expanded Live Activity that triggers an intent to wake up the app in the background and update the Live Activity.
Hope the user opens the app on their own.

Both of these options are unideal, so instead I ask for WhenInUse Location Services permission and start a monitoring for significant locations. One of these will be fired not long after the train departs (within about 1km or 5 minutes). That trigger will open the app in the background, update the Live Activity based on the current time, then go back to sleep.

Persisting the trip

There’s an edge case I wanted to handle with the Live Activity lifetime.

If the system kills the app in the middle of a trip, the Live Activity in theory should continue uninterrupted since it’s an App Extension. But in the case of Shinkansen Live, I’m expecting to update the Live Activity while the app is backgrounded. This means there’s a potential flow where:

The app is in the background with the Live Activity running.
The app is killed by the system.
The Live Activity continues to run.
The system cold launches the app in the background.

At this point, I could decide to query the Live Activities framework to see if there’s a Live Activity running and if so, restore the ticket model layer and UI. However, I prefer not to treat the Live Activity as the source of truth for the model layer.

I added support with the Sharing library to persist the ticket model automatically on changes. On the above flow, I use the persisted ticket model to restore the UI and Live Activity and AlarmKit state, ensuring the ticket data is still valid.

Handling dismissal

One final bit of UX that’s not mission critical but is very user friendly is to respond to user-initiated Live Activity dismissals from the lock screen. If the user swipes from right to left on your Live Activity, the system dismisses it. When your app next runs, it will receive an update from Activity.activityStateUpdates stream (if you’re monitoring it).

If the app detects a user-initiated dismissal, I consider that their trip has ended, clear the ticket, and go back to the app home screen. There’s an argument that it’d be safer to simply toggle the Live Activity, but since my app’s only purpose is to show a Live Activity, I don’t think it makes sense to build in more complexity to keep multiple states.

Animations

The focused nature of this app allowed me a bit more breathing room to experiment with custom screen transitions and multi-stage animations.

Root level transitions

At the bare minimum, I usually try to use a default opacity transition for root level views when they aren’t covered by system transitions like a navigation push or sheet presentation.

For Shinkansen Live, I added a little bit of extra scale effect to the usual opacity transition of the initial screen, both on cold launch and when returning from the trip screen.

For the trip screen, I first animated the card down from the top with some scale and opacity, then fade in the other sections with some scale.

Scanning animation

The most fun was doing the scanning animation. Once the selected image is downloaded and displayable, I animate it in with a bit of 3D effect. Then I use the coordinate results of the text observations to animate those boxes onto the image.

This animation serves a few purposes in my opinion:

Visually expresses to the user what the app is actually doing.
Buys time for the parser to do its job.
Feels fun and playful in a way that is motivating for users to want to go through the trouble of submitting their ticket image.

My final bit of (self) user testing made me realize that even though I’d built in a way to update ticket values that were missing or erroneously parsed, I had no in-app UI for actually doing the field checking. Depending on their input source, the user would have to hold up their physical ticket next to the app’s virtual ticket to double check the fields. Or if they’d used a screenshot, they’d have to flip back and forth between Photos app.

As my last big task, I added support for showing the original image inline with the ticket in a modal overlay.

This setup uses matchedGeometryEffect and was a nightmare to work through. In the end it’s not perfect, but the speed conceals some of the jankiness. matchedGeometryEffect has a lot of undocumented incompatibilities with other modifiers, so it was just hours upon hours of reordering modifiers and building and running to check what had changed. I came out of the experience with little new demonstrably true observations I can share here, unfortunately.

Card dragging

Whenever there’s a card-looking UI on screen, I want it to be interactable even if there’s no real gesture that makes sense.

I created a custom drag gesture with rubberbanding that allows the user to drag the ticket a little bit in any direction. When released, it snaps back with a custom haptic that mirrors the visual.

AlarmKit

AlarmKit is new in iOS 26 and I thought it might be a good fit for Shinkansen Live’s use case.

Alarm setting in trip screen and full screen alarm notification

The integration was mostly straightforward, but it added another layer of complexity to the reducer implementation to ensure that it was added, changed, and removed for all the relevant cases:

Arrival time exists or doesn’t exist.
Arrival time is updated manually by the user.
System time is too close to arrival time to set an alarm.
Journey is ended by the user before arrival time.
Live Activity is dismissed from the lock screen.

And more.

Permission dialog

A last minute annoyance with AlarmKit was testing localization: I found a bug where the localized text for the AlarmKit permissions dialog was not being used on iOS 26.0. But the bug was fixed for iOS 26.1. And no, the bug nor the fix were mentioned in any official SDK release notes.

Localization

One of my favorite usages for coding agents is doing Localization setup. Note I’m specifically not talking about LLMs doing the actual translation, but instead:

doing the initial conversion from inline strings to string keys e.g, loaded.end-journey-button.
adding localizer comments to each string key e.g. Button to end the current journey.
maintaining default values for when string interpolations are required e.g.

Button(
    String(
        localized: "loaded.arrival-alert.status.minutes-before",
        defaultValue: "\(mins) min before",
        comment: "Status showing minutes before arrival (for menu items)"
    )
) { ... }

The initial conversion uses a custom markdown document with some basic rules. It takes about an hour for the first run and then a few more passes with a human in the loop to ensure the xcstrings file is clean.

Dynamic Type

The app still lays out pretty well with most levels of Dynamic Type. I only use semantic font qualifiers. All content is in a scroll view that’s usually fixed.

Landing screen with various Dynamic Type sizes

Trip screen with various Dynamic Type sizes

VisionKit Camera

I’m using VNDocumentCameraViewController as the integrated camera view for scanning. The UX is a little weird because there’s no way to limit the input (output?) to one photo. The result is the user can take a bunch of photos of their ticket before they tap “Done” and the app will only read the first.

Project stats

By the numbers

Development time: ~6 days including App Store materials
Lines of Swift (excluding tests, static data, etc.): 8,388

Daily Devlog

Dec 16, 2025 — Initial layout & OCR foundation
Dec 17, 2025 — Live Activities, AlarmKit, failure states
Dec 18, 2025 — Bilingual OCR, loading animation, app icon, localization
Dec 19, 2025 - Parser rewrite, settings view
Dec 20, 2025 — Polish, persistence, supported formats view, App Store prep
Dec 23, 2025 — Ticket image modal, App Store materials, v1.0 Release

Conclusion

Part of the appeal of this idea is that the scope could only creep so much. I honestly didn’t leave much on the TODO list for a version 1.1 besides endless optimization potential for the parser.

As usual this post was brain-dump style. If there’s any part you connected with and would like me to explore further, feel free to give me a shout.

Shinkansen Live: Scan Your Ticket, Get a Live Activity

Wed, 24 Dec 2025 05:53:39 -0600

Today I’m releasing my latest iOS app: Shinkansen Live or 新幹線ライブ in Japanese.

Shinkansen Live on the App Store

Shinkansen Live app icon

The concept is simple: you scan your Shinkansen ticket or receipt and you can see the details of your trip in a Live Activity on your lock screen and Dynamic Island.

Scan flow in 3 panels: scanning, ticket, lock screen

Here’s a quick screen capture of the main flow:

Motivation

I took a quick Shinkansen trip from Omiya (north Tokyo) to Karuizawa (Nagano) last week to do some co-working with my friends Jens and David. I had pre-purchased a reserved seat with the Eki-net, JR-East’s Shinkansen app (on iOS, it’s a web app wrapper). Similar to when I have a physical ticket (but somehow worse?) I found myself opening the app repeatedly to check my ticket’s listed attributes for:

departure time: in the 30 minutes or so lead up, before I’d entered the gates.
train number: to cross reference and check the platform I should leave from.
car number: when it was time to ascend to the platform and look for where on the platform I should line up.
seat number: when the train pulled up and I was boarding.

Both of my train apps Eki Bright and Eki Live have Live Activities support that I use frequently. I only ride the Shinkansen a few times a year, but while riding up to Karuizawa that day, I wondered, couldn’t I just OCR the Shinkansen ticket info from screenshot and stuff it into a Live Activity?

And so as soon as I arrived at Sawamura Roastery in Karuizawa, I got to work on prototyping a new app. My goal was to have a prototype by the ride home. With some extra polish it ended up taking a few more days of work.

Co-working vibes at Sawamura Roastery in Karuizawa

Features

The structure of the app is essentially a landing screen, a processing screen, and a trip-in-progress screen. The Live Activity requires its own multiple states and layouts of UI. For polish, I needed an error screen, an about screen, and a screen explaining what ticket formats are accepted.

Supported input formats

Landing screen showing input options

My initial scope was just handling screenshots from Eki-net (JR-East’s app) and SmartEX (JR-Central’s app), and in retrospect this probably would have better line to draw in the sand for version 1. However, I added support for scanning physical tickets too since the app seemed like it would be too specialized without physical tickets, probably the majority use-case.

And so, you can scan your physical ticket with the camera, choose a screenshot from the Photo Library, paste an image from the system pasteboard, or create an empty ticket if you want.

OCR and parsing

Scanning a ticket with OCR in progress

As of version 1.0, the app uses on device VisionKit to recognize text in the image and custom algorithm to do error recovery and parse out the relevant attributes from the ticket. I’ll discuss the development aspects of this decision in a future post, but for now, I’ll say that the merits of using OCR over multi-modal LLMs are that OCR is very fast, maintains privacy, and is accurate enough for a V1.

Trip in-progress

Trip in-progress screen showing ticket details

Once the ticket is scanned and parsed, you land on the trip screen for the remainder of your journey.

I recreated a facsimile of the legendary Shinkansen ticket. While doing research into ticket formats, it was surprising to see how different the information layouts are depending on where and by what means they are purchased, but the aesthetic is generally the same.

For the case of physical tickets, parsing is imperfect, so I wanted to ensure users could recover from minor errors like a missing time or train number. Therefore, all fields are user editable by tapping.

Editing screen with editable ticket fields

I also include the input image that a user can reference in an expanded view. This makes it easier to double check values and fix mistakes.

Live Activity

Live Activity in Dynamic Island compact, expanded, and lock screen views

Finally, the whole point of all this is to have a functional Live Activity. The Live Activity has a before and during layout. Before departure we show the departure time, train number, car number, and seat number (for reserved seats). During the trip, we show the arrival station and time.

Due to a technical limitation with Live Activities, I use Location Services to monitor for significant location changes in the background, and use that to wake up the app and update the Live Activity when the departure time has passed. On the technical side, this means I don’t need to run a push notification server or do any other networking from the app.

Arrival alarm

I’m a chronic sufferer of a disease called Scope Creep (this is a joke), so I couldn’t help but add an optional arrival alarm feature. This feature uses the new iOS 26 AlarmKit framework.

Alarm setting in trip screen and full screen alarm notification

Animations and transitions

I spent a unreasonable amount of time working on the animations and transitions for this app. Since there’s comparatively not a lot of screens or unique transitions to handle, it felt like a good opportunity to push the limits and make the upload experience more delightful. After all, there’s not a ton of benefit to cost when you consider needing to download an app, and then screenshot or photo your ticket in order to get that slight benefit of not needing to unlock your phone or take your ticket out of your pocket. Hopefully some fun animations add to the motivations to get over that mental hump.

Scanning transitions and animations

Image popup transition

Outlook

Shinkansen Live is free on v1.0 release. I have no idea whether the App Store listing will get views, whether the listing will convert to downloads, whether the idea will resonate with people to try, and whether any one-time users will keep the app on their devices and remember to use it. I don’t use the Shinkansen enough to estimate this well.

Regardless, I’m glad the app exists now. I hope it saves at least a few people that little extra friction in an otherwise smooth Shinkansen journey.

You Are a Frameworks Engineer at Big Tech Corp

Fri, 19 Dec 2025 15:33:38 -0600

You are a Frameworks engineer at Big Tech Corp (BTCorp).

You got the job out of university a few years ago and have mostly enjoyed it.

You’ve gotten used to the pace. Yearly release cycles.

You just finished your last release cycle working on Framework X, a new key feature in BTCorp’s lineup that was internally forecasted to sell lots of hardware.

You’re proud of what was shipped, but know there’s still a lot of work to be done.

You’ve just been reassigned to a new team though. In fact, all of your colleagues working on Framework X have been reassigned. Word is that Framework X is essentially soft deprecated and will never receive any new features or bug fixes. Word is that it was poorly received by tech bloggers and marketing is already starting to scrub it from any materials they can.

You’ve been reassigned to Framework UI. Framework UI is a new UI framework for BTCorp’s hardware. It’s been a skunkworks project for a few years, but finally has the blessing of a VP. After all, Framework CoreUI, the UI framework that’s been around for decades, has an awful reputation outside the company for being too hard to use. BTCorp has been losing developer mindshare and it’s finally affecting the bottom line.

You and all your colleagues are used to Framework CoreUI though. There’s plenty of knowledgeable individuals inside the company that have built or used Framework CoreUI; anyone inside the company can read Framework CoreUI’s source code; and there’s plenty of internal documentation at the source level.

You’re happy to be on the Framework UI project. It seems like an interesting technical challenge. Supposedly there’s a lot of executive team members that are monitoring it closely.

You could spin this into a coveted L5 promotion.

You learn that no one on the skunkworks team (or BTCorp at large) has built software with Framework UI that has shipped to users outside the company yet, but that doesn’t bother you. BTCorp is well known for being secretive about new projects.

You meet with your new team and get assigned your function. You’ll be responsible for everything to do with this function and have a year to get it implemented, documented, tested, and successfully presented at the next World Wide Developers Meetup (WWDM).

Your function is called PaintColor. It takes a Path and Color instance. There’s another team working on Color.

Your PaintColor function is responsible for drawing color on a path. Any color on any path.

You work with your tech lead to understand the requirements. They tell you you should be calling through to Framework CoreUI under the hood. Most of Framework UI is actually just a wrapper over Framework CoreUI.

You’re told the underlying goal is to abstract away a lot of the complexity of Framework CoreUI so third-party developers are less intimidated by BTCorp’s tech stack.

You get to work and quickly realize that this isn’t going to be as easy as you thought!

You’re not just calling through to the existing Framework CoreUI PaintColor function. Framework UI is in BTCorp’s oldest programming language. And you learn there’s a new system in place in Framework UI that defers draw calls. There’s a new concurrency system being tested but it’s not ready yet.

You dig some more and find that going down to the byte level there’s some very complicated mapping you need to do to convert colors. It’s no problem because you’re well versed in algorithms and data structures (you passed the coding test during your interview process at BTCorp after all).

You spend the next couple months getting the implementation perfect. There’s tons of transforms and bit shifting. It’s annoying you’ve had to rewrite it from scratch a few times already (the concurrency system keeps changing), but it’s okay because this is your function and you’re committed to getting it right.

You’re satisfied with the implementation. In your automated testing, it displays all your test colors fine.

You’ve got plenty of time before the World Wide Developers Meetup.

You start documenting your function.

/// Paints a color on a path.
///
/// - Parameters:
///   - Path: a path.
///   - Color: a color.
function PaintColor(Path, Color)

You check the existing documentation for other functions in Framework UI and yours is far and beyond the most detailed. You were originally thinking about going into more detail about the edge cases, but you figure there’s still so much about Framework UI that’s in flux that it’s best not to overcommit.

You hand off your documentation to the documentation team to copyedit and give final approval. They’re already very busy and understaffed, so they tell you to check back in about a month.

You start working on your 30 minute presentation for WWDM. You’re told to emphasize how easy it is to paint colors on paths using Framework UI.

You put together the simplest example you can in your first code sample on slide 24: PaintColor(Path.square, Color.default).

You see nothing on the screen. Hmm… what’s going on? you think.

You start debugging. Color.default is an alias for Color.blue, the default system color. Color.blue doesn’t show up either.

You try the other semantic colors: Color.yellow, Color.red, Color.green. All of them display fine. It’s just Color.blue that’s not painting.

Your function’s only purpose is to paint colors and there’s a color it won’t paint.

Your earlier testing was only on Colors initialized from random hexadecimal strings. Semantic colors were only introduced recently by the team in charge of colors.

You dig deeper and find the exact low-level byte-code representation of Color.blue that was chosen by the team. You put that directly into your function and it’s broken.

You dig even deeper and realize this goes far beyond your understanding of the system. As far as you can tell though, it’s only this one color value that maps incorrectly into the Framework CoreUI painting function.

You start freaking out a little bit. You realize that you’re going to need some help from the experts in the language team, the concurrency team, and Framework CoreUI team to get this solved.

You go talk to your manager. You explain what you’ve uncovered. Those teams are busy, your manager says. The WWDM presentation is the most important thing right now, your manager says, let’s focus on that.

You go to lunch and meet with some friends on other teams. You explain the situation as a hypothetical and ask what they would do. No one inside BTCorp is going to use Framework UI, so who cares, they say.

You decide to stop by to see the Color team on the way back from lunch. You ask them if they have authority to tweak the semantic color values and they say no, you’ll have to talk to the design team.

You head over to the design team’s office, interrupting a designer at your level and they seem annoyed. You explain in too much detail the function you’re working on and the situation with that exact shade of blue. They don’t understand any of the terms you’re using and besides, all this seems like your fault.

You ask the designer if they would please just shift the semantic blue color one bit in either direction. It would be imperceptible to users, you say with an unearned certainty that can’t hide your desperation. The design team will not relent. It has to be that blue. This was decided months ago and is already used in all the promotional materials.

You go back to the team in charge of the Color implementation. You ask, could we add a new semantic color to the API called Color.paintColorBlue that’s one bit off from blue? It will make it easier for all third-party developers to work around the bug during the beta period. The Color team’s manager does not think this will be aesthetically pleasing in the API and may cause confusion. All affected third-party developers can just redefine the color themselves with the hexadecimal initializer, they say.

You give up with workarounds for now, confident you’ll find time and resources to fix the bug before the final release that’s still a couple months away.

You finish writing your WWDM presentation, using Color.red to show off your PaintColor function. At the end of the presentation, you add a quick aside that Color.blue and Color.default are currently broken.

You get a note back from the WWDM presentation review team that says they’ve cut the section about PaintColor being broken. The policy is not to mention any known bugs or malfunctions, as it makes BTCorp look bad and undermines confidence in the new Framework UI framework.

You ask the documentation team if you can still add a “known issue” to the Framework UI beta release notes. They respond that it’s too close to WWDM and they don’t have time. They haven’t reviewed your main documentation yet either, but making any change will move it to the back of the queue. And you’re planning on fixing the bug before the main release, so adding any note about it for just the beta period doesn’t make sense.

You record your WWDM presentation.

Your presentation is lauded as one of the best at WWDM. All third-party developers are really excited about Framework UI. From what everyone can see, it’s so much easier to paint colors on paths now. All the examples from the presentations, including yours, look like such an advancement over Framework CoreUI.

You attend a meet-the-framework-engineers session at WWDM, where third-party developers line up for hours to ask you and your colleagues questions. You’re terrified someone’s going to call out your bug. Luckily, Framework UI has only been available for a few hours, so only one developer asks you about the bug. You tell them to try restarting their computer and to get back in line. By the time they get to the front, you’ve gone home.

Your next few months in the beta period after WWDM are hectic. You’ve been pulled away to fix more important bugs in Framework UI. Management is panicking because it’s become increasing clear that Framework UI is not yet capable of being used for real software.

You watch as Framework UI is released on time anyway. Long time third-party developers are angry they’ve invested time in learning it and trying to build apps with it before the beta period ends. But management still considers it a success, because the narrative among the tech press is positive and new third-party developers are onboarding at record pace. Veteran third-party developers are simply using Framework CoreUI as they always have, just like the developers inside BTCorp.

Your performance review season begins, and at the meeting your manager doesn’t mention anything about the Color.blue bug you spent months agonizing over being unable to fix. Your manager does, however, mention a peer review from the design team that states you were “hard to work with”. Regardless, you weren’t going to get a promotion anyway, but your manager now has something documented to justify it.

Your colleagues called it: it’s been a few months since launch and no developer inside BTCorp has attempted to use Framework UI for production work. Most haven’t even read the Getting Started page that the documentation team worked so hard on.

You decide to jump ship and quit BTCorp, using the outwardly successful launch of Framework UI to secure your next gig at Competing Tech Corp.

You check the third-party developer message board on your last day at BTCorp. You’re curious to see if there’s any feedback about Framework UI as you haven’t really kept up with the launch.

You scan the first page and see a message on the board with a title that mentions Color.default not working. The original poster tracked down the issue to Color.blue and even mentioned the same workaround.

You respond anonymously:

`Color.blue` is not supported in `PaintColor` at this time. 

If you're interested in this functionality, please submit a report to the external bug tracker.

You hit “Send” and not a moment later, the original poster has already responded, exasperated that they’ve already spent hours chasing down the bug, asking why supporting Color.blue would need to be considered as a separate feature, and why you couldn’t have added this to the documentation.

You close your laptop and leave your badge at the reception desk on your way out.

SwiftUI Group Still(?) Considered Harmful

Fri, 12 Dec 2025 10:37:39 -0600

A number of years ago, I internalized a SwiftUI axiom after getting burned on what appeared at first as a heisenbug.

The axiom:

Never use Group (with only a few exceptions).

TL;DR: I still think this is a useful axiom, although at some point over the last several iOS updates the behavior of Group is less harmful when applied naively. But there are still some reasons why it’s useful to treat it with caution.

`Group` distributes its modifiers amongst its subviews

Group is documented as a wrapper View. From the official docs (emphasis mine):

Use a group to collect multiple views into a single instance, without affecting the layout of those views, like an HStack, VStack, or Section would. After creating a group, any modifier you apply to the group affects all of that group’s members.

Apple specifically calls out what “affects all of that group’s members” means in the next paragraph, with an accompanying code sample:

Group {
    if isLoggedIn {
        WelcomeView()
    } else {
        LoginView()
    }
}
.navigationBarTitle("Start")

The modifier applies to all members of the group — and not to the group itself. For example, if you apply onAppear(perform:) to the above group, it applies to all of the views produced by the if isLoggedIn conditional, and it executes every time isLoggedIn changes.

This burned me in the past because I had a screen pattern like the (very simplified version) below:

struct ContentView: View {
    @State var isLoading = true

    var body: some View {
        Group {
            if isLoading {
                ProgressView()
            } else {
                DataLoadedView()
            }
        }
        .onAppear {
            fetchData()
        }
    }
    
    private func fetchData() {
        // make network request
        // set `isLoading = false` in completion handler
    }
}

With Group’s documented behavior, the onAppear modifier is essentially distributed across the Group’s views:

Group {
    if isLoading {
        ProgressView()
            .onAppear {
                fetchData()
            }
    } else {
        DataLoadedView()
            .onAppear {
                fetchData()
            }
    }
}

The bug was (at the time) that fetchData will be called twice: once when ProgressView appears, and once again when DataLoadedView appears.

Of course, there are many new modifiers and patterns since iOS 13 or 14 or whenever I was bitten by this (probably before the documentation was added). Regardless, after learning about this behavior it sort of makes sense. And so I learned my lesson that Group should not be used in this case.

What’s changed with `Group`

I’d locked this knowledge away and hadn’t considered it in years. However, coding agents seem to love to use Group to write the exact buggy code pattern I just illustrated above. I was curious enough to investigate it again.

Well it turns out that in iOS 26 and maybe even as far back as iOS 15, in most cases Group no longer distributes its onAppear or task calls amongst its subviews. Read on for the caveats.

From my testing, onAppear, onDisappear, task, and maybe other modifiers seem to have been special-cased by Apple to work at the Group-level and not be distributed to subviews like they used to. Note that this means the documentation for Group (that I quoted above) is now incorrect.

`onAppear` and `task`

Consider the following View:

struct ContentView: View {
    @State var isLeft = true
    var body: some View {
        Group {
            if isLeft {
                Rectangle().fill(.red).frame(width: 50)
                    .frame(maxWidth: .infinity, alignment: .leading)
                    .onAppear {
                        print("left")
                    }
            } else {
                Rectangle().fill(.blue).frame(width: 50)
                    .frame(maxWidth: .infinity, alignment: .trailing)
                    .onAppear {
                        print("right")
                    }
            }
        }
        .onAppear {
            print("group")
            Task {
                try? await Task.sleep(for: .seconds(1))
                isLeft.toggle()
            }
        }
    }
}

According to the docs, this code should loop between the left and right rectangles. However, as of iOS 26 (and as far back as I can test, iOS 15), it runs the onAppear modifier once and stays showing the right blue rectangle.

The console:

left
group
right

Screen-level Views

I stumbled on this lonely bug report from June 2020:

Inconsistency in how Group’s onAppear and onDisappear are called - Using Swift - Swift Forums

Something that I noticed today and didn’t expect it is that if a Group is not the root view of the screen, its onAppear is called per each child, while for a Group that is the root, the method is called once, regardless of the number of its children.

struct ContentView: View {
    var body: some View {
        Group { // Group is the root of the screen, onAppear is called once
            Group {  // non-root view, onAppear is called once per each child
                Color(.red)
                Color(.yellow)
            }
            .onAppear { print(".:. onAppear2") }
            .onDisappear { print(".:. onDisappear2") }
            
            Color(.blue)
            Color(.purple)
        }
        .onAppear { print(".:. onAppear1") }
        .onDisappear { print(".:. onDisappear1") }
    }
}
/* Output:
.:. onAppear1
.:. onAppear2
.:. onAppear2
*/

My guess is that this was reported during iOS 13, right before iOS 14 beta.

I tested this code as well, and it turns out the OP’s example no longer prints onAppear2 twice on iOS 15 or iOS 26.

List

The only case I’ve found (so far) where the documented onAppear (and task) distributed-across-subviews behavior still exists is when Group is within a List:

struct ContentView: View {
    @State var isLeft = true
    var body: some View {
        List {
            Group {
                if isLeft {
                    Rectangle().fill(.red).frame(width: 50)
                        .frame(maxWidth: .infinity, alignment: .leading)
                        .onAppear {
                            print("left")
                        }
                } else {
                    Rectangle().fill(.blue).frame(width: 50)
                        .frame(maxWidth: .infinity, alignment: .trailing)
                        .onAppear {
                            print("right")
                        }
                }
            }
            .onAppear {
                print("group")
                Task {
                    try? await Task.sleep(for: .seconds(1))
                    isLeft.toggle()
                }
            }
        }
    }
}

The console:

left
group
right
group
left
group
right
... (repeats forever)

Regular modifiers with Group

As a check that Group still distributes its modifiers in the general case, I created a simple custom modifier that prints on init and applied it to the Group:

struct PrintModifier: ViewModifier {
    init() {
        print("modifier init")
    }

    func body(content: Content) -> some View {
        content
    }
}

struct ContentView: View {
    @State var isLeft = true
    var body: some View {
        Group {
            if isLeft {
                Rectangle().fill(.red).frame(width: 50)
                    .frame(maxWidth: .infinity, alignment: .leading)
                    .onAppear {
                        print("left")
                    }
            } else {
                Rectangle().fill(.blue).frame(width: 50)
                    .frame(maxWidth: .infinity, alignment: .trailing)
                    .onAppear {
                        print("right")
                    }
            }
        }
        .modifier(PrintModifier())
        .onAppear {
            print("group")
            Task {
                try? await Task.sleep(for: .seconds(1))
                isLeft.toggle()
            }
        }
    }
}

The console:

modifier init
group
left
modifier init
right

As we can see, PrintModifier is being applied to each subview independently as is documented. We’ve somewhat proven to ourselves that in the general case, Group still distributes modifiers.

When is Group useful?

When is Group still the right choice?

Honestly, not very often!

Distributing a lot of modifiers across sibling views

If you really really really need to apply one or more modifiers independently to a set of sibling views and you need the sibling views to stay legible to their current parent container, Group is the right choice.

In the below toy example with Form:

Form {
    Text("Title")
    Group {
        Text("Subtitle")
        Text("Description")
    }
    .foregroundStyle(.secondary)
}

I would prefer simply duplicating the modifier manually:

Form {
    Text("Title")
    Text("Subtitle")
        .foregroundStyle(.secondary)
    Text("Description")
        .foregroundStyle(.secondary)
}

In my experience, the conditions that lead to Group being useful for this case are exceedingly rare. In a quick search of my current codebase, I have only a couple examples.

This one just barely makes sense as it applies 4 modifiers to these 2 slightly different conditional subviews. Not quite large enough to justify separating out into named views; not quite small enough to duplicate the modifiers inline.

Group {
    switch routeOption {
    case .departure, .departureOnly:
        HStack(spacing: 3) {
            Image(systemName: "arrow.up.right").imageScale(.small).bold()
            Image(systemName: "train.side.middle.car")
        }
        .padding(.horizontal, 10)
    case .arrival, .arrivalOnly:
        HStack(spacing: 2) {
            Image(systemName: "train.side.middle.car")
            Image(systemName: "arrow.down.right").imageScale(.small).bold()
        }
        .padding(.horizontal, 11)
    }
}
.font(.caption)
.foregroundStyle(.secondary)
.padding(.vertical, 10)
.background(Material.ultraThick, in: RoundedRectangle(cornerRadius: 4))

Overcoming the 10-subview limit (no longer applies)

Before Swift 5.9’s variadic generics, ViewBuilder could only handle 10 non-enumerated subviews. This example is still in the Group docs but no longer applies:

// No longer applies as of Swift 5.9
var body: some View {
    VStack {
        Group {
            Text("1")
            Text("2")
            Text("3")
            Text("4")
            Text("5")
            Text("6")
            Text("7")
            Text("8")
            Text("9")
            Text("10")
        }
        Text("11")
    }
}

Comprehensible Later: A Read-it-later App for Language Learners

Sat, 15 Nov 2025 05:57:31 -0600

This post is a short retrospective on Comprehensible Later, my working-title for a read-it-later iOS app prototype I worked on last week. Although it’s currently in private beta on Test Flight, I want to share the motivation and technical challenges I ran into while working on it.

Share Extension presented from Safari, main app article list, and article detail screens

TLDR: Comprehensible Later is an iOS app for saving articles natively written in a language you’re learning, with automatic translation via LLM to a simpler version of your target language. The goal is to give you more interesting things to read at a level you can understand without needing to pause to look up every other word.

Original level and simple level for a Japanese article

Original level and simple level for an English article (from this blog)

What is Comprehensible Input

Comprehensible Input is part of a language acquisition framework first introduced by Dr. Stephen D. Krashen. The framework states that language is separately acquired and learned. Acquisition happens by ensuring ample input (reading or listening) with the important caveat that the input is comprehensible at the learner’s current level. Learning happens through comprehensive study of rules and vocabulary. From this summary:

When we receive comprehensible input, the conditions are met for our brain to be able to use its natural ability to acquire language, without having to do anything else. There’s no need to study, review vocabulary, or practice anything. Watching and reading itself results in acquisition.

In some senses, this method seems intuitive, not in the least since almost all children acquire language skills before they begin formal teaching. In a second-language context, graded readers – books written for various non-native language levels – have existed for over a century. Wikipedia even has a simple English language variant for many common articles. I’ve occasionally used the modern Satori Reader service for Japanese graded texts.

But I think the important part is recognizing exactly how basic you need to make some input in order for it to be understandable, especially at the absolute-beginner level. In a since-removed introductory YouTube video from the creator, he shows a session of an instructor sitting with a zero-level beginner student, pointing at vivid images in a travel magazine and gesturing heavily while explaining the contents very slowly in the target language as the primary means of bootstrapping.

Language is so multi-dimensional that it’s incredibly time consuming – both as a creator and a consumer of materials – to get the exact level of material that is both comprehensible but challenging enough to increase your overall ability. Then add another dimension of motivation: as a reader, how do you find materials with a subject matter that’s interesting to you and will keep you motivated to push through word-after-word, page-after-page, day-after-day?

This led to a hypothesis:

Graded readers are usually close to the correct difficulty to facilitate learning, but do not have an audience wide enough to support a variety of interesting subject material.
Native materials cover an infinite range of interesting topics, but are infeasible to read until the latest stages of language acquisition.
One of the most commonly accepted use-cases for LLMs is text summary and translation.

What if we used LLMs to translate any native article on-demand to the user’s exact target language level?

The barriers to this being feasible are:

Can an LLM properly translate from e.g. Native Japanese to JLPT N4 level Japanese in a “natural” way – where it is simultaneously challenging, comprehensible, and accurate?
Can the translation happen fast enough to fit within a user’s desired language-learning workflow?
What additional resources are required to facilitate language learning? In-line dictionary lookup? An SRS system? Customized word lists?
What unique points are there to each target language that increase the interface complexity? For example, for Japanese learning, should we include furigana for all potentially unknown kanji?
Does it also make sense to allow translation from e.g. Native English to simple Japanese (if the target language is Japanese)?

Specification for the prototype app

I’ve used 2025 frontier LLMs to write and translate simple versions of text before, so I’m confident it’s either possible now at a minimally acceptable translation quality or will be in the very near future.

What I wasn’t confident about is whether it’s cost prohibitive or time prohibitive to use the highest quality reasoning models to do the translation.

In retrospect, I should have spent at least a little more time doing bench testing on the API versions of various models on a wide array of sample articles. Instead, I took the less (more?) pragmatic route of jumping into the implementation for an app prototype that I could start using ASAP in context, as well as distribute to a few friends.

My initial thought was that my main source of content would be blog posts and news articles I come across from my everyday feed scrolling. But I also felt I should support translating raw text too, like that from social media posts.

I considered a Safari Extension to replace existing text on a webpage with the simplified translation, similar to how the built-in translation function in Safari works. But my gut-feeling was that this would be too limiting for language learning use cases. Even reading a text at a simpler level of a target language still takes enough time that it would be better to ensure the user doesn’t feel obligated to read everything at once. Additionally, this wouldn’t work for native text outside of Safari.

My next thought was a Share Extension. Share Extensions are old iOS technology, but still highly used and useful. In a share extension I could display the translated article content in a dedicated modal and have full control over its presentation and layout.

However, I also wanted to support the read-it-later use case. Personally, I stumble upon articles when doing feed scrolling sessions when I have a few minutes on the train but don’t necessarily have the time to read the whole article, even in English, at that time. I use Instapaper for read-it-later for English articles and I felt this would be a similarly useful use case to model my app after.

With that in mind I got to work on the actual prototype with the following initial spec:

A native app that:
- keeps a list of articles imported from URLs or as raw text.
- has a detail view that shows both the original and translated versions of the text.
A share extension that:
- immediately processes the shared URL or text and displays it in the share modal.
- allows the user to optionally save the translated text in the app for later.

Note that as of iOS 18.4, there’s actually another option for the interface: TranslationUIProviderExtension. iOS users can replace Apple’s Translation app with another translation app, meaning the Translate menu tooltip can open a third-party app. I have mine set to DeepL. Due to the limitations I’ll discuss later (namely, translation processing time), it doesn’t make sense yet to implement Comprehensible Later as a Translation Extension.

Implementation

I worked through several iterations of a detailed implementation spec with Claude and Codex then set them off to work getting the foundations of the app in place. This wasn’t exactly vibe coding because I specified technologies and packages to use up front and guided their output along the way. But I was still aiming to have the agents create the clay that I’d be molding in a distinct second phase of development.

Packages

The key packages that would make this closer to a weekend prototype and not a months-long project were:

swift-readability - wrapper for Firefox’s reader-view parsing library for stripping down a full page HTML to its essential content.
AnyLanguageModel - use any LLM API with Apple’s Foundation Models SDK interface.
swift-markdown-ui - display the full Markdown spec in SwiftUI (note: I later replaced this).
Demark - convert HTML-to-Markdown.
Ink - convert Markdown-to-HTML.
sqlite-data - SQLite wrapper for local article storage and observable data layer for the app.

Data flow

The initial data flow for articles imported via URL was the following:

Input URL
    ↓ URLSession
HTML Data
    ↓ String(data:)
HTML String
    ↓ Readability
Clean HTML
    ↓ Demark
Markdown
    ↓ AnyLanguageModel
Translated MD
    ↓ swift-markdown-ui
Display

The initial flow for raw text:

Input Text
    ↓ AnyLanguageModel
Translated MD
    ↓ swift-markdown-ui
Display

However, due to limitations with the swift-markdown-ui package, the final version of the prototype uses this flow:

Input URL
    ↓ URLSession
HTML Data
    ↓ String(data:)
HTML String
    ↓ Readability
Clean HTML
    ↓ Demark
Markdown
    ↓ AnyLanguageModel
Translated MD
    ↓ Ink
HTML
    ↓ WebView
Display

With the bones of the app architecture and dependencies in place, I began testing and optimizing the data flow.

Readability

I found a small bug in the Readability Swift wrapper where the baseURL parameter was inaccessible to the URL-based initializer Readability().parse(url:options:). This prevented relative image tags from getting properly resolved to a full address. For example, on my website image tags look like /images/example.jpg and are resolved by my browser automatically to be either https://twocentstudios.com/images/example.jpg (the real server) or http://localhost:4000/images/example.jpg (my local machine).

Luckily, the baseURL parameter was accessible in the Readability().parse(html:options:baseURL:) initializer. As a workaround I simply needed to fetch the page data myself with URLSession.

Demark

Demark has two different HTML->Markdown parsing implementations: heavy-and-accurate or fast-and-inaccurate. Since the HTML is getting pre-processed by Readability in advance of being passed to Demark, I’m using the fast-and-inaccurate version that doesn’t load the full page in a headless WKWebView.

AnyLanguageModel

As of iOS 26, Apple’s local Foundation model is slow, not-ubiquitously available on devices, and (arguably) barely functional for most use cases, especially mine. Within a few years I expect it may be useful. Similarly, my impression is that any other MLX-compatible models runnable on an iOS device are not yet accurate or fast enough for my use case.

Therefore, I grabbed both an OpenAI and Gemini API key and wired them up to AnyLanguageModel for testing. I ran a few trials with the top-tier, mini, and nano variants and decided on defaulting to the mini variant as a compromise between speed, cost, and accuracy. Specifically, Gemini Flash 2.5 is the current default, but I suspect I could spend several weeks creating and running benchmarks across the dozens of closed and open models.

AnyLanguageModel made it easy to build a user settings-based model switcher with very little code adjustments required on my side. Technically, Gemini ships an OpenAI-compatible endpoint so I could have kept even more of the same codepath. During debugging, I realized that AnyLanguageModel wasn’t passing through the instructions parameter to OpenAI, so I submitted a quick PR and Mattt had it merged and version bumped by the next day.

In a later mini-sprint, I added a full settings screen that allows switching model provider, model, target language, target difficulty, adding custom translation instructions, and even fully rewriting the system prompt. Of course, I would never include all these settings in a production app, but it’s useful for my trusted beta testers to tinker if they so choose.

The translation settings screen

By default, my (simple) system prompt is:

Faithfully translate the native-level input markdown text into the target language with the target difficulty level.

Be creative in transforming difficult words into simpler phrases that use vocabulary at the target difficulty level. Combine or split sentences when necessary, but try to preserve paragraph integrity.

The output format should be standard Markdown including all supported markdown formatting like image/video tags. Preserve all structure from the input (paragraphs, lists, headings, links, images, videos). DO NOT ADD COMMENTARY.

Target language: \(targetLanguage)
Target difficulty level: \(targetDifficulty)
Additional notes: \(additionalNotes)

I’ll discuss my impressions of the effectiveness of this prompt a little later on.

Something I noticed almost immediately during testing was that requests were taking at minimum 30 seconds and sometimes over 60 seconds to complete. It didn’t really depend on model size either. I found the same performance characteristics for both OpenAI and Gemini APIs direct from first-party servers. I thought it might be the streaming API or perhaps some configuration in AnyLanguageModel I was not in control of, so I switched back to the single-request version. It didn’t help. I also began testing the same prompt and inputs from the API sandbox pages like OpenAI’s playground and Google’s AI Studio and saw basically the same results.

Although the slow translation speed is a pretty substantial blocker, I felt like, at least temporarily, I could work around it in the UX by leaning into the read-it-later nature of the app. I added support for Apple’s Background Tasks API so there was a greater chance that articles added early in the day would be ready to read by the time the user opened the app.

App UI

With the translation flow in place, I began shaping the app UI.

The list of articles was simple enough. I held off on adding lots of important, but not urgent, contextual actions like archiving and deleting from the list view.

I did add both “import from pasteboard” and “import from free text” buttons to the toolbar.

The article list view showing toolbar buttons for importing from pasteboard and free text

The import URL/text screen

I spent more time on the article detail view. Initially, it displayed the title, import state, and translated article. My focus for adding actions was to facilitate debugging primarily for myself and secondarily for my beta testers. This meant buttons for copying the original article text, copying translated article text, deleting an article, opening the original link, and retrying the translation (with different settings).

After some initial usage, I realized I wanted to see the original text and the translated text side-by-side so that I could compare the language usage by sentence and paragraph.

Article detail view with options to display original and translated text

Supported actions for article detail

Debug info viewer

However, the most time-consuming and impactful change was the markdown display system. This was a tough decision, but I think ultimately necessary for the first version.

Originally, I was planning to use swift-markdown-ui to display the translated markdown text in SwiftUI. This implementation was basically plug-and-play, rendered exactly as I wanted, supported images out of the box, and was performant. However, the one fundamental and unsolvable issue is that SwiftUI Text only supports paragraph level copy support and does not support character-level or word-level selection. For language learning, I absolutely need the ability to select a word and use the context menu tooltip action “Look Up” or “Translate” or “Copy” buttons. swift-markdown-ui would not be able to support this and I needed to research other solutions.

I spent nearly a full day researching and experimenting with other Markdown solutions. My second preference was to convert Markdown to AttributedString either natively or with a package, then display the AttributedString in a SwiftUI-wrapped UITextView with selection enabled but editing disabled. However, both the native and package versions of AttributedString initialization failed at properly respecting whitespace, newlines, and supporting images. My estimation was that it’d take significantly more time for me to grok the full Markdown spec, all the underlying packages, and then implement the required patches than I was willing to spend for a prototype.

Therefore, I pivoted to using a browser-based target view instead. iOS 26 was blessed with WebView, a modern SwiftUI-native implementation of the UIWebView and WKWebView UIKit views before it. With a WebView as the new target, I used Ink to convert the LLM output Markdown back to HTML, added a barebones stylesheet, and loaded these contents. I don’t love using a WebView for this use case since it’s comparatively heavy, has plenty of rendering quirks (like occasional white background flashes), and requires a full screen layout. But at the moment it’s the least-worst option.

Unfortunately, the slow translation speed meant some of the complexity of creating a fully-featured Share Extension was in vain; it didn’t make sense for the user to wait 30-60 seconds for the share extension to load a preview of the article content like I’d originally planned.

My initial vision was to load a one-page preview of the translation as quickly as possible. Then, I’d allow the user to tap a button to continue viewing the full translation in line. Or at any time they could tap a button to save the article URL (or raw text) to the main app to read later. I was planning on having an “open in app” button too, but as far as I can tell it’s not supported to open an app directly from a Share Extension.

I kept the full functionality of the share extension intact in case I can solve the translation speed issue in the future. But as another workaround, I added an Action Extension. An Action Extension appears in the bottom section of the system share sheet. Like a Share Extension it can also present custom UI, however since I already have a Share Extension I made my Action Extension have no UI and immediately save the URL to the app.

iOS share sheet showing both the Share Extension and Action Extension for quickly saving articles

Share Extension with translation complete

Import flow

App Extensions can share data on device with the main app using an App Group. When the user indicates they want to add the URL or raw text to the app, the Extension serializes an Article model to a json and writes a new file to the App Group. The main app monitors the shared App Group directory for new files. When it detects a new file, it adds the Article to the app’s SQLite database. If the Article already finished translation, it will include the translated markdown and no further processing is necessary. Otherwise, it will be queued for processing.

I chose not to share the SQLite database between the main app and the extensions because, since the app and extensions are separate processes, there are myriad issues with using SQLite in this way. Since data sharing is one way (from extension to app) there’s no need to introduce that complexity.

Adding articles from the main app instance skips the file encoding/decoding step and simply writes a new Article to the database.

The processing code is admittedly a bit fragile, but in testing has worked well enough that I haven’t felt an immediate need to rewrite it. It uses an enum Status stored alongside each Article in the database in order to manage the translation queue, including failures. SQLiteData supports observation, so both the article list view and the article detail view are always up to date on an Article’s status.

Localization

Localizing a prototype would be something I’d never consider doing before the advent of coding agents. The actual act of translation between a base language and another language is insignificant compared to the amount of additional tooling and operational complexity of introducing localization keys, adding comments, handling interpolation, handling pluralization rules, handling error messages and other strings generated deep in business logic, and handling the indirection involved in looking up the values for the keys. The new xcstrings file’s autogeneration definitely helps ease the burden. But it’s at least an order of magnitude more work in my opinion.

All that said, coding agents can automate enough of this work that I added full localization support for Japanese for one of my beta testers who wanted to try the app for converting English to simple English. I’m still cognizant of the ongoing support complexity full localization adds to a prototype, but for now it’s not a decision I regret.

Various screens with Japanese localization

Impressions so far

What I’ve learned so far is that the prompt needs to be more customized to each target language and should probably go as far as including an allow-list of words to use, especially for the most basic target difficulties.

I’ve found the models have a hard time with native Japanese news articles. Something about the language is just so dense that my first prompt attempt does not push the model to simplify enough.

Similar to what I’ve found with even commercial apps like Instapaper, a large percentage of sites now have enough paywall or otherwise reader-hostile javascript that it’s not enough to fetch a simple URL directly from the source. I’m not ready to handle the endless, unforgiving work of handling all the edge cases of the open web, so URL fetching is going to be best effort for the foreseeable future.

The Readability library itself is not perfect at parsing out text from pages that aren’t obviously written as “articles”. This isn’t all that different from the built-in Safari reader mode which isn’t universally supported across the entire web.

Seeing some of my blog posts in super-simple English was really fun. One of my ongoing goals is to write simpler without giving up my voice, so seeing how an LLM breaks up my sentences and phrases and clauses is enlightening (of course, not at all related to the use case the prototype was built for).

For Japanese, there’s some unpredictability on how the LLM deals with kanji. Usually it includes kanji as is, but sometimes it will add the reading in parentheses directly after for literally every word. For example, “果物（くだもの）を食べる（たべる）”. Native ruby/furigana support would be ideal, and possibly easier using HTML than AttributedString.

What’s next?

Comprehensible Later is on Test Flight in private beta with myself and a few friends. I’m planning on collecting feedback and evaluating the app’s potential for wider release. It could take another generation or two of LLM. It could take as long as waiting for local models to improve. Or the entire concept could be flawed. I’m not sure yet. But that’s what the prototype is for.

Regardless of the result, it was of course a good learning experience to see what it’s like to build a read-it-later service for iOS in 2025.

An E-Ink Desk Companion With M5Paper

Fri, 10 Oct 2025 13:39:28 -0500

The M5Paper on my desk displaying my daily TODO list

I’ve had an M5Paper since my days in the R&D Department at Cookpad that I’ve never known what to do with. My friend Dave has been doing a lot of microcontroller projects in his spare time, and it inspired me to get out the M5Paper again.

M5Paper ESP32 development kit with E-Ink display marketing image

M5Paper is an all-in-one development kit device that wraps an ESP32 microcontroller, a rechargeable battery, some environment sensors, and sticks on a 4.7” capacitive-touch E-Ink display. I bought it for about ¥8000 a couple years ago, but the improved version S3 goes for $59 USD at the time of this writing.

My new idea was to have the M5Paper on my desk mirroring my current TODO list for the day.

Feasibility research

I keep a daily markdown file with my TODOs at the top and then notes about the day’s work below it. I’ve been doing this for the last 2 years. Previously I’d use one markdown file for an entire year and use it as a rolling TODO list and scratch pad (this actually worked surprisingly well).

Daily notes in Obsidian with TODO items at the top

I’ve found that my daily TODO list often falls off my radar as soon as the Obsidian window gets buried behind all the others. I end up leaving TODOs unchecked or, even worse, completely forget to do a task. I realized the E-Ink display of the M5Paper was one way to keep my TODO list in my literal peripheral vision without requiring any additional TODO list management, including manual duplication or otherwise altering my current workflow.

The Obsidian vault I use for my daily notes and project notes is sourced from a folder synced with Dropbox. Therefore, I realized I could have the M5Paper run the following:

Wake up and connect to WiFi
Calculate the current date
Use the Dropbox API to fetch a file called 2025-10-10.md within a hardcoded folder path (i.e. the current date)
Display the contents of the file on the E-Ink display
Go to sleep for some amount of time (an hour?) to conserve battery

Workflow diagram showing the M5Paper fetch and display cycle

I had Claude Code and Codex help me along the prototyping journey.

First prototype

The first big decision was what environment to use for programming the ESP32. There’s the beginner-friendly UiFlow2 web IDE, the Arduino IDE, the PlatformIO plugin for VSCode, and then the more command line heavy route.

After flashing the firmware with UiFlow2 (this required downloading an out-of-date macOS app), I decided to use MicroPython with command line tools. I’ve used PlatformIO and Arduino IDE before but I wanted to see if MicroPython was more ergonomic than the embedded C++ variant.

The command line tools I ended up using were esptool and mpremote. Another reason I picked these was so my coding agents could be more helpful in being part of the development loop, although maybe the models have more training data for Arduino code?

The Dropbox part was a big unknown for me. I was banking on being able to fetch from my existing folder hierarchy with the Dropbox API. If it turned out to be impossible I probably would have given up since I’m not willing to change my entire personal knowledge-base setup.

Luckily, it was really easy to create a Dropbox App that worked only for my account. I could even generate a hardcoded API token with the right permissions that lasted 4 hours for testing. With that in place, Claude Code generated the first attempt at the code needed to get something from Dropbox onto the screen.

The next step was understanding how deployment to device was supposed to work and then later automating it. This was especially error-prone because I was intentionally putting the device into deep sleep, which would cause it to become unresponsive to the base commands. In the end, I have a Makefile with a deploy command that looks like this:

DEVICE_PORT = /dev/cu.usbserial-0214257D
ESPTOOL = python3 -m esptool --chip esp32 --port $(DEVICE_PORT) read_mac
MPREMOTE = python3 -m mpremote connect $(DEVICE_PORT)

.PHONY: deploy test list clean help reset rtc-check rtc-sync rtc-init repl-mode

deploy: ## Deploy main.py, secrets.py, and bm8563_rtc.py to M5Paper (full deployment)
	@echo "🔄 Deploying to M5Paper..."
	@echo "⚡ Resetting device..."
	@$(ESPTOOL) > /dev/null 2>&1
	@echo "⏳ Waiting for boot..."
	@sleep 4
	@echo "📁 Copying files..."
	@$(MPREMOTE) cp main.py :main.py > /dev/null 2>&1 && echo "  ✓ main.py copied" || echo "  ⚠️ main.py failed"
	@if [ -f secrets.py ]; then $(MPREMOTE) cp secrets.py :secrets.py > /dev/null 2>&1 && echo "  ✓ secrets.py copied" || echo "  ⚠️ secrets.py failed"; fi
	@if [ -f bm8563_rtc.py ]; then $(MPREMOTE) cp bm8563_rtc.py :bm8563_rtc.py > /dev/null 2>&1 && echo "  ✓ bm8563_rtc.py copied" || echo "  ⚠️ bm8563_rtc.py failed"; fi
	@echo "🔄 Final reset..."
	@$(ESPTOOL) > /dev/null 2>&1
	@echo "✅ Deployment complete!"

This essentially just copies over the Python files from my Mac to the M5Paper, but with a few extra steps to make sure the device is ready to be written to. Hard-won knowledge, but after getting this in place, the development loop is about as fast as it can be.

Putting the device in deep sleep blocks the REPL from connecting normally. Therefore, I also have a Makefile command for temporarily replacing the deployed contents with a dummy file.

repl-mode: ## Replace main.py with a stub so the device boots straight to the REPL
	@echo "🛠  Entering REPL mode..."
	@echo "⚡ Resetting device..."
	@$(ESPTOOL) > /dev/null 2>&1
	@echo "⏳ Waiting for boot..."
	@sleep 4
	@echo "🗑  Installing REPL stub..."
	@$(MPREMOTE) cp repl_stub.py :main.py > /dev/null 2>&1 && echo "  ✓ stub copied" || echo "  ⚠️ stub copy failed"
	@echo "🔄 Final reset..."
	@$(ESPTOOL) > /dev/null 2>&1
	@echo "✅ Device will now stay idle for REPL access."

Going beyond the prototype

I had my first working prototype up and running after a few hours which was great. However, there were some lingering issues the next day:

The dev-use-only Dropbox API token I had used expired after 4 hours. I needed to use the full OAuth flow to get a refreshable token.
The real time clock (RTC) was often getting confused after wake-ups or power events and either returning to the unix epoch or defaulting to an incorrect hour during the correct day.
Japanese characters showed up as rectangular boxes.
I wanted a way to refresh the contents manually by pressing the side button.

Dropbox API token

The Dropbox API token issue turned out to be the easiest solve of the above problems. Claude Code wrote me a quick script I could run on my Mac that used my Dropbox app secret credentials to return a refreshable token. I added that token to the secrets.py file on the M5Paper and now I had a long-term solution.

Dropbox app settings showing API token configuration

Clock issues

As usual, anything with clocks or calendars is the hardest problem. 1. My lack of knowledge about the M5Paper itself combined with 2. the difficulty of isolating a test environment that allowed me to observe the device without interrupting it – prolonged the debugging experience.

Through a lot of agent churn, I finally learned that the M5Paper has two clocks: ESP’s on-chip RTC and a dedicated chip external RTC connected to the lithium battery. Setting the external RTC with a one-off script during dev time would save a lot of hassle and complexity in the main loop.

Japanese characters

I sometimes leave notes in Japanese. I noticed these characters appeared on the display as unrenderable unicode blocks like ▯▯▯. Leaving them this way was not a deal breaker, but through a quick exploration of the M5 API, I found there was already a pre-loaded Japanese font I could use that solved the problem. The only down side is that the granularity of font sizes with the Japanese font makes it so I have to choose between a-little-too-small and a-little-too-large (I chose large).

Contents refresh and power management

I was originally hoping I could use the M5Paper for long stretches on battery power. Unfortunately, I think its age has left the battery in a vestigial state. Plus, I have a new charging port available for it, which makes keeping it plugged in all the time no longer an issue.

I’ve therefore set the refresh logic going forward as:

On battery power: refresh once an hour.
On USB power: refresh once every 5 minutes.
Also refresh on side button tap.

On redraws, the whole screen flickers black and white, then draws the text (quickly) line by line. This would be a little distracting to see in my field of view every 5 minutes, especially since the TODO list does not change that often (and I am the only one changing it). Therefore, I also added some diffing logic. Although the list is refreshed every 5 minutes, it will only redraw if the contents on screen have changed.

The M5Paper does have a capacitive touch screen, but in my preliminary investigations, the deep sleep interrupt does not work the same way as it does for the hardware button. Since I’ve recently changed the refresh logic to account for unlimited USB power, I could go further and keep the ESP32 awake, add scrolling, and add other capacitive touch buttons to the display. But for now I’m happy with the light feature set.

The final small task (mostly for ongoing debugging) was to add a battery level indicator to the bottom of the screen as a screen-width progress bar. I’m only using the battery level indicator to get a feel for how the internal battery handles daily usage going forward. This knowledge will inform future decisions about product direction (heavy air quotes on “product” since this is just for me).

Conclusion

If you’ve never done embedded systems development before, hopefully I’ve shown that building personal-use hardware is within reach for most software developers. There’s an infinite web of tangential devices and use cases out there (looking at you, Raspberry Pi 4 in a box in my closet). Hopefully this has given you, the reader, some ideas for fun weekend projects whose results you can enjoy every day.

The completed M5Paper desk companion in action

Let's Write a Train Tracking Algorithm

Mon, 22 Sep 2025 15:00:00 -0500

I delivered a 20-minute presentation on September 20th at iOSDC Japan 2025.

If you prefer video:

Japanese (conference): YouTube
English (post-conference recording): YouTube

Other materials:

GitHub: train-tracker-talk - Open source code and presentation materials
Blog: I Presented At iOSDC 2025 - More about the conference and presentation context
App Store: Eki Live - The app discussed in the presenation

This post is a deconstructed version of the talk with the slide images above and my speaker notes in English below.

Lately I’ve been working on an app called Eki Live.
Today I’m going to talk about a part of that app.
So what do I mean by train tracking algorithm?
Well, when riding a train, it’s useful to know the upcoming station.

On the train, we can see the train information display or listen for announcements.

But would it also be useful to see this information in your Dynamic Island?

In my talk, we’ll first review the data prerequisites we’ll need for the algorithm.
Then, we’ll write each part of the algorithm, improving it step-by-step.

We need two types of data for the train tracking algorithm:
Static data that describes the railway system of greater Tokyo.
And Live GPS data from the iPhone user.

Railways are ordered groups of Stations.
In this example, we can see that the Minatomirai Line is made up of 6 stations.

Trains travel in both Directions on a Railway.
Coordinates make up the path of a Railway’s physical tracks.

This map shows the Railway data we’ll be using.

We collect live GPS data from an iPhone using the Core Location framework.
We store the data in a local SQLite database.

A Location has all data from CLLocation.
Latitude, longitude, speed, course, accuracy, etc.

A Session is an ordered list of Locations.
A Session represents a possible journey.
Green is for fast and red is for stopped.

I created a macOS app to visualize the raw data.
In the left sidebar there is a list of Sessions.
In the bottom panel there is a list of ordered Locations for a Session.
Clicking on a Location shows its position and course on the map.

Our goal is to write an algorithm that determines 3 types of information:
The Railway, the direction of the train, and the next Station.

Here is a brief overview of the system.

The app channels Location values to the algorithm.

The algorithm reads the Location and gathers information from its memory.

The algorithm updates its understanding of the device’s location in the world.

The algorithm calculates a new result set of railway, direction, and station phase.
The result is used to update the app UI and Live Activity.

Let’s start by considering a single Location.
I captured this Location while riding the Tokyu Toyoko Line close to Tsunashima Station.

Can we determine the Railway from just this Location?

We do have coordinates that outline the railway…

First, we find the closest RailwayCoordinate to the Location for each Railway.
Then, we order the Railways by which RailwayCoordinate is nearest.

Here are our results.

The closest RailwayCoordinate is from the Toyoko Line at only 12 meters away.
The next closest RailwayCoordinate is from the Shin-Yokohama Line at 177 meters away.

We did it!
Our algorithm works well for this case.
But…

Let’s consider another Location.
This Location was also captured on the Toyoko Line.

But in this section of the railway track, the Toyoko Line and Meguro Line run parallel.
It’s not possible to determine whether the correct line is Toyoko or Meguro from just this one Location.

The algorithm needs to use all Locations from the journey.
The example journey follows the Toyoko Line for longer than the Meguro Line.

First, we convert the distance between the Location and the nearest RailwayCoordinate to a score.
The score is high if close and exponentially lower when far.
Then, we add the scores over time.

The score from Nakameguro to Hiyoshi is now higher for the Toyoko Line than the Meguro Line.

We did it!
Our algorithm works well for this case.
But…

Let’s consider a third Location.
This Location was captured on the Keihin-Tohoku Line which runs the east corridor of Tokyo.

Several lines run parallel in this corridor.
The Tokaido Line follows the same track as the Keihin-Tohoku Line

But the Tokaido Line skips many stations.

If we only compare railway coordinate proximity scores, the scores will be the same.

Let’s add a small penalty to the score if a station is passed.
If a station is passed, that indicates the iPhone may be on a parallel express railway.
Let’s also add a small penalty to the score if a train stops between stations.
If a train stops between stations, that indicates the iPhone may be on a parallel local railway.

Using this algorithm, the Keihin-Tohoku score is now slightly larger than the Tokaido score.

Let’s consider two example trips to better understand penalties.
For an example trip 1 that starts at Tokyo station…

The train stops at the second Keihin-Tohoku station.
The Tokaido score receives a penalty since the stop occurs between stations.

As we continue…

The Tokaido score receives many penalties.
Therefore, the algorithm determines the trip was on the Keihin-Tohoku Line.

For an example trip 2 that also starts at Tokyo…

The train passes the 2nd Keihin-Tohoku station.
And the Keihin-Tohoku score receives a penalty.

As we continue…

The Keihin-Tohoku score receives many penalties.
Therefore, the algorithm determines the trip was on the Tokaido Line.

We did it!
Our algorithm works well for this case.
There are many more edge cases.
However, let’s continue.

For each potential railway, we will determine which direction the train is moving.

Every railway has 2 directions.
We’re used to seeing separate timetables on the departure board at a non-terminal station.

For example, the Toyoko Line goes inbound towards Shibuya and outbound towards Yokohama.

Let’s consider a Location captured on the Toyoko Line going inbound to Shibuya.

Once we have visited two stations, we can compare the temporal order the station visits.
If the visit order matches the order of the stations in the database, we say that the iPhone is heading in the “ascending” direction.

The iPhone visited Kikuna and then Okurayama.

This ordering does not match the database, so we consider it “descending”.
In the database, “descending” maps to inbound.
Therefore, we know the iPhone is heading inbound to Shibuya.

We did it!
Our algorithm works well for this case.
But…
It could take 5 minutes to determine the train direction.
Can we do better?

Let’s use the Location’s course.
Remember that course is included with some CLLocations by Core Location.
Several points moving at a decent speed are required before Core Location adds course to a CLLocation.
And course itself has its own accuracy value included.

Core Location provides an estimate of the iPhone’s course in degrees.

Note that this is not the iPhone’s orientation using the compass.
The course value should be the same regardless of whether the iPhone is in a pocket or held in a hand facing the rear of the train.

The course for the example Location is 359.6 degrees.
It’s almost directly North.

First, we find the 2 closest stations to the Location

Next, we calculate the vector between the 2 closest stations for the “ascending” direction in our database.
For the Toyoko line, the “ascending” direction is outbound (as mentioned earlier).
Therefore the vector goes from Tsunashima to Okurayama.

We need to take a quick sidebar to talk about the dot product.
Do you remember the dot product from math class?
We can compare the direction of unit vectors with the dot product.
Two vectors facing the same direction have a positive dot product.
Two vectors facing in opposite directions have a negative dot product.

Next, we calculate the dot product between the Location’s course vector and the stations vector.
If the dot product is positive, then the railway direction is “ascending”.
If the dot product is negative, then the railway direction is “descending”.

The dot product is -0.95.
It’s negative.
Negative means “descending”.
And “descending” in our database maps to inbound for the Toyoko Line.
Therefore, the iPhone is heading to Shibuya.

We did it!
Our algorithm works well.
Let’s move on to the last part of the algorithm.

Finally, we can determine the next station.

The next station is shown on the train information display.
We’ll call this the “focus station phase” going forward.
This includes the station name (e.g. Kikuna) and its phase (e.g. Next).

The display cycles through next, soon, and now phases for each station.

On a map, here is where we will show each phase.

We calculate the distance d and direction vector c from the Location to the closest station.
We show the closest station S or the next station in the travel direction S+1 depending on d and c.

When the closest station is in the travel direction, the phase will be “next”.

A Location less than 500m from the station will be “soon”.

A Location less than 200m from the station will be “now”.

Even though the Location is within 500m from the closest station, the station is not in the travel direction.
Therefore, the phase will be “next” for the next station in the travel direction.

We did it!
Our algorithm works well.
But…

GPS data is unreliable.
Especially within big stations.
Especially when not moving.
Here is an example Location stopped inside Kawasaki station that has an abysmal 1 km accuracy.

Let’s create a history of Locations for each station.
For each station, let’s categorize each Location according to its distance and direction.

In this example, “approaching” points are orange, “visiting” points are green, and the departure point is “red”.

Focus station algorithm version 2 has 3 steps.

In step 1, we categorize a Location as “visiting” or “approaching” if it lies within the bounds of a Station.
Our rule is that only 1 Station per Railway will store a unique Location in the visitingLocations or approachingLocations array.
Usually, this is not an issue, but some Stations on the same Railway are within 200m of each other.
To disambiguate, we always choose the closest Station.

If the Location is outside the bounds of any Station that already has visitingLocations or approachingLocations as non-empty, we set the firstDepartureLocation for that Station.
It’s okay for a Location to be set as firstDepartureLocation for Station A while also being in a visitingLocations or approachingLocations array of Station B.
Additionally, there is special handling for the startup case where a railway has no Locations set yet. In this case, we try to find the closest Station opposite the travel direction and set its firstDepartureLocation.
We can then consider that Station the user’s departure station and use it to determine the focus station.

In step 2, we use the station history to calculate the phase for each station.

This is a departure phase for Minami-Senju station.
The StationDirectionalLocationHistory has only a firstDepartureLocation.

This is an approaching phase for Kita-Senju station.
Note: this would still count as an approaching phase even if there were only 1 Location in the approachingLocations array.

This is a visiting phase.
Note: this would still count as a visiting phase even if there were only 1 Location in the visitingLocations array.

This is a visited phase.
You can see the firstDepartureLocation in red.

In step 3, we look through the station phase history for all stations to determine the focus station phase.

In an example, when the latest phase for Kawasaki station is visited, then the focus phase is “Next: Kamata”

In another example, when the latest station phase for Musashi-Kosugi station is visited and Motosumiyoshi station is approaching, then the focus phase is “Soon: Motosumiyoshi”

Using a state machine gives us more stable results.

We did it!
Our algorithm works well…

But can we tell the difference between a visited station and a passed station?
Remember, we need this information to calculate a potential penalty for the railway score.

If the train is stopped within a station’s bounds for more than 20 seconds then we consider it visited.

If the train is moving within a station’s bounds for more than 70 seconds then we also consider it visited.
This case is for stations with bad GPS reception.

Otherwise we consider the station as passed.

Now I’d like to demo the SessionViewer macOS app I created.
I’ll show a journey from Kannai station to Kawasaki station on the Keihin-Tohoku line.
It takes some time for all Locations to be processed by the algorithm (top right).
But while it’s processing, I can start playback to see the journey at 10x speed (top right).
In the inspector (right sidebar), you can see the algorithm’s results updating.
Keihin-Tohoku line has the highest score (top right).
The direction is northbound (top right).
The latest phase for each station is shown (middle right).

When we reach the last Location in the Session, we can see the full Station history (middle right).
We can see the phase history for any station by clicking its current phase.
When I click on a station, I can see on the map the Locations that were used to calculate its phase.

The 5 iOS apps I created to collect this data are open source on GitHub.
The macOS app and algorithm are included as well.

The algorithm is still being improved!

But if you want to try it, Eki Live is on the App Store now.
The app starts up automatically in the background and shows the next station in the Dynamic Island.

Thanks for reading this presentation.
If you have questions or comments, feel free to reach out.

twocentstudios

Print Design Mindset to App Design Mindset

The print design era

The complexity catches up

The mindset shift

Making the shift

Introducing Eki Bright 2.0

Closing the Loop on iOS with Claude Code

Disclaimers before we start

Step 1: Building

Prerequisites

Move DerivedData location to your project folder (optional)

Document project file & scheme

Get simulators

Install xcsift

Building

Clearing DerivedData

Step 2: Installing & Launching

Prerequisites

Document the app binary location for your scheme

Find your bundle identifier

Installing the app on the simulator

Launching the app on the simulator

Navigating by URL (bonus)

Step 3: Reading Console & Log Output

Blocking vs. non-blocking

--terminate-running-process

Launching the app on simulator and reading the output

Blocking console/print direct (when you know output volume is reasonable)

Blocking console/print to file (safer for unknown or expected heavy output)

Non-blocking console/print

Non-blocking Logger/OSLog

Non-blocking console/print & Logger/OSLog

Step 4: Controlling & Viewing the iOS simulator

Prerequisites

AXe

Image Magick (optional)

FFmpeg (optional)

Reading from the simulator

Accessibility info via describe-ui

Screenshots

Video

Manipulating the simulator with taps and swipes

Strategies for increasing tap accuracy

Putting it all together

Step 5: Building, Installing, Launching, Reading Output on a Physical Device

Prerequisites

Get devices

Build for device

Install on device

Launch on device

Blocking console/print capture on device

Non-blocking console/print capture on device

Logger/OSLog capture on device (requires manual sudo)

Final thoughts

How to parameterize names, ids, etc. for these commands

Why not include traditional testing?

Don’t sleep on simctl

What does the future hold?

Going forward

Corrections

Shinkansen Live - Developing the App for iOS

Contents

Overall development strategy

OCR and parsing the ticket image

Why OCR? Why not multi-modal LLMs?

Overall strategy

OCR prototype

OCR for real Shinkansen ticket images

Using spatial data

Transforms

Levenshtein distance

English and Japanese mode OCR

Unparseable fields fallback UX

Defense-in-depth parsing system

OCR testing system

Live Activities

View guidelines and tips

Use non-semantic font sizes for compact and minimal

For dynamically updating times, prepare to spend a lot of time in trial and error

`--terminate-running-process`

Accessibility info via `describe-ui`

Don’t sleep on `simctl`

`Group` distributes its modifiers amongst its subviews

What’s changed with `Group`

`onAppear` and `task`