twocentstudios

Count Biki - Developing the App for iOS

Tue, 31 Oct 2023 14:16:28 -0500

Continuing my series of posts about my latest iOS app, Count Biki, I’ll discuss the more interesting parts of app at version 1.1.

Count Biki app icon, featuring the character Count Biki

Please check out the other posts in the series:

Count Biki - Drill Japanese Numbers - the motivation behind the app and the solution I’ve begun to explore from the learner (user) perspective
Count Biki - App and Character Design - the design process and specifics of creating the Count Biki character
This post - a guided tour of the codebase and implementation details

Japanese learners can download the app from the App Store.

Like many of my other apps, Count Biki is open source on GitHub. The commit/tag referenced in this post is v1.1, so the main branch will not fully align with the post’s content.

Stats

Before starting digging into the details, I’ll go over some stats about the app for context.

The app has 6 screens (5 with behavior).
The app has around 3600 lines of Swift code.
The app has a minimum deployment target of iOS 16.4.
The app supports only portrait mode and the iPhone form factor. There’s no technical reason for this. It was mostly to reduce complexity, promotional material requirements, and testing for the first release.
The app uses about 5 packages via Swift Package Manager: ComposableArchitecture, Collections, AsyncExtensions, DependenciesAdditions, ConfettiSwiftUI. ComposableArchitecture relies on several other pointfreeco packages, which I also use directly.
The app uses a basic xcodeproj file.
The app was archived with Xcode 17.0.1, which includes support for Swift 5.9, iOS 17, and (colloquially) SwiftUI 4, although I don’t use any iOS 17 specific APIs.
The app has a SwiftUI App entrypoint.
The only network requests made by the app are via StoreKit 2.
The app embeds an scnassets file for SceneKit-related files.
The app uses the licenseplist library installed via Homebrew for generating a license file for packages.
The app embeds no analytics framework.

Architecture

The app is built all-in with The Composable Architecture (TCA) 1.0. This includes passing and scoping Stores through the view layer, creating Reducers with State and Actions for most screens, and creating dependencies in the style of the swift-dependencies library.

This post is not a tutorial on using TCA. However, I will do my best to describe my experience learning TCA by writing this app, and mention the convenient points and the points that tripped me up while developing it.

I started rewriting an existing app with TCA back in 2020, but never finished it. I got stuck due to performance problems and the rewrite fell off my radar. Since then I’ve used SwiftUI extensively for both production and prototyping. The architecture of most of those projects has been either loose MVVM or everything-in-the-view style. This includes Goalie, which I consider MVVM style.

I started off development for Count Biki by prototyping the listening quiz feature. This is the core of the app and the most complex feature (although the in app purchase support is arguably more complex).

the listening quiz screen

Even as the most complex screen, it’s not particularly complex. But getting started was relatively slow because I was referencing the TCA 1.0 guided tour videos (as they were being released), the TCA case studies, and the OSS isowords codebase.

Even though TCA is more ergonomic than it was when I first started experimenting with it a few years ago, there were a lot more concepts to digest this time around, particularly around navigation. BindableAction, @Bindable, @PresentationState, the Destination reducer pattern.

Shared state

The most difficult conceptual problem I had with TCA was the techniques for sharing state across an app.

Parent and child reducer intercommunication

My preconception of Redux-like systems such as TCA was that there was one big bag of state and any reducer could grab any part of it. The scoping part of TCA – paring down the state and actions to hand off to a subsystem – tripped me up. It wasn’t clear to me whether changes to state made in child stores would change that same state in the parent part. In other words, I think it’s similar to the sort of implicit learning curve there was while learning SwiftUI, namely:

how does a parent communicate to a child?
how does a child communicate to a parent?
where is the source of truth for a piece of state?

I know the answers to the SwiftUI variant of these questions much better than the TCA variant, but they were certainly hard-won answers.

Quiz and settings example – source of truth in parent state

A simple example of my sharing state confusion is with my listening quiz and settings screen. The system has the following behavior:

The quiz screen needs read-only settings values.
The settings screen allows read/write access to settings values.
The quiz presents the settings screen modally.
The settings values are persisted to disk.

The quiz and settings screens (parent and child)

In view terms, the quiz screen is the parent and the settings screen is the child. But the quiz reducer needs access to the settings values immediately, which means that settings state can’t be modeled as optional view state. The “is the settings view visible?” part of the state and the “what are the settings?” part of the state can’t be combined in this case.

I modeled this by:

having a full copy of the settings values owned by the quiz reducer.
keeping a copy of the settings values as optional destination state.
loading the settings values from the dependency on initialization of the quiz state.
copying the current settings values to the settings state when the settings button is tapped as representation that the view is presented.
updating the “source of truth” version of settings values when the settings reducer sends an updated version, and ensuring the latest value is persisted to disk.

struct ListeningQuizFeature: Reducer {
    struct State: Equatable {
        var speechSettings: SpeechSynthesisSettings // 1
        @PresentationState var destination: Destination.State? // 2
        // ...

        init() {
            @Dependency(\.speechSynthesisSettingsClient) var speechSettingsClient
            self.speechSettings = speechSettingsClient.get() // 3
        }
    }
    
    struct Destination: Reducer {
        enum State: Equatable {
            case settings(SettingsFeature.State) // 2
        }
        // ...
    }

    var body: some ReducerOf<Self> {
        Reduce { state, action in
            switch action {

            // 4
            case .titleButtonTapped:
                state.destination = .settings(.init(
                    topicID: state.topicID,
                    speechSettings: state.speechSettings,
                    sessionChallenges: state.completedChallenges
                ))
                return .none
            
            // 5
            case let .destination(.presented(.settings(.delegate(.speechSettingsUpdated(newSpeechSettings))))):
                state.speechSettings = newSpeechSettings
                do {
                    try speechSettingsClient.set(newSpeechSettings)
                } catch {
                    XCTFail("SpeechSettingsClient unexpectedly failed to write")
                }
                return .none
            }
        }
    // ...
}

This strategy of keeping source-of-truth state in a parent, passing a copy to the child reducer, and playing back changes from child to parent to ensure the state is in sync is seemingly the strategy shared in the SyncUps example app.

I’m not 100% happy with my strategy. For this simple app, it’s fine. But it perhaps unnecessarily spreads responsibility for the settings across multiple reducers.

About and IAP example - source-of-truth in a dependency

Another way to go about sharing state between parent, child, or sibling reducers is:

keep the source-of-truth state in a dependency
mutating the state through the dependency
set up a one-way binding to copy the dependency state into the reducer state from any number of reducers

My first impression of this strategy was that it goes against the ethos of TCA/Redux. Especially for cases where the state is not owned by an external subsystem under only loose control by the application, like from an Apple framework. However, reading this discussion made me consider dependency-managed state as more often the preferred option.

Concretely, this means a dependency with a get/set/observe interface. As a very generic example:

struct CurrentUserClient {
  var currentUser: () -> User?
  var setCurrentUser: (User?) async -> Void
  var currentUserStream: () async -> AsyncStream<User?>
}

Then each reducer that reads/writes/observes the current user can use CurrentUserClient like so:

Create a reducer-local derived copy of the state as var currentUser: User? in UserReducer.State.
In State.init, read in currentUser from the dependency.
Add a Action.setCurrentUser(User) action.
On receiving the setCurrentUser action, update the state iff the current user has changed.
Add a .onChangeOf(\.currentUser) modifier to the reducer to write updated values to the dependency’s setter.
In Action.onTask, set up a long running async stream that sends new values back into the system as actions through Action.setCurrentUser.
The reducer should make changes to its local copy of State.currentUser.

The get/set/observe-style state allows both push and pull state management in reducers. For example, a pull-style read-only version would only need (1) (2), then an Action.refreshButtonTapped that copies the current value from the dependency into local reducer state.

Unfortunately, I’ve yet to find a more automated version of encapsulating the read/write/observe system.

The closest I got to implementing this type of system was while implementing my About and TransylvaniaTier (In app purchase) features.

the about and IAP screens (as parent and child)

The about screen is the parent. The IAP screen is the child. The about screen observes the purchases state from a dependency and updates its own state and its child state in response to changes. The key parts of the full reducer are clipped out below:

struct AboutFeature: Reducer {
    struct State: Equatable {
        var appIcon: AppIconFeature.State
        var transylvaniaTier: TransylvaniaTierFeature.State

        init() {
            @Dependency(\.tierProductsClient.purchaseHistory) var purchaseHistory
            appIcon = .init(isAppIconChangingAvailable: purchaseHistory().status == .unlocked)
            transylvaniaTier = .init(tierHistory: purchaseHistory())
        }
    }

    // ...

    var body: some ReducerOf<Self> {
        // ...
        Reduce { state, action in
            switch action {
            case let .onPurchaseHistoryUpdated(newHistory):
                state.appIcon.isAppIconChangingAvailable = newHistory.status == .unlocked
                state.transylvaniaTier.tierHistory = newHistory
                return .none
            case .onTask:
                return .run { send in
                    for await newHistory in purchaseHistoryStream() {
                        await send(.onPurchaseHistoryUpdated(newHistory))
                    }
                }
            }
            // ...
        }
    }
}

I will probably refactor the about and IAP reducers to be more independent in the future. I will probably want to present the IAP screen from more places throughout the app, which will require it to handle its own purchase history state updates.

For the record, I found writing TCA reducers without inter-system state sharing to be very straightforward and exhibit all the benefits of TCA (testability, side-effect isolation, previewability, etc.). Of course, this app has not had multiple contributors nor survived years of releases, so take my experience as you will.

During the exploratory phase of development, what I really wanted to do was have one reducer with one state and one action enum shared across all the views in my app. Of course this doesn’t scale and perhaps would cause performance problems sooner than later. But after I was more confident in the view structure, it’d be obvious how to split up the reducers and communicate state changes between them, avoiding a lot of rework. I’m sure there’s a way to do this, but while I was learning TCA I didn’t feel confident enough to stray from the case studies.

Navigation was one of the later additions to the TCA 1.0 release. Since the app only has a few screens, and I did at least a little reconfiguration of the UX, I never felt like I put the navigation tools through their paces.

Going “by-the-book”

I consider navigation “by-the-book” as storing each piece of navigation state in the reducer, usually with a separate Destination reducer.

I did this in a few places in the app that have obvious boundaries of behavior: The topics screen, the quiz screen, the settings screen, the about screen, etc.

However, in a few places within NavigationStack, I decided to take the lazy way and use raw NavigationLinks. The obvious downside to this is that the navigation state can only be directly manipulated by the user and not programmatically (notably through deep linking). In practice, I haven’t found this to be an issue yet. I found the TCA NavigationStack modeling to feel heavy enough at the outset that I didn’t want to delay my release implementing it by-the-book.

The topics screen in particular felt like there was little benefit in creating an additional reducer to handle a screen of static data. I wanted the flexibility to keep all that topic view structure and logic in one place, even though in reality it’s covering both TopicCategory and Topic entities. (Aside: it’s kind of similar how SwiftUI forms give you the ability to configure a picker as inline or as a completely separate screen added to the stack, and the presentation parts are abstracted away from the developer.)

As I get more comfortable with TCA, I’ll probably go back and move some of the navigation primitives into the TCA world.

Scoping stores

When implementing store scoping within the view layer, I found myself struggling a bit to craft the correct syntax to express the relationship between the parent and child features.

When going by-the-book, the store scoping feels like boilerplate (there’s not a lot to think about). But when I did want to play outside the sandbox a bit, I realized I was in over my head and couldn’t even get the scoping statement to compile.

The frustrating part was that I knew I either had to:

look at a lot more store scoping examples as the raw syntax
go back and rewatch the pointfree episodes about the theory behind store scoping

Or perhaps both? At this point, I can definitely tell I’m missing a key piece of my understanding of the composable part of TCA, but I’m not sure yet how to fill in that missing piece. Or at least what the most efficient way to do so is.

In the end, I mostly avoided the issue entirely by either going more by-the-book with the structure of my features or completely eschewing TCA navigation as discussed in the previous section.

Dependencies

I consider TCA to have 4 interdependent domains of complexity:

Reducers - composing the app logic by defining State and Action, and modifying state and kicking off side-effects on each action.
View-layer integration - using ViewStore and scoping Store.
Dependencies - the distinct layer of world state and side-effects.
Testing Reducers - exploiting the malleability of the dependency protocol in order to isolate the reducer layer from the dependency layer, and exploiting the one-way dependency of the view layer on the reducer layer to ignore the view layer.

The TCA philosophy is to divide responsibilities amongst the reducer, view, and dependency layers to enable maximum testing ability of the reducer layer, in both unit and integration tests (for some definition of “unit” and “integration”). UI testing is reasonable but discouraged. Dependency testing is rarely mentioned, but is of course possible depending on how you the developer choose to design your dependencies.

Writing dependencies in the pointfreeco-style

The pointfreeco-style of dependency design is unique (in my experience). And there are three distinct attributes:

The dependency interface as a struct containing one or more variables of functions.
Writing the dependency implementation as a class/actor in a way that you can create an instance of it within a static initializer of the dependency interface. When wrapping a system class as a dependency, often the system class is used directly in the static initializer.
Treating this dependency interface as a singleton throughout your application, including being initialized independently at app startup and having the same lifetime as the app.

My impression is that none of the above three attributes are absolute requirements to use TCA or the swift-dependencies library. However, most of the official examples contain the three attributes. (One notable exception is swift-dependencies-additions, which has its own internal system of proxies to enable a more convenient interface for simple getters and setters).

For reference from the TCA case studies, an example of (1), the dependency interface as a struct (SpeechClient):

struct SpeechClient {
  var authorizationStatus: @Sendable () -> SFSpeechRecognizerAuthorizationStatus
  var requestAuthorization: @Sendable () async -> SFSpeechRecognizerAuthorizationStatus
  var startTask: @Sendable (SFSpeechAudioBufferRecognitionRequest) async -> AsyncThrowingStream<SpeechRecognitionResult, Error>
}

And an example of (2), creating an actor and then using it in a static initializer:

private actor Speech {
  private var audioEngine: AVAudioEngine? = nil
  private var recognitionTask: SFSpeechRecognitionTask? = nil
  private var recognitionContinuation:
    AsyncThrowingStream<SpeechRecognitionResult, Error>.Continuation?

  func startTask(
    request: SFSpeechAudioBufferRecognitionRequest
  ) -> AsyncThrowingStream<SpeechRecognitionResult, Error> {
      // ...
    }
  }
}

extension SpeechClient: DependencyKey {
  static var liveValue: SpeechClient {
    let speech = Speech()
    return SpeechClient(
      authorizationStatus: { SFSpeechRecognizer.authorizationStatus() },
      requestAuthorization: { /* ... */ },
      startTask: { request in await speech.startTask(request: request) }
    )
  }
}

Regarding (3), the “global-ness” of dependencies, in theory, the store-scoping initializers allow you to override dependencies all the way down a reducer hierarchy. For example:

Store(initialState: SettingsFeature.State()) {
    SettingsFeature()
} withDependencies: { deps in
    // TODO: Does this create a new instance each time an action is sent to the reducer?
    deps.speechSynthesisClient = .init() 
}

Or at the reducer-level:

// TODO: Does this create a new instance each time an action is sent to the reducer?
Reduce { state, action in
    // ...
}
.dependency(\.speechSynthesisClient, .init())

However, the feasibility of initializing a fresh dependency with data from the system is unclear to me. Especially since it seems like the above code would initialize a fresh instance of speechSynthesisClient each time a reducer is run (thousands of times per app run!).

There are a few other topics on the discussion boards that mention the global lifetime of dependencies:

For more complex applications than my own, I consider this limitation to be something to be carefully considered and architected around.

The related problem of doing some sort of one-time dependency initialization and setting up relationships between dependencies at app startup is also unclear to me. I asked about it on the TCA discussions board, and discussion #1287 touches on it briefly, but due to a change in my app requirements, I no longer needed to follow up on it.

There is certainly a wealth of information in the discussion boards I’ve yet to fully digest in the context of a more complex app. (Just a reminder that I’m writing this blog post for myself too as a way to process my thoughts and understanding on these inter-related concerns.)

Heisenbugs encountered while overriding dependencies

withDependencies is most often used as a developer tool – for testing, SwiftUI previews, or temporary debugging.

There were a few times where I ran into heisenbugs where my overridden dependencies at the app root or in previews weren’t getting overridden. Or they were getting overridden in a parent reducer but not a child. These situations felt nearly impossible to debug without doing a full code-review of swift-dependencies and TaskLocals.

Overriding dependencies for use with SwiftUI Previews

A key piece of the ListeningQuiz screen is the text-to-speech playback for each question. AVSpeechSynthesizer does not work on SwiftUI Previews or even the iOS simulator. TCA allowed me to easily mock out this dependency on just Previews and the simulator so it didn’t become a blocker for quick UI development. I could spend as much time as I normally would developing off-device when I wasn’t doing TTS-related tasks.

extension SpeechSynthesisClient: TestDependencyKey {
    static var previewValue: Self {
        @Dependency(\.continuousClock) var clock
        return Self(
            availableVoices: { [.mock1, .mock2] },
            defaultVoice: { .mock1 },
            speak: { _ in
                // Simulate 2 seconds of speech time
                try? await clock.sleep(for: .seconds(2))
            },
            speechRateAttributes: {
                .init(minimumRate: 0.0, maximumRate: 1.0, defaultRate: 0.5)
            },
            pitchMultiplierAttributes: {
                .init(minimumPitch: 0.5, maximumPitch: 2.0, defaultPitch: 1.0)
            }
        )
    }
}

The use of dependencies in Count Biki

After all that foreword, I can talk at least a little bit about how I used dependencies in CountBiki.

Stateless clients

SpeechSynthesisClient is responsible for wrapping AVSpeechSynthesizer, Apple’s top-level API for text-to-speech.

My tendency for designing clients is to separate configuration and side-effects. In other words, clients should be stateless by default. For example, most NetworkClient APIs are designed like:

// Set configuration on app init or when current user is updated
NetworkClient.shared.setSessionToken("123abc")
NetworkClient.shared.setCacheSize(1024*10*10)

// Make an authenticated network request in some other part of the app
let users = try await NetworkClient.shared.fetchFriends()

In the above example, the NetworkClient is both a source-of-truth for network-related state and a way to perform side-effects.

For most use cases, this style of API is very reasonable. The benefit is that the caller of fetchFriends needs zero knowledge of the NetworkClient configuration details.

I prefer to have NetworkClient be stateless. At the callsite, the above example would look like this instead:

// `networkConfiguration` is passed around a subsystem via dependency injection or generated from existing state
let networkConfiguration: NetworkClient.Configuration = ...
let users = try await NetworkClient().fetchFriends(networkConfiguration)

One benefit of stateless clients is that it’s trivial (or at least more-so) to add multi-account support and use the same NetworkClient instance.

Another benefit is that it avoids the dependency initialization catch-22 we discussed earlier. A NetworkClient initialized at app startup and with the same lifetime as the app has no initialization race condition for needing its configuration set to a valid state before making calls.

The NetworkClient example is probably a bad one though. The underlying URLSession is stateful by design. It maintains its own cache, so wrapping it with a stateless interface could actually increase the chance of bugs, or completely break caching functionality. Another downside is that functions that could be encapsulated within the NetworkClient such as seamlessly renewing expired session tokens.

All of that is to say I used the stateless client pattern for SpeechSynthesisClient.

struct SpeechSynthesisSettings: Equatable, Codable {
    var voiceIdentifier: String?
    var pitchMultiplier: Float?
    var volume: Float?
    var rate: Float?
    var preUtteranceDelay: TimeInterval?
    var postUtteranceDelay: TimeInterval?
}

struct SpeechSynthesisUtterance {
    var speechString: String
    var settings: SpeechSynthesisSettings
}

struct SpeechSynthesisClient {
    var speak: @Sendable (SpeechSynthesisUtterance) async throws -> Void
    // ...
}

SpeechSynthesisSettings are persisted using their own client SpeechSynthesisSettingsClient. The top-level feature is responsible for fetching the settings and configuring an utterance to provide to the stateless SpeechSynthesisClient.

The implementation of SpeechSynthesisClient is admittedly weak. I should be using an actor.

Separating settings

I erred on the side of splitting persisted settings into separate clients, even though they all use the same underlying storage. Specifically: SpeechSynthesisSettingsClient and TierPurchaseHistoryClient. There’s some unnecessary indirection at this point.

Wrapping StoreKit

Implementing StoreKit for a consumables tip jar was a bit of a headache.

Consumables act differently than subscriptions, so there were a few misunderstanding before I landed on an API for the TierProductsClient dependency.

The API I wanted for consumers of the dependency was similar to the read/write/observe API I mentioned previously. However, several of the mutations happen within the dependency itself while it’s observing internals of StoreKit.

I wanted to use Combine’s CurrentValueSubject. Swift Concurrency does not have a built-in replacement for CurrentValueSubject, and using CurrentValueSubject safely within the confines of Swift Concurrency is not a fully endorsed activity (especially with compiler warnings set to maximum). I didn’t want to add another package dependency, however AsyncExtensions seemed to provide exactly the tool I needed.

Topics

I somewhat regret my Topic interface. A Topic is a wrapper for generating question/answer pairs. I thought my interface would be generic enough for all kinds of topics.

struct Topic: Identifiable, Equatable {
    enum Skill {
        case listening
        case reading
    }

    enum Category {
        case number
        case money
        case duration
        case dateTime
        case counter
    }

    let id: UUID
    let skill: Skill
    let category: Category
    let title: String
    let description: String
}

A Topic itself is only a bag of data attributes. However, I create a wrapper struct internally called TopicGenerator when defining each Topic that also provides a question generator function. This allows me to expose Topic to the rest of the app as behaviorless and Equatable, but still behavior associated with each Topic and defined inline with it.

private struct TopicGenerator: Identifiable {
    var id: UUID { topic.id }
    let topic: Topic
    var generateQuestion: @Sendable (WithRandomNumberGenerator) throws -> (Question)
}

The WithRandomNumberGenerator part turned out to be pretty useless. I thought it might be useful for testing or otherwise seeding and keeping some control over values generated, but in the end it was just another field I had to work around.

An example of a TopicGenerator definition within the code:

TopicGenerator(
    topic: Topic(
        id: Topic.id(for: 201),
        skill: .listening,
        category: .duration,
        title: "Hours",
        description: "1-48時間"
    ),
    generateQuestion: { rng in
        let answer = rng { Int.random(in: 1 ... 48, using: &$0) }
        let postfix = "時間"
        let displayText = "\(answer)\(postfix)"
        let acceptedAnswer = String(answer)
        return Question(
            topicID: Topic.id(for: 201),
            displayText: displayText,
            spokenText: displayText,
            answerPrefix: nil,
            answerPostfix: postfix,
            acceptedAnswer: acceptedAnswer
        )
    }
)

I’m defining a static ID for each Topic for a future where user answers are persisted and need to be associated back to a Topic internally.

Another part of the interface I went back and forth on was whether I should keep the generated question and user answer as Int or whether all answers should converge to String. I kept the latter method (String) but realized during development that allowing access to the numerical values would give more flexibility in comparing and reformatting values.

struct TopicClient {
    var allTopics: @Sendable () -> IdentifiedArrayOf<Topic>
    var generateQuestion: @Sendable (UUID) throws -> (Question)
}

The TopicClient itself simply abstracts away looking up a Topic by its ID and generating a Question for it.

A downside to this stateless design is that all TopicGenerators must have the input signature (they can’t take distinct parameters). Additionally, I can’t guarantee within the TopicClient that subsequently generated questions aren’t the same.

BikiAnimation

There’s a unique friction point we sometimes encounter with SwiftUI being a declarative system but needing to interface with imperative events.

One of these friction points is animations.

For my Count Biki SceneKit model, I have 1 idle animation and 2 brief animations that play for a second or two each. One animation is triggered when the user gets a question right. One animation is played when the user gets a question wrong.

What’s the right way to model this in SwiftUI?

This problem is easier to solve for something like the confetti animation that plays when the user gets a question right. We can model this as an integer that counts up by one each time the animation should play. The SwiftUI view will playback the animation any time that value changes (whether it be by 1 or 100).

struct State: Equatable {
    var confettiAnimation: Int = 0
    // ...
}

// Change `confettiAnimation` when a correct answer is submitted
Reduce { state, action in
    switch action {
    case .answerSubmitButtonTapped:
        // ...
        state.confettiAnimation += 1
    // ...
    }
}

// `ConfettiCannon` will observe changes to this value
VStack { /* ... */ }
    .confettiCannon(counter: viewStore.confettiAnimation)

// Or we can use `.animation` with any `Equatable` value
VStack { /* ... */ }
    .animation(.default, value: viewStore.confettiAnimation)

But for 2 different types of animation, how can we model it? Well we could just have 2 different counter variables and keep increasing the number of variables and the number of change observers. But this doesn’t scale well.

Another way to model this is with a UUID wrapper:

struct BikiAnimation: Equatable {
    enum Kind {
        case correct
        case incorrect
    }

    let id: UUID
    let kind: Kind
}

struct State: Equatable {
    var bikiAnimation: BikiAnimation?
    // ...
}

Reduce { state, action in
    switch action {
    case .answerSubmitButtonTapped:
        // ...
        state.bikiAnimation = .init(id: UUID(), kind: .correct)
    // ...
    }
}

SceneView { /* ... */ }
    .onChange(of: bikiAnimation) { newValue in
        switch newValue?.kind {
        case .correct:
            sceneState.playCorrect()
        case .incorrect:
            sceneState.playIncorrect()
        case nil:
            break
        }
    }

Testing

One of TCA’s biggest wins is ease of testing. I dabbled in testing, but as of the 1.1 release I’ve only written 1 test. My reasons for this lack of tests are the usual excuses: it’s an MVP; I wanted to ship and get user feedback ASAP; the view-layer and dependency-layers were actually more error prone than the reducer-layer logic; the structure of the app was constantly in flux during initial development.

Regardless of these excuses, I appreciate the testability facilitated by TCA and I’ll certainly be adding more, not fewer, tests over time to the project.

SwiftUI

Although the focus of this post has been its architecture and relationship to TCA, I do want to mention a few parts of the view layer that stand out in some way.

BikiView

BikiView is the wrapper for the Count Biki character avatar derived from a SceneKit .scn file.

Wrapping stateful view architecture like SceneKit or UIKit in SwiftUI can sometime be tricky.

The SceneState struct holds onto the loaded SCNScene containing Biki, and also handles loading and playing back the animations on demand from the separate .dae files.

Note that adding this view to a hierarchy non-trivially increases CPU usage and energy usage for the device. For now, I think this is an okay trade off, but SceneKit’s relative resource intensiveness is something to be aware of.

Styling with base iOS classes

Apple’s human interface guidelines define system colors and system typography.

When I’m not working closely with a designer, it’s much easier to rely on the Apple-provided design system than create my own one-off design system. It takes a lot of the burden off of me as an individual to ensure proper accessibility, support dark mode, ensure a base-level of visual quality, and spend time designing my own controls.

The tradeoff is that the visual style is a lot less fun and unique than it could/should be.

For now, I feel okay about this compromise. After ensuring a base level of functionality, I feel more comfortable spending time exploring visual style adjustments.

Even after 5 versions, I still feel using the Apple design system primitives can be unintuitive. For example, many of the system colors are still defined only in UIKit.UIColor and not SwiftUI.Color – for example UIColor.secondarySystemBackground. SwiftUI.Color has the limited primary and secondary variants.

The meaning of tint and accentColor is subtly different, as are their SwiftUI modifiers (.accentColor() was deprecated and .tintColor() added in iOS 15).

Semantic/Dynamic Type font styles like Font.title and Font.body don’t always map well to app styles, and there’s never been much guidance from Apple on how to do so.

Aside: I’ve never worked with a designer who understood or considered these semantic font styles or Dynamic Type. The most Apple advertises Dynamic Type to designers is these three sentences from the HIG:

Consider using the built-in text styles. The system-defined text styles give you a convenient and consistent way to convey your information hierarchy through font size and weight. Using text styles with the system fonts also supports Dynamic Type and the larger accessibility type sizes (where available), which let people choose the text size that works for them.

Laying out the quiz screen without a scroll view

It’s arguably best practice to use a scroll view for every screen in your app by default.

The quiz screen is designed to be rapid fire, and wouldn’t really make sense to force the user to scroll up and down constantly while they are trying to power through questions.

However, I was of course going against the laws of physics trying to fit in all the UI elements I wanted while still supporting all small screen sizes and accessibility Dynamic Type sizes. This was not an easy problem to solve, and for the version 1.1 release I had to cheat a bit by limiting the accessibility Dynamic Type size to the second level. I feel shame doing so, but at the moment it’s better to artificially enforce a limit rather than have a broken layout.

The listening quiz screen at maximum supported accessibility size

Adding commas dynamically to the TextField

Text input is way more difficult than it seems. There’s a ton of complexity wrapped around the simple SwiftUI interface of TextField($text).

Answering a quiz question with only numbers is much simpler than all the functionality needed for a multi-lingual, editable text field. However, since TextField already exists, I preferred to use it with as few customizations as possible before implementing my own text input system from scratch.

However, my beta users ran into problems when answering questions with long numbers like 125400000. Without comma separators, it quickly becomes impossible to know exactly which number you’ve entered.

Although it’s certainly possible, I felt uncomfortable modifying the TextField’s string while the user is typing to add separators, thinking that there’d be edge cases (especially with moving the cursor) I didn’t have time to thoroughly test. As a pragmatic (and hopefully temporary) solution, I instead added formatted text that mirrors the input string right above the TextField.

A temporary solution for formatting the input string without interrupting the default TextField handling

Conclusion

These were the various concerns running through my head while developing this simple app.

Usually when working in a team I write these kind of explanations in pull request descriptions. But in this case I’ve saved everything until the first release.

I don’t like to share “solutions” to problems before I’m completely sure they’re robust enough to survive the long term. That’s why the slant of the above text is more “here was X problem and how I solved it” than “here’s exactly how to solve X problem”.

I also share some of these thoughts day-to-day on Mastodon, so feel free to follow me there. And if you want to dig into the codebase, here’s one more link.

Thanks for reading this post or the whole series.

Count Biki - App and Character Design

Mon, 30 Oct 2023 14:36:58 -0500

Continuing my series of posts about my latest iOS app, Count Biki, I’ll discuss the planning, design, and character design details.

Count Biki app icon, featuring the character Count Biki

Please check out the other posts in the series:

Count Biki - Drill Japanese Numbers - the motivation behind the app and the solution I’ve begun to explore from the learner (user) perspective
This post - the design process and specifics of creating the Count Biki character
Count Biki - Developing the App for iOS - the high-level implementation details

Japanese learners can download the app from the App Store.

Like many of my other apps, Count Biki is open source on GitHub.

Planning

I made a big list of every number-adjacent concept in the language and the different modes I wanted to practice.

Brainstorming which topics I could cover and how I could test them

This was a bit overwhelming at first.

I did a bit of prototyping in a Swift Playground to figure out which of Apple’s APIs – speech recognition, numeral-to-text transliteration – were feasible to use.

Starting in Swift Playgrounds to test out the system APIs

I decided to start with the flow of:

generate a random number
speak the number out loud
the user types the number on a 10-key keyboard

My first sketches looked like this:

The first UI sketch in my notebook

Prototyping

And the first prototype looked like this:

The first prototype

I made a fun little animated barber pole to imply that review sessions were infinite:

And cleaned up the design a little bit:

The cleaned up prototype

Designing an original character

You may have noticed in the top right corner of the above sketch, there’s a little character I marked “The Count”.

A sketch of The Count

I wanted to riff on Duolingo’s use of characters to motivate the learner and keep the app lighthearted. Duolingo uses a tool called Rive (although I didn’t know this at the time). I’ve been dabbling in Blender for 3D modeling for the last few years, so I thought I’d experiment with creating a 3D modeled character with animations and rendering it in the app with Apple’s native SceneKit framework.

Of course, this little maneuver set me back a couple weeks 😅. And to be clear, adding an animated character to the app was way more of an “I’m interested in learning something new” rather than “the app will be useless without this feature”. However, this did make naming and branding and making an icon for the app way easier!

Animated character proof of concept

Before spending an awful lot of time perfecting a character and its animations, I wanted to get a proof of concept working that covered the entire use case end-to-end, namely:

a 3D-character model
textured with either physically-based shaders or UV-unwrapping on a bitmap
rigged with at least one bone
with multiple animations defined in Blender
with lighting and a camera
exportable from Blender and importable with Xcode
displayable in a SceneView in SwiftUI
with each animation triggerable on demand from a SwiftUI button

This is a lot of new concepts for me! Some of the Blender tasks I’d done once or twice under the guidance of random YouTube tutorials, but the SceneKit integration was completely unknown.

I started by modeling a character with only 3 objects, UV-unwrapping him, and painting the texture.

Modeling and UV-unwrapping the prototype character

Then I rigged him with a single bone and added 3 animations (doing my best to understand how the non-linear animation editor worked).

Rigging and animating the prototype character

Here’s the “you got the question incorrect” animation rendered out of Blender for reference:

Next was probably the hardest part of the entire process: the exporting/importing dance. Like DenverCoder9, I found only a few faint traces on the internet of how I’d go about getting both a model and its separate animations out of Blender and into SceneKit. I naively followed the instructions from the thread, made some tweaks to the SceneKit file, tweaked the camera and lighting, mocked out a SwiftUI view with some buttons, and the result was:

I noticed that the animations were more subdued in app version than they were in the Blender version. I wasn’t sure why, but I decided to move on since it was a gamble of whether the subtle differences in the production version of the model would surface the same problem.

Animated character production version

Now that I had a working proof of concept, it was time to do the hard work of actually designing the real character.

First, I did a short interlude of powering through a few character modeling and animation tutorials on YouTube. This long tutorial from Imphenzia was particularly influential and useful.

Like many ideas, I can’t say exactly where or when all the parts of the Count Biki branding came to me. I knew I wanted to riff on the Count von Count character from Sesame Street. However, I also wanted to incorporate Japanese counters. I always thought 匹 (ひき hiki), the small animals counter, was kind of cute sounding. The version for 3 small animals is pronounced biki (びき).

Why a rabbit? I can’t remember for sure, but the ironic part is that rabbits are traditionally not counted with 匹 (ひき hiki), but 羽 (わ wa) due to rumored historical reasons. Regardless, a cute, vampiric rabbit felt like a unique take.

I did some Dribbble hunting for references and mood board creation, did a few notebook sketches (that are much too embarrassing to post publicly), then began 3D modeling.

My first attempt was… not great. Luckily, a few friends gave some very useful feedback and my next iteration was much better.

Revised model of Count Biki rendered in Blender

I specifically modeled Biki sitting so I could simplify the rig. Rigging – adding virtual bones that stretch the polygons and are easier to animate – was my next step.

I can’t emphasize enough how complex rigging is. It may seem like an afterthought, but it’s an entire division at 3D studios, and for good reason. Below is Biki’s humble rig showing his suit in weight painting mode.

Biki with his rig bones visible

With all that in place, I moved on to the animation part. Even though I’d never done keyframe animation in Blender for a 3D character before, getting the basics done felt surprisingly natural to me. This was probably the most fun part of the entire process.

Animating Biki one keyframe at a time

The final result for the “you got the answer correct” animation looked like this straight out of Blender:

Getting the model and animations out of Blender… this part was dreadful. I tried so hard to use Apple’s newest, most blessed workflow of USDZ, but some combination of the Blender exporter and the Xcode importer made this impossible. In the end, I used the same combination of one DAE (Collada) file with just the model and armature, and one DAE file per animation.

I imported the model file into Xcode, creating a new SceneKit scene, then painstakingly re-added the camera, lights, and materials, all of which look and work differently (read: worse) in SceneKit than they do in Blender. I couldn’t get Biki to look as clean in the app as he does in Blender.

Biki moved into his new home in an SCN file

I exported the animations next. When playing them back in the app, I found that some polygons on the suit were getting caught during the animation. This was somewhat expected due to my inexperience modeling and rigging and weight painting. I powered my way through these issues and ended up with the final result: Biki in the test app, playing his idle animation and reacting to questions being answered right and wrong.

App UX Design

After integrating Biki and doing a little cleanup, the listening quiz screen looked like this:

A beta version of the listening quiz screen including an integrated Biki

The listening quiz screen was still the “base” app screen. You could switch to a short list of topics from a modal settings screen, but since this didn’t seem sustainable for the dozens of topics I wanted to target for v1.0 release, I started preparing to have a new root app screen.

For this app, I was basically doing the static UI design step straight in SwiftUI. The main reason for this is that I was primarily using the build-in SwiftUI components and styling for lists, etc., and therefore it would have taken me more time to make pixel perfect mockup in Figma than it took to throw together some SwiftUI code in Xcode.

I made a topic categories view mockup with some additional features I didn’t plan on implementing right away. I did this in order to see how the screen could adapt to these planned features. The v1.0 doesn’t include favorites, recent topics, or the reading skill.

A SwiftUI-based mockup for the topic categories screen with some yet-to-be-developed features

Back on the listening screen, I had to make some changes to support the screen no longer being the root of the app. The session settings screen needed a way to exit the current session.

The session settings screen with an end session button

Next, I added more voice settings for rate and pitch. And a test voice button for easy preview.

Adding a very simple counter for correct and incorrect answers to the listening quiz screen and the session settings screen rounded out all the main UI elements I wanted for the first release.

The listening quiz screen for v1.0

The session settings screen for v1.0

Although I knew it’d delay my first release, I wanted to get an app info screen out of the way. This was mostly because I wanted my in app purchase tip jar system in place out of the gate.

I approached the design of the info screen by checking out what was included in some similar apps, doing a rough UI sketch in my notebook, then jumping straight into coding it up in SwiftUI.

I landed on this:

The top of the app info screen for v1.0

I followed the same design process for the in app purchase tip jar screen. I got really into fleshing out the “lore” of the app, and I really wanted to 3D model all the whimsical objects I introduced, but I knew it’d take some time and wouldn’t really impact my first batch of users enough to warrant delaying the release any further.

The unlocked variant of the Transylvania Tier screen for v1.0

With the info screen complete, I was ready to release the app! Well, I still needed to do all the App Store preparation: designing screenshots, writing the description, etc.

Putting together the app screenshots in Figma

My first submission was denied by the App Store reviewers. (I think it was an issue with the in-review in app purchases not being available from the Store Kit 2 API on the reviewer’s device). I resubmitted with a few more guards, and luckily it was approved and released!

Some initial user feedback helped me realize that my “end session” button shouldn’t be so buried, especially in the infinite session mode. I did some design concepting with paper sketches and directly in SwiftUI (which in retrospect was probably slower than leaning on Figma).

Honestly, the listening quiz screen is very busy. It has a lot of elements and it’s tough to look at it with fresh eyes to understand which parts could be removed.

I think that including the Count Biki avatar makes fitting in the other elements more difficult, but I’m planning to keep him in for at least a few versions to gather feedback about him.

I’d also like to remove the progress bar completely, but when I add time attack mode, I think it will be necessary, so I don’t want to eliminate the progress bar now and have to solve the design problem again later.

Rejected concepts for redesigning the listening quiz

In the end, I moved the progress bar closer to the keyboard and surfaced the end session and settings buttons with clear labels.

Updated design of the listening quiz for v1.1 to allow quicker access to ending the current session

Conclusion

This is just the beginning of the design journey. I’ve intentionally leaned towards a generic iOS design, but I would love to make the app more stylized and personable. New topics and new features will undoubtedly require rethinking my original design assumptions. And I’ll need to decide whether I want to double-down on the character Count Biki and add more animations, new outfits, etc.

Thanks for reading this design walkthrough, and if you’re a Japanese learner or developer, please check out the app on the App Store or the source on GitHub.

The next post in the series is Count Biki - Developing the App for iOS.

Count Biki - Drill Japanese Numbers

Sun, 29 Oct 2023 14:49:00 -0500

I’m proud to present my latest iOS app, Count Biki.

Count Biki app icon

Count Biki a self-guided utility app for drilling numbers of all shapes and sizes in Japanese.

The listening quiz screen: the screen you'll spend the most time on

In this series of posts I’ll talk about:

This post - the motivation behind the app and the solution I’ve begun to explore from the learner (user) perspective
Count Biki - App and Character Design - the design process and specifics of creating the Count Biki character
Count Biki - Developing the App for iOS - the high-level implementation details

Japanese learners can download the app from the App Store.

Like many of my other apps, Count Biki is open source on GitHub.

Background

Counting to ten is one of the first things we learn in a foreign language. Each language has its own rules about how numbers combine to form larger numbers, how they combine with objects, and how they associate with the concept of time.

When I first moved to Japan, every couple shops or restaurants I’d visit wouldn’t have a display at the cash register. The cashier would speak my 4 or 5 digit total aloud at native speed and I’d be lucky to get the first digit. I’d panic and fumble through the transaction, then forget about it until the next time. It was tough skill to practice.

Japanese has unique readings for days of the month, e.g. July 1st, October 9th, etc. We learn them in the 101-level course, but in daily conversation I’d rarely need to recall each specific reading with native accuracy to get my point across. The knowledge gracefully slipped away as it usually does.

Fast forward to now, and there are more language learning apps than ever. Even ones that help with numbers. However, I wanted a more streamlined way to practice all types of numbers, and I wanted to focus my development on Japanese and, as a learner, ensure I could practice the uniquely difficult parts of the language.

In cases like dealing with money, the numbers are essentially random. Therefore, I didn’t want to rely on just hard coding a couple dozen entries. Computers are great at:

generating random numbers
generating speech from text
generating spelled out text from numbers

All the rules about our language systems are already encoded into our devices. I saw my role as being able to surfaces these tools in a convenient wrapper so we can drill the particular weak points of numbers endlessly.

Available Topics

First in the UX flow, the learner choses a topic category. In the first release there are 4 categories: numbers, money, time durations, and dates & times.

The topic categories screen

Within each topic category, there are several topics.

The topics screen within the money category

My goal in dividing up topics and categories was to remove the burden of configuring complicated sets of ranges and digit sliders from users. Of course, infinite customization can be added later. But for my first release, I wanted to make some guesses as to how the app can be immediately useful to many different skill levels, and also suggest scenarios that learners may not even recognize as unique and useful.

In the numbers category, the app covers ranges of numbers mostly based on the number of significant digits you need to input. It’s rare to encounter someone reading you a 9-digit number with every digit as non-zero. People generally work with significant digits, maybe 3 or 4. So although there’s a “Master” category that covers 5 significant digits, the higher order drills for 100 million (億) and 10 trillion (兆) still only use 3 or 4 significant digits (with the remainder being zeros).

In the money category, the app adds the 円 suffix and tailors the number ranges to cover common situations like the convenience store or restaurant. This category has been my most used so far.

In the time durations category, the app covers the main groupings from seconds to years. Although I considered adding some topics to mix them, I’d like to do some more research to determine what the most common pairings in daily life situations are. This also adds complexity to the user input section (what’s the lowest friction way to enter in multiple unrelated numbers? The system date picker? One text box?).

In the dates & times category, the app covers similar variants as the time durations. However, these represent individual moments in time. And there are many variants that are commonly used. For example, 24-hour time is very common in the language, as is AM/PM (午前、午後). Days of the month and months have several exception readings. The app even (for fun) has an experimental topic for converting Japanese calendar years to Gregorian calendar years (this is not easy!).

Listening Quiz

At launch, the only quiz skill supported is listening.

The listening quiz screen

behind the scenes, the app generates a number based on the topic as the question
the on-device text-to-speech engine speaks the question
the learner types their proposed answer in the input box
the learner taps the submit button to check their answer
if the answer is correct, the app plays some animations (Count Biki nods, confetti is thrown, and the device plays haptics) and the flow starts over
if the answer is incorrect, the app shows a red bar and play some other animations
the learner can tap the play button to replay the question
the learner can tap the show answer button to give up and show the answer

The app keeps track of the number of questions answered correctly, incorrectly, and skipped. But as of the initial release, this data is only kept for the duration of the session and is not persisted.

The only session mode available at launch is infinite drill mode. The app will keep generating questions forever, and it’s up to the learner to decide when they’d like to quit or a switch to a different topic. I chose this mode as the default intentionally, as it matches well with the ethos of the app: it’s a utility for additional practice. Although it adds some burden to the learner to choose their own curriculum, there’s power in “staying out of the learner’s way”). Eventually, I’d like to add a time attack mode and question attack mode to give learners the option of deciding on bounded study sessions in advance.

Voices

The app uses iOS’s built-in text-to-speech (TTS) engine. Although the on-device TTS is not perfect and requires some user configuration, I felt it was a good starting point in keeping the complexity, cost, and potential failure points low.

The iOS binary has a standard-quality TTS voice predownloaded for most languages. For Japanese, this voice is called Kyoko. As of iOS 17, there are 4 available Japanese voices: Kyoto, Otoya, Hattori, and O-ren. Kyoto and Otoya have a separate “enhanced” quality voice.

iOS spoken content voices accessibility settings in iOS 17

As of the initial release of Count Biki, the learner can choose any of the voices they’ve downloaded from the iOS settings to use in their listening quiz. Using alternate voices requires going deep into the iOS accessibility settings and downloading ~70MB voice files. We’ve included instructions within the Count Biki settings that show the step-by-step of how to add voices, but unfortunately there’s no way to make this process more automated yet.

The in-app explainer for how to download voices from iOS system settings

As AI-powered TTS engines become commodities over the next few years, I’ll be looking to increase the variety, accuracy, and aesthetics of the TTS component of the app.

The iOS TTS engine does include APIs for changing the speaking rate and pitch. I find the default rate to be a little quick. In the Count Biki app settings the app includes sliders for modifying both the rate and pitch. The extreme settings are definitely extreme, but I didn’t want to artificially limit them for the initial release.

Voice settings within the session settings screen

Quiz results

Generally, seeing the results of a quiz is important for a few reasons. You want to spend more time practicing questions you’ve missed. You want an overview of your progress over time in mastering a topic.

Going along with the utility ethos of the app, I’ve only included the most basic quiz results; the results are displayed inline on the listening quiz screen and slightly more detailed on the session settings screen.

Results as displayed on the session settings screen

As much as I wanted to create an elaborate results system for the initial release, the more I considered it, the more work I realized it would take to make it useful and not get in the way. It also seemed like it’d end up as an endless rabbit hole of complexity that would make implementing other skills and topics more difficult.

The naive solution of showing a big list or correct or incorrect questions doesn’t seem like it’d add a lot of value on its own. I’d rather spend more time considering a few key metrics that I could extract for the user that are 1. understandable at a glance and 2. immediately actionable. I’m regretfully offloading the burden of progress tracking onto the user while the app is still taking shape.

App info

Most apps have an info/settings page that collects all kinds of disparate non-essential functions. Count Biki will probably end up with its mutable settings spread out across a few different contexts, so this page is more about tips, support, and legal info.

The info screen

In app purchases

When I was first charging for my apps, the App Store didn’t have in app purchases. It was actually still the era of the “lite version” - a completely separate binary and listing on the App Store with only a subset of features that would link users to pay for and download the full version.

Since I’m starting to dip my toes in the water in building a self-sustaining independent app business, but my app isn’t at subscription-level value yet (in my opinion), I figured I’d try out the tip jar payment model. The plan is that almost all features are free upon first release, but features added in the future will require a pay-what-you-want-once upgrade. I called this “Transylvania Tier” as a nod to the vampire theme.

The tip-jar screen for unlocking 'Transylvania Tier'

There is one unlockable feature: alternate app icons.

The tip-jar screen for unlocking 'Transylvania Tier'

Conclusion

I’m hoping Count Biki will prove to be an invaluable addition to learners’ study materials. There are plenty more topics and features that complement the theme and goal of the app that I’ll continue to chip away at over time and as I get more feedback from learners.

Thanks for reading this little checkpoint summary, and if you’re a Japanese learner or developer, please check out the app on the App Store or the source on GitHub.

The next post in the series is Count Biki - App and Character Design.

twocentstudios

Count Biki - Developing the App for iOS

Stats

Architecture

Shared state

Parent and child reducer intercommunication

Quiz and settings example – source of truth in parent state

About and IAP example - source-of-truth in a dependency

Reducers with no state sharing

Sidebar: starting with one reducer

Navigation

Going “by-the-book”

Scoping stores

Dependencies

Writing dependencies in the pointfreeco-style

Heisenbugs encountered while overriding dependencies

Overriding dependencies for use with SwiftUI Previews

The use of dependencies in Count Biki

Stateless clients

Separating settings

Wrapping StoreKit

Topics

BikiAnimation

Testing

SwiftUI

BikiView

Styling with base iOS classes

Laying out the quiz screen without a scroll view

Adding commas dynamically to the TextField

Conclusion

Count Biki - App and Character Design

Planning

Prototyping

Designing an original character

Animated character proof of concept

Animated character production version

App UX Design

Conclusion

Count Biki - Drill Japanese Numbers

Background

Available Topics

Listening Quiz

Voices

Quiz results

App info

In app purchases

Conclusion