Pyrus.io

Running games in the browser with SwiftWasm

2021-05-15T14:00:00+00:00

Running games in the browser with SwiftWasm

A quick shout-out to Toronto Game Jam 2022 for being the inspiration to push this project forward and help me also find time to blog. It’s my favourite coding event of the year!

Introduction

I’ve been experimenting recently with running some simple games written in Swift in the browser. There’s still a long way to go but results have been promising:

Two proof-of-concept games running on the browser (Links below)

This blog is a summary of progress and learnings so far.

Origin Story

Why Swift?

Over the last several years I’ve been passively developing a SpriteKit game in my free time. While making a game in something like Unity might make more sense if my goal was to finish a game, I’ve been mostly interested in using my language of choice, Swift. It gives me ways to explore the mechanics of Swift that I won’t find as often in my day-to-day.

Why a custom engine?

One of the ways that I’ve been able to explore functional programming is through hobby game development and I’ve been fascinated with the way it might work with an Entity Component System (ECS). My first attempts were mediocre, at best. However, I found new inspiration last year in Point-free’s The Composable Architecture (TCA). So I ended up rewriting my hobby game from the ground up with a brand new ECS inspired by TCA.

One big focus of this new engine was to eliminate use of Apple’s proprietary GameplayKit and isolate the use of SpriteKit library to being a rendering system only. This would open the door to better cross-platform support.

Exploring SwiftWasm

After a highly successful migration to my new engine, the vision for how to support cross-platform became increasingly clear. I was curious to explore the SwiftWasm project and see where it was. It was a pleasant surprise to see there has been some incredible development. Here’s a few thing I found so far:

Setup instructions are super easy to follow. The Carton CLI makes creating and developing your Swift web projects really simple. The SwiftWasm team has done an increadible job of making the process to set up and run your code straight forward.
The JavascriptKit library makes interacting with the DOM quite intuitive. If you want to “write Javascript in Swift” effectively, you can do this. It’s a bit verbose, and about as “safe” as what writing TypeScript feels like, but it absolutely works!
While it is possible to import and use most of the Foundation library, its footprint on your WASM binary is MASSIVE. As I started to explore importing modules I quickly learned that the use of Foundation was preventing me from including more than a small module’s worth of my own code.
There were 2 places I needed to rework my code to overcome relying on Foundation
1. My engine was using JSONEncoder/JSONDecoder. This was pretty easy to move to a separate module
2. I was using some geometry functions like cos and sin. Thankfully, Apple has developed a Swift Numerics Package and RealModule contained what I needed at a much smaller footprint
Not including Foundation also means no access to types like Date and Data. For the timebeing this has not been a requirement for my needs. My general thoughts are that because we are running inside of a browser, the way we interact with dates and data will be different, and require leveraging Javascript Object equivalents for now

Organizing the engine for cross-platform support

I won’t go too deep into the engine’s architecture in this blog post but there’s a few high-level concepts to understand. If you are familiar with Point-free’s TCA this will probably look familiar:

The architecture breaks down to State, Action and Environment. State is your entire game state, Actions are all the events that cause your game’s state to change and Environment effectively represents the world outside of your game. In this engine, the renderer that draws everything to the screen is considered part of Environment. Environment would also be where things like networking is managed.
Your State and Actions are the same regardless of what platform you are running on, but the Environment is what changes dramatically. How you render game objects will be completely different. How you take in user inputs (a small fraction of all your game actions) will require some straight forward mapping.
Reducers are the game logic. Reducers are the things that actually change your game’s State based on incoming Actions.
Reducers are composable, which means you can selectively choose the parts of game logic you want to bundle together

Therefore, if we write different Reducers for the platform rendering and do a little bit of input mapping, we can keep the rest of our game logic cross-platform. We only bundle in our platform-specific Reducers at the very end in their own isolated executables.
What this means, in practice, is that the engine currently has both SpriteKitRenderingReducers which leverage Apple’s SpriteKit for rendering on Apple platforms and WebRenderingReducers which leverage Pixi.js for rendering on web platforms. (In future I could look into writing my own basic rendering libraries, but it’s not a focus at the moment, and I would still need per-platform rendering reducers)

Results and observations

As you can see, it works! You can play the games I showed above here:

Play Wild Breakout here

I added mobile controls but its much easier with a keyboard

Play Asteroids here

This was the first experiment, but Breakout for TOJam2022 helped me push things

Notes

I am leveraging the browser’s requestAnimationFrame to run my loop. I feel like I still have a lot to learn about timing. On web but it’s not as elegant or consistent as SpriteKit’s update function and matching the timing between platforms has mostly been trial and error so far.
When playing from an iPhone I’ve experienced freezing, but never on my laptop. I haven’t had a chance to deep dive into what is happening and determine if its resource limitations issue or my own code.
Debugging on web has been tricky. I found myself swapping to SpriteKit debugging when dealing with runtime errors in order to hit breakpoints and get useful stack traces.

Conclusions

Initial results have been super promising and I only feel encouraged to explore more uses for SwiftWasm in future. Development and deployment was very straight forward. While there might be an atypical bandwidth requirement for using Swift to run your average website, it’s acceptable for an application.

SwiftWasm seems most useful for applications that don’t depend on Foundation. We can’t forget browser limitations; This is not quite like running Swift on Apple or Linux platforms. I’ve learned to question dependence on Foundation in all my code. It comes as an import statement by default in Xcode, but do I really need it? Could other Swift packages out there make themselves more SwiftWasm ready by simply isolating Foundation requirements to a separate module? Apple’s own open source Swift Numerics Library was a great alternative for my needs.

What’s Next

If you think any of this is cool, I suggest sponsoring the SwiftWasm team’s work. It’s incredible how far the project and tooling has come and I think we should show more love and support from the community.
If you are interested in exploring more about my game engine, RedECS, check it out on GitHub. It’s a hobby project for me, progressing at a casual hobby pace, but I’m trying to start versioning my updates. I’m continuing to slowly chip away at achieving parity between Web and SpriteKit. Texture and font rendering are big ones to add soon.
My long term dream is to have my little RPG game running in-browser. I will probably follow my heart on what it takes supports this when prioritizing engine developments.

More progress - new enemy sprites imported and autopilot AI completing the level on my behalf! pic.twitter.com/h7JD3VbhZO
— Kyle Newsome (@kylnew) August 30, 2021

Interviewing Developers; What is the right process?

2018-10-24T14:00:00+00:00

Having both taken a fair number of interviews as well as conducted them, I wanted to write a blog that weighs the value of different interview approaches from both points of view.

TLDR; Do I test coding or not?

In my opinion, yes, at least some coding should probably be done. However, conducting code challenges in a reasonable and consistent way is of the utmost importance. What the entirety of your process looks like will vary by seniority and from organization to organization, which is okay. What we really need less of in this industry is poorly developed, inconsistently delivered hazing rituals masquerading as good technical skill assessments.

How important is it to code?

Does a developer need to code during your interview process? From personal experience, I have felt I learn much more during the conversational parts of the interview, but need to see some code as a final gut check. Let me break down a few things I do and don’t think work consistently well.

Whiteboard interviews w/ code

I’m all for diagramming and architecture talks by whiteboard, but getting anyone to write code becomes painful. There are so many things that can go wrong in these situations that have nothing to do with candidate competency. It just doesn’t feel fair. I have both done whiteboard problems as well as conducted them. I have both felt hazed during a process and felt like I’m hazing candidates due to the dynamics of a whiteboard code challenge. I’m strongly against them, but I won’t outright condemn because there is no one-size-fits-all interview process.

Laptop coding with onlookers (in person or remote)

As I mentioned above, a lot of non-code related issues can stem from whiteboarding. One of them being that a whiteboard is not a keyboard. So our next inclination is usually to do a coding problem on a laptop. Laptop coding is a lot better than whiteboard coding, but all the other whiteboard problems can still exist. The unnatural tension and pressure that arise when having someone look over your shoulder can still be a bit of a bother. Different interviewers can also influence the code result depending on the level of hints provided. I’ve also had interviewers make incorrect or confusing observations that caused me to take pause.

These kinds of tests should not be confused with an assessment of the candidate’s pair programming skills either. It is not a natural day to day problem or conversation that you are having during the test.

Take-Homes

I’ve found that take-home tests are both some of the fairest and most practical challenges to offer a candidate. Secondly, they turn out to be a great way to assess how much someone really cares about the job opportunity. If they forget to even submit the test or return you something very quick and sloppy, it tells you something. When they respond in good time with clean code and an email of questions/decision justifications it tells you something. I’ve also seen it demonstrate a Senior developer’s long-outdated knowledge of a platform that I’m not sure would have been as clear by any other means of testing.

Laptop coding - private work session

While I have not personally taken or conducted a private work session interview, it seems like an interesting balance between live laptop coding interviews and a take-home. If you don’t like the idea of someone going home and maybe seeking help or not being sufficiently time-boxed, then giving them a room to work in alone for an hour might be a good compromise. You may want to be prepared to award partial-points if the problem is hard enough to require close to an hour.

Technical interview questions

Technical interview questions are probably one of the most important portions of an interview process. As a candidate, I have felt the most heard as myself when I answer technical interview questions and as an interviewer, I’ve felt I learn the most about others. It’s hard to gauge what people are passionate about from code alone, but a technical interview question that leads to a strong answer or anecdote from a candidate can really help you see them more clearly.

Evaluating the GitHub Profile

I think a GitHub profile, or really any online presence can be a huge positive for a candidate. However, the lack of one tells you nothing about someone. At best it can be used for bonus points or a gut check but it doesn’t stand alone as a good evaluation tool.

Ask the candidate to show some of their code and discuss

Asking a candidate to talk through some of their own code is a great way to have a technical conversation and see some code simultaneously. In the past, I’ve paired this with a take-home test where the candidate speaks to their take-home code during the face-to-face interviews. I’ve also been asked during an interview to talk through some of the code on my GitHub that was 2 years old. It was a little scary, but the interviewer was open to me criticizing and suggesting improvements too. This is probably the right frame of mind to have if you will be putting someone on the spot to justify code they wrote a while ago.

Culture/Team-fit assessment interview

What does a team-fit assessment process look like? I can’t tell you precisely. Some of us are assessing team-fit the whole time, but others like to focus more deeply on how every new individual affects the culture and ask behavioral questions. I am not an expert in culture fit, but I don’t think it’s something easy to assess in a few short interviews. I agree that it matters, though. A weak or unmotivated developer on a team can distribute pressure to other team members which affect their morale and productivity.

If a hire ends up being a mess from a fit perspective, assess your interview process but also don’t be too hard on yourself. Not all bad fits reveal themselves immediately or reflect a fault in your process. This is one of those situations where the saying “Hire slowly, fire quickly” applies. If someone isn’t working out, you should have a probation policy to remove them quickly.

Developing a better interview process

With some of these considerations in mind, how do we put them together into an effective interview process? The details of an interview process will differ depending on the seniority of the candidate, but here are some guiding principles that carry through no matter what:

Start with interview planning

First and foremost, having a well structured and consistent interview process outweighs the value of any given question asked. If you don’t have an even ground to test people on, you’re not even trying to do your interview process well. It’s also not respectful of your candidate’s time to be ill-prepared. You’d be better off waiting a week to decide on your process, test it internally and refine it before the first interviewee even steps through the door.

Determine what you need

Try to be realistic about what kinds of problems you need this person solving day-to-day. Jumping straight to hardcore algorithms for an entry-level developer may be appropriate for some jobs if you want to have a really high bar to entry, but most likely you need to see someone functioning on something that looks closer to your codebase than a sorting algorithm. No specific level of technical difficulty will be appropriate for any given company, so this should be part of your interview planning process to agree on what matters.

Respect your candidates to get their best

Years ago I took an interview with a company that greeted me with a small welcome gift of a bottle of water, a couple snack bars and an interview agenda with each of my interviewer’s names, titles, and photos. It blew me away. I felt so valued by the process that I didn’t even care how it went after I left and wanted to recommend applying to other developers. In fact, I didn’t get the job but still took the time to thank them for making me feel so welcome.

While we don’t all have to greet our candidates with gift baskets, the part of the process that actually impressed me the most was the agenda. It showed a lot of respect for my time and candidacy. I felt more energized and excited for the interviews to follow.

Respect your interview process as you would your code base

If you currently have no interviewing policies for developers formalized at your organization, I would start by making a git repository and storing all documentation around your interview process there. Add all your questions, code challenges, etc… If the process needs to change, then formally introduce those changes in pull requests and discuss.

Treat those resources as the only process that your employees follow during interviews. Give every person on your team a place to go to be thoroughly briefed on how to participate in the interview process. When your process is consistent and strong, your candidates notice and it reflects on the passion for quality in your organization.

Serverless or Not? That is the question

2018-06-18T14:00:00+00:00

This post is an anecdote about about playing with cloud functions. Over the last few days I’ve been weighing the pros/cons of these services and hope my experiences might help others’ own investigations. This is about a hobby project I am working on, which is a great place to try things. There’s not much code in this post but the project is open source on GitHub.

Developer on a Dime

If you’re anything like me you probably have some combination of too many hobbies and ideas than both time and money can afford. This post primarily focuses on the money limitation and less so on time. I was interested in developing a web application with backend services while maintaining a cost of zero dollars.

Now, I’m not so cheap that I am unwilling to pay money for services. I simply want to pay a variable rate for what I use. While paying 5-10 dollars a month for an instance on some cloud service is not outrageous and has never stopped me in the past, these things add up when you have enough things you want to experiment with in isolated environments. This is a challenge for hobbyists and small business owners alike.

I will gladly pay when my web traffic or compute demands need it, but why pay for dead air?

Enter Serverless

After attending AltConf in San Jose recently, I was inspired by a talk from IBM on their Serverless Swift capabilities. Serverless, for the uninitiated, is IBM’s cloud functions service running Apache OpenWhisk. This is one of many cloud function services available from providers but the only one I know of supporting Swift code. The general idea is that instead of running entire servers in the cloud, you execute isolated blocks of code on demand and pay only for the exact amount of compute time you are using. No need to pay for idle server time. No need to load balance or scale. In principle, it sounds like a straight forward and cheap way to reliably do work in the cloud and pay per use down to the millisecond. If the amount of work to be done is relatively small, it’s free. As we can see below, 5 million executions of 500ms can cost as little as zero dollars.

This got me thinking about how I might take advantage of some free cloud computing with Swift.

My Serverless App Concept

I decided to make a small search engine for Swift packages which I creatively named the Swift Package Directory. Visitors can search for swift packages, see popular ones, or add their own by providing the github repository name. For v1 of the website that seemed like plenty.

There are 4 main functions for this application.

The search function that returns relevant packages from the database
The add function that puts content in the database
The popular function that returns packages with the most github stargazers from the database
A long running function that updates all packages in the database. This one is doesn’t need to face externally and can run on a schedule.

The Zero Dollar Database - Cloudant

I played with a view different ideas on how to manage the database requirement of this project. Some creative ideas came to mind, like using github similarly to the way Cocoapods manages it’s repo. Ultimately I wanted to go a route that was the lowest friction and the greatest flexibility.

IBM Cloud offers a free tier of their Cloudant NoSQL databases, which I decided would suit my needs well enough. There was some example code from IBM available that went this route too. It’s somewhat limited on write/reads per second though so I was going to need a way to mitigate that. I’m certainly okay with paying for my real database usage, but there’s no need inflate that demand by not proactively caching resources where possible.

The Zero Dollar Website - Netlify

If I could set these four functions up in the cloud with a connected database solution, then all I would need is a static HTML website. Netlify offers a free tier for static websites that includes HTTPS and custom domains, so I decided my static assets would live there.

The Zero Dollar Cache - Cloudflare

In order to avoid aggressively demanding resources and hitting limits too quickly on Serverless or Cloudant, I expected that I would need to put a caching layer over top of this whole operation. Given this search directory would not be updated too frequently, I would put the whole website and API behind a CDN that clears regularly. Cloudflare offers a free tier with SSL support, so that kept with the plan of paying as little as possible.

Execution

You can follow along with the code I wrote here. A cloud function comes down to a main function that takes Input and a completion block. That block is called with either Output or an Error. For example, the search function’s signature looks like this:

Getting running

Getting set up with Serverless and Swift was not a trivial task. IBM’s Serverless is built using OpenWhisk, which has it’s own special way of executing Swift code. I learned quickly that in order to have my code run correctly I needed to make sure it was built with an OpenWhisk Docker container that adds new functions to your Main.swift file before deploying (See that repo’s epilogue.swift & spm-build/_Whisk.swift if you want to know what gets added to your code). Some headaches ensued in learning this but on the other side it worked. My deploy.sh file was derived from IBM’s documentation on how to use this Docker container as a convenient build tool. The deploy process is not the fastest though because the container currently dumps all the build dependencies at the end of each execution. Rebuilds are from scratch.

After writing out my functions it became abundantly clear there was a lot of duplicate code in each. I certainly didn’t want to suddenly have a maintenance nightmare out of four functions. That would seem like a serious step backward from writing normal server-side code. Frankly, I would pay money NOT to have to do maintenance like that. Not to fret, however, Serverless Swift let’s you include all the packages you want, so including a ‘Core’ package with most of the code was doable. The real challenge becomes managing a folder structure. I ended up with a SPDCore Package at my root and my “actions” (i.e. serverless functions) in a actions/functionName structure. Unfortunately for Swift development, package dependencies require a tagged git commit. This meant I had to commit, tag and bump version dependencies in my actions’ Package.swift files every time I wanted to build with the latest code. I hope that in a future version of Swift Package Manager that this requirement is dropped for locally sourced dependencies.

Testing locally

One thing I liked about taking the Serverless approach was that each of my functions was an executable I could test individually in XCode. It was nice to not have HTTP in the way; just inputs and outputs. Running in XCode did require me to add a little bit of MacOS only code to my Main.swift to execute the function outside of OpenWhisk, but it was very minimal. Given I was always communicating with Cloudant, I required a connection at all times, which isn’t perfect. There may have been ways for me to set up a database locally too, but it wasn’t a rabbit hole I went down on my Serverless adventure.

Running remotely

The local and remote experiences were mostly identical. I needed to input some environment variables from their management console instead of in my XCode scheme but that was about it. Testing the functions, the first response after idle time was usually a little bit cold, taking between 500ms-1000ms but things were more responsive between 50ms-70ms thereafter. This was just a measurement of the execution time, I believe, not total HTTP response time. Cloud functions may not be ideal for things that should be super fast and are prone to testing a user’s patience. With CDN caching in place I figured my function’s average response times wouldn’t be too slow though.

Going Live

Finally, I put together a static HTML site to access these serverless functions. I pushed some code to Netlify and was up and running in no time. The only thing left to bring it all together now was getting everything behind a CDN and running on HTTPS. I wanted to use my custom domain name so I had some DNS configuration to figure out. That’s when things went a little south…

Roadblock - IBM Account upgrade

I believed I would need to be managing some DNS settings through IBM, which required providing a credit card. I’ve since realized nothing I needed was in DNS but that’s hindsight. I put my information in and was denied for security reasons. After contacting customer service by phone I was asked to try again. On my second and third attempt I was denied again. IBM’s security layer requires a 1 hour waiting period between each attempt so this was starting to test my patience. Every screen change I was being re-prompted to enter a credit card now. My free services still worked, but my account was in a bad state. Exchanges continued by email, phone and Twitter over 30 hours before I was finally granted access. IBM was determined to rectify the issue after I had given up, so I give them credit.

However, in that waiting period of limited access, I naturally grew a bit frustrated. Afterall, I had just poured hours in to this new service. Suddenly I felt denied what I needed to finish my assessment. I started to feel a little foolish for taking a direction with my code that might make me too beholden to one service provider. I’ve seen the virtues of containerization before, so was this path really a good idea?

Is serverless worth it if its less transferable than a docker container? What was I losing by not taking a more traditional approach to backend? I was determined not to let myself suddenly be beholden to Serverless. What has resulted makes for a good side-by-side comparison against tranditional server setup. It also left me with a codebase ready for both.

Pivot - Wrapping the code in a Swift server

With the assumption that IBM’s Serverless may not work out, I devised a plan to serve what I had made so far while trying to make the least change possible to my serverless code. I decided to use Vapor, a server-side Swift framework. Using docker I set my project up in a container. Thanks to separating most of the core serverless code into it’s own module, my Vapor server could simply import this existing module as-is. I wrapped the logic in a few HTTP routes and it was ready to go. Better yet, Vapor was able to serve the static files itself rather than needing to worry about serving my static assets elsewhere. I stuck with using IBM’s Cloudant for the database, as my free tier IBM account was still operational and, like I said, the goal was to not touch code I had already written.

Next, I was faced with the challenge of how to deploy this code. Given that I was going to need a server running 24/7 at this point, my ambitions of a near-$0/month bill would have to be set aside, at least temporarily. I care enough about the site I’m trying to build not to let implementation/cost details keep it out of production forever.

Deploying Manually

My first thought was to try deploying the code to a cloud provider manually. I went with Digital Ocean because I knew they offer droplets for $5/month, which I can deal with. After launching an instance, I tunneled in via SSH, pulled down my git code and got it running pretty quickly. The problem with this approach, however, was all the manual work I immediately started dreading as I went through the motions.

How am I going to deploy changes for this and then restart the server?
How do I set up the right port access?
How will I maximize uptime and restart if it goes down?
How will I set up HTTPS?

All these issues and more were suddenly my burden. It was faster to set up initially, but I was unsettled about the increased responsibility. Operations related issues like these are my least favorite of software engineering problems, especially when I’m not being paid to solve them like on hobby projects.

Deploying through Heroku

I had never tried Heroku until this week but I had heard great things about the simplicity of their interface and the peace of mind it brings. So I tried packaging my site up in a docker container and deploying the code. This process was quite straight forward and, as I had hoped, it gave me peace of mind that my Digital Ocean approach lacked.

With Heroku I had to deploy the whole Docker container. This was initially 2GB of transfer and my changes, which affected the whole .build folder meant ~400MB uploads on every new deploy. I tried cutting the size down by having the project build on launch but it was too slow to start within the 60s Heroku provides.

The free instance of Heroku goes to sleep every 30 seconds and the next tier is $7/month. A little pricier but probably worth it for the luxury. If I wasn’t trying an experiment in paying as little as zero dollars, I would play with Heroku more.

Back to Serverless

After passing time fiddling with 2 alternate solutions for a day, IBM got in touch with me and helped resolve my issues. I was happy with the progress and learnings enough to want to write this post so I was determined to see a Serverless implementation through to the end.

Zero Dollar Load Testing - Loader.io

Before finally launching my website I wanted to do some load testing in order to confirm my site was ready to be resilient. Loader.io is a great little service for testing sending thousands of clients per minute at a URL and measuring average response time. They offer a free tier that lets you send clients to up to 2 unique URLs simulateously. Paid tiers let you add more clients and urls, which would be nice, but out of scope for my hobby budget. I did a few tests on Serverless with and without the CDN cache operational, as well as on Heroku for comparison.

Load testing observations

Cloudant db delivers exactly what it promises for the free tier. No more no less. For example, the 5 queries/second limit means maximum 300 requests/minute. When I sent 481 requests/minute at the server 171 requests were rejected
Overloading was causing timeouts on functions. Running a whole 60s before getting cut off. This probably means there are some issues in my code, but I also lowered the timeout on my functions to 5 seconds because I don’t want to be paying for those 55 seconds.
Using my CDN I could achieve 2000 requests per minute for 2 URLs.
Response times during load testing weren’t super impressing via Serverless or Heroku. I needed to do some more investigation into this. Serverless had more variability but both reponded on average around 300-400ms.

Load reality

These tests give me a decent idea of how many requests per minute I can handle when data is not in the cache. It also demonstrates pretty effectively why a cache is necessary. In reality there’s going to be a combination of both happening. Information like the popular listings will cache quickly, since it’s on the front page. Given the wide variety of possible search terms, there will be the most uncached queries delivered for search, which is a bit concerning from a response time perspective. I’m not as worried about # of queries per second right now.

Choke points

Overall the reponse times from the uncached services is not ideal whether testing via Serverless or Heroku. This observation led me to investigate the database a little bit further. My first suspicion was that I was not using CouchDB properly. However, I did set up indices for queries and tests against the database directly performed around 20ms on average. So somewhere between the database and the service things are slowing down. It’s probably not very efficient to be connecting to the database over HTTP, interpreting it in JSON in the service then returning it again in JSON to the client. In a more traditional server setup we’d maintain a more direct connection to our database. I decided to try testing a simple ping API endpoint that does no databasing or HTTP communication at all. Just returns {success:true}.

Even a simple ping is taking 180ms on average. So HTTP work with the database is not helping, but the cloud function itself is not speedy at even it’s slimmest. It even responded in over 900ms at it’s worst. The variable response times are hard to deny.

Pros & Cons

With that, I quickly want to round up some quick pros and cons about using Serverless.

Serverless Pros

No scaling, load balancing needed
Minimalistic approach to getting work done in the cloud
No large deployment artifacts
Easy to trigger any way youd like e.g. API call, git push, scheduled, another function
Pay only for what you use. Variable costs start at $0.

Serverless Cons

Trickier to develop offline
Many manual tasks for things you’d do in code e.g. API Endpoint setup, Environment vars
Once multiple functions want to share behavior or environment variables, it gets more complicated/arduous
Frontend and backend aren’t easily developed side-by-side
Somewhat unrealiable response times
Still feels a lot like a server work to me.

Conclusions

For the time being I will be running Swift Package Directory off of Serverless. It’s been a fun experiment and I don’t have much to lose by sticking it out. Should this experiment grow in popularity enough to have demanding server needs I will definitely think about moving to a more traditional set up. Lucky for me, I’ve already done the leg work of designing this project to function either serverless or inside of a Docker container on a service like Heroku.

Probably one of my favorite features about Serverless right now is the triggered functions capability. I see use cases for little bits of triggered code. Some ideas boil down to a repetitive task, nothing else. Even if I migrated the API to Heroku I’d potentially keep scheduled tasks like the bulk update as cloud functions. Those seem like the best use cases. This is a new tool for our toolbelt more than a replacement to everything we know so far. Traditional HTTP APIs with demanding response times are probably not going to be the first and best use case from what I’ve seen so far.

If you have any questions or suggestions for improvement I’d love to hear them.

You can see more in the GitHub repo here

Epilogue: Features I’d like to see from Serverless

Be able to deploy a git repo of one or more functions
~~Global environment variables that apply to all functions in a group~~ Apparently this can be done with Serverless packages (subgroups) but I had trouble hooking package actions up with the API layer
Better metrics
~~Better HTTP error status code and response control~~ Available
Better logging features. See all printed output.

Adventures in MVVM with RxSwift

2018-06-01T14:00:00+00:00

In the last couple of days I found some time to work on a simple iOS To-Do app. It’s written using RxSwift and MVVM architecture. The code is open source on GitHub.

I’ve experimented with RxSwift in several other projects and enjoyed it. This was the first time I’ve tried making an iOS app from scratch with MVVM and the RxCocoa extensions. I implemented a few features that would challenge RxCocoa. I thought inline editing in a collectionview and undo/redo would work. Here are some quick thoughts about it.

It feels clean, but there’s more work involved upfront

I’m a fan of the lightweight view controller that is primarily focused on bindings and isolated viewmodel code. Setting things up feels a little slow at first compared to the gains we are typically used to seeing with MVC. However, the clean separation of logic lays down some really nice foundations that give me a sense of maintainability and ease of testing. The undo/redo functionality worked itself really nicely into the viewmodel layer which felt like a nice abstraction.

Working with UI as you are editing is sometimes tricky

I ran in to two notable areas of difficulty that related interaction with UI while editing content simultaneously

Using the RxCocoa basic example of connecting a tableview, you get left with a table that reloads it’s rows on every change
While typing in a textfield, relaying the text changes immediately would cause a feedback loop from the ‘updated’ signal.

The first problem was reasonably straight forward to fix. You’re simply better off using the items(datasource:) function and being your own data source. However, this burderns you with the responsibility of diffing your own changes. There’s a GitHub repo RxDataSources that make this easy. I ran into some issues with this repo, likely due to version mismatching so I ended up using an existing diffing algorithm I’ve written.

The second problem seems to have no ideal solution. I’ve tried two solutions now:

If textfield.isFirstResponder == true, then I didn’t update the textfield since I could assume it’s up to date. This seemed like the easiest solution to not fighting UIKit. However, I’ve realized since posting this blog earlier today that this causes the current editing cell to ignore changes during Undo/Redo.
My current solution is to not update the textfield from its delegate methods. If textfield.isFirstResponder == true then when the model updates I copy the previous textfield.selectedRange over and nudge it in the appropriate direction. This way the cursor position is maintained while actively typing.

This goes to show some of the nuanced challenges of RxCocoa extensions when you dig in deeper.

Next Steps

I’m enjoying working on this project so far and might work on a few more things like connecting a webservice and syncing list results. I hope this might serve as a useful example to someone, feel free to ask questions or give feedback here or on the code’s GitHub repo

Developer Hubris

2018-05-24T14:00:00+00:00

Sometimes I joke that the best program I ever wrote was a VBScript Macro in Microsoft Excel just before I made a career change. It certainly wasn’t the most elegant code I’ve ever written but it really got the job done. The macro helped cut hours of administrative formatting work that I got tired of doing in my day to day. When I shared the tool at work it spread across the organization quickly without me having to lift a finger. Even after I left I heard the macro made it as far as departments in Spain. I wrote it over a weekend in my room, thanks to the discovery that I could Google almost anything and find an answer to get me unstuck. It was the catalyst of what has become my career in software.

Sometimes I find myself dreaming of developing a tool in just a few days and having it spread with such fanfare. What I’m not so sure of is whether no opportunities like this still exist or I’m almost blinded by experience. My mind jumps too deeply into the technicals and makes work for itself that is divergent from reaching the end goal any way possible.

I may be understating the total time it took to develop my first useful tool. After all, it took over a year at my job to understand the problem space and feel enough pain to want to attack it. However, I’m not interested in discussing problem discovery as much as solution implementation. Once I had the idea to fix something with code I got the job done fast. Some days I fear that the experienced developer in me, if faced with the same problem, would start thinking too abstract. In 1 to 2 weeks time I would deliver something of equal usefulness but with a cleaner codebase that no one may ever actually see or need to improve on.

Maserati Problems

I’m not sure where I first heard the term “Maserati Problem”, but it seems to have stemmed from the startup movement in tech. In layman’s terms, it is a problem you create based on the assumption of future success. i.e. What if customers experience XYZ after we have 10k daily active users?. If the solution to that hypothetical adds weeks to the project, then now may not be the time to solve the problem. Developers can be prone to creating Maserati problems both in personal projects and workplaces. We might be even worse at this when it comes to personal projects because there’s no one to slap us across the face and assert what is most important.

Quality Engineering vs Stubbornness

There is no silver bullet solution to this problem. It is simply a matter of continuing self awareness. We must remember to not be too proud of our code. We are here to solve problems first. Sometimes there is a necessary trade off between delivering on time and getting it perfect in the first try. It is up to you to understand the structure of your organization and cater appropriately to the business needs. This varies greatly depending on company size, company stage, team size, your job description and more.

In personal projects it’s okay to do something for the love of code alone but if that wasn’t the initial goal, then where are you now? I try my best to state one of two possible goals in personal projects - Learn something or ship something, but don’t expect both. My personal track record of sticking to this doesn’t always work out, but I try.

Closing Thoughts

I find myself admiring both expert coders and scrappy coders that ship products. Some apps that impress me get developed by self described designers who did whatever it took to make their interface vision work. Whether those works have long term viability varies. However, assuming that the initial product is well received enough there will be time, motivation and maybe even money to make it better.

iOS9’s Mail.app has weird swipe gestures

2015-10-30T14:00:00+00:00

While doing some research on swipe gesture patterns in iOS apps recently I revisited the default Mail app installed by Apple. I’ve been a user of Mailbox since its release a few years ago so the default app is mostly just an icon on my home screen now.

At the time when I switched, swipe gestures were limited in the default app, but they have expanded over the last few major updates to iOS. I was surprised to return most recently to find some gestures that I find a bit sloppy. Watch this small recording I made:

Did you catch all that? I certainly didn’t. Let’s break down what’s going on.

Let’s break this down

First I pull to the right to reveal a blue area with a ‘Mark As Unread’ ability. If I swipe only about half way, it is simply revealed as a button. If I swipe beyond half way, the blue area races to catch up with my finger and automatically executes if I release.

Next, I pull to the left, which has a similar reveal effect, however, this time there are 3 differently coloured options, ‘More, Flag, Trash’, revealed. If I swipe beyond half way, the ‘Trash’ option catches up with my finger and automatically executes if I release. The other 2 options do not have a swipe/release way of being executed.

At the very end of the video, I actually didnt even intend to swipe all the way and trash that mail item, but it enforced the point I’m about to make.

Reveal and long swipe gestures in one is a bad recipe

Having actions that first start as ‘reveal then tap’ suddenly changing into a ‘long swipe’ is quite confusing. I’m going to give the first swipe gesture a break because there was one possible action and was just 2 ways to do the same thing. However, I think the animations and colours could still be improved to better signal that releasing from a long swipe is going to execute the action though.

The left directional swipe which reveals three actions that collapse in to one ‘trash’ option on a long swipe, is just messy though. Not only does it suffer from confusing transitional animations but the fact that I could be intending to flag something and instead accidentally trash it by just swiping a little too far is a recipe for user error.

What works better

There are a few things I would do here to improve the experience but also make as little change as possible.

First, I would improve the transitional animations from reveal to long swipe. If the back panels are already coloured, then the icon needs to animate somehow, possibly with a checkmark or some indicator that the state is already changing before your finger releases. Take the Mailbox app, for example, which circles the icon with an arrow when you hold it. This is a slightly different type of action, but conveys the same notion; the icon can tell us more about how our gestures are affecting state.

Secondly, I would get rid of the long swipe gesture to auto trash items, or reduce the actions to a single option at a time. Mailbox introduces the idea of short and long swipes changing the executed action all together, which is accompanied by a colour and icon change. This could be used to toggle between ‘Flag’ and ‘Trash’ states but I wouldn’t put more than two possible states in a single swipe before things get too granular. This is why just sticking to reveals is best and most consistent without rocking the boat.

Use swipe gestures with caution

I’m a big fan of swipe gestures on table cells to reveal and execute actions. It’s a great way to get interaction without requiring buttons all over your list items. However, they need to be used with care and restraint. I would consider them to be intermediate to advanced user actions, so it’s important to think of how the audience will learn these gestures and adopt them over time. Sensitive gestures that have not been carefully introduced may leave some wondering what just happened and how on earth to undo (An action hidden behind an even more obscure gesture - literally shaking your phone).

General impressions on Swift 2.0

2015-10-26T08:00:00+00:00

I’ll be the first to admit that I am not the first person jumping on to Apple’s Swift language. I was, admittedly, a bit apprehensive about jumping in after the release of v1.0 over a year ago. My first impressions were the language and its syntax, while very promising, was just not complete enough yet.

However, with the release of 2.0 several months ago and announcement of a forthcoming open source release, my excitement level significantly increased. I know some developers like myself are just taking their first looks at the language so I wanted to summarize a few thoughts from my first experiences.

Within the last month I decided to delve heavily in to Swift on a new personal project. Overall the transition has been quite easy, given an existing knowledge of Objective-C, and I’m really positive on the future of what Swift will become. The rest of this blog is about four features that got me the most excited.

Clean and simple syntax

The obvious big win in Swift is a much cleaner syntax. What might not be quite as clearly understood at first glance, is how Swift works a lot like how we understand Objective-C, but with many more assumptions made. Property declarations were kind of out of hand in Objective-C. You were declaring a class’ properties in 2 or 3 places across a .h and .m file. It was getting better, but Swift just puts that all behind the curtains.

A simple var myVariable = 1 is all it takes to get a strongly referenced internal Integer that will be set to 1 on initialization. If thats not what you need, it’s mostly about appending words like private, weak, lazy to indicate different behaviour.

The Swift playground, holy crap, the playground

You may not know, but typing $ swift in to your Mac’s terminal will open a Swift REPL. A great way to start toying with the language, but XCode’s Swift playground takes the feedback loop to a whole new level. Not only can you get immediate feedback on simple statements, you can render views in isolation and play around with visual attributes. You can even review how visual elements changed over time! I found that using the Swift playground was an amazing way to learn about the CAAnimation library.

The Swift Playground gave me a great way to dive in to creating bezier paths for layers to follow during animation. It turned something I might have done in Objective-C over a couple days in to an afternoon of exploration.

Optionals make Swift more expressive than Objective-C

It’s nice how Objective-C ignores messages to nil and all, but sometimes that’s just not what you want. It also stinks when you have to go on a witch hunt to figure out what’s nil that absolutely shouldn’t be. The addition of optionals and the ! for indicating when one definitely shouldn’t be nil gives Swift a much clearer way of expressing what statements truely are and aren’t required to succeed.

Enums are awesome

One of the coolest features about Enums in Swift is the ability to use Strings and Tuples. It makes for an incredibly flexible system for implementing any special type that has a limited set of states. I recently used it to list all of my Sound effects in a clear syntax and avoid listing sound file names like "ping.wav" in various files.

It’s worth taking the dive

I’m sure if you take some time to read the language’s documentation and play around for a few hours, you’ll get pretty excited too. I really see no reason to write any more Objective-C than I need to at this point. The syntax alone makes Swift very attractive and it’s starting to feel very comfortable.

Postcard - A Post Mortem

2015-05-25T08:00:00+00:00

It hasn’t been easy to write these words, but I have decided to discontinue the Postcard project. I’ve fallen behind on maintenance and due to some sweeping new policy changes in Facebook’s platform as of May 2015, the live iOS and Android development versions are broken and it’s been the final kick in my ass necessary to admit Postcard is unsustainable.

Post Mortem TLDR;

The iOS and Android projects are released as open source code, as-is with broken/partial functionality
- iOS: github.com/bitwit/postcard-ios
- Android: github.com/bitwit/postcard-android
I learned a few things

What happened?

Postcard was an interesting project for me. It started from a frustration with ever changing policies and lack of standardization in social media and ironically will be shutting down for exactly the same reasons. I really have to say that social media management is a tough thing to tackle for any single developer. It’s hard enough to depend on code that you didn’t write yourself and it’s even harder to depend multiple on APIs. It takes a lot of maintenance just to stand still. I can’t only blame changes to external services as the downfall to this project, however. This was just the final straw that got me to admit I couldn’t go forward any longer.

The Cost of Postcard

The real costs and opportunity costs of my time were too high. Most simply put, Postcard was downloaded about 6000 times and made $750 USD total. The Apple license and App.net license cost me $100 a year each and I’d already paid twice for them both. I quickly had to get back to other work after launch.
Lack of the right plan

The lack of profit for Postcard is certainly unfortunate and it’s really hard for me to say what it could have been but I know I was backwards on at least a few things from a planning perspective.
- I heavily pursued trying to get media coverage for day one but I had no clue what to do next. It’s really not enough to just get some burst coverage. You need steady coverage and a bigger marketing plan to stay visible. I was fortunate enough to get covered on TechCrunch and get a nice day 1 boost, but it quickly went downhill1.
- I’m not sure I really looked at it like a business properly until after I launched and got feedback.
- I did not do enough market research. Market research isn’t just about looking at your competitors and thinking you can do things better than them. It’s about really understanding the who, how and why of the people that you are building for. I didn’t really know how to speak to or price effectively for my market.
- I think I went after making a version for Android too soon before I even had enough success with the iOS version. It wasn’t a good direction to be focussing time and energy.
- Opportunities me and requests for other custom (paid) integrations but not enough traction on what I had already built. I felt like I really hadn’t designed things in a nimble or responsive way. There should have been a leaner development cycle than 8 months.
Maintenance (or lack thereof)

With other obligations, I fell behind on maintaining both my iOS app and WordPress plugin. Most things worked well for a while but iOS8 made some big changes. It brought on a lot of necessary code adjustments that really put a drag on all the time I’d invested in making things work perfectly for iOS7.

The WordPress plugin also got stale as a major update and even a new forthcoming JSON API made the custom work I had done seem like a large investment made in something soon to be obsolete. The WordPress plugin also suffered from the same lack of understanding my users as the Postcard application itself.

When Facebook made their sweeping changes to applications, effective as of May 2015, and silently broke what I had in the app store, it was everything I needed to flip the proverbial table and admit there was no continuing Postcard2. More deep code changes stood in the way of keeping Postcard even standing still and that seemed like a bad place to be one year and three months after release.

The good news to anyone interested in possibly carrying the torch is that I’ve posted both the iOS and Android projects to GitHub as-is. It’s not my intention to maintain or develop them any further. I hope that others might find some use in reading the code. If there’s a ‘small’ way in which I can help, try reaching me on Twitter at @kylnew.

iOS: github.com/bitwit/postcard-ios
Android: github.com/bitwit/postcard-android

On to new things

I am in good spirits as I bring Postcard to a close. It is just one of those things that began to weigh on my shoulders as I became increasingly uncertain on how to proceed. I don’t want to be that developer who never admits their project is over when it’s over. Both myself and my users deserve better. Looking back I’m thankful for several opportunities Postcard has afforded me:

Postcard taught me a lot about product development. It taught me about just how important the marketing research portion is and that I wouldn’t take something big like this on alone again.
Postcard let me scratch my design and UX itch. I’m not saying it’s the nicest app ever, but I spent a lot of time trying to give it a simplicity and charm that I’m proud of. The only thing I didn’t design was any of the icons or logo.
Postcard introduced me to many new people. It’s the reason I have my current job with SIDEKICK.pro.
Postcard got me playing with and learning Android development. It was an interesting experience having never written any Java before or even using a modern Android phone. I’m really glad I had a chance to give it a shot and break out of only building for iOS or web.

I’m thankful to everyone who gave Postcard a shot and sorry to have to let it go. I wouldn’t do it if I didn’t believe it was to build something even better in the future.

If you want to say hi, I’ll be on Twitter @kylnew and I’ll be blogging at BitWit.ca.

Pyrus.io

Running games in the browser with SwiftWasm

Running games in the browser with SwiftWasm

Introduction

Two proof-of-concept games running on the browser (Links below)

Origin Story

Why Swift?

Why a custom engine?

Exploring SwiftWasm

Organizing the engine for cross-platform support

Results and observations

Notes

Conclusions

What’s Next

Interviewing Developers; What is the right process?

TLDR; Do I test coding or not?

How important is it to code?

Whiteboard interviews w/ code

Laptop coding with onlookers (in person or remote)

Take-Homes

Laptop coding - private work session

Technical interview questions

Evaluating the GitHub Profile

Ask the candidate to show some of their code and discuss

Culture/Team-fit assessment interview

Developing a better interview process

Start with interview planning

Determine what you need

Respect your candidates to get their best

Respect your interview process as you would your code base

Serverless or Not? That is the question

Developer on a Dime

Enter Serverless

My Serverless App Concept

The Zero Dollar Database - Cloudant

The Zero Dollar Website - Netlify

The Zero Dollar Cache - Cloudflare

Execution

Getting running

Sharing function code

Testing locally

Running remotely

Going Live

Roadblock - IBM Account upgrade

Pivot - Wrapping the code in a Swift server

Deploying Manually

Deploying through Heroku

Back to Serverless

Zero Dollar Load Testing - Loader.io

Load testing observations

Load reality

Choke points

Pros & Cons

Serverless Pros

Serverless Cons

Conclusions

Epilogue: Features I’d like to see from Serverless

Adventures in MVVM with RxSwift

It feels clean, but there’s more work involved upfront

Working with UI as you are editing is sometimes tricky

Next Steps

Developer Hubris

Maserati Problems

Quality Engineering vs Stubbornness

Closing Thoughts

iOS9’s Mail.app has weird swipe gestures

Let’s break this down

Reveal and long swipe gestures in one is a bad recipe

What works better

Use swipe gestures with caution

General impressions on Swift 2.0

Clean and simple syntax

The Swift playground, holy crap, the playground

Optionals make Swift more expressive than Objective-C

Enums are awesome

It’s worth taking the dive

Postcard - A Post Mortem

Post Mortem TLDR;

What happened?

The Cost of Postcard