furbo.org by Craig Hockenberry

Luna Display

What if I told you that you could add a Retina Display to your MacBook Pro for under $100? And what would you think when I showed how it plugs into your computer?

The business card underneath this hardware gives you some hints, but where did this magical device come from and how does it work?

This story all starts out in The Iconfactory office in Minnesota where we do custom app development. It’s a co-working space that we share with the fine folks at Astro HQ, the makers of the popular AstroPad app.

Troy Gaul showed me the video from the Kickstarter page, Matt Ronge asked me if I’d give it a try (“Hell yeah!”), and Savannah Reising sent me the prototype hardware you see above.

I’ve had two displays on my development machines since the 1990’s. By now, it’s fully ingrained in my habits and makes for a very efficient workflow. So much so that it’s the capability I miss the most when I travel, especially when I have an iPad with a kick-ass display. It’s not hyperbole to say that this new product, called Luna Display, solves this problem completely.

It’s also got me thinking about how to reorganize my home office, because this is a great setup no matter where you are!

The prototype hardware arrived late last Saturday, so here is how my Sunday morning started…

Setup

I put the USB-C hardware you saw above into my wife’s MacBook Pro. My laptop is a bit older and uses Mini DisplayPort for attaching external displays (Astro HQ is working on this interface but the prototypes aren’t quite ready.) The device immediately started blinking. Who doesn’t love a blinking light on new hardware?

I then downloaded the Luna Display app for the Mac. It’s clearly labeled as a “Technology Preview” and I’m fully aware of what that means :-) For example, there are some menu options in this version, like “Reload Codec Config” and “Luna Device Reset”, that tell me engineers are still tweaking things. Like any Kickstarter project, anything I write about now may not be in the shipping product.

Even that blinking light, which stopped after I launched the app, might not be in the final product. But I hope it is, because it’s simple feedback that the Mac side isn’t running. And it blinks!

Earlier in the week, I got a TestFlight invite for the Luna Display app on iOS, so I got that all set up. After launching this app, you get a message saying “Look at your Mac Screen”. Man, I love it when developers think about how you’re going to use something!

On the Mac, there was a dialog asking if PRO BABY could connect via Wi-Fi. That’s my 9.7” iPad Pro, so of course I clicked “Allow”, but it’s good to know that they’re thinking about who gets to see what’s on my Mac.

At this point, Luna Display is all set up and I’ve got dual displays. This is awesome!

Tweaking Things

The Mac app presents two buttons: “Enable HiDPI” and “Display Arrangement”. Getting the Luna Display in the right place was the first thing I wanted to do, so it was very helpful to have that second button.

The first button presented a message to “Install the Luna System Extension”. This extension is needed to put the display into a full Retina resolution, but years of happy marriage have taught me that you don’t install with a password on your spouse’s laptop. So I declined and can’t wait to do it when my miniDP prototype arrives!

Since the Luna Display is running on an iPad, touches and Pencil input work as you’d expect. In fact, they work a little too well, because I caught myself moving my finger over to the MacBook sitting next to it. Maybe Microsoft is onto something here, after all.

I was also curious if I could use multiple iPads simultaneously, I’m an iOS developer so conceivably I could have a MacBook with six displays of varying resolution and speediness. As we’ll see next, there’s a good reason you’re limited to only one iOS device at a time.

A Networked Display

The first thing on my mind, and probably yours, is how’s the display quality? And the answer to that question leads us to the most important part of this technology: the link between your Mac and iPad.

When we renovated our 1920’s bungalow in Laguna Beach, Wi-Fi was just getting started and the speeds weren’t great. As a result, we have a lot of gigabit Ethernet and not a lot of motivation to upgrade a bunch of old Airport Extremes. The network isn’t terribly fast, but most of the devices that can outrun it can be hardwired.

Except now I have a Mac that’s sending screenfuls of compressed pixels to my iPad’s Retina Display over an aging network. And guess what?

The display looks pretty damn good! If I look closely, there’s some blockiness as I move the windows around quickly, but things like the iTunes visualizer look much better than I expected.

I have some experience flinging pixels between a Mac and an iOS device so the effectiveness of Astro HQ’s LIQUID technology is not lost on me. It’s clearly something they’ve honed over the years with their Astropad product. It also explains that oddly named menu item we saw above: “Reload Codec Config”.

There’s also a “Vitals” window that you can open on the Mac to see how well this technology is working. I love to geek out on graphs and the data shown above helped me understand what was going on under the hood: it’s all about the bandwidth. The more you can give Luna Display, the better it looks. You want those blue throughput lines to be as high as possible. The bump in graphs is while I was moving a window around furiously on the Luna Display — I saw similar results while running the iTunes visualizer.

To make my wireless network worse than it normally is, I started a download Xcode on another Mac connected to the same access point. This chewed up about 50 Mbps, and the quality of the Luna Display decreased dramatically.

So yes, someone else in your office could ruin your productivity. Bummer.

A Wired Display

It was at this point I remembered that you could also use a USB connection between the Mac and iPad. Let’s give that a try!

As soon as the cable went in the display quality was perfect. Wowza.

And those blue graph lines you saw above? They were literally off the chart with the USB connection.

(Something tells me that the price to upgrade my network will be way more than this Kickstarter is costing me.)

Practically speaking, I think this is a likely configuration if you’re using these devices for an entire workday. The USB cable lets you share the power load and recharge during extended use. It also completely avoids the problem of your coworkers watching 4K video on YouTube all day.

In the likely event that you’ll take your Luna Display while you travel, make sure to pack a cable, because we all know how great hotel Wi-Fi can be.

A Display With an OS

Everything else behaves just as you’d expect and acts like any other display you’d plug into your Mac. The only difference with Luna Display is that there’s an operating system controlling what gets sent to your second screen.

When you put the Luna Display app into the background with the iPad’s home button, the display disappears from the Mac after a few minutes. All iOS apps have a limited time in the background and Luna Display is no different. When this timeout occurs, you’ll see the Mac’s display reconfigure and windows will move to new spaces. As soon as the iOS app is brought back to the foreground, all your windows move back to their original locations.

Quirks

All new software has its quirks, but I encountered surprisingly few of them while testing Luna Display. This product is much more mature than I expected from a Technology Preview.

One thing I didn’t expect to see was “Luna Display” as the color profile for the display. Considering that I’m running on the first “Display P3” device that Apple shipped, that wider gamut should be utilized. Changing the profile didn’t appear to have any effect, either. I’m kind of a stickler for this kind of thing :-)

In talking with the Astro HQ developers, the reason is because this version uses an 8-bit color pipeline. They are working on 10-bit color and wider gamuts since it’s a great feature for designers using AstroPad.

There were a few times where things disconnected unexpectedly or got hung up. Nothing serious, and it was easy to recover from (it’s like accidentally unplugging a display: the Mac handles that gracefully.)

Even so, the developers are working to improve this situation, which is mainly caused by congested Wi-Fi. They still have some work to do on their UDP-based network protocol when other HTTP requests start hoarding bandwidth.

As I’ve only had the device for a few days and been testing on my wife’s computer, I didn’t do any battery life measurements. I did notice that plugging in the USB cable between the Mac and iPad caused the iOS device to charge, decreasing the battery life of the MacBook Pro.

In Conclusion

I know the folks behind this device. They’ve been shipping great products on macOS and iOS for many years, and I have great confidence that they will iron out any of the kinks in Luna Display.

I also know that I’ll never have a MacBook or an iPad without this tiny bit of magic. Please join me in backing the Kickstarter.

Written on August 22nd, 2017 for Development, Miscellaneous

Swift Changes Considered Harmful

I like Swift. We’re using it for new products like Linea, which was written from the ground up with the new syntax. In spite of this, I still consider the language harmful.

The best way to understand the issues is by imagining that you’re an iOS developer who wants to write their first macOS app.

You find this sample code and it has exactly what you need:

PhotoEditor: Crafting Modern Cocoa Apps

The sample demonstrates several Cocoa technologies and features, including:

Storyboards

Unified window titlebar

Full-sized content view with behind-toolbar blurring

NSAppearance (“dark mode”)

Sandboxing

Resume (state restoration)

Modernized event tracking

Modernized drag & drop

UI validation

The best part is that these bits are fresh — it was published in October 2016 for Xcode 8 running on macOS Sierra. Perfect!

Build and Not Run

Everything is great until try to build and run that project. Xcode displays 12 issues that you’ll need to deal with before you can try out the sample.

A bunch of the errors are due to private protection levels on instance variables. Unfortunately, there’s no automatic fix for this issue. But even though you’re no expert in Swift, you find that changing the definitions to public seems to clear things up.

A harder problem is an error about a method not overriding its superclass:

override func observeValue(forKeyPath keyPath: String?, of object: AnyObject?, change: [NSKeyValueChangeKey : AnyObject]?, context: UnsafeMutablePointer?) {
    ...
}

When you try to do the automatic fix for this issue, the code changes, but the error remains:

override func observeValue(forKeyPath keyPath: String?, of object: Any?, change: [NSKeyValueChangeKey : AnyObject]?, context: UnsafeMutableRawPointer) {
    ...
}

This is a problem you’ll have to solve with Google. After a few failed attempts, you finally land on a solution at Stack Overflow:

override func observeValue(forKeyPath keyPath: String?, of object: Any?, change: [NSKeyValueChangeKey : Any]?, context: UnsafeMutableRawPointer?) {
    ...
}

So what does fixing all this sample code tell us?

Learning with Swift is Hard

I can hear many of you saying, “just fix the auto migration tool!” But that’s not the issue here.

Swift is an easy language to learn and has some wonderfully innovative tools to help beginners. But as soon as you’re beyond the basics, the constant Swift syntax changes become an impediment to learning.

For public versus private issue above, I’m left wondering if this distinction is important in the design of macOS apps. Did I break something? And why do some of the instance variables work OK as private?

The problem with Key-Value Observation is an even bigger concern. As an iOS developer, it’s likely that you’ve not been exposed to this essential part of macOS app design. You’re forced to fix an issue where you have absolutely no experience. You don’t know what you don’t know.

And the part that’s broken is watching changes to contentLayoutRect: the mechanism that deals with everything from positioning vibrant content to supporting full-screen windows. It’s at the core of the window system modernization that started in Yosemite. For many developers, this is the reason they downloaded the sample code in the first place.

Take a look at the Stack Overflow page again. There’s some good information there, but it’s completely hidden by Swift code that’s no longer relevant.

Code Migration is a Crutch

It’s gotten to the point where every time I come across some information written in Swift, especially on Stack Overflow, I cringe. I know that I’m going to have to not only figure out the code, but also which version was used for the answer.

For many of us, sample code is the best documentation possible. It not only shows you an API, but also how it works in context. This sample code appears on a lot of places besides the web: it’s in WWDC videos, PDFs of presentations, books, and downloadable projects like the one used above.

And none of this code can be fixed easily with a migration tool.

It’s up to the author of the original information to make a revision when the next version of Swift rolls out and breaks things again. Some Stack Overflow answers will get this treatment, but most will not. Do you think Apple is going to edit the videos with non-functioning Swift code?

In my last book, all the sample code was done in Objective-C. I wanted to make the projects in Swift, but after seeing the language change significantly three times, I realized that any such examples would be outdated in a matter of months.

And that’s a shame.

It’s OK to Screw Up

Brian Kernighan has a bit of experience with programming languages. This comment from a recent talk on successful language design sums up the situation nicely:

If you design a language and other people use it, you very quickly learn that you screwed up. … this tension between changing something because you got it wrong, or because you have a better idea, versus trying to keep it stable, so that people who learn something don’t have to keep up with your language.

…

Swift keeps changing, and people have code that works, and “oops” it doesn’t work anymore with the next version. And that’s a problem. And how do you stop that?

I think the answer to this question is to realize that everything in our business has shortcomings. Perfect is the enemy of good.

With the recent announcement for Swift 4, it feels like folks are still searching for perfect, when what many of us want is just a great and stable language.

To that end, I invite you to duplicate this Radar.

Written on February 17th, 2017 for Advice, Development

A New Way to Work

Up until the middle of last year, my iPad spent most of its time next to my comfy chair. I’d bring it into the office whenever I needed to test an app on the device, but for the most part it stayed in our living room for reading and browsing.

What changed? I added this to the home screen on my iPad Pro:

Gedeon Maheux and Troy Gaul worked on Linea for over a year. What started as an internal tool for our artists, became something that has fundamentally changed the way I do development work.

A simple start

I’ve been coding professionally for over 40 years and have always kept a notebook near my desk. It’s an essential tool that lets me capture thoughts and ideas in a way that lists on a computer cannot.

As I started beta testing Linea, my trusty paper got used less and less. My first work with the app was doing a rough outline for a chapter in my book on color management:

There’s nothing here that you couldn’t have done in the standard Notes app. It took me awhile to realize that Linea has a superpower you don’t get with the built-in app…

Discovering layers for ideas

To be honest, I didn’t use my Apple Pencil much until Linea came around. The Notes app didn’t do anything I couldn’t do on paper, and drawing apps were overkill for my needs.

This all changed when I discovered the power of simple layers for managing ideas. A few pictures are worth a thousand words, so let’s go through the process with a recent example: the testimonials section of Linea’s website.

In our line of work, we often get a design with no guidance on how to construct it. Here’s what the original design comp looked like:

It’s our job to figure out how to make these designs look and work right. And Linea is a powerful tool for that task.

Start by breaking it down

The first step is to break the design into pieces. I did a rough sketch of what I wanted on Linea’s bottom-most layer:

So far, that’s pretty underwhelming. What’s the big deal?

Well, we’re about to encounter one of the three hardest things in computer science: cache invalidation and naming things.

Another layer for names

This is where Linea really starts to shine. Because it’s a hard problem, your first naming choices are rarely the best. If you’re like me, it usually takes a few tries to get the right ones. When you use a separate layer for your names, they’re super easy to change without wiping out the lines of the underlying outline:

Because selecting colors in Linea is so easy, I like to highlight important names. While debugging the page, the two states of the img elements had a matching CSS outline.

It’s also when Linea starts to feel like a miniature whiteboard on your desk. The Touch Eraser is one of those things that is so natural that you use it without breaking your train of thought.

A notes layer above all

After getting the names you want, it’s time to start building! This is when I create another layer for things I want to keep track of. Again, you can add or remove notes without destroying the work on the other layers.

My two favorite things in CSS are animations and Flexbox layout. And as much as I love them, you’ll see above that I can never remember the names of the timing functions or container properties.

(If you’re a web developer, take a look at the #testimonials source code on the Linea website: the images in the .picker Flexbox animate just by changing the CSS class on the <img> elements. Simple and elegant.)

Helping us stay organized

I also have a bad habit of keeping my drawings around forever — a lot of thought went onto the page and it’s hard to it throw away. Linea’s project management makes it easy to keep track of the old work and keep my desk clean.

It was a pleasure to learn that other developers are seeing the same benefits that I have. My friend Gus Mueller has this to say:

“Linea has been my go-to app for sketching out ideas and solving visual programming problems since the day I started using it. It’s now an indispensable tool for my work.”

My iPad Pro is still a great device for reading in a comfy chair, but now it makes a daily journey to my office. If you’re a professional developer, I bet Linea will become a fixture on your desk, too.

I invite you to take a look at the product website to learn more.

Written on January 31st, 2017 for Advice, Design, Development

Making Sense of Color Management

After over two years of work, my book on color management is now available for purchase. People I know and respect are saying some really nice things about my writing. It makes me happy when doing hard things pays off.

Previously, I’ve written about how displays are changing and how color affects the things we build for these devices. Whether you build on the web, or with native apps, your work in the coming years will be affected. You can learn more about the book at the Iconfactory.

But don’t just take my word for it. Here’s what Mike Krieger, Cofounder and CTO at Instagram, has to say about the book:

For years, engineers and designers have mostly stumbled through color management, understanding it ‘just enough’ to ship an asset to the web or an app. As devices move to wide color, a deeper and more practical understanding is vital, and Craig’s book provides exactly that. In fact, I’ve put his advice to use for Instagram.

As a developer, you might be interested in taking a look behind the curtains at the book’s mini-site. You’ll find additional articles, new markup for the web, and sample code for both iOS and macOS. The book provides essential background for these examples, but it will give you a taste of what you’ll be learning.

I also have some more advanced code examples for iOS in the works. If you work with raw pixels in bitmaps, process photos with Core Image, or do analysis with vImage, this will be right up your alley, so stay tuned!

Written on December 9th, 2016 for Design, Development

Apple’s Lenovo

To bring long-term value for clients, companies need to continually reinvent themselves.

This quote comes from an IBM announcement that it was selling its PC business in 2005.

IBM’s business was moving towards services, and it was less dependent on hardware sales. But it still needed hardware to provide services for its clients.

The deal let IBM focus on its future. Lenovo got the ThinkPad brand, engineering talent, and a sales channel. This was one of those rare win-win scenarios they teach in business school.

Apple is no stranger to reinventing itself. Thirty years ago, it was selling a $15,000 printer.

Laserwriter

It was a beautiful device that produced stunning output. It launched the desktop publishing revolution and established Adobe as a central figure in the software ecosystem.

And today you can’t buy a printer with an Apple logo.

Someday soon, we’ll look back wistfully at those beautiful displays Apple used to make. With the announcement of the new LG partnership and displays, we’re seeing another part of Apple’s business being outsourced.

Maybe it’s time that Apple does the same thing with the Mac Pro business described by Marco Arment.

Like IBM, Apple is in a bind. Its future and the bulk of its revenue depend on mobile devices. Yet we can’t make these products without the horsepower provided by the Mac.

Besides Apple’s own internal needs, there are many professionals creating content for mobile devices. These folks need fast and capable computers to create apps, music, and videos. We all need powerful Macs to enrich the mobile ecosystem.

John Gruber said it best, “It’s the heaviness of the Mac that allows iOS to remain light.” But will iOS pick up this heavy load by the end of this decade? I don’t see it happening with my own work.

Licensing just the operating system was a disaster for Apple. Professional customers don’t have the time to build and maintain their own Hackintoshes. Any partnership to build Mac hardware would need to be structured so that it benefits Apple, the partner, and customer alike.

Just like IBM and their clients have benefitted from Lenovo.

Written on November 5th, 2016 for Business, Observation