Turning a MacBook into a touchscreen with $1 of hardware (2018)

232 pointsposted 9 hours ago
by HughParry

101 Comments

wildrhythms

3 hours ago

"We've done tons of user testing on this, and it turns out it doesn't work. Touch surfaces don't want to be vertical. It gives great demo but after a short period of time, you start to fatigue and after an extended period of time, your arm wants to fall off. it doesn't work, it's ergonomically terrible."

-Steve Jobs, 2010

https://www.businessinsider.com/steve-jobs-touch-screen-mac-...

roboy

an hour ago

I had several PCs with touch screen and this absolutely true. Even intermittent use is not something I did, it’s just too inconvenient to ever become a habit, so the few times it’d be great, I don’t think about it being there because it’s not in my active list of affordances.

satvikpendem

21 minutes ago

I wonder what people will say when Apple releases the touch screen MacBook later this year then.

trash_cat

2 hours ago

Nobody forces you to use touchscreen exclusively?

neya

an hour ago

Yeah, but this was also strategically in Apple's interest to sell the iPads with nerfed up iPad OS as a separate line up. I love Steve Jobs and all, but this did NOT age well. The millions of people using Surface and Surface Pro will absolutely disagree with this take.

dotBen

7 hours ago

I wouldn't want a touchscreen MBP even if it was free, anyone else feel similar?

I don't get the draw - we already optimize for keyboard commands to avoid living our fingers over to a touchpad. Why would I want to start clicking on my screen?

If you're using your computer for tasks (rather than entertainment) and you're not a visual designer, I don't get why Apple are apparently going to be putting them into the new MBP line later this year.

AussieWog93

an hour ago

I feel like the point isn't "there should be a touch screen MacBook" but more "holy shit we simulated a working touch screen by looking at reflections coming off the glass, isn't that cool".

Waterluvian

7 hours ago

I don’t understand touchscreens on laptops that aren’t designed to fold flat. It’s got the feel of finger painting an unconstrained birthday balloon.

samplatt

3 hours ago

You're essentially holding a large tablet upright, but all the weight is taken up up the base. Rather than finger-painting, try holding it on both sides like a tablet or gamepad and operating with thumbs.

Scrolling/controlling checkboxes and switches feels GREAT. Depends entirely what you're using it for.

dwedge

5 hours ago

I had a laptop that folded to 180, without touch screen, and had a webcam hidden under F7 so it either looked straight up your nose or showed your huge fingers

bitmasher9

7 hours ago

Sometimes, if I’ve been using my iPad for awhile and switch over to my MBP, I might reach out and touch the screen out of habit. I can’t be the only one.

antonymoose

6 hours ago

I had the opposite problem when work issued me a ThinkPad - I would accidentally brush my screen with my caveman knuckles once a day and somehow nuke a dozen lines of code.

bruckie

3 hours ago

My kids do this all the time. They also use the touchscreen in conventional laptop configuration (not folded-flat tablet mode) on their Chromebooks all the time. It's bizarre to me. I'm always trying to get them to use the keyboard, but they don't care. Example: enter password on the keyboard, then tap the log in button on the screen with their finger, rather than just pressing enter. ¯\_(ツ)_/¯

egeozcan

38 minutes ago

My 3 year-old never had access to tablets or phones but sees us using phones all the time. So when he gets curious and touches the laptop screen (something he does with our phones occasionally as well), he's shocked to see it doesn't react.

dwedge

5 hours ago

While I'm the same and totally agree with you, the few times I've been using touchscreen I find the habit sticks so hard that for days I keep touching my macbook screen, so there is definitely some subconscious desire for this (or I would have defaulted to using the trackpad even if my brain thought touch was available)

post-it

7 hours ago

> I don't get why Apple are apparently going to be putting them into the new MBP line later this year.

Apple has apparently being going to put a touchscreen in a laptop every year since the iPad came out, and it's never materialized.

xmprt

7 hours ago

Previously, those were rumors from enthusiasts who wanted to see it. Now it's an internal leak so there's a lot more credibility to those rumors.

hu3

7 hours ago

No, there were internal "leaks" in the past as well.

https://archive.ph/xOgtp

xmprt

5 hours ago

> Based on current internal deliberations, the company could launch its first touch-screen Mac in 2025

It looks like those leaks aren't too far off what I'm saying. Deadlines slipping by 1-2 years isn't way off especially for such a new/different product direction. And the rumor also said "could" which means even internally, it wasn't a strong claim.

jacobgkau

6 hours ago

> January 11, 2023

> Based on current internal deliberations, the company could launch its first touch-screen Mac in 2025

Even if it didn't come to pass, just a few years ago is a more relevant leak than the every-year-since-the-iPad-released "rumors."

hu3

5 hours ago

Yes and it's an article about a leak 3 years ago. And there were more "leaks" before that. I just can't be bothered to research and link the obvious to argument against an "opinion".

thesuitonym

7 hours ago

Macs are definitely not optimized for keyboard commands. If you feel the software you use is keyboard optimized, odds are it's not really Mac software.

allthetime

3 hours ago

Absolute base line example.

Copy text in terminal

Mac: command+c

Linux/windows: ctrl+shift+c (unless you want to cripple proper ctrl+c functionality in which case you can (maybe) activate it from a UI menu)

The command key on Mac is somewhat magical and engages in all sorts of productivity and finger efficiency related context switching so that you can do more with less physical movement.

I’m genuinely curious who you think does it better

jonhohle

4 hours ago

macOS has been one of the best keyboard OSes for over a decade, maybe longer. Nearly everything is bindable without additional software or third party apps. This can be done on globally or app-specific. A lot of this comes from the deep script ability that used to be a priority but has fallen by the wayside in recent years.

9dev

7 hours ago

Huh? Of all the wonky shit about my Mac, the flawless keyboard navigation is really none of that

dwedge

5 hours ago

I'm surprised you feel that way. I fight with my mac every day for one reason or another. At least it's not as bad as the days where some software used Cmd + letter and some used ctrl +letter, but for instance Cmd tab will switch to the wrong window when I go back and I have to use the mouse. Window switching in general is a lot harder if you only have the keyboard because the laptop is docked without a magic trackpad

codesnik

5 hours ago

strange. Some keyboard shortcuts in os x are kinda weird and not intuitive to linux or windows users, but they are there. It's totally possible to use mac without trackpad. even cmd+tab switcher has a lot of hidden (but googlable) things: while still holding cmd after initial cmd+tab, you can close apps with q, switch to other apps with tab and (cmd+)shift+tab or left/right arrows, show app windows with down, etc. There's also a cmd+` for switching between one app's windows. I still find that distinction weird from usability perspective, but it's not too hard to adapt to it.

__MatrixMan__

4 hours ago

Really? I find that on MacOS apps are very inconsistent about whether popping open a menu shows me hints for selecting items in that menu. Those same apps are consistent about it on Linux.

And then there's the bonkers window manager which can't move focus directionally (e.g. Super + left) and so you have to fall back to Cmd + tab tab tab tab but even then there's no consistency about whether you're switching between app instances or windows instances within the same app...

JSR_FDED

4 hours ago

Cmd+Shift+Tab goes in the opposite direction.

devmor

7 hours ago

“Flawless” is absolutely the opposite of how I’d describe the third class keyboard navigation in MacOS.

It’s actually more intuitive to use a magic keyboard on the iPad than on the desktop OS.

sitharus

6 hours ago

What do you find missing from macOS keyboard navigation?

I've been using macs since the 90s so I'm quite used to it, so I'd love to know what I've been missing out on.

devmor

4 hours ago

I have a lot of complaints but I would say my three big gripes are:

- Window navigation within (rather than between) open programs. Mainly if one is on an external monitor, this is just a nightmare and I end up using expose and clicking the window instead.

- Window positioning (I installed 3rd party software called Rectangle for this last year so it’s kind of solved but if we’re talking about the vanilla experience this is a big one)

- Having to switch focus to the dock and navigate one by one through shortcuts to open them instead of the Super+Dock position shortcuts that Windows and KDE expose

sitharus

an hour ago

Interesting, those are problems I don't have, I guess due to my work and workflow.

Command-` works for window switching as I expect, probably simply due to being used to it so I know exactly how It works.

Window positioning is an interesting one. I can't stand windows being positioned through tools, I stack them like you would with papers and shuffle through so the edge overlap is really important. Probably showing my age there!

And I never use the dock. Spotlight gets me everything I'd need from there.

allthetime

3 hours ago

Window navigation is just ctrl+direction

nandomrumber

3 hours ago

> Window navigation within open programs

Isn’t table cmd + ~

platevoltage

5 hours ago

At least we have trackpads that are worth a crap.

devmor

4 hours ago

I do adore the macOS trackpads, they blow anything else I’ve used out of the water.

The keyboard (physically) is also just very pleasant to type on.

paxys

5 hours ago

Specific applications sure, but the base OS interface is unusable without a trackpad.

Jeremy1026

3 hours ago

Cmd+Space to open spotlight, type in the first 3 or 4 letters of whatever you're trying to do (an application to open, or a system setting to change) and then Return gets me about where I need to go most of the time. Cmd+Tab and Cmd+` for window selection. I don't do much else on the OS itself so my bases are covered.

radley

7 hours ago

It just feels ancient and weird now that I can tap every screen I own, except my Mac. I don't want to replace the Mac's keyboard & mouse with a touchscreen, I would simply like it to support touch.

(This also made me realize the impending obsolescence of the Studio Monitor XDR: no touch support.)

emmelaich

2 hours ago

Sometimes I feel the urge to do some art, and the bigger surface might allow that. Perhaps in lieu, make a bigger trackpad.

2muchcoffeeman

3 hours ago

Why? Better than a track pad.

Use a Surface Pro some time. If you are just casually browsing or reading a website. I find it much nicer to just tap on a link or swipe to scroll.

christophilus

3 hours ago

I have 2-3 old touchscreen laptops lying around. The touchscreen is useless to me. Worse than useless. If I ever use it, it’s accidentally, and I end up annoyed.

eitally

7 hours ago

I very much would want a touchscreen for my use cases.

Wowfunhappy

5 hours ago

I wouldn't want a touch screen to become the primary input device, but I think it would be useful on occasion. Not entirely unlike how we still have touchpads even though we try to use keyboard commands.

paxys

5 hours ago

It would make sense if the screen folded over. In a laptop form factor a touch screen is just annoying because it keeps pushing the screen back.

commandersaki

3 hours ago

Yes, I feel like it'll be a degrade in quality if they do this with any of their current line up. If they want to make a Macbook Ultra or whatever with it, that's fine -- I would have no interest in it.

harr01

6 hours ago

The benefit of a touchscreen MBP is that Apple will be forced to make their screens more protective.

skwallace36

5 hours ago

as long as it works well, would rather have it than not (but don't want to pay extra, so yeah... leave it out)

platevoltage

5 hours ago

I agree. I've never wanted a touch screen on my laptop. My screen gets smudged enough already.

lysace

5 hours ago

The ergonomic aspects are horrible. I believe there's actual research on this from the 70s/80s/90s.

fortran77

6 hours ago

All my Windows laptops have touch screens and I love it. What is the problem with having another input method available? You only use it when it’s appropriate.

nothrowaways

7 hours ago

Touch screens are not pleasant for laptops. I prefer not to have them.

mrtksn

7 hours ago

It's actually quite pleasant user experience for scrolling. Some interactions are better with a pointer, others are better with touch.

You can try it on an iPad with Magic Keyboard attached, it's very good to be able to do precision through the trackpad and then casually move large things on the screen with your fingers.

prmoustache

7 hours ago

Honestly I just hate having fingerprints on a screen. And I use pageup/pagedown mostly which to me is better than scrolling.

Trackpad is nice for a device you can lay flat on a table or keep on one hand while sitting on the sofa, not too much when the device has a keyboard permanently attached to it and it cannot fold. I know I have a thinkpad like that and I never use the touchscreen.

LtWorf

6 hours ago

Yesterday someone online told me I'm a boomer because (among the many other issues I mentioned) I said that apple computers lack page up/down keys which is annoying.

genthree

6 hours ago

Option-up/option-down?

Two keys rather than one, but makes up for it by not being way off in some oddball part of the keyboard. You can one-hand it pretty easily, since there's an "option" right next to the arrow keys.

MarsIronPI

3 hours ago

As a stolid classic-era Thinkpad user I don't have a dog in this fight, but it seems to me that the strain of having to hold down another key as I scroll rapidly would get tiring rather quickly. Perhaps if there was a Cmd lock it would be fine.

nothrowaways

7 hours ago

Agree for iPad. But for a laptop trackpads ftw!

gavinsyancey

7 hours ago

You don't have to use it.

criddell

7 hours ago

As long as there's a way to maintain the current display density, that would be just fine.

However, like on Windows, I suspect macOS would increase the tap target size on lots of the touchable elements. Even if I don't use the touchscreen, I would still have to pay the touch target real estate tax in my applications.

hntway90904

6 hours ago

That's a fair ask. My dream would be a simple toggle in something like control center for macOS that can flip between "touch mode" and "desktop mode" with most of the under the hood stuff being the same and just UI changes for the task. No doubt this would create new hurdles for software devs but again I'm dreaming here. Windows 10 actually had this with "tablet mode" in the notification center but I think they already soured people on the touch Windows thing by this point. I think Apple could reasonable do it better if they had the will but they'd much rather you buy and iPad for touch and a mac for desktop and everyone who doesn't want an extra device for certain use cases is left out in the rain.

dataflow

6 hours ago

Note on Windows you can disable the touch device and it goes back to the old density. Don't know if Macs support it.

nothrowaways

7 hours ago

you will accidentally touch the screen more often that you think.

wek

5 hours ago

This is cool. Simple prototype. Is it dependent on lighting ... what if you are outside or backlit or glare etc...?

BugsJustFindMe

2 hours ago

Ignoring whether touchscreen laptops are actually a good idea, I "OOF"ed out loud at this line.

> Filter for skin colors and binary threshold

Skin has an extremely broad range of colors that are also lighting dependent. I'd have gone with background subtraction.

Jabrov

6 hours ago

What a super neat application of computer vision. Cool writeup. Thanks for sharing the code and making it open source too!

ianberdin

6 hours ago

I was laughing so much. Thank you. Unexpected tech!

long-time-first

7 hours ago

This is amazing. They should start to install upward looking cameras to implement this officially.

coef2

3 hours ago

People use their laptops under various lighting conditions. I can imagine it would be difficult (or likely impossible) to bring this PoC to a solid production level technology. It looks like a fun project though.

joaohaas

5 hours ago

As other people mentioned this is obviously not something I would want in my notebook... but I can still appreciate the cool tech!

I can also definitely see this kind of thing being used in things budget outdoor displays, specially if the UI is made to accommodate the lack of accuracy, and the camera is positioned on the side (since these displays are usually vertical).

starkparker

4 hours ago

Difficult to capture reflections across a large screen while also dealing with outdoor lighting, glare, and moisture. The touchscreen part isn't usually what makes outdoor signage expensive compared to IP65, temperature control, and a secure housing, all of which would still need to apply here.

This looks like a neat option for retrofitting, and I suspect it'd work for some non-screen glass applications too. A combined IR/visible light solution would be interesting too, since I suspect those are complimentary (IR touch has issues with radiant light, while this wouldn't; this would have issues with low/no light, while IR wouldn't).

solfox

8 hours ago

Love it! I appreciate the ethos of doing more with existing hardware. Adding an actual touchscreen would add real COGs to a macbook, and many potential failure points. Using the existing camera hardware + software seems to produce a "good enough" result for most people for casual use. I'm sure with some time and eng, Apple could make the "hack" shippable. But it doesn't earn product managers the big big bonuses, so it'll never happen.

dhbradshaw

3 hours ago

Don't love touchscreens that much.

But I did love my Toshiba Satellite. It was like writing on paper!

Down with capacitive screens and long live Active Digitizers!

JaredCampbell

7 hours ago

It feels real because of the dirty touchscreen.

parpfish

3 hours ago

I don’t want to encourage people touch my screen and smudge things up.

spidermonkey23

8 hours ago

Using an external webcam is that not more than $1? cool project though; reminds me of how you could use a Wii remote to create a interactive whiteboard.

t1234s

8 hours ago

Is there a coating you can apply to the glass to help with smudge marks?

rappatic

8 hours ago

Oleophobic coating is standard on phones and tablets, which is part of why they don’t pick up fingerprints as easily.

Some brands offer coating you can DIY yourself (eg ProofTech OLEOPEL) but these seem mostly designed for phone screens. I don’t know whether they’d be as effective on laptop screens

dotancohen

7 hours ago

And this is why modern screens (and eyeglasses) should be cleaned with a damp microfiber cloth and no aggressive cleaners. These coatings are fragile.

I do carefully clean the nosepads with soapy water, however.

ForOldHack

7 hours ago

I think I could do this for less than 15 cents: four small peices of double sided tape, and the tiny mirror, and two hair pins... but the software? Priceless.

callamdelaney

6 hours ago

The reason we buy macbooks is because they aren't touchscreens.

egypturnash

8 hours ago

I wonder how well this would work with my bright blue fingernails that are about .5" longer than my finger.

I then wonder how much recalibration I would have to do when one of them broke and I was poking directly at the screen.

Synthetic7346

7 hours ago

You could still use the keyboard and track pad

jojobas

4 hours ago

You chose to suffer not just with touchscreens. That said, it would probably work just fine.

egypturnash

2 hours ago

It's honestly kind of amazing how many things seem to never be tested by anyone with long fingernails. My favorite was the time I picked up a video game controller and physically could not use it because the buttons were in a stylish little recess that meant my thumbnail would have to pass through the plastic housing before I could get the button to make contact.

jojobas

an hour ago

Why would product designers make products worse for most due to self-inflicted conditions of the few?

brcmthrowaway

7 hours ago

Checking this profile of a random hacker in 2018, of course they are now working on AI.

anishathalye

7 hours ago

I was working on AI in 2018 too :)

At that time, I was quite interested in adversarial examples and ML security.

rox_kd

6 hours ago

Neeeeveeeer!!! please let macbooks be as they are .. why would I ever choose to put fingers on that beautful screen ... I don't get it!

bigyabai

6 hours ago

> why would I ever choose to put fingers on that beautful screen ... I don't get it!

I'm not sure, but bare in mind that the iPad is almost as large of a market as the Mac at this point, and the iPhone has long surpassed Mac revenue. Touching your computer is a very popular sentiment among the grimy-handed public.

platevoltage

5 hours ago

There is a reason why the MacBook hasn't been replaced by an iPad with a keyboard and trackpad attached to it. Very different use case.

hoherd

3 hours ago

You're right. Those are completely separate millions of dollars to be made. Best not let them overlap at all, even if some of us intuitively touch our laptop screens to scroll a window now and then, especially when carrying it around.

bigyabai

an hour ago

What is that reason? Clearly Apple can put desktop chips in the iPad, their desktop apps already support ARM, and the iPad hardware can absolutely run macOS. What is the advantage of iPadOS, instantially?

als0

8 hours ago

Still an amazing hack today and I love it. However, I heard Apple are developing a touch screen MacBook this year, and I simply don't get why they're doing that. I don't know what's worse, the ergonomics or the fingerprints.

bardackx

8 hours ago

i used to have a bad touch screen laptop like 7 years ago and back then it made the mobile development more pleasant (both native and web based)

Etheryte

8 hours ago

So you mean they're developing the iPad, an insanely popular device, and you're not sure why they would make such a device?

nsxwolf

7 hours ago

I have been around touch screen Windows laptops for I don’t know how many years now, and I have never felt even the slightest compulsion to touch the screen.

bigfishrunning

7 hours ago

It might be a generational thing; my kids get touchscreen laptops from their school, and they interact with them almost exclusively by touching the screen. I agree, I'd much rather use a mouse (or even better, a trackball; i wish most laptops still had those)