Disclosure: Fullstack Academy, the coding bootcamp, has previously sponsored Hacker Noon.
The concept that underpins “virtual reality” — the idea of making people feel they’re somewhere other than where they actually are — goes back to the 19th century in the form of technology like stereoscopic viewers. But our current understanding of VR and even AR as computer-simulated or computer-augmented versions of reality originated with the military, manufacturing, media, and other massive industries. Furthermore, the technology of the future — from VR to sentient robots — has til now been the stuff of science fiction. The average person, therefore — especially those who haven’t grown up with access to technology or any familiarity with its concepts — likely still sees these powerful tools as either pie in the sky or as behemoth technologies available only to moneyed corporations and the government.
But today’s developers are changing all that. Every seven weeks, I see Fullstack Academy’s coding bootcamp students creating apps using VR, AR, machine learning, facial recognition, and more. So that not only is the technology available to any developer able to learn it, but it is becoming available to any user who wants to pick up the technology these developers are putting down. Below are just some of the ways our students are bringing sci-fi tech into the mainstream — and I think their ideas are going to take.
AR & VR
There’s a lot of talk about VR in education — to immerse students _actually_sitting in a classroom in Oklahoma or Nigeria in a virtual rainforest in South America, or to bring the Ming Dynasty or the First World War to life for them. And that’s great. But what about truly incorporating these technologies into everyday life? Thus far we haven’t seen too much of that — but these apps could change that.
augMenu
You know how Chinese restaurants are the best because they put so many pictures on the menu? Or how no matter what kind of restaurant you’re in, when someone at the next table over gets something that smells amazing, you are forced to ask them what it is, and then you tell the waiter you’ll have what they’re having? Shouldn’t there be a way for you to preview ALL the menu items like that? There is. It’s called AR. Well, in this case it’s called augMenu, a mobile app that allows you to see an augmented reality version of any menu item before you order it. Use your phone to scan any item from the menu and it will render in glorious detail, solving this whole how-do-I-know-if-it-looks-good quandary.
Blacklight
Our experiences are ephemeral. We see a concert, share a moment with a stranger on the subway, fight with our code — and then that moment passes. Social networks — especially Facebook and Instagram — have tried to tackle this issue by cataloguing our experiences and letting everyone else access them. But that access is confined to those apps and it’s not interactive. You have to be on Instagram to see which desserts are popular at this restaurant, and what you see there isn’t affected in any way by your current environment. But what if others’ catalogued memories could be more integrated into your immediate experience? Blacklight is making progress by enabling users to associate a memory with a location on a map, and then letting other users dynamically access those memories — videos, pictures, a snippet of text — via an augmented reality app, as they move through the location.
ScavengAR
In addition to exposing children to far-away places, AR is a great way to connect adults more deeply to their own communities. Maybe the building you live in is from the 1920s, and a famous writer once lived there. Or maybe that CVS on the corner was built after a fire destroyed the whole block, and the crime was never solved. Whatever subject you fancy — history, architecture, true crime, literature, music — stories are embedded in the world around you, and AR can help expose them. ScavengAR is an augmented reality, city-wide scavenger hunt application that does just that, offering users a tour of the neighborhood and local trivia.
DataVR
If you aren’t good at visualizing or if you’re not an abstract thinker, data-heavy fields can prove really frustrating. VR can help you see right in front of your eyes the information you’re having trouble conceptualizing and organizing in your brain, opening new avenues of understanding to countless individuals. DataVR is a brief exploration of this idea. It starts you off with classic bar charts, scatterplots, and cluster analyses, and finally combines two cutting-edge technologies by offering an immersive visualization of real-time cryptocurrency transactions.
Machine Learning & AI
The idea of sentient robots who can be programmed to learn on their own — and of course then become more intelligent than the humans who programmed them and wage a war on humanity — is at the heart of sci-fi classics everywhere from I, Robot to The Terminator to The Matrix all the way up to the Westworld of today. (Have you started season 2??)
But the reality of experiments in machine learning and AI comes down to hysterical machine misinterpretations of human whimsy (as in these neural network-created guinea pig names), reflections of our own vitriol (see the chatbot who learned racism from people on the internet), and finally: Incredibly helpful bots who can make our lives better by doing the tedious tasks we don’t have time to. (Wait, that does sound like I, Robot…) Below are some examples of the latter.
tripHub
Traveling with a group of friends is awesome. Planning travel with those friends is often not. Most people just hire a travel agent and outsource the job — but what if you can’t afford that? Or what if you don’t have time to manage the travel agent? Here’s where a virtual assistant armed with machine learning can really help. tripHub, created by Grace Hopper Program students, enables you and your friends to simply message about foods, activities, and experiences you like, and from there, the bot will learn your preferences and recommend similar options wherever you’re headed. It can also create polls to see how everyone’s feeling about its recommendations, quote prices, and tell you exactly how to get tickets or book a stay, leaving you with all the fun and very little of the fuss.
SousAIs
Theodore Roosevelt was quoted as saying, “Do what you can with what you have where you are.” It’s an apt quote to apply to cooking — but what if you have no idea what you can do with the can of corn, three strips of bacon, and loaf of sourdough in your apartment? Don’t worry — your virtual assistant will know. Tell “Sous” — as in “sous chef” — exactly what ingredients you have, and she’ll find you a recipe you can make with it. Reverse-engineering a recipe from a set of ingredients is exactly the sort of thing a robot with access to the entire internet is much better suited to than a dad or a college student or a developer.
Facial Recognition Technology
Facial recognition technology is exactly the sort of Big Brother, CCTV, Snapchat-is-stealing-your soul issue that reinforces the idea of big, scary technologies being used on us by forces we can’t control. (Is Snapchat stealing your soul? Jury’s out.) But the truth is we can absolutely put facial recognition technology to work in our everyday lives — and for good, not evil. The following projects from grads of Fullstack Academy’s Software Immersive and their Web Development Fellowship, in partnership with the City of New York, prove it.
Kapture
How many times have we discovered after the fact that a school shooter had mental health concerns they’d never shared with anyone who could help? Emotional well-being is incredibly important, especially in young people, but the way most schools are set up, teachers don’t have the time to grade papers, update classroom materials, develop lesson plans, interface with parents and administration, and connect one-on-one with every student regularly enough to know when something is wrong. That’s where a program like Kapture comes in. It can scan students’ faces to identify patterns of facial expressions and deviations from the norm, and then alert instructors to changes, so instructors can be aware of potential problems and offer help.
Face-It!
Not everything in life has to be useful, per se. In fact, studies have shown that the more humans just play around, the better we are at problem-solving and empathizing, and there’s no doubt we’re happier. Face-It!, a sort of mash-up of “Simon Says” and “Bop It,” where all the commands are facial expressions and you have to “grab” the coins on the screen, helps you do exactly that. The app shows you an emoji, and then recognizes whether your face matches: Are you winking? Cheese-ing really hard? Making an angry face? The app can tell, and it’s glorious.
Bootcampers Keep Pushing the Envelope
In order for “sci-fi” technologies like AR/VR, machine learning, and facial recognition to move more into the mainstream, ordinary people have to realize how accessible they are and how we can use them for good — to improve our day-to-day lives, help us feel good, and teach us about our world. Fullstack Academy and Grace Hopper students are precipitating that shift in our understanding by consistently pushing the envelope and finding novel ways to incorporate these underutilized technologies. We encourage everyone to be curious, try new things, and solve the problems you see around you, even if that means letting the robots help.