Bridging the Gap Between Technology and Humanity with Connective Media

We’ve all experienced awkward situations where, upon receiving a text message, we’re unsure of the sender’s intention. Of course, it isn’t the phone’s fault — how would a phone know what the sender was feeling?

Human emotion is something machines can neither convey nor contextually understand very well… yet.

But in the not-too-distant future, that will change — and students in the Technion-Cornell Dual Degree in Connective Media program at the Jacobs Technion-Cornell Institute are exploring how, by building technologies that reflect a more human reality.

“Emotion is something that’s hard to study and hard to build into an application,” said Claire Opila ’17, a Connective Media candidate with a background in psychology and a desire to leverage technology to help people struggling with anxiety or depression.

“There’s a lot of literature about how people with depression post to Twitter or Facebook differently,” said Opila. “You could use early warning signs or signals for someone who is inclined toward depression, and help them before they hurt themselves.”

Based on whether a person is typing fast or slow, for instance, a machine can be taught to recognize patterns that indicate the typer’s mood. Opila worked with a team to program a keyboard capable of detecting emotional states.

She thinks that if a machine could know a person was suffering, it could alert prevention hotlines or mobilize other support resources. Though Opila and her team are still working on a prototype, she knows the project could have far-reaching social implications, and she is conducting market research to determine where a product like this could have the biggest impact.

The keyboard is but one example of what the future of human-computer interaction might look like — and exactly the type of exploration that the Connective Media program hopes to foster.

Mor Naaman, associate professor of Information Science at the Jacobs Technion-Cornell Institute explained how the Connective Media program exposes students to different theories within the realms of psychology and technology and examines how and where the two subjects intersect.

Students in the program also take courses in design, information policy, and technical systems, which help them understand the larger social, ethical, and legal ramifications of the products they are building. “By taking a range of courses in different disciplines within the program, students gain a broader view of the human user, and how technologies can be built to better serve them,” said Naaman.

With such comprehensive studies, students constantly think about how their projects might affect others and society at large. In the case of Opila’s project, Naaman was privy to research that would help her team build an application, and was able to point her team to industry practitioners that could enhance and support their project.

Over the course of two semesters, students in the Connective Media program become in-depth experts of new technologies. Many take this deep understanding forward and found startups when they graduate.

“We’d like the innovation to be meaningful,” said Naaman. “We want students to create something new that hasn’t been built before, but also to think about why it’s useful and what it can do.”

Within the Connective Media program, students engage with peers from a wide range of backgrounds who are eager to help each other succeed.

“I learn as much as I do from my peers as I do from the faculty,” said Opila. “The one thing we all have in common is an interest in people and building technologies for their unique needs.”

From Pallette, a device controlled by the tongue that is potentially a breakthrough communication tool for quadriplegics and the mobility impaired, to a project aimed at helping Riders for Health motorcyclist workers in Lesotho track medical samples and monitor the transport process, people are the focus of every connective media project.

Connective Media students don’t take traditional exams, but they do receive intense technical training. “Evaluation is an ongoing process, and students really showcase their knowledge through their projects,” explained Namaan. “It’s a unique program in the sense that it provides strong basics in technology, and teaches students how to develop and build their ideas.”

For students like Opila, this is exactly the type of training they’re looking for, which blends technical and problem-solving skills with real-world issues they care about. “That’s why the Connective Media program gives you an advantage where other straight computer science programs wouldn’t.”