Class: Physical Computing
Professor: Danny Rozin
“Access” - physical limitation vs. meaningful interactive experience: an interactive visual and sound experience between a visually impaired and a hearing impaired person.
Project Statement | Defining Objectives:
Project Goals:
create a physically interactive system that: demonstrates clear and engaging physical interaction; focuses on sensing of relevant actions of the user it is designed for, and on clear, prompt, and effective response; should be iterative (a system where the user sees the system’s response, and takes more action in response, in a continued loop).
explore creating interactive experiences for accessibility
make physical limitation become a crucial part of a meaningful interactive experience
learn more about design and "equitability"
Role: Individually owned project
Design Challenge
" How can I create a meaningful interactive system where physical limitation enhances users' experience and inspires equitability in experience design? "
Ideation:
The brainstorming process includes finding inspirations for:
meaningful interactions: interesting and thoughtful concepts that inspire meaningful interactions (either new concepts or existing concepts that demand more exploring and creativity to tap into a richer interactive experience, getting people to think and wonder)
physical sensors useful for the interactive system
interesting, engaging supporting materials (additional props that would enhance experience?)
possible multimedia elements involved: sound, visual, light, etc.?
Please check out more on my ideation process blog post.
Design Process:
*Building overall interactive experience:
*Choosing technology options:
p5 js, JavaScript
Arduino micro controller, C
Flex sensor, force sensitive resistor, distance sensor, DC motor
Tensor-flow PoseNet (machine learning real-time human pose estimation)
*Choosing supporting materials:
Basic materials include:
Sensors: a force sensor, a flex sensor,
Fabric: different types of fabric used to hide sensors inside
I built some prototype from the above materials and begin the testing process for technical components (sensors), and some physical interactive components with potential users (what types of fabric would be the most effective, the sizes of different components, etc.)
*Testing individual technical components:
DC motor:
Distance sensor:
Force sensor:
Design Exploration + User Validation:
*User Test 1:
In this first user test, the technical components didn't have to work. It is the first opportunity to let the potential users try out the interactive system I designed, and see if it makes sense. I used paper cardboards and scrap fabric to make prototypes.
📝User Test 1 key take-aways:
When given the fabric pad to touch, users enjoyed touching the soft and furry fabric.
For those users who are blindfolded, they felt more comfortable sitting down.
For the other users who are shown moving, mesmerizing kaleidoscopic visuals on the screen, they were engaged.
Users were curios about looking at the visuals through a pair of diffraction goggles. However, the goggles seemed to be redundant and potentially could make the experience become too complicated due to too many props involved.
User test 2:
The second user test was the time to test all components, including technical components and make sure everything worked as intended. For example, if it was a sound component, then sounds would play. It was a visual components, visuals would show.
📈Improvements from User Test 1:
All technical components worked well.
Visual and sound components were sent properly through working sensors.
Took away diffraction goggles to simplify the interaction and avoid confusion for users.
In the case of this project, there were both sound and visual components that were controlled by sensors. These sensors, in turn, were controlled by the act of touching with human hands. This second user test was also time to obtain any feedback on components and interactions that were key to the whole experience.
📝User Test 2 key take-aways:
Users enjoyed the furry touch-pad
Users asked for the diffraction goggles to return.
Users asked for more sound experiences
Both users desired better and more effective responsive system between the two
Both users needed more instructions to effectively interact
User test 3:
This was the last and final test. As of this stage, all major problems have been addressed. This test was mainly to address any final minor issues, and to fine-tune any aspects that would enhance the experience.
📈Improvements from user test 2:
All interactive components requiring physical touches have been made more sturdy.
In addition to added visual effects, diffraction goggles were brought back as an option for the users for their experience. This piece of supporting prop turned out to be a highly potential enhancing element to the experience (for the hearing-impaired person). Seeing the moving and transforming graphic in front of a curved widescreen through the diffraction goggles truly gave the users an immersive feeling of looking through a mesmerizing kaleidoscope.
The idea of creating lots of movements of graphic elements, lots of shapes transforming with changing colors, also came from the intent to create an engaging, mesmerizing experience such that it draws full attention of the visually capable person in the experience to focus on enjoying and be stimulated by what is designed specifically for someone with primary physical ability being his/her vision (as opposed to hearing, or any other physical abilities, for example).
More sounds have been added to enrich sound experience for visually impaired users.
A more effectively corresponsive system between A and B established. Visual and sound effects became easier to correlate between the two. This improvement was the greatest "breakthrough" in this interactive project. However, there are still drawbacks of the system that need to be taken into account as improvement areas in the future. These areas are further mentioned in Reflection section.
Final Design:
“Access”: An interactive visual and sound experience between a visually impaired and a hearing impaired person.
Message: “There is no physical ability limitation to a meaningful interactive experience.”
Technology tools: p5 js, JavaScript, C, python, Arduino micro controller, flex sensor, force sensitive resistor, Tensor-flow PoseNet (machine learning real-time human pose estimation
Reflection
🎢 Final design: Improvement areas (feedback throughout design process):
The experience was far from an "effective communication system": Establishing an effective line of communication is not an easy task. Even though this was not necessarily the focus or the end-goal of the final experience, I realized humans' natural desire to have an effective responsive system for communication or interaction. I received a lot of good questions from users throughout the design process and during the show around such drawback in the final design.
Feedback system: Digging deeper into the above drawback, what is still lacking from the final design is sufficient amount of feedback between A and B. In the end, A is informed when a sound effect is sent successfully, but A also wants to know "what type of sound effect is that?" or "did really B receive it?" "how would I know that B has received the sound?". Similarly, B wonders "what type of visual effect am I sending?" and "how do I know if I am sending too much or too little? (in other words, am I doing a good job in creating an equally good effect in response to the other person?). These questions were not only eye-opening for me to consider next time when I design a similar system, they gave me an appreciation for designing interactive experiences specifically for accessibility.
Although the final design is far from perfect, I am glad I took on the challenge to create something meaningful and inspiring. The fact that this was an individual challenge made it even more worthwhile as an accomplishment in terms of a learning process.
📝 Overall key take-aways:
Ideation: unlike previous group projects, I decided to work on final project by myself. One undeniable benefit was that I had the freedom to be creative with my own inspirations. It is definitely more appropriate to work alone when you know what you really want to work on or there are unique ideas that inspire you and set your passion "on fire" 🔥. This is particularly crucial, in my opinion, because I was more determined to make my project work no mater how much effort and what challenges involved, due to strong interest and passion for the end-results.
Scale: Despite the freedom of creativity, it was important for me to scale the scope of my project appropriately to where it fits timeline requirement without sacrificing my originally intended concepts.
Design process: Talking to a lot of potential users gave me insights that significantly and positively impacted how my final product turned out in the end. During this design exploration process, I had the opportunity to show it and test it with one of our classmates who is blind - Antonio who reminded me how important it was to create an effective mutually responsive communication system between the two users. Having said that, it was also easy to be carried away with more and more potential "add-ons" from user test feedback. I managed to take in feedback that helped me stay focused on originally intended end-goals and inspirations.
Learning curve: I thoroughly enjoyed stretching myself to learn new techniques, coding knowledge, prototype fabrication skills, and interactive experience design. If a group project teaches me how to collaborate and learn from others while contributing mostly my stronger skill set, an individual project teaches me to learn and work on my weaknesses and get out of my comfort zone.
Seeking critics: 🌈when working on a project individually, it is important to seek critics from outside. But, what is even more important is how to receive feedback (positive or negative) gratefully. I presented my work-in-progress to a panel of critics, including peers and professors. It was not easy to receive criticism, especially when the project has become your "baby", and when a lot of thoughts and efforts have been invested. However, it was crucial for me to take in all feedback with utmost positivity and appreciation. If not all, even a small improvement usually comes out of it. Most important thing is to make sure i am not in my bubble of comfort and disregard opportunities to be improved, especially when those opportunities come from honesty and genuine intents to help me get better.
Project management: 🤹🏻♂️Managing timeline and tasks to be executed, making sure the right order of steps were taken towards project completion, were all crucial during an individual project. At times, I was behind schedule, feeling exhausted at the same time, or shipping schedule of supplies and materials was unexpectedly delayed. I had to come up with a lot of plan B's for some of those instances. Trying the best possible, and knowing when to be content with alternatives, next-to-perfect outcome can sometimes be a good mindset to have. In the end, it was great learning experience being your own project manager.
Thank you to all my family, friends, and professors at NYU for your support throughout the process. Thanks to everyone for stopping by to check out my project at the ITP Winter Show and showing so much interest with many thoughtful questions, interesting observations and feedback. I am grateful for such a fun and resourceful learning opportunity!
Cheers to a continued journey of learning and creating!🚀
Comments