Press Releases

Manifestation of Virtual Assistants and Robots into Daily Life: Vision and Challenges

Similar to how the smartphone and Internet have significantly changed our daily lives, artificial intelligence (AI) applications have started to profoundly affect our everyday lives as well. Two major products of this relatively recent trend are
of 11
All materials on our website are shared by users. If you have any questions about copyright issues, please report us to resolve them. We are always happy to assist you.
Related Documents
  1   Manifestation of Virtual Assistants and Robots into Daily Life: Vision and Challenges Reza Rawassizadeh 1 , Taylan Sen 2 , Sunny Jung Kim 3 , Christian Meurisch 4 , Hamidreza Keshavarz 5 , Max MŸhlhŠuser  4  and Michael Pazzani 6   1  Department of Computer Science, Metropolitan College, Boston University. US 2 Department of Computer Science, University of Rochester, US 3  Department of Health Behavior and Policy, School of Medicine, Massey Cancer Center, Virginia Commonwealth University, US 4   Department of Computer Science, Telecooperation Group, Technical University of Darmstadt, Germany 5  Faculty of Electrical and Computer Engineering, Tarbiat Modares University, Iran 6  Department of Computer Science, University of California Riverside, US   Similar to how the smartphone and Internet have significantly changed our daily lives, artificial intelligence (AI) applications have started to profoundly affect our everyday lives as well. Two major products of this relatively recent trend are virtual assistants and home robots. They have similar functional characteristics: both interact with users through conversational agents and attempt to imitate human behavior. Home robots host a virtual assistant and have mechanical capabilities as well. There are many discussions about risks, challenges and the future vision associated with the proliferation of AI at the industrial level. These discussions, however, have not yet widely extended to the user level within the context of daily lives. In this article, we provide a review to discuss the benefits, risks, challenges, open questions and the future vision of using virtual assistants and social robots in daily lives. Key words:  Robot, Virtual Assistant, Conversational Agents, Ubiquitous Computing 1. INTRODUCTION AND BACKGROUND On July 10 th , 2017, Alexa, AmazonÕs virtual assistant may have saved an assault victimÕs life by calling 911 after a suspect accused his girlfriend of cheating and threatened to kill her if she called the police 1 . While the device may have misinterpreted suspectÕs questioning of his girlfriend Ò  Did you call the sheriff? Ó as a request to call 911, this event is a harbinger of the rising  pervasiveness and power that AI will bring to our daily lives. The notion of robot assistants is as early as ancient history. According to the Greek myth, Hephaestus, the god of invention and technology, built Talos, a robotic statue. Just as Alexa may have protected suspectÕs girlfriend by calling 911, the purpose of Talos was to protect Europa at the request of Zeus himself. While the concept of automata is as old as civilization, it has not been until recent history that artificial intelligence (AI) has begun to be concretely defined and realized. In 1948, Norbert Wiener formalized the term cybernetic as Ò the scientific  study of control and communication in both living things and machines Ó [1]. This definition has since been refined, and now describes several disciplines within computer science, including AI and robotics. In the 1950s, Alan Turing, devised an exper-iment, called the ÒImitation GameÓ (a.k.a. Turing Test) [2], as a way of answering whether machines can indistinguishably imitate human behavior, and therefore truly think. Turing predicted that such machines would exist by the year 2000. Now, with the proliferation of chatbots and virtual assistants, many consider this a remarkably achievable task. Imitation of human  behavior is one of the oldest goals in the field of computer science. These definitions were used to define AI as Ò the capability of a machine to imitate intelligent human behavior  Ó [3]. Improvements in the production of inexpensive sensors, algorithmic advancements, networking facilities, and hardware miniaturization increasingly train the spotlight on cybernetic applications, including virtual assistants , chatbots , and  social robots . They are different  AI applications, they mimic one or more of the cognitive capabilities of humans, most notably simulating naturalistic dialogue. Natural language interfaces (NLI) are being adapted very rapidly by corporations such as Amazon, Apple, Google, Samsung, etc., and it has been predicted that the next major shift in user experience designs will be in NLI interfaces including conversational agents in robots and virtual assistants [4].   One report [5] shows that 44 percent of smartphone users use virtual assistantsÑgreater than the 34 percent of smartphone users who have installed web search applications. Virtual assistants are often integrated into devices referred smart display, speakers (e.g. Amazon Echo, Google Home and Apple Homepod), smartphones and desktops. 1   Voice-activated device called 911 during attack, New Mexico authorities say [accessed 19th Jul 2019]    2   The term Òsocial robotÓ is defined as a robot that can communicate, understand and relate to users. Social robots may host virtual assistants, but also possess mechanical capabilities, such as the ability to move their body parts. Existing home or social robots are focused on four major applications [6], health care and therapy, education, public spaces and work environment, and home settings. One commercialized example of these robots is Jibo 2 . There are also humanoid robots, which are program-able and thus can operate as social-robots such as NAO robot 3 . However, due to their high cost, humanoid robots are currently not affordable for widespread home use.  Notwithstanding, social or home robots and virtual assistants are gaining a momentum in the marketplace. Due to the increasing computational power, an aging population, and increasingly individualized lifestyles, we envision home robots and virtual assistants will significantly change our daily lives. New research possibilities will open across multiple disciplines and expand toward AI applications. Table 1 presents primary characteristics of current consumer devices along with their limitations which we envision will  be overcome in coming virtual assistants and social robots. There are many more other ubiquitous devices, which are used for single or limited number of purposes, such as IoT thermostats, smart rings and smart mirrors. However, we list ones which are widely in use in the consumer market. Besides, this table only focuses only on device characteristics of these devices and other aspects such as security are not discussed. Table 1: A comparison between user interaction characteristics of consumer market ubiquitous devices.   In the following, we will discuss the relevant academic literature as well as analyze trends in related future markets from crowd sourcing platforms (i.e. Indiegogo and Kickstarter), and then we propose our overview on these technologies. In partic-ular, our contribution in this paper is two-fold. First, we analyze and list the risks and benefits of existing systems. Next, we discuss challenges, future directions and visions associated with these technologies. We focus on AI applications for personal use in home   settings and not industrial applications. This style of classifying the discussion (first describing risks/benefits and then the future vision) is inspired by other scholarly works that examined a technological product by first providing a survey about risks and advantages and then discussing the vision and future direction [6, 7]. 2   Jibo robot homepage: [accessed 20 th  July 2019]   3  Nao robot homepage: [accessed 20 th  July 2019]   Home Robot   Smartphone   Wearables (Smartwatch/ AR glass)   Smart Speaker / Smart Display   Proximity to the user  Since they have me-chanical capability and they can follow the user it could be high. However, still it has not been stud-ied. High, but not as high as smartwatch and similar weara- bles such as fitness trackers. Smartwatch is very high, AR glasses could be high only if the user wear them. Unlike smart-watch the user will not wear the AR glass continuously during the day. Very low, because they are static and not mobile. Application Varia-bility  Limited due to its immaturity and lack of de-facto standard operating systems. Very high and smart-phones have the largest market in comparison to other devices. High for AR glasses and limited for smartwatches, due to their bat-tery and size limitations. Limited due to its immatu-rity and the lack of a de-facto standard operating system. Interaction Modal-ity  Mainly voice, touch could be supported as well. Mainly touch and voice. Mainly touch for smartwatches and hand gesture for AR glasses. Mainly voice, touch could be supported for smart display as well. Connectivity  Can operate discon-nected from the In-ternet. Most of the time Connected to the Internet. Can operate disconnected from the Internet. Always connected to the In-ternet. Sensor Variability  Limited to the main application of the robot functions. Multiple sensors and available for different types of applications Limited due to the size and bat-tery limitations. Very limited due to the reli-ance on mainly using voice interaction. Mechanical Capa-bility  yes no no Very limited, e.g. rotating its camera toward the user.  3   2. RISKS & BENEFITS OF EXISTING SYSTEMS Studies about applications of social robots [8] and virtual assistants [9] discuss two major categories of services, task automa-tion and user interaction . Often there is no distinct borderline between these two services with most existing systems support-ing both. Each of these services are associated with risks and benefits, which we describe in the following sections. 2.1 . Task Automation One of the oldest expectations from robots and computers in the home setting, has been   helping users in their daily life challenges, such as cooking foods and washing dishes. There are successful examples of robots which help with domestic tasks, including vacuum cleaning and lawn mowing robots. We categorize these services as task automation . While task auto-mation embodies promising applications as mentioned above, there are several potential associated risks. First, the replacement of human labor with mechanical labor may lead to an increased chance of inactivity and obesity among users. Another widely discussed risk is unemployment among professional assistants who offer similar services, e.g. home gardeners, which is still a topic of discussion among scientists and sociologists [10, 11]. Second,   mechanical errors of a robot with a hard body coming into contact with a relatively fragile human body can be catastrophic, and thus require a high level of risk assessment and caution. There are recent reports that investigate the harm a robot can physically cause to their owners [12], which necessitates further research on safety. Third, an area of task automation introduce benefit and harm are health applications. The health care industry has recently shown research in health care robots and health related virtual assistants [13]. For instance, there are robots designed to address sexual needs as well as emotional needs. These technologies can be useful for individuals who are lacking in building inter- personal relationships and intimacy. This, however, raises some ethical questions [14], including the concerns that erotic as-sistants may increase the social isolation among users. In addition to these challenges, data collection toward satisfying certain needs may expose users to privacy and security risks, especially by systems which compile extensive datasets filled with inherently personal, and often confidential, user information, such as user information and exposing recorded voice communications recorded in their most private spaces (e.g.  bedroom) to untrusted parties. To mitigate these challenges, designers and policy makers should consider unintended health risks,    security and safety   is- sues , ethical dilemmas  and the  privacy  of their system. For instance, considering privacy risks while implementing their algo-rithms and design their system completely independent from a network [15], which means securing all collected data locally. 2.2. User Interaction Both virtual assistants and social robots typically include a software component that has  personality , support a naive inte- gration of emotions, and   use mainly voice interfaces  for interaction through natural language-based communication. These factors will likely allow virtual assistants and social robots to avoid usability issues pertinent to mobile health applications [16]. For example, mobile health applications often use graphs and numbers to represent information, despite the fact that graph illiteracy is a significant problem even in developed countries. This indicates that several groups of usersÑincluding those with low numeracy and those with minor visual impairment, e.g. elderly usersÑare likely to be underserved by these applications. On the other hand, with respect to the accessibility, virtual assistants and robots use conversational agents to interact with users [17]. Therefore, instead of presenting graphs and numbers, these systems can chat or talk to users in much the same way as physicians or health-care providers would. Interaction based on the natural language can be used to facilitate communication with some underserved individuals , includingÑespeciallyÑthose with visual or cognitive impairments (be-cause graphical user interface independence), and individuals who cannot read numbers or graphs. We use mobile-health applications as just one example of an area in which these systems can offer advantages to underserved communities. However, many other types of applications can be deployed for underserved communities, with the same attitude.   Another good example in this line of work, are robots and virtual assistants that are designed to assist elders with their loneliness. Loneliness among the elderly is a significant challenge in developed countries with individualistic cultures. How-ever, there are ethical considerations regarding the  psychological attachment   to man-made assets, such as elder care robots, which diminishes interpersonal interactions. An increasing number of studies revealed negative impact of usersÕ attachment to smartphones and video games [18]. This controversial posit is at least partially rooted in the entertainment capability of com- puters. The gaming industry, for instance, has provided evidence of strong human attachment to artificial pets like Tamagotchi 4 . There are notable similarities between artificial pets and virtual assistants, especially where psychological attachment is con-cerned. Indeed, many believe that the risks associated with human-machine interactions outweigh the advantages. Humans tend to bond with other humans who exhibit empathy, emotional intelligence, and abundant social and cognitive resources. Robots and virtual assistants could satisfy needs for companionship. There are have been some regarding child interaction with 4   Tamagotchi toy home page: [Accessed 20 Jul 2019]  4   robots [19], smart speakers [20] and virtual assistants [21]. However, the longitudinal consequences for human psychology are not yet well known, as these technologies are still in their infancy stage. Another example is virtual assistants used as coaches or professional therapists, e.g. to mitigate mental health [22]. They can be used to mitigate or cure psychiatric, cognitive, or physical impairments. They can communicate with users and provide feedback toward achieving certain goals, often using natural language to simulate human speech. As with a human coach, they can deliver feedback proactively or reactively. These are promising services. However they are also associated with the poten-tial risks of mass unemployment  , and the possibility of biasing users toward certain products, services, or perspectives . Unlike devices which provide graphical user interfaces, existing implementations of conversational agents show less motivation to-ward application development openness. As a result, existing systems are limited to the default functionalities provided by their vendors. Biases can influence the mindset of users. For example, a corporation intent to benefit from its technologies by delivering content that favors its organizational interests rather than social norms of the user community, or consider a certain expression of pop-culture, which is sometimes against values and norms of a society. For example, a virtual assistant randomly  plays music may stream sponsored music tracks that promote violence and drug use. Table 2 presents the risks and benefit we have described and identified by using these devices and technologies. To sum-marize this discussion, we believe the increasing ubiquity of these applications is a double-edged sword. They should be designed with an eye toward ethics, user rights, and caution to unintended or unanticipated consequences. Nevertheless, we cannot propose a useful remedy before these conversational agents will be used on a large scale in different cultures and communities. As these systems proliferate into the consumer market, more information will become available and thus more risks and benefits can be identified. Description Robot Virtual Assistant Risk Lack of physical human activities because of daily task automation !    _ Mechanical safety related issues !   _ Ethical challenges related to reducing interaction among humans !   !  Physiological and mental attachment to human-made artifacts to satisfy the need for companionship !   !  Mass unemployment !   !  Biasing toward certain product, services or perspectives _ ! Benefit Improving mental and physical health !   !  Substituting expensive human based tutoring and coaching !   !  Assisting users in their daily physical needs, such as moving a heavy object !   _ Enabling underserved individuals to benefit from digital health technologies _ !  Satisfying the need for companionship !   !   Table 2: risks and benefits associated with proliferation of virtual assistants and robots.   3. FUTURE VISION AND OPEN QUESTIONS There are three identified challenges that vendors address in the voice-recognition domain: first improving speech recog-nition and command processing; second offering support for different languages, different accents, and bilingual users; and third understanding conversational contexts and establishing rapport 5 . Vendors have recognized these challenges, and some  proposals and efforts are promising. Deep learning algorithms, for instance, have enabled tremendous advances in speech recognition. These three challenges are well-recognized and the community is working towards mitigating them. In addition, we believe there are other challenges, which are not widely recognized by the community and we introduce them here. The following section highlights these open questions and future vision that have been often relatively understudied. 3.1. Communication & Conversation Existing   virtual assistants operate through predefined conditional ÑÒif x, then yÓÑ rules. For instance, AmazonÕs Alexa uses a list of predefined skills that users can download and run on their devices 6 . Existing systems usually cannot understand questions outside their knowledge base [23]. Traditionally when a user poses a question that does not exist in the knowledge  base of the agent, the agent answers with either a variant of ÒI donÕt understandÓ or else pastes the userÕs question directly into a web search. This limits the usability and reliability of these systems. However, there are promising algorithms and challenges that are in progress to enable dynamic, on the fly answer reconstruction. For example, CoQA [24] proposes a dataset 5   How conversation (with context) will usher in the AI future. [accessed 20 th  Jul 2019]   6   Amazon Alexa skill repository [accessed 20 th  Jul 2019]    5   to enable algorithms dynamically learning questions/answers, rather than extracting questions/answers from a static dataset. Sounding Board [25] models the user utterance by using multidimensional representation and content-oriented conversation segments. It creates a randomness in conversation and adjusts the answer to the user mental model. Another example is the conversational model of ERICA the robot [26], which analyzes the focused words of usersÕ utterances and constructs the response based on the focused words. As a result, it provides answers with improved accuracy than traditional methods. Research works are moving fast toward supporting more creative answering and information extraction. There might be two other approaches toward mitigating this challenge. (i) The agent can establish a ÒdialogueÓ with the user to collect more information, responding to unexpected questions by saying, for example: ÒI donÕt understand your question, can you provide more details? and  ÒI will try to learn this topic from the web, thought it will take some times. Please ask me this question again later.Ó. (ii) The agent can search knowledge bases outside their own, such as analyzing information from the web. Note that this is different than simply reasserting the userÕs question as a web search query. A stunning theorem was developed (GšdelÕs incompleteness theorem), which provides a profound link between the con-creteness the cybernetic systems described by Wiener, and the philosophical question of what machines can be proven to accomplish. In summary, GšdelÕs Incompleteness Theorem provides that given a concrete mathematical system (with some easy to satisfy properties), there exist true statements that cannot be proven in finite time. In other words, while it may be true that artificial intelligence is capable of developing novel, creative material, it might not be possible to formally prove that they have such a capability. In interpreting GšdelÕs incompleteness theorem, there are two schools of thought about Artificial In-telligence. One group including Lucas and Hofstadter [27] describes that since machines are limited to a predefined formali-zation, decision making will be limited to the grammar of that formalization and machines cannot step outside of their formal-ization limits. On the other hand, there are scientists such as Norvig and Russell [3] who do not agree with this argument and state a computer can invent a new formalization, and therefore it can implement creativity. Thus, the capability of creative communication by the AI and constructing answer without an accessing a knowledge base remains an unanswered question. 3.2. Context Sensing and Personalization Existing systems collect limited contextual and sensor data, often neglecting most available sensors on the device. Many systems, such as Google Assistant or Siri, run on the smartphone. Except calendar, location and email, they do not use other personal data or smartphone sensor data [23]. At the time of writing this paper, there are no known virtual assistants that benefit from contextual data. Some social ro- bots, such as Kuri 7 , may designed to collect contextual data.    Never-theless, they are not produced in large scale due to their unsuccessful marketing campaign. Collecting these data might be associated with  privacy risks as well, but contextual data collection will assist per-sonalization of the services these systems provide.   Social robots and also Internet of Things devices have two distinct differences from the traditional context-sensing systems found in smartphones and wearables. First, unlike smartphones and wearables, they are usually  shared among household members . Therefore, they should identify a user from a small group of users (e.g. family or guests). This sort of identification allows more effective personaliza-tion. Second, unlike smartphones and wearables, social robots and smart speakers are not constantly attached to a userÕs body.  Accord-ingly, these devices can observe user activities and collect data from a third-person perspective . This enables more accurate activity recog-nition   and mitigate data quality challenges existed in wearable de-vices [28]. For example, a robot can be used to track weight-lifting and other physical activities, which include using weights. Currently, wearable and mobile devices perform activity tracking,  but this comes with several significant limitations. One is that these devices are incapable of collecting certain data about weight lifted by the user or details of their activities. Because they cannot accurately monitor users from a third person per- spective . Google Fit Workout in Wear OS, for example, prompts users to enter workout and weight-data manually (see Figure 1a). A robot, on the other hand, could unobtrusively, but closely, follow a user to collect the same data and it obviates the need for manual data entry. For example, by using its camera and an image-recognition algorithm it can recognize the type of activity with high precision (see Figure 1b).   Some may argue that because certain AI technologies, like smart speakers, are non-mobile, they are incapable of collecting enough contextual information to be useful. However, there are promising efforts to the contrary. Laput et al. [29] proposed a static but powerful sensing device that can collect data from its contextÑfor instance, data concerning activities a user performs in the kitchen. Even without mobility, these systems can collect useful data in their target environments. Furthermore, Cohen 7  Kuri production has been suspended [accessed 20th Jul 2019] FIGURE 1: (a) a screenshot of Google Fit Workout from An-droid wear smartwatch, the user should manually enter the amount of weight used for the bench press. (b) a robot can monitor the user behavior to recognize its activities while staying in the close proximity of the user.  
Similar documents
View more...
Related Search
We Need Your Support
Thank you for visiting our website and your interest in our free products and services. We are nonprofit website to share and download documents. To the running of this website, we need your help to support us.

Thanks to everyone for your continued support.

No, Thanks

We need your sign to support Project to invent "SMART AND CONTROLLABLE REFLECTIVE BALLOONS" to cover the Sun and Save Our Earth.

More details...

Sign Now!

We are very appreciated for your Prompt Action!