News & Politics

Mobile Vision Mixer. A System for Collaborative Live Mobile Video Production. Ramin Toussi. Department of Computer and Systems Sciences.

Description
Mobile Vision Mixer A System for Collaborative Live Mobile Video Production Ramin Toussi Department of Computer and Systems Sciences January 2011 Advisor: Oskar Juhlin Second Advisor: Arvid Engström Examiner:
Published
of 45
All materials on our website are shared by users. If you have any questions about copyright issues, please report us to resolve them. We are always happy to assist you.
Related Documents
Share
Transcript
Mobile Vision Mixer A System for Collaborative Live Mobile Video Production Ramin Toussi Department of Computer and Systems Sciences January 2011 Advisor: Oskar Juhlin Second Advisor: Arvid Engström Examiner: Fredrik Kilander This masters thesis corresponds to 30 credits. IV Summary. Mobile phones equipped with video cameras and access to high-bandwidth networks enable a new generation of mobile applications: live amateur video production. There are already mobile technologies that allow capturing, editing and broadcasting live video from mobile phones; however some parts of production techniques still remain exclusively in the hands of professionals; multi-camera filming in a live broadcast of sporting events is an obvious example for such. In this thesis, a system is described that is developed to address these needs for amateur video producers. Specifically, it focuses on providing a real-time collaborative mobile environment that can be used by mobile users who are interested in making live video from various angles only by using their phones. A user study was also conducted to evaluate the system and see how people would use the system to collaboratively produce a live video. Results from the study prove that although producing a live mobile video is not always easy and straightforward, features like live preview and being able to actively communicate with each other are of great importance and help. Acknowledgments This project would have never become possible without the contribution of several people. First and foremost I would like to thank my advisor and MobileLife centre director, Oskar Juhlin. Thank you for the great support, for trusting me and guiding me through the project. I also want to thank Arvid Engström, my second advisor and Mobility Studio director at MobileLife, whose knowledge and talent have always inspired me. Arvid also led the team while evaluating the prototype in Malmö and Göteborg. When it comes to the evaluation of the work, acknowledgments should go to Alexandra Weilenmann and Emelie Dahlström as well, with whom I had fantastic experiences. Great thank you to Mahak Memar and Mudassar Ahmed Mughal, my friends and colleagues at SICS and MobileLife who helped us running tests by acting as remote operator in Stockholm. I also have to mention all those anonymous young people who volunteered to participate in our test sessions. Emelie wrote a separate report about the evaluation. It was written in Swedish; but later on, she helped me translate some parts of it to English, for this thesis and the paper. Tack så mycket Emelie! The project, the dissertation and all other related reports and papers received some invaluable comments from other contributors to this effort. First and most of all I want to thank Goranka Zoric (Goga) for the fruitful discussions we had when writing the paper. She really inspired me by her knowledge and patience. This also includes Kari Gustafsson, Michael Kitzler and Per Sivborg with whom we collaborated a lot for patenting the idea; the thesis and particularly its technical parts was excellently aspired by these people. While implementing, I took advice and technical support from a couple of people from external companies. Among them, I mention Bambuser and MediaLooks who provided us with resources about their services. Also, they were both smart and fast in responding to my technical questions. Specially, I want to mention Måns Adler from Bambuser and Hanno Roding from MediaLooks particularly. VI A very special thanks to Fredrik Kilander, my teacher at DSV, KTH Interactive System Engineering program manager and this thesis examiner. I believe I was so lucky to meet you and have you as my examiner. You most impressed with all your kindness, patience and commitment to your job and to your students. The present dissertation also received several excellent comments from you. Thank you so very much. Acknowledgments as well to all people at MobileLife, SICS and Interactive Institute, for every wonderful moment we shared together. And last but not least, I have to thank my family and friends from the bottom of my heart. My parents, for always being there; my father who has always been a real father, supportive, caring and compassionate. To mum, for all the goods you have in your soul, your love and devotion. My brother and his wife, for all the laughter we had together and that you will be amazing parents soon! Contents 1 Introduction Research Problem Methodology Contribution Layout Background Video in HCI and CSCW Video Production Professional Production Amateurs and semi-professionals practices Comparison and conclusion Mobile Broadcasting, More Mobility Related Work System overview Inspiration and Implication for Design Employed Technologies and Components Ideal Architecture Use Scenario First Attempts and Lessons Learnt Implemented Architecture Bambuser API Broadcasting with Adobe Flash Media Live Encoder Combiner Process Switch Process Vision Mixer Mobile Application Communication Further Technical Improvements VIII Contents 4 System Evaluation Method and Setting Study Results Problems Found Discussion Conclusion and Further Work Further Work References List of Figures 1.1 SKY Sport24 news channel production control room A typical combination of live streams The Mobile Vision Mixer application prototype running on a Nokia N86 8MP phone FLV playback with DirectShow in GraphEdit Vision Mixer Architecture Mobile Vision Mixer in Operation Simplified Vision Mixer Architecture Mobile Vision Mixer in Operation, in a Simplified Architecture Combiner Flash component layout Abstract Model of Communication and Data Flow in Mobile Vision Mixer Conceptual Design of the Ideal Video Combiner and Switch Component Integration Evaluation in Malmö Codirecting with MVM List of Tables 3.1 Some of the available metadata fields contained in a typical video object returned by Bambuser getvideos API function 1 Introduction This thesis is reporting on the Mobile Vision Mixer (MVM)system, an application prototype than can provide mobile users with a real-time collaborative environment by which they can make live broadcast with their own footage of any event. It can be specifically useful for mobile users who are interested in video practices. This work can be recognized as a significant step forward in mobile video; between July and December 2010 it gained some press interests 1, designated the title of innovation while a patent is also pending and is expected to be finalized soon 2. Having features like video cameras and high-bandwidth network access integrated into the recent(2010) mobile technologies, mobile users are now enabled to have a firsthand ability of social media creation. With this, mobile phones are now beyond communication and passive media consumption devices. This integration, by taking advantage of a sheer peculiar characteristic of mobile devices, being available everywhere and all the time, has ended up with the emergence and development of new services for immediate publishing and sharing of live image and video [16, 26, 27]. ComVu Pocket Caster, launched in 2005 which later on was renamed Livecast is the pioneer in live mobile video publishing. In the years that followed, more services were introduced like Qik, Kyte, Flixwagon, Floobs, Next2Friends, Stickam, Ustream and Bambuser 3 ; among which, Qik and Bambuser are the most widely used [22]. Employing this sort of capture-and-share-straightaway [23] services allows people to instantly share their captured mobile images through manageable web pages instead of using s, paper prints and web publishing [23, 26]. Mobile phones in this way enhance a shared experience among the spectators of a live event. Moreover, in distributed events like car rally or bicycle racing, this experience will become even more enjoyable [16, 18]. However, results from previous studies and research show that although these live mobile broadcasting services are available for indi- 1 e.g.: 2 European Patent Office, under Rule 19(3) EPC 3 2 1 Introduction viduals, allowing people only to broadcast from their mobile devices is not enough. This includes situations in which a group collaborates to create a live show such as sporting events or live TV interviews. Accordingly, challenges still remain for the designers of these services to provide their users with features that so far have been exclusively in the hands of professionals [22]. The production of live TV shows usually takes place under a time critical condition [19] and needs to be highly coordinated among the entire production team members. On the other hand, events like team based sports might be distributed over a large area or could happen so fast (as it is in ice hockey and football) to be covered only by one camera; hence the need for the real-time coordination of several cameras is extremely felt [19]. In such multi-camera settings, each camera starts filming from a position that is defined by the director; their corresponding video streams are simultaneously transmitted to the production control room. There, the director by having multiple views of both live and pre-recorded items on an array of monitors, can manage a suitable selection and combination of streams to provide the spectators of the final broadcast with the best viewing experience. The main role in the production room is played by the director or Vision Mixer(VM) who also has control over the switching operation, ProcAmps(Processing Amplifiers) like brightness, contrast, hue and saturation as well as the instant replays and handling the communication between team members [19, 25]. Video production in this sense, should be considered as an interactional process which demands an extensive collaboration to provide the director with efficient direction capabilities [14]. The spectators consequently, would enjoy the final outcome of this collaboration, the seamless broadcast of the event from multiple angles, without ever being aware of what happens behind the scenes [19]. Figure 1.1 shows SKY Sport24 news channel production control room. In this picture, the director and his staff together with the mixing console, video sources, preview monitors and other equipments are visible. With the advancements in digital and mobile technology, contribution from amateurs in video production has been made possible. Phones with advanced functionalities are coming that can promote current amateur mobile video technologies by establishing distributed real-time collaborative environments [16, 18, 25]. The area has recently gained more interest in research and a growing field of practice is visible as well; yet to enable users to experiment a real video work, more research and studies are expected [19, 22]. There are already mobile technologies for capturing and sharing of multimedia content; some current devices like Nokia N-Series, Apple iphones and Android phones also support basic post-recording editing functions such as rough cuts and ordinary transitions that are useful for individual spectators who want to capture and 1 Introduction 3 Fig SKY Sport24 news channel production control room4 share moments with others. However, no solution exists to address more advanced requirements of mobile video users. It has also been argued that with the increasing interest in this area and to support amateurs with a more robust and effective collaboration on the Internet, applications like a mobile mixer need to be designed to allow viewers to collaborate in such a shared experience [16,18,25]. The Mobile Vision Mixer(MVM) prototype targets these needs by providing users with freedom, liveness and coordination of the task. This work, mainly addresses the mobility and collaborative aspects of amateur video making. MVM consists of a mobile application and a remote service setup for choosing and receiving a group of four live video streams from users broadcasting an event with their mobile phones. The mobile application represents live preview of each stream to its user. The mobile user then, can select a video stream from the preview for broadcasting to the Internet. This broadcast can immediately be publicly shared via social networks or a personal webpage. Figure 1.2 depicts a possible mix of streams from four different cameras in an ice hockey match. MVM is presented in the current discourse. It is an innovative example in which a collaborative mobile environment enables a group of users to co-produce and broadcast a live footage only by using their mobile phones. MVM is developed as a functional prototype to probe the feasibility of mobile collaborative live video production applications and to investigate how people would use such systems to make a video. While working, some room for further work was also discovered. Some invaluable results were also found, like an agenda of system properties and other challenges in developing similar and more advanced systems. 4 1 Introduction Fig A typical combination of live streams The system consists of a backend mixing and switching application, a web service and a mobile application for any mobile phone that supports Adobe Flash Platform [6]. Bambuser is been used to address the live streaming needs. The system in this sense, includes four mobile camera operators streaming from their mobile phones; while the user who is running MVM on his phone (the director) sees a quadruple live preview of them (mixing) and can select one for broadcast at any moment (switching). The system is created to allow mobile users coordinate a live mobile video production through a multi-camera setting. Previous studies also reveal that live TV shows or sporting events become more attractive and understandable to viewers if multi-angle shots like wides and mediums or detailed and overviews are provided [14, 18, 25]. MVM can address this need likewise by presenting the seamless switching capability. Figure 1.3 depicts a running instance of MVM. MVM can make co-directing a live broadcast enjoyable by getting all participants joining the production task; having live preview of every stream, will provide the director with a better understanding of the collaboration. The author hopes that this work could influence the design for mobile video services. 1.1 Research Problem Current technologies do not allow amateur video producers to collaboratively make and broadcast a live footage. Addressing this problem is the main focus of this dissertation; in this way and by presenting a system prototype, the thesis belongs to 1.2 Methodology 5 Fig The Mobile Vision Mixer application prototype running on a Nokia N86 8MP phone the Artifact Development type 5. Chapter 2 justifies why such a system is required and what aspects were found missing in the previous works; but briefly speaking, bringing extreme mobility to video producers, providing mobile video users with a collaborative tool, all together in a mobile system, is the main research problem in this thesis. 1.2 Methodology Considering the nature of the problem that I was supposed to create a functional prototype in a reasonable time (February - May 2010), it was realized that the traditional methods of user centered design which expects the HCI designers to begin by observing the target group, defining their requirements and designing for them might be time costing. Alternative solution then, was to put more effort on designing and implementing the prototype; yet the main strain toward this thesis can be divided into the following steps: 1. Pre-studies 2. Design and the implementation of the prototype 3. System evaluation The first step constituted some literature review, checking similar works, getting familiarized with related technologies and trying out possible solutions to make the prototype. 5 Thesis Information, Version 4, May 3, available at last visited: December 10, 2010 6 1 Introduction Next step focused on the actual implementation of the system, which is described later and in details in chapter 3. System evaluation as the final phase was expected to provide essential feedback about users experimenting the system. To do this, user studies were conducted while test sessions were video recorded for further analysis. The study method as well as results found, are presented in chapter Contribution The effort was done as part of my master thesis final project at MobileLife research centre 6 in Stockholm, Kista between January and July I was mainly responsible for designing and developing the entire MVM system. The evaluation of the project however, was not in the scope of my task when the project started; yet, later on it was decided and done under the same working group s supervision, morevideo! 7 at MobileLife. Needless to say that the main contribution of the thesis is the MVM prototype itself, a mobile system that offers its users to co-produce a live video. The successful launch of the system showed the possibility of similar and more sophisticated systems. While working, my colleagues and I also learned a lot; particularly with the user studies, seeing people in collaboration and figuring out some other related aspects. 1.4 Layout After presenting an introduction to the main idea of the work in this chapter, the rest of the dissertation is outlined as follows: The discourse continues by chapter 2 for giving a background of the topic mentioning the importance of video in HCI 8 and CSCW 9, followed by an overview of professionals and amateurs video production interests and practices. Mobile broadcasting services as well as some examples from previous works are also presented in the same chapter. Chapter 3 then gives an overview of the design and implementation of the system. Technical aspects of the system are discussed in detail by presenting the architecture of the system, how it is implemented, use scenario and system components. Expected technical improvements are also briefly presented Human Computer Interaction 9 Computer Supported Collaborative Work 1.4 Layout 7 The evaluation process, addressing the two conducted user studies is briefly discussed in chapter 4. The results of the evaluation as well as the problems found with MVM are also described in that chapter. The discourse is finally concluded with chapter 5. Some expected work to improve the system are also discussed in this chapter. 2 Background When it comes to video production in CSCW, video as a social media gains multiplied interest from its researchers as it could be considered both as a mediated channel for communication and a topic of concern [25]. Technologies have been developed to provide a certain amount of collaboration between members of a video production team; they are usually supplied along with fixed devices like TV studios, production rooms or custom fitted buses which provide the production team with some degree of fluidity. The concept of mobility however, has been widely ignored in CSCW [24], while with the recent advancements in mobile technologies more effort in research on live interaction with visual content is expected. This movement includes developments and improvements in processing power of mobile phones and pervasion of high-bandwidth Internet mobile networks like 3G and recently 4G (2010); at the same time, high speed network access and storage costs are decreasing [19]. In the consumer demand, the situation is even worse; there actually exist no real practices to provide ordinary people with collaborative video productions, even though several products have been developed that allow some sort of fluidity between individuals on fixed devices, which limits the users to their domain [24, 25]. These issues are discussed briefly in the following subsections. First, video as a visual social media and its role in HCI and CSCW is discussed. Then, the video production is presented in two different ways, first on what happens in professional production and then what amateurs
Search
Similar documents
View more...
Related Search
We Need Your Support
Thank you for visiting our website and your interest in our free products and services. We are nonprofit website to share and download documents. To the running of this website, we need your help to support us.

Thanks to everyone for your continued support.

No, Thanks