Science

Cross-Media Information Spaces and Architectures (CISA)

Description
1. 2 December 2005 Cross-Media Information Spaces and Architectures Prof. Beat Signer http://www.beatsigner.com Web & Information Systems Engineering Lab Department…
Categories
Published
of 101
All materials on our website are shared by users. If you have any questions about copyright issues, please report us to resolve them. We are always happy to assist you.
Related Documents
Share
Transcript
  • 1. 2 December 2005 Cross-Media Information Spaces and Architectures Prof. Beat Signer http://www.beatsigner.com Web & Information Systems Engineering Lab Department of Computer Science Vrije Universiteit Brussel WEB & INFORMATION SYSTEMS ENGINEERING
  • 2. 01001000100 0101001100010010100010010 0100100010010001000100001001000 100010001100001001000110010101010110 1100010010100010010010010100010101011001 101001100001100 101010100 00101000100 110101010101 00100 01010101000 1001 1111111111 111 0000000010 0 11111111 0101001011 010101010 111010101 001010010110 Fluid Cross-Media Information Spaces
  • 3. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 3June 9, 2017 Prof. Dr. Beat Signer Interactive Paper, Cross-Media Information Architectures Sandra Trullemans Personal Cross-Media Information Management Audrey Sanctorum User-defined Cross-Device and Cross-Media Interaction Reinout Roels MindXpres: Extensible Content- driven Presentation Tool CISA Human-Machine & Human-Information Interaction Information Systems & Management Information Visualisation & Navigation WEB & INFORMATION SYSTEMS ENGINEERING CROSS-MEDIA INFORMATION SPACES AND ARCHITECTURES (CISA)
  • 4. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 4June 9, 2017 Dr. Ahmed A.O. Tayeh Open Cross-Media Authoring, Fluid Document Formats WEB & INFORMATION SYSTEMS ENGINEERING CISA Human-Machine & Human-Information Interaction Information Systems & Management Information Visualisation & Navigation CROSS-MEDIA INFORMATION SPACES AND ARCHITECTURES (CISA) Lars Van Holsbeeke Smart Environments, Implicit Human-Computer Interaction Jan Maushagen Learning Analytics, Adaptive Persuasive ICT Tools Payam Ebrahimi Data Physicalisation Framework
  • 5. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 5June 9, 2017 Former Members CISA Human-Machine & Human-Information Interaction Information Systems & Management Information Visualisation & Navigation CROSS-MEDIA INFORMATION SPACES AND ARCHITECTURES (CISA) Dr. Lode Hoste Multimodal Interaction, Midas and Mudra Framework Prof. Dr. Bruno Dumas Multimodal Interaction, Cross- Media Information Architectures Brecht De Rooms Implicit Context-aware Human- Computer Interaction, HCII Cristian Vasquez Paulus Community Semantics, Structured Data on the Web
  • 6. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 6June 9, 2017 CISA Human-Machine & Human-Information Interaction Information Systems & Management Information Visualisation & Navigation RSL Hypermedia Metamodel MindXpres Paper-Digital Interfaces PimVisTangible Holograms
  • 7. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 7June 9, 2017 What is Wrong with Digital Documents?  Existing document formats are based on the simulation of paper affordances on desktop computers  How to manage mixed-media "documents" in open and fluid cross-media information spaces?  on the data level - context-sensitive adapatation - cross-media transclusion  on the visualisation and navigation level - zoomable user interfaces  on the cross-media interaction level - fluid multimodal cross-media interfaces  Remediation of the "paper simulation" approach  WYSIWYG is only one out of many options! CISA Human-Machine & Human-Information Interaction Information Systems & Management Information Visualisation & Navigation
  • 8. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 8June 9, 2017 Cross-Media Information Spaces and Architectures Information Systems & Management Information Visualisation & Navigation Information Visualisation nd NavigationHuman-Machine & Human-Information Interaction CISA RSL [4] MindXpres [7] OC2 [5] associative file system [2] data-driven storytelling [10] XIMA [12] cross-device interaction [8] iServer/iPaper [3] ViDaX [15] data physicalisation [30] map-based interaction [17] non-linear storytelling [25] PaperProof [27]iGesture [21] Query by Sketch [28] PaperSketch [29] Context Modelling Toolkit [9] Midas [19] SpeeG2 [22] PaperPoint [24] Print-n-Link [16] digital libraries [11] Mudra [20] mixed reality [26] ArtVis [14] open cross-media linking [1] PimVis [13] DocTr [6] EdFest [23] source code visualisation [18]
  • 9. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 9June 9, 2017 Research Methodology Conceptual Modelling General Frameworks Applications User Studies
  • 10. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 10June 9, 2017 PARC User Interface (Mid 70s)  Desktop metaphor  WIMP interaction  "filing cabinets" with hierarchical folders  Paper paradigm  "What You See Is What You Get" (WYSIWYG)  what about richer document formats that existed at that time? Xerox Star 8010 [http://www.digibarn.com/collections/systems/xerox-8010/]
  • 11. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 11June 9, 2017 Digital Documents as a Paper Simulator? Vannevar Bush Ted Nelson Most people don't understand the logic of the concept: "What You See Is What You Get" is based on printing the document out ("get" means "get WHEN YOU PRINT IT OUT"). And that means a metaphysical shift: a document can only consist of what can be printed! [...] No overlays [...] – PAPER UNDER GLASS. When data of any sort are placed in storage, they are filed alphabetically or numerically, and information is found (when it is) by tracing it down from subclass to subclass. It can be in only one place, unless duplicates are used [...] The human mind does not work that way. It operates by association. As We May Think, Atlantic Monthly, July 1945 Geeks Bearing Gifts: How the Computer World Got This Way, Mindful Press 2009
  • 12. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 12June 9, 2017 Evolution of Interfaces date January 1984 September 2016 + 32 years price $2,500 $2,299 x 0.92 CPU 68000 Motorola 8 MHz 0.7 MIPS quad-core Intel core i7 4 GHz 238,310 MIPS x 500 x 340,442 memory 128 kB 16 GB x 125,000 storage 400 kB floppy drive 2 TB fusion drive x 5,000,000 monitor 9" black and white 512 x 342 68 dpi 27" colour 5120 x 2880 218 dpi x 3 x 84 x 3.2 devices mouse keyboard mouse keyboard same same GUI desktop WIMP desktop WIMP same [partly based on Beaudouin-Lafon 2004] original Macintosh 27-inch iMac comparison
  • 13. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 13June 9, 2017 RSL Metamodel and iServer Global Information Systems Group, ETH Zurich
  • 14. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 14June 9, 2017 Links Users Layers Plug-ins Structure Active Content Global Information Systems Group, ETH Zurich
  • 15. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 15June 9, 2017 Bridging the Paper-Digital Divide
  • 16. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 16June 9, 2017 Paper-Digital Integration Amazon Kindle Anoto Digital Pen and Paper Technology Augmented / Interactive PaperElectronic Paper
  • 17. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 17June 9, 2017 Cross-Media Link Server (iServer) Global Information Systems Group ETH Zurich
  • 18. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 18June 9, 2017 Interactive Paper Applications  Enhanced Reading  paper-digital maps  interactive festival brochures  educational materials  auction brochures  scientific publications  …  Enhanced Writing  photo album  researcher's notebook  mammography annotation  query by sketching  …  Paper-based Interfaces  PaperPoint  document proof-editing  digital libraries  video analysis tool  educational games  Art Installations  Lost Cosmonaut  Generosa Enterprise  Interactive Tabletops  photo browser  collaborative sketch tool
  • 19. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 19June 9, 2017 Interactive Festival Guide (EdFest) Global Information Systems Group ETH Zurich
  • 20. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 20June 9, 2017 Visual and Interaction Design  Visualisation of interactive areas and functionality  design patterns  Interaction design  online versus offline input processing  lack of modal dialogues  multimodal interaction  Findings might be appli- cable to traditional GUIs  e.g. cross-application widget interaction Global Information Systems Group, ETH Zurich
  • 21. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 21June 9, 2017 Paper-Digital Maps Szabolcs Becze
  • 22. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 22June 9, 2017 RSL-based Associative File System signer My Pictures Vancouver ER 2010 Granville Island Vancouver Harbour Public Market Keynote J. Thorp Olga and Lamia Dinner ER 2010 multiple classification structural link navigational link My Videos Vancouver Aquarium ER 2010 Keynote cross-media association My Presentations ER 2010 WISE 2009 cross-media transclusion single slide transclusion (via OOXML RSL selector) Gregory Cardone
  • 23. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 23June 9, 2017 RSL-based Associative File System ... Gregory Cardone
  • 24. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 24June 9, 2017 Personal Information Management (PIM)  Keeping, organising and re-finding information  digital and physical  Study of human- information interaction  files, piles, mixtures, …  OC2 PIM model  based on RSL hypermedia metamodel  Cross-Media PIM system  explicit as well as implicit associations between entities
  • 25. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 25June 9, 2017 Object-Concept-Context (OC2) Framework object layer structural link concept layer context layer navigational link associative link weighted link Object 1 Object n Object 2 Context 1 Context n Concept 1 Concept 2 Concept n extent link entity link Links selector resource (1,*)(1,*) (1,1) (0,*) (0,*) (0,*) RefersTo HasTargetHasSource partition Entities (0,*) link Navigational Links link Extent Links partition ResourcesSelectors context Contexts link Structural Links link Associative Links resource Physical Objects resource Digital Objects resource Concepts resource Objects HasAssoc Target (0,*) (1,*) HasExt Target HasExt Source HasAssoc Source (1,1) (1,*) (0,*) (0,*) (0,*) (1,1) disjoint partition translation OC2 conceptual model RSL-based metamodel
  • 26. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 26June 9, 2017 PimVis Tim ReynaertAudrey Sanctorum
  • 27. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 27June 9, 2017 PimVis Setup
  • 28. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 28June 9, 2017 PimVis Context View Audrey Sanctorum
  • 29. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 29June 9, 2017 PimVis Document View Audrey Sanctorum
  • 30. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 30June 9, 2017 PimVis Focus View Audrey Sanctorum
  • 31. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 31June 9, 2017 Document Tracking (DocTr) Framework  Unified document (object) tracking solution  plug-ins for QR codes, RFID tags, SIFT algorithm, …  tracking of documents across organisational structures  avoid fragmentation when using multiple tracking setups Document Management PendingPool DataModel IdentificationMechanism OC2 GUI REST Input Interface REST Output Interface Third-party Applications DocTr Delegator Tracking Setups
  • 32. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 32June 9, 2017 Cross-Device and Cross-Media Interfaces  Rapid prototyping platform for cross-device and cross-media integration  End-user authoring  customised distribution of user interface components  mashup tool for digital and physical (IoT) components
  • 33. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 33June 9, 2017 ArtVis: Information Exploration  Advanced visualisation techniques in combination with a TUI  explore Web Gallery of Art  faceted browsing  phidgets-based TUI  RFID-tagged physical objects  Three main components to explore, analyse and browse the information Bram Moerman
  • 34. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 34June 9, 2017 ArtVis ... Bram Moerman
  • 35. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 35June 9, 2017 ArtVis ... Bram Moerman
  • 36. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 36June 9, 2017 Video: ArtVis Bram Moerman
  • 37. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 37June 9, 2017 Mixed Reality "The perceived congruence concerns both visual as well as tactile perception. A perfect hologram will appear embodied but trying to touch it will destroy the illusion." Microsoft HoloLens Jansen, 2014
  • 38. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 38June 9, 2017 Tangible Holograms Timothy J. Curtin
  • 39. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 39June 9, 2017 Tangible Graph Hologram
  • 40. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 40June 9, 2017 Collaborative Hologram Interaction
  • 41. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 41June 9, 2017 Data Physicalisation  Tangible user interfaces (TUIs) for interacting with digital data and services  e.g. ArtVis  Physical objects used for output  often static encoding of digital information  How can we achieve dynamic data physicalisation with dynamic affordances  e.g. 3D visualisation with additional dimensions represented by physical variables such as temperature or texture  exploration of big data sets  tangible holograms as an experimental platform
  • 42. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 42June 9, 2017 Robotic Arm Prototype  Lego Mindstorms-based 6DOF arm prototype  motors can be locked or set to float mode - useful for bidirectional I/O  replaceable sphere for different types of non-visual feedback  Challenges  inverse kinematics (final system will be body mounted) - use robotics toolbox (MATLAB)  limited HoloLens hand tracking  3D printing of final arm Timothy J. Curtin
  • 43. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 43June 9, 2017 Data Physicalisation Framework  Inspired by research agenda of Jansen et al.  Understanding the design space  how do users map data to physicalisations  Understanding the perceptual effectiveness  e.g. what are the just-noticeable differences that a physical variable can convey?  Framework for dynamic data physicalisation  data processing with device-independent output - TextDIO: text for static or dynamic reports - PhysDIO: data points, data variables and physical variables  text and visualisation drivers for representation on specific devices - e.g. tangible hologram prototype
  • 44. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 44June 9, 2017 HCI and HCII in Smart Environments
  • 45. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 45June 9, 2017 Context Modelling Toolkit (CMT)  Multi-layered context modelling approach  programmer, expert user and end user  Beyond simple "if this then that" rules  Client-server architecture  server: context reasoning based on Drools  client: sensor input as well as applications End User Expert User Functions Actions Template Filled in template Situation Situations Facts Rule Programmer Rule (4) (5) (6) (7) (8)  Increase trust via intelligibility
  • 46. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 46June 9, 2017 Context Modelling Toolkit (CMT) …
  • 47. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 47June 9, 2017 Cross-Domain Internet of Things (IoT)  Middleware for interoper- ability of smart behaviour across IoT platforms  Cross-domain conflict resolution  End-user authoring and intelligibility
  • 48. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 48June 9, 2017 SpeeG2: Multimodal Text Input User Speech recognition (Microsoft SAPI 5.4) Skeletal tracking (Microsoft Kinect) 5 4 2 3 SpeeG2 GUI 6 1 Sven De Kock
  • 49. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 49June 9, 2017 Video: SpeeG2 Performance (21 WPM) Sven De Kock
  • 50. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 50June 9, 2017 INFEX: Cross-Device Information Exchange
  • 51. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 51June 9, 2017 INFEX Architecture
  • 52. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 52June 9, 2017 Open Cross-Media Linking  Integration of a new resource type includes  development of a data plug-in  implementation of a visual plug-in  registration with the resource plug-in repository  Link browser loads plug-ins on demand  internal or external (third party application) visualisation  Third-party applications use the link browser for visualisation  similar to the integration of third-party applications with web browsers  Non-monolithic annotation model and authoring tool  resource-specific plug-ins
  • 53. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 53June 9, 2017 Visual Plug-ins Annotation/Link Browser & Editor Client Application showEntity(), getSelectedEntity() showEntity()
  • 54. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 54June 9, 2017 Cross-Document Linking local visual plug-ins external visual plug-ins communicate Link Browser
  • 55. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 55June 9, 2017 Document Retrieval and Discovery  Use explicit as well as implicit document relationships
  • 56. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 56June 9, 2017 PaperPoint Presentation Tool  Mobile PowerPoint presentation Tool  interactive PowerPoint paper handouts  Support for non-linear presentations  refer back to previous slides  dynamically integrate slides  skip slides
  • 57. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 57June 9, 2017 PaperPoint Features  Enhanced mobility during presentations  Improved presentation overview for the presenter  see content of current and nearby slides (orientation)  Annotation of static slide content  highlighting of content  dynamic creation of new content at presentation time  Improved control of embedded media (e.g. video)  Digital whiteboard  Multi-pen support  collaborative presentations  brainstorming
  • 58. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 58June 9, 2017 What is Wrong with Slideware?  Simulation of physical slides  limited space due to the slide concept  linear navigation from slide to slide  difficult to reuse content and embed rich media types  MindXpres addresses these issues  unlimited canvas with zoomable user interface  non-linear navigation and associative linking  content-based approach with automatic visualisation
  • 59. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 59June 9, 2017 MindXpres Presentation Tool  Extensible prototyping platform for novel presen- tation concepts  content-based approach - separation of content and pre- sentation (automatic visualisation)  cross-media content reuse  non-linear navigation
  • vol0903a07.pdf

    Jul 23, 2017

    Metta Bhavana

    Jul 23, 2017
    Search
    Related Search
    We Need Your Support
    Thank you for visiting our website and your interest in our free products and services. We are nonprofit website to share and download documents. To the running of this website, we need your help to support us.

    Thanks to everyone for your continued support.

    No, Thanks