Graphics & Design

A Natural Language Like Description Language

Description
A Natural Language Like Description Language
Published
of 12
All materials on our website are shared by users. If you have any questions about copyright issues, please report us to resolve them. We are always happy to assist you.
Related Documents
Share
Transcript
  375Proc. 10 th  Australasian Conferenceon Information Systems, 1999 A Natural Language Like Description Language Eva Heinrich and Elizabeth KempInstitute of Information Sciences and TechnologyMassey UniversityPalmerston North, New Zealand{E.Heinrich, E.Kemp}@massey.ac.nzJon D. Patrick Basser Department of Computer ScienceUniversity of SydneySydney, Australia jonpat@cs.usyd.edu.au Abstract Computer programs for the analysis of human behaviour captured in multimedia data formatcommonly provide mechanisms to describe the behaviour recorded. Yet these programs do notsatisfactorily fulfil the need for a description mechanism which allows the production of richdescriptions of behaviour in a flexible way and which facilitates the correct and complete retrieval of descriptions according to their meaning. In this paper a new coding language for the natural languagelike description of human behaviour, the Flexible Structured Coding Language (FSCL) is introduced.This coding language supports the rich description of behaviour, it is flexible in its use, and it allowsthe correct and complete retrieval of behavioural descriptions according to their meaning. Keywords Coding Languages, Analysis Of Multimedia Data INTRODUCTION The analysis of human behaviour recorded on video, audio, and image or described in textdocuments is often assisted by the use of computer programs which can handle input data of multiple media types: OBSERVER (Noldus, 1993; Noldus Information Technology, 1998a and1998b), CABER (Patrick, 1985a and 1985b), NUDIST with CVideo (Richards et al., 1991),Atlas/ti ( Weitzman et al., 1995), Code-A-Text (Cartwright, 1998), Learning Constellations(Goldman-Segall, 1993), FERAL (Carter, 1997), MacShapa (Sanderson et al. 1994; Sanderson,1997), PAC (Heinrich et al. 1998).To assist in the analysis of the large amounts of information contained in the multimedia data thecomputer programs commonly provide a mechanism to produce descriptions of the recordedbehaviour. These descriptions, in text or other computer accessible data formats, are linked tospecific positions in the behavioural multimedia data and form the basis for further analysis steps. Thebehaviour descriptions, stored in a database, are queried to extract descriptions of selectedbehaviours. The query results form the basis for the quantitative, qualitative, and sequential analysisof the recorded behaviour (Sanderson et al., 1997).  376 There are two principle ways to formulate the descriptions of the behavioural data. One is the use of natural language in the form of textual input where the resulting descriptions are commonly calledannotations, or memos for longer textual descriptions (Weitzman et al., 1995). The other is to use aformalised coding scheme where the resulting descriptions are called code instances and the processof description is called coding (Weitzman et al., 1995).This paper highlights the strengths and weaknesses of both types of description languages, the naturallanguage and the formalised coding scheme, as they are used in current multimedia programs. Then anew flexible structured coding language, FSCL, is introduced which attempts to combine thestrengths of both existing approaches while at the same time avoiding their weaknesses. LANGUAGES FOR THE DESCRIPTION OF BEHAVIOUR Natural Language for the Description of Behaviour The strengths of the use of natural language as a description language are its expressiveness andflexibility, its immediate availability to every analyst with access to a word processing facility, and thefamiliarity with and understanding of natural language for every analyst. This is reinforced by theemphasis of textual description mechanisms to support the analysis process (Weitzman et al., 1995;Sanderson et al., 1997).The weakness of natural language as a description language becomes apparent only if one wants tomake use of a computer system to analyse the descriptions. Natural language with its lexicalambiguity, ambiguous sentence structure, and context dependency of meaning is very complex(Smeaton, 1997). Even if the meaning of a sentence is immediately apparent to a human reader it canbe extremely difficult for a computer system to detect this meaning. Good computer programs fornatural language processing are very complex, specialised programs which are still far away frombeing able to analyse all sentences correctly (Smeaton, 1997). The risk of retrieving descriptions of behaviour with a different meaning from that intended might be acceptable if these descriptions areclosely inspected by the analyst with incorrectly retrieved descriptions discarded. Where the retrievalof descriptions is used as the basis for the calculation of quantitative measurements and therefore thedetailed inspection of each description is not part of the analysis process the risk of retrievingdescriptions with the 'wrong' meaning cannot be tolerated.The current systems that support behaviour analysis rely on text search to assist in the analysis of natural language descriptions (NUDIST, FERAL, Code-A-Text). None of the systems uses naturallanguage processing techniques, presumably because of their complexity. The problem with the textsearch approach regarding the meaning of text can be illustrated with the following examplesentences.Bill hit Jim and ran.For the human reader it is obvious what this sentence describes: Bill is performing two activities, hehits and runs. Assuming one wishes to use a text search method to search a set of behaviouraldescriptions for the behaviour 'Bill ran' . Searching for the character sequence 'Bill ran' will not get amatch with the example sentence even though it describes the behaviour requested. With this textsearch strategy one can achieve only an incomplete match with behaviour descriptions. Changing thetext search strategy by including a wildcard operator into the text string and searching for 'Bill*ran'will retrieve the example sentence. Yet a new problem becomes obvious if one considers thefollowing example sentence:  377 Bill hit Jim and Jim ran.The application of the text search 'Bill*ran' to this sentence produces a match that would incorrectlyidentify the sentence with the behaviour 'Bill ran' when it is obvious to the human reader that this isinvalid.These examples show that applying a text search strategy cannot deal with the content of evensimple natural language sentences. Structured Languages for the Coding of Behaviour For formalised coding schemes the strengths and weaknesses depend very much on the definition of the coding scheme and the coding process. Coding schemes, in current implementations of systemswhich support the analysis of recorded behaviour, usually allow the definition of words or wordcombinations as codes and the combination of these codes into hierarchical structures(OBSERVER, NUDIST). In the coding process these codes are applied to the behavioural datasequences either as single codes (NUDIST, FERAL) or in predefined, very restricted combinationsof codes (OBSERVER).These coding systems express meaning mainly within single codes (NUDIST, FERAL) and only in alimited way in the combination of codes (OBSERVER). They have the strengths of simplicity of use,and relative flexibility for changing the coding scheme.The restrictions of these coding schemes can be highlighted considering the previous examplesentences. To express meaning within one single code the whole sentence 'Bill hit Jim and Jim ran'could be introduced as one single code. To analyse the relationships within the code, which is buildby a natural language sentence, the same reference problems are experienced as with naturallanguage descriptions previously mentioned. If instead one defines codes for each individual element,like 'Bill, 'Jim', 'hit', 'ran' and 'and', more flexibility would be gained by combining these codes intosentences. At the same time one could not determine the relationships between these single codesand retrieval would suffer, again, from the same problems, as in analysing natural language sentencesusing text search techniques.A very different approach for the definition of a coding scheme was taken in CABER. Here adomain specific formal language is defined which attempts to address all observable behaviour withinthis domain. The resulting coding sentences produced with CABER contain rich meaning andbecause the definition of the formal language considers the relationship between single codes it canbe readily accessed by a computer system. The disadvantages of this approach lie in the need toincorporate every observable behaviour in the specification of the formal language and the inflexibilityof this formal language concerning extensions and modifications. A further disadvantage is the needfor the coder or observer to learn the domain specific formal language. Comparison of Natural Language Description and Formalised Coding The consideration of natural language as a description language has shown that the weakness of thisapproach lies in the complexity of accessing the rich meaning in the description sentences in anautomated way with the assistance of a computer system. The potential advantage of using aformalised description language lies in the possibility of extracting meaning in an automated way.From the observations on existing coding approaches a table is presented here indicating the featuresthat are present or absent in the various approaches (Figure 1). In the design of a coding approach,expressiveness and flexibility are related to ease of computerised extraction.  378 THE FLEXIBLE STRUCTURED CODING LANGUAGE (FSCL) Goal of FSCL The motivation behind the definition of FSCL was to combine the advantages of all three descriptionapproaches outlined in the previous sections while avoiding their disadvantages. This was to beachieved by designing a natural language like description language which is •   easy to understand by the human analyst; •   not limited to a specific domain but generic and therefore applicable to many studies; •   highly expressive yet simple enough to be ‘understood’ by a computer program; •   highly flexible to do justice to changing requirements during the analysis process; •   capable of expressing both full natural language like sentences and abbreviated sentences in thestyle of single codes or simple combinations of codes. Concept of FSCL Any language consists of two main components: the vocabulary and the grammatical rules tocombine words of the vocabulary into meaningful sentences. The approach taken with the design of FSCL is to allow a flexible definition of the vocabulary and to provide a fixed, generic grammar  forexpression and extraction of meaning. The flexible definition of the vocabulary, that is the freedom tointroduce any word at any time into the coding language, aims at providing a natural language likeenvironment in which the user has the words available to describe any occurring behaviour. With theNaturalIndependent StrictLanguage (1)Codes (2)Grammar (3)Expressiveness+  − +Flexibility++  − Computerised − − +extraction of meaning(1)a description in natural language, either as annotation or asa single code containing a natural language sentence(2)a system of codes that can be combined in anentirely unrestricted manner(3)a combination of codes to a coding sentence formalised via agrammar with a fixed vocabulary    379 explicit definition of a generic grammar, which is much simpler than the implicit grammar of naturallanguage, the coding language is provided with a way to combine words into meaningful sentences.At the same time, due to the relative simplicity of the grammar, it is able to automatically extract fromthe database meaningful sentences by computational techniques.The element of FSCL which allows the combination of a flexible vocabulary with a fixed genericgrammar structure is the introduction of categories . All words in the vocabulary belong to one of sixcategories and the grammar is defined on these categories. The grammar defines how the differentelements of a sentence, like subject, verb, object, can be combined whereas the categories definewhich words of the vocabulary can be used as subject, verb or object. By defining the grammar oncategories and not on individual words it is possible to specify the grammar in a generic way. Thismeans that the same grammar can be used across studies of different behavioural domains with theonly modification needed to use the coding language to describe new behaviours is in the definition of suitable vocabulary items. Categories of FSCL The categories of FSCL serve the purpose of grouping the words of the vocabulary. The grammarrules of FSCL that define how to combine words into sentences are defined on these categories. Byknowing to which category a word belongs, a computer program can determine in which way thisparticular word can be used within a sentence. New words can be introduced into the vocabularywithout having to modify the grammar rules as these only apply to the categories which remainunchanged.The categories of FSCL have been defined in accordance with ‘word classes’ as defined in theEnglish grammar. Word classes have generally replaced the traditional categorisation of words into‘parts of speech’ (Wardhaugh, 1995; Jackson, 1980). While there is some variation in the definitionof the word classes the main classes are defined consistently by most authors:Nouns, pronouns, verbs, adjectives, adverbs, conjunctions, and prepositions (Wardhaugh, 1995);and as well, (full-)verb, determiner, operator-verb, interjection, and enumerator (Leech et al., 1982);numerals (Jackson, 1980); and, auxiliary verbs (Freeborn, 1995).Based on these word classes the following categories have been defined for FSCL: P Person/ThingA   ActivityC ConceptK ConjunctionD DescriptorR Preposition The word class ‘nouns’ has been divided into the two categories of Person/Thing and Concept. Thisdistinction was made to support the generation of the two sentence types. While words of bothcategories could function as objects in a sentence only words of the category Person/Thing could beactors and only words of the category Concept could be used in the place of concepts in concept object   clauses, which are allowed as abbreviated sentences. The word classes ‘adjectives’ and‘adverbs’ have been combined into the category Descriptors, which also contains enumerators.
Search
Related Search
We Need Your Support
Thank you for visiting our website and your interest in our free products and services. We are nonprofit website to share and download documents. To the running of this website, we need your help to support us.

Thanks to everyone for your continued support.

No, Thanks