{"title":"Finding information about mental health in microblogging platforms: a case study of depression","authors":"Max L. Wilson, Susan Ali, M. Valstar","doi":"10.1145/2637002.2637006","DOIUrl":"https://doi.org/10.1145/2637002.2637006","url":null,"abstract":"Searching for online health information has been well studied in web search, but social media, such as public microblogging services, are well known for different types of tacit information: personal experience and shared information. Finding useful information in public microblogging platforms is an on-going hard problem and so to begin to develop a better model of what health information can be found, Twitter posts using the word \"depression\" were examined as a case study of a search for a prevalent mental health issue. 13,279 public tweets were analysed using a mixed methods approach and compared to a general sample of tweets. First, a linguistic analysis suggested that tweets mentioning depression were typically anxious but not angry, and were less likely to be in the first person, indicating that most were not from individuals discussing their own depression. Second, to understand what types of tweets can be found, an inductive thematic analysis revealed three major themes: 1) disseminating information or link of information, 2) self-disclosing, and 3) the sharing of overall opinion; each had significantly different linguistic patterns. We conclude with a discussion of how different types of posts about mental health may be retrieved from public social media like Twitter.","PeriodicalId":447867,"journal":{"name":"Proceedings of the 5th Information Interaction in Context Symposium","volume":"11 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-08-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122265617","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Characterizing relevance with eye-tracking measures","authors":"J. Gwizdka","doi":"10.1145/2637002.2637011","DOIUrl":"https://doi.org/10.1145/2637002.2637011","url":null,"abstract":"Relevance, a fundamental concept in information search and retrieval, is 80-years old [4]. The recent decades have been ripe with work that brought a much better understanding of this rich concept. Yet, we still don't know which cognitive and affective processes are involved in relevance judgments. Empirical work that tackles these questions is scarce. This paper aims to contribute toward better understanding of cognitive processing of text documents at different degrees of relevance. Our approach takes advantage of a direct relationship between eye movement patterns, pupil size and cognitive processes, such as mental effort and attention. We examine gaze-based metrics in relation to individual word processing and reading text documents in the context of a constricted information search tasks. The findings indicate that text document processing depends on document relevance and on the user-perceived relevance. Statistical analyses show that relevant documents tended to be continuously read, while irrelevant documents tended to be scanned. Most eye-tracking-based measures indicate cognitive effort to be highest for partially relevant documents and lowest for irrelevant documents. However, pupil dilation indicates cognitive effort to be higher for relevant than partially relevant documents. Classification of selected eye-tracking measures show that an accuracy of 70-75% can be achieved for predicting binary relevance. These results show a promise for implicit relevance feedback.","PeriodicalId":447867,"journal":{"name":"Proceedings of the 5th Information Interaction in Context Symposium","volume":"212 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-08-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131478423","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Measuring and improving data quality of media collections for professional tasks","authors":"Myriam C. Traub","doi":"10.1145/2637002.2637056","DOIUrl":"https://doi.org/10.1145/2637002.2637056","url":null,"abstract":"Carrying out research tasks on data collections is hampered, or even made impossible, by data quality issues of different types, such as incompleteness or inconsistency, and severity. We identify research tasks carried out by professional users of data collections that are hampered by inherent quality issues. We investigate what types of issues exist and how they influence these research tasks. To measure the quality perceived by professional users, we develop a quality metric. This allows us to measure the suitability of the data quality for a chosen user task. For a chosen task, we study how the data quality can be improved using crowdsourcing. We validate our quality metric by investigating whether professionals perform better on the chosen research task.","PeriodicalId":447867,"journal":{"name":"Proceedings of the 5th Information Interaction in Context Symposium","volume":"28 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-08-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132193966","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"On choosing an effective automatic evaluation metric for microblog summarisation","authors":"S. Mackie, R. McCreadie, C. Macdonald, I. Ounis","doi":"10.1145/2637002.2637017","DOIUrl":"https://doi.org/10.1145/2637002.2637017","url":null,"abstract":"Popular microblogging services, such as Twitter, are engaging millions of users who constantly post and share information about news and current events each day, resulting in millions of messages discussing what is happening in the world. To help users obtain an overview of microblog content relating to topics and events that they are interested in, classical summarisation techniques from the newswire domain have been successfully applied and extended for use on microblogs. However, much of the current literature on microblog summarisation assumes that the summarisation evaluation measures that have been shown to be effective on newswire, are still appropriate for evaluating microblog summarisation. Hence, in this paper, we aim to determine whether the traditional automatic newswire summarisation evaluation metrics generalise to the task of microblog summarisation. In particular, using three microblog summarisation datasets, we determine a ranking of summarisation systems under three automatic summarisation evaluation metrics from the literature. We then compare and contrast this ranking of systems produced under each metric to system rankings produced through a qualitative user evaluation, with the aim of determining which metric best simulates human summarisation preferences. Our results indicate that, for the automatic evaluation metrics we investigate, they do not always concur with each other. Further, we find that Fraction of Topic Words better agrees with what users tell us about the quality and effectiveness of microblog summaries than the ROUGE-1 measure that is most commonly reported in the literature.","PeriodicalId":447867,"journal":{"name":"Proceedings of the 5th Information Interaction in Context Symposium","volume":"9 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-08-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116746167","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
M. Bron, Titia van der Werf, Shenghui Wang, M. de Rijke
{"title":"A social bookmarking system to support cluster driven archival arrangement","authors":"M. Bron, Titia van der Werf, Shenghui Wang, M. de Rijke","doi":"10.1145/2637002.2637046","DOIUrl":"https://doi.org/10.1145/2637002.2637046","url":null,"abstract":"Cultural heritage materials are increasingly being made available through standard search facilities. However, it is challenging to automatically organize these materials in a way that is well aligned with users' specific interests. We report on the development of a social bookmaking system to collect human annotations that are used to measure the performance of three different clustering algorithms. We find that there is a discrepancy between the latent structure present in the data and the clusters annotated by humans. However, it is difficult to detect such discrepancies explicitly.","PeriodicalId":447867,"journal":{"name":"Proceedings of the 5th Information Interaction in Context Symposium","volume":"61 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-08-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130044618","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Categorising search sessions: some insights from human judgments","authors":"Tony Russell-Rose, Paul D. Clough, Elaine Toms","doi":"10.1145/2637002.2637035","DOIUrl":"https://doi.org/10.1145/2637002.2637035","url":null,"abstract":"The session is a common unit of interaction that is used in search log analysis. By analysing sessions, it is possible to identify distinct classes of searcher behaviour that can be used to design search applications that better support groups of users based on their expected behaviours. This paper describes an online card sort experiment to investigate how people distinguish between search sessions (i.e., how they group them) to gain insights into their organising principles and to inform the future use of automated approaches, such as clustering. Results show patterns of user behaviour to be the most common way of grouping sessions.","PeriodicalId":447867,"journal":{"name":"Proceedings of the 5th Information Interaction in Context Symposium","volume":"44 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-08-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121867650","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"YASFIIRE: yet another system for IIR evaluation","authors":"Xing Wei, Yinglong Zhang, J. Gwizdka","doi":"10.1145/2637002.2637051","DOIUrl":"https://doi.org/10.1145/2637002.2637051","url":null,"abstract":"We present a system that supports Interactive Information Retrieval user studies on the Web. Our system provides support for user and task management, for processing web-based task specific interfaces and for Web-event logging. It also offers functionality useful to IIR studies that capture eye-movement on Web page elements. The system complements logging functionality offered by a typical usability/eye-tracking software packages and is designed to act in concert with such software.","PeriodicalId":447867,"journal":{"name":"Proceedings of the 5th Information Interaction in Context Symposium","volume":"8 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-08-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115041719","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Users' criteria of video digital libraries from a public affairs context","authors":"Boryung Ju, Dan E. Albertson","doi":"10.1145/2637002.2637031","DOIUrl":"https://doi.org/10.1145/2637002.2637031","url":null,"abstract":"This paper reports on a user-centered analysis of video digital libraries. Video digital libraries enable \"in the loop\" retrieval and playback from centralized and organized collections. As a time-based and multi-channeled format, video digital library systems warrant different considerations for design and information delivery. The purpose of the present study is to collect and initially analyze users' criteria of video digital libraries as part of their interactive experiences. Fifty-two journalism and political science college majors were surveyed, resulting in a total of 242 individual collected responses. Content analysis was performed on the survey responses, and the emergent coding method produced 28 criteria (subcategories) under 5 major categories. Criteria corresponding to Retrieval functions of video digital libraries emerged as the highest priority of the participants, based on its frequency across the responses for the major categories. Criteria corresponding to the User Interface, Collection Quality, User Support, and Organization of Collection followed respectively, in terms of frequency. Cohen's Kappa was .87, indicative of high-level of inter-coder reliability. Findings of the present study provide an initial baseline for design and evaluation of video digital libraries and motivate further research.","PeriodicalId":447867,"journal":{"name":"Proceedings of the 5th Information Interaction in Context Symposium","volume":"67 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-08-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129128888","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"User intent behind medical queries: an evaluation of entity mapping approaches with metamap and freebase","authors":"João Palotti, V. Stefanov, A. Hanbury","doi":"10.1145/2637002.2637043","DOIUrl":"https://doi.org/10.1145/2637002.2637043","url":null,"abstract":"This work focuses on understanding the user intent in the medical domain. The combination of Semantic Web and information retrieval technologies promises a better comprehension of user intents. Mapping queries to entities using Freebase is not novel, but so far only one entity per query could be identified. We overcome this limitation using annotations provided by Metamap. Also, different approaches to map queries to Freebase are explored and evaluated. We propose an indirect evaluation of the mappings, through user intent defined by classes such as Symptoms, Diseases or Treatments. Our experiments show that by using the concepts annotated by Metamap it is possible to improve the accuracy and F1 performances of mappings from queries to Freebase entities.","PeriodicalId":447867,"journal":{"name":"Proceedings of the 5th Information Interaction in Context Symposium","volume":"5 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-08-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116532022","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"E-mail categorization using partially related training examples","authors":"Maya Sappelli, S. Verberne, Wessel Kraaij","doi":"10.1145/2637002.2637014","DOIUrl":"https://doi.org/10.1145/2637002.2637014","url":null,"abstract":"Automatic e-mail categorization with traditional classification methods requires labelling of training data. In a real-life setting, this labelling disturbs the working flow of the user. We argue that it might be helpful to use documents, which are generally well-structured in directories on the file system, as training data for supervised e-mail categorization and thereby reducing the labelling effort required from users. Previous work demonstrated that the characteristics of documents and e-mail messages are too different to use organized documents as training examples for e-mail categorization using traditional supervised classification methods. In this paper we present a novel network-based algorithm that is capable of taking into account these differences between documents and e-mails. With the network algorithm, it is possible to use documents as training material for e-mail categorization without user intervention. This way, the effort for the users for labeling training examples is reduced, while the organization of their information flow is still improved. The accuracy of the algorithm on categorizing e-mail messages was evaluated using a set of e-mail correspondence related to the documents. The proposed network method was significantly better than traditional text classification algorithm in this setting.","PeriodicalId":447867,"journal":{"name":"Proceedings of the 5th Information Interaction in Context Symposium","volume":"70 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-08-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124466441","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}