Interactive Ir User Study Design Evaluation And Reporting
DOWNLOAD
Download Interactive Ir User Study Design Evaluation And Reporting PDF/ePub or read online books in Mobi eBooks. Click Download or Read Online button to get Interactive Ir User Study Design Evaluation And Reporting book now. This website allows unlimited access to, at the time of writing, more than 1.5 million titles, including hundreds of thousands of titles in various foreign languages. If the content not found or just blank you must refresh this page
Interactive Ir User Study Design Evaluation And Reporting
DOWNLOAD
Author : Jiqun Liu
language : en
Publisher: Morgan & Claypool Publishers
Release Date : 2019-06-04
Interactive Ir User Study Design Evaluation And Reporting written by Jiqun Liu and has been published by Morgan & Claypool Publishers this book supported file pdf, txt, epub, kindle and other format this book has been release on 2019-06-04 with Computers categories.
Since user study design has been widely applied in search interactions and information retrieval (IR) systems evaluation studies, a deep reflection and meta-evaluation of interactive IR (IIR) user studies is critical for sharpening the instruments of IIR research and improving the reliability and validity of the conclusions drawn from IIR user studies. To this end, we developed a faceted framework for supporting user study design, reporting, and evaluation based on a systematic review of the state-of-the-art IIR research papers recently published in several top IR venues (n=462). Within the framework, we identify three major types of research focuses, extract and summarize facet values from specific cases, and highlight the under-reported user study components which may significantly affect the results of research. Then, we employ the faceted framework in evaluating a series of IIR user studies against their respective research questions and explain the roles and impacts of the underlying connections and "collaborations" among different facet values. Through bridging diverse combinations of facet values with the study design decisions made for addressing research problems, the faceted framework can shed light on IIR user study design, reporting, and evaluation practices and help students and young researchers design and assess their own studies.
Interactive Ir User Study Design Evaluation And Reporting
DOWNLOAD
Author : Jiqun Liu
language : en
Publisher: Springer Nature
Release Date : 2022-05-31
Interactive Ir User Study Design Evaluation And Reporting written by Jiqun Liu and has been published by Springer Nature this book supported file pdf, txt, epub, kindle and other format this book has been release on 2022-05-31 with Computers categories.
Since user study design has been widely applied in search interactions and information retrieval (IR) systems evaluation studies, a deep reflection and meta-evaluation of interactive IR (IIR) user studies is critical for sharpening the instruments of IIR research and improving the reliability and validity of the conclusions drawn from IIR user studies. To this end, we developed a faceted framework for supporting user study design, reporting, and evaluation based on a systematic review of the state-of-the-art IIR research papers recently published in several top IR venues (n=462). Within the framework, we identify three major types of research focuses, extract and summarize facet values from specific cases, and highlight the under-reported user study components which may significantly affect the results of research. Then, we employ the faceted framework in evaluating a series of IIR user studies against their respective research questions and explain the roles and impacts of the underlying connections and "collaborations" among different facet values. Through bridging diverse combinations of facet values with the study design decisions made for addressing research problems, the faceted framework can shed light on IIR user study design, reporting, and evaluation practices and help students and young researchers design and assess their own studies.
A Behavioral Economics Approach To Interactive Information Retrieval
DOWNLOAD
Author : Jiqun Liu
language : en
Publisher: Springer Nature
Release Date : 2023-02-17
A Behavioral Economics Approach To Interactive Information Retrieval written by Jiqun Liu and has been published by Springer Nature this book supported file pdf, txt, epub, kindle and other format this book has been release on 2023-02-17 with Computers categories.
This book brings together the insights from three different areas, Information Seeking and Retrieval, Cognitive Psychology, and Behavioral Economics, and shows how this new interdisciplinary approach can advance our knowledge about users interacting with diverse search systems, especially their seemingly irrational decisions and anomalies that could not be predicted by most normative models. The first part “Foundation” of this book introduces the general notions and fundamentals of this new approach, as well as the main concepts, terminology and theories. The second part “Beyond Rational Agents” describes the systematic biases and cognitive limits confirmed by behavioral experiments of varying types and explains in detail how they contradict the assumptions and predictions of formal models in information retrieval (IR). The third part “Toward A Behavioral Economics Approach” first synthesizes the findings from existing preliminary research on bounded rationality and behavioral economics modeling in information seeking, retrieval, and recommender system communities. Then, it discusses the implications, open questions and methodological challenges of applying the behavioral economics framework to different sub-areas of IR research and practices, such as modeling users and search sessions, developing unbiased learning to rank and adaptive recommendations algorithms, implementing bias-aware intelligent task support, as well as extending the conceptualization and evaluation on IR fairness, accountability, transparency and ethics (FATE) with the knowledge regarding both human biases and algorithmic biases. This book introduces a behavioral economics framework to IR scientists seeking a new perspective on both fundamental and new emerging problems of IR as well as the development and evaluation of bias-aware intelligent information systems. It is especially intended for researchers working on IR and human-information interaction who want to learn about the potential offered by behavioral economics in their own research areas.
Methods For Evaluating Interactive Information Retrieval Systems With Users
DOWNLOAD
Author : Diane Kelly
language : en
Publisher: Now Publishers Inc
Release Date : 2009
Methods For Evaluating Interactive Information Retrieval Systems With Users written by Diane Kelly and has been published by Now Publishers Inc this book supported file pdf, txt, epub, kindle and other format this book has been release on 2009 with Computers categories.
Provides an overview and instruction on the evaluation of interactive information retrieval systems with users.
Resources In Education
DOWNLOAD
Author :
language : en
Publisher:
Release Date : 2001-04
Resources In Education written by and has been published by this book supported file pdf, txt, epub, kindle and other format this book has been release on 2001-04 with Education categories.
Simulating Information Retrieval Test Collections
DOWNLOAD
Author : David Hawking
language : en
Publisher: Springer Nature
Release Date : 2022-06-01
Simulating Information Retrieval Test Collections written by David Hawking and has been published by Springer Nature this book supported file pdf, txt, epub, kindle and other format this book has been release on 2022-06-01 with Computers categories.
Simulated test collections may find application in situations where real datasets cannot easily be accessed due to confidentiality concerns or practical inconvenience. They can potentially support Information Retrieval (IR) experimentation, tuning, validation, performance prediction, and hardware sizing. Naturally, the accuracy and usefulness of results obtained from a simulation depend upon the fidelity and generality of the models which underpin it. The fidelity of emulation of a real corpus is likely to be limited by the requirement that confidential information in the real corpus should not be able to be extracted from the emulated version. We present a range of methods exploring trade-offs between emulation fidelity and degree of preservation of privacy. We present three different simple types of text generator which work at a micro level: Markov models, neural net models, and substitution ciphers. We also describe macro level methods where we can engineer macro properties of a corpus, giving a range of models for each of the salient properties: document length distribution, word frequency distribution (for independent and non-independent cases), word length and textual representation, and corpus growth. We present results of emulating existing corpora and for scaling up corpora by two orders of magnitude. We show that simulated collections generated with relatively simple methods are suitable for some purposes and can be generated very quickly. Indeed it may sometimes be feasible to embed a simple lightweight corpus generator into an indexer for the purpose of efficiency studies. Naturally, a corpus of artificial text cannot support IR experimentation in the absence of a set of compatible queries. We discuss and experiment with published methods for query generation and query log emulation. We present a proof-of-the-pudding study in which we observe the predictive accuracy of efficiency and effectiveness results obtained on emulated versions of TREC corpora. The study includes three open-source retrieval systems and several TREC datasets. There is a trade-off between confidentiality and prediction accuracy and there are interesting interactions between retrieval systems and datasets. Our tentative conclusion is that there are emulation methods which achieve useful prediction accuracy while providing a level of confidentiality adequate for many applications. Many of the methods described here have been implemented in the open source project SynthaCorpus, accessible at: https://bitbucket.org/davidhawking/synthacorpus/
Understanding And Evaluating Search Experience
DOWNLOAD
Author : Maria Stone
language : en
Publisher: Springer Nature
Release Date : 2022-05-31
Understanding And Evaluating Search Experience written by Maria Stone and has been published by Springer Nature this book supported file pdf, txt, epub, kindle and other format this book has been release on 2022-05-31 with Computers categories.
This book is intended for anyone interested in learning more about how search works and how it is evaluated. We all use search—it's a familiar utility. Yet, few of us stop and think about how search works, what makes search results good, and who, if anyone, decides what good looks like. Search has a long and glorious history, yet it continues to evolve, and with it, the measurement and our understanding of the kinds of experiences search can deliver continues to evolve, as well. We will discuss the basics of how search engines work, how humans use search engines, and how measurement works. Equipped with these general topics, we will then dive into the established ways of measuring search user experience, and their pros and cons. We will talk about collecting labels from human judges, analyzing usage logs, surveying end users, and even touch upon automated evaluation methods. After introducing different ways of collecting metrics, we will cover experimentation as it applies to search evaluation. The book will cover evaluating different aspects of search—from search user interface (UI), to results presentation, to the quality of search algorithms. In covering these topics, we will touch upon many issues in evaluation that became sources of controversy—from user privacy, to ethical considerations, to transparency, to potential for bias. We will conclude by contrasting measuring with understanding, and pondering the future of search evaluation.
Advances In Information Retrieval
DOWNLOAD
Author : Claudia Hauff
language : en
Publisher: Springer Nature
Release Date : 2025-04-02
Advances In Information Retrieval written by Claudia Hauff and has been published by Springer Nature this book supported file pdf, txt, epub, kindle and other format this book has been release on 2025-04-02 with Computers categories.
The five-volume set LNCS 15572, 15573, 15574, 15575 and 15576 constitutes the refereed conference proceedings of the 47th European Conference on Information Retrieval, ECIR 2025, held in Lucca, Italy, during April 6–10, 2025. The 52 full papers, 11 findings, 42 short papers and 76 papers of other types presented in these proceedings were carefully reviewed and selected from 530 submissions. The accepted papers cover the state-of-the-art in information retrieval and recommender systems: user aspects, system and foundational aspects, artificial intelligence and machine learning, applications, evaluation, new social and technical challenges, and other topics of direct or indirect relevance to search and recommendation.
Geographies Of The Book
DOWNLOAD
Author : Professor Charles W J Withers
language : en
Publisher: Ashgate Publishing, Ltd.
Release Date : 2012-11-28
Geographies Of The Book written by Professor Charles W J Withers and has been published by Ashgate Publishing, Ltd. this book supported file pdf, txt, epub, kindle and other format this book has been release on 2012-11-28 with Science categories.
The geography of the book is as old as the history of the book, though far less thoroughly explored. Yet research has increasingly pointed to the spatial dimensions of book history, to the transformation of texts as they are made and moved from place to place, from authors to readers and within different communities and cultures of reception. Widespread recognition of the significance of place, of the effects of movement over space and of the importance of location to the making and reception of print culture has been a feature of recent book history work, and draws in many instances upon studies within the history of science as well as geography. 'Geographies of the Book' explores the complex relationships between the making of books in certain geographical contexts, the movement of books (epistemologically as well as geographically) and the ways in which they are received.
Advances In Information Retrieval
DOWNLOAD
Author : Nazli Goharian
language : en
Publisher: Springer Nature
Release Date : 2024-03-22
Advances In Information Retrieval written by Nazli Goharian and has been published by Springer Nature this book supported file pdf, txt, epub, kindle and other format this book has been release on 2024-03-22 with Computers categories.
The six-volume set LNCS 14608, 14609, 14609, 14610, 14611, 14612 and 14613 constitutes the refereed proceedings of the 46th European Conference on IR Research, ECIR 2024, held in Glasgow, UK, during March 24–28, 2024. The 57 full papers, 18 finding papers, 36 short papers, 26 IR4Good papers, 18 demonstration papers, 9 reproducibility papers, 8 doctoral consortium papers, and 15 invited CLEF papers were carefully reviewed and selected from 578 submissions. The accepted papers cover the state of the art in information retrieval focusing on user aspects, system and foundational aspects, machine learning, applications, evaluation, new social and technical challenges, and other topics of direct or indirect relevance to search.