Laboratory Experiments in Information Retrieval

Sample Sizes, Effect Sizes, and Statistical Power

Nonfiction, Computers, Database Management, Information Storage & Retrievel, Science & Nature, Mathematics, Statistics, General Computing
Cover of the book Laboratory Experiments in Information Retrieval by Tetsuya Sakai, Springer Singapore
View on Amazon View on AbeBooks View on Kobo View on B.Depository View on eBay View on Walmart
Author: Tetsuya Sakai ISBN: 9789811311994
Publisher: Springer Singapore Publication: September 22, 2018
Imprint: Springer Language: English
Author: Tetsuya Sakai
ISBN: 9789811311994
Publisher: Springer Singapore
Publication: September 22, 2018
Imprint: Springer
Language: English

Covering aspects from principles and limitations of statistical significance tests to topic set size design and power analysis, this book guides readers to statistically well-designed experiments. Although classical statistical significance tests are to some extent useful in information retrieval (IR) evaluation, they can harm research unless they are used appropriately with the right sample sizes and statistical power and unless the test results are reported properly. The first half of the book is mainly targeted at undergraduate students, and the second half is suitable for graduate students and researchers who regularly conduct laboratory experiments in IR, natural language processing, recommendations, and related fields.

Chapters 1–5 review parametric significance tests for comparing system means, namely, t-tests and ANOVAs, and show how easily they can be conducted using Microsoft Excel or R. These chapters also discuss a few multiple comparison procedures for researchers who are interested in comparing every system pair, including a randomised version of Tukey's Honestly Significant Difference test. The chapters then deal with known limitations of classical significance testing and provide practical guidelines for reporting research results regarding comparison of means.

Chapters 6 and 7 discuss statistical power. Chapter 6 introduces topic set size design to enable test collection builders to determine an appropriate number of topics to create. Readers can easily use the author’s Excel tools for topic set size design based on the paired and two-sample t-tests, one-way ANOVA, and confidence intervals. Chapter 7 describes power-analysis-based methods for determining an appropriate sample size for a new experiment based on a similar experiment done in the past, detailing how to utilize the author’s R tools for power analysis and how to interpret the results. Case studies from IR for both Excel-based topic set size design and R-based power analysis are also provided.

View on Amazon View on AbeBooks View on Kobo View on B.Depository View on eBay View on Walmart

Covering aspects from principles and limitations of statistical significance tests to topic set size design and power analysis, this book guides readers to statistically well-designed experiments. Although classical statistical significance tests are to some extent useful in information retrieval (IR) evaluation, they can harm research unless they are used appropriately with the right sample sizes and statistical power and unless the test results are reported properly. The first half of the book is mainly targeted at undergraduate students, and the second half is suitable for graduate students and researchers who regularly conduct laboratory experiments in IR, natural language processing, recommendations, and related fields.

Chapters 1–5 review parametric significance tests for comparing system means, namely, t-tests and ANOVAs, and show how easily they can be conducted using Microsoft Excel or R. These chapters also discuss a few multiple comparison procedures for researchers who are interested in comparing every system pair, including a randomised version of Tukey's Honestly Significant Difference test. The chapters then deal with known limitations of classical significance testing and provide practical guidelines for reporting research results regarding comparison of means.

Chapters 6 and 7 discuss statistical power. Chapter 6 introduces topic set size design to enable test collection builders to determine an appropriate number of topics to create. Readers can easily use the author’s Excel tools for topic set size design based on the paired and two-sample t-tests, one-way ANOVA, and confidence intervals. Chapter 7 describes power-analysis-based methods for determining an appropriate sample size for a new experiment based on a similar experiment done in the past, detailing how to utilize the author’s R tools for power analysis and how to interpret the results. Case studies from IR for both Excel-based topic set size design and R-based power analysis are also provided.

More books from Springer Singapore

Cover of the book China and the West at the Crossroads by Tetsuya Sakai
Cover of the book DNA Replication by Tetsuya Sakai
Cover of the book Brassinosteroids: Plant Growth and Development by Tetsuya Sakai
Cover of the book Jatropha, Challenges for a New Energy Crop by Tetsuya Sakai
Cover of the book Earthquake Engineering for Nuclear Facilities by Tetsuya Sakai
Cover of the book The Use of First and Second Language in Chinese University EFL Classrooms by Tetsuya Sakai
Cover of the book Mobile Learning in Higher Education in the Asia-Pacific Region by Tetsuya Sakai
Cover of the book Machine Translation by Tetsuya Sakai
Cover of the book Shadow Banking and the Rise of Capitalism in China by Tetsuya Sakai
Cover of the book Student Equity in Australian Higher Education by Tetsuya Sakai
Cover of the book Application of Geographical Information Systems and Soft Computation Techniques in Water and Water Based Renewable Energy Problems by Tetsuya Sakai
Cover of the book Principles of Regional Science by Tetsuya Sakai
Cover of the book Optical and Wireless Technologies by Tetsuya Sakai
Cover of the book China’s Conquest of Taiwan in the Seventeenth Century by Tetsuya Sakai
Cover of the book Electro-Osmosis of Polymer Solutions by Tetsuya Sakai
We use our own "cookies" and third party cookies to improve services and to see statistical information. By using this website, you agree to our Privacy Policy