Lecture

Designing Experiments and Measuring Learning

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

In course

CS-411: Digital education

This course addresses the relationship between specific technological features and the learners' cognitive processes. It also covers the methods and results of empirical studies: do student actually learn due to technologies? In fall 2025, P. Dillenbourg w ...

Description

This lecture covers the design of experiments in educational research, focusing on the challenges unique to the social sciences. Topics include hypothesis formulation, variables in experiments, controlling for individual differences, and statistical tests for analyzing learning outcomes. The instructor discusses the Ballmer peak hypothesis, experimental designs, and strategies to mitigate biases in research. The lecture emphasizes the importance of construct validity and the impact of selection bias on research outcomes.

Instructors (2)

Pierre Dillenbourg

A former teacher in elementary school, Pierre Dillenbourg graduated in educational science (University of Mons, Belgium). He started his research on learning technologies in 1984. In 1986, he has been on of the first in the world to apply machine learning to develop a self-improving teaching system. He obtained a PhD in computer science from the University of Lancaster (UK), in the domain of artificial intelligence applications for education. He has been assistant professor at the University of Geneva. He joined EPFL in 2002. He has been the director of Center for Research and Support on Learning and its Technologies, then academic director of Center for Digital Education, which implements the MOOC strategy of EPFL (over 2 million registrations). He is full professor in learning technologies in the School of Computer & Communication Sciences, where he is the head of the CHILI Lab: "Computer-Human Interaction for Learning & Instruction ». He is the director of the leading house DUAL-T, which develops technologies for dual vocational education systems (carpenters, florists,...). With EPFL colleagues, he launched in 2017 the Swiss EdTech Collider, an incubator with 80 start-ups in learning technologies. He (co-)-founded 4 start-ups, does consulting missions in the corporate world and joined the board of several companies or institutions. In 2018, he co-founded LEARN, the EPFL Center of Learning Sciences that brings together the local initiatives in educational innovation. He is a fellow of the International Society for Learning Sciences. He currently is the Associate Vice-President for Education at EPFL.

Patrick Jermann

After studies in Geneva (TECFA) and Pittsburgh (LRDC) I joined EPFL in 2003 to coordinate eLearning projects and conduct research in the field of Computer Supported Collaborative Learning (CSCL). Starting 2013 I am responsible for MOOCs production at the Center for Digital Education (CEDE).Former Associate Editor for the IEEE Transactions on Learning Technologies and former Member of the Editorial Board for the International Journal of Computer Supported Collaborative Learning (iJCSCL). Specialties: Interaction analysis, research methods, statistical methods, prototyping, software development, pedagogical design.

Official source

Related concepts (32)

Statistical hypothesis testing

A statistical hypothesis test is a method of statistical inference used to decide whether the data at hand sufficiently support a particular hypothesis. Hypothesis testing allows us to make probabilistic statements about population parameters. While hypothesis testing was popularized early in the 20th century, early forms were used in the 1700s. The first use is credited to John Arbuthnot (1710), followed by Pierre-Simon Laplace (1770s), in analyzing the human sex ratio at birth; see .

Electronic assessment

Electronic assessment, also known as digital assessment, e-assessment, online assessment or computer-based assessment, is the use of information technology in assessment such as educational assessment, health assessment, psychiatric assessment, and psychological assessment. This covers a wide range of activities ranging from the use of a word processor for assignments to on-screen testing. Specific types of e-assessment include multiple choice, online/electronic submission, computerized adaptive testing such as the Frankfurt Adaptive Concentration Test, and computerized classification testing.

Educational assessment

Educational assessment or educational evaluation is the systematic process of documenting and using empirical data on the knowledge, skill, attitudes, aptitude and beliefs to refine programs and improve student learning. Assessment data can be obtained from directly examining student work to assess the achievement of learning outcomes or can be based on data from which one can make inferences about learning. Assessment is often used interchangeably with test, but not limited to tests.

Criterion validity

In psychometrics, criterion validity, or criterion-related validity, is the extent to which an operationalization of a construct, such as a test, relates to, or predicts, a theoretical representation of the construct—the criterion. Criterion validity is often divided into concurrent and predictive validity based on the timing of measurement for the "predictor" and outcome. Concurrent validity refers to a comparison between the measure in question and an outcome assessed at the same time.

Test statistic

A test statistic is a statistic (a quantity derived from the sample) used in statistical hypothesis testing. A hypothesis test is typically specified in terms of a test statistic, considered as a numerical summary of a data-set that reduces the data to one value that can be used to perform the hypothesis test. In general, a test statistic is selected or defined in such a way as to quantify, within observed data, behaviours that would distinguish the null from the alternative hypothesis, where such an alternative is prescribed, or that would characterize the null hypothesis if there is no explicitly stated alternative hypothesis.

Related lectures (64)

Understanding Statistics & Experimental Design

Explores unequal variances, replication, power, effect size, biases, and their impact on research outcomes.

Statistics & Experimental Design

Explores conditional probability, Framingham studies, effect size, t-test, and sampling error in statistics.

Red Influence: Attractiveness, Desirability, and Status

Explores the effects of red on attractiveness, desirability, and status, emphasizing statistical analysis and the challenges of replication and publication bias.

Hypothesis Testing and Confidence Intervals: An Overview

Covers hypothesis testing, confidence intervals, and their applications in statistics.

Student Evaluation of Teaching: Effective Strategies