Mediaspace scheduled maintenance: Aug 25, 2026 07:00 - 12:00 AM. During this time, videos will be temporarily unavailable. Check status updates.

Lecture

Designing Experiments and Measuring Learning

In course

CS-411: Digital education

This course addresses the relationship between specific technological features and the learners' cognitive processes. It also covers the methods and results of empirical studies: do student actually learn due to technologies? In fall 2025, P. Dillenbourg w ...

Description

This lecture covers the design of experiments in educational research, focusing on the challenges unique to the social sciences. Topics include hypothesis formulation, variables in experiments, controlling for individual differences, and statistical tests for analyzing learning outcomes. The instructor discusses the Ballmer peak hypothesis, experimental designs, and strategies to mitigate biases in research. The lecture emphasizes the importance of construct validity and the impact of selection bias on research outcomes.

Instructors (2)

Pierre Dillenbourg

A former teacher in elementary school, Pierre Dillenbourg graduated in educational science (University of Mons, Belgium). He started his research on learning technologies in 1984. In 1986, he has been on of the first in the world to apply machine learning to develop a self-improving teaching system. He obtained a PhD in computer science from the University of Lancaster (UK), in the domain of artificial intelligence applications for education. He has been assistant professor at the University of Geneva. He joined EPFL in 2002. He has been the director of Center for Research and Support on Learning and its Technologies, then academic director of Center for Digital Education, which implements the MOOC strategy of EPFL (over 2 million registrations). He is full professor in learning technologies in the School of Computer & Communication Sciences, where he is the head of the CHILI Lab: "Computer-Human Interaction for Learning & Instruction ». He is the director of the leading house DUAL-T, which develops technologies for dual vocational education systems (carpenters, florists,...). With EPFL colleagues, he launched in 2017 the Swiss EdTech Collider, an incubator with 80 start-ups in learning technologies. He (co-)-founded 4 start-ups, does consulting missions in the corporate world and joined the board of several companies or institutions. In 2018, he co-founded LEARN, the EPFL Center of Learning Sciences that brings together the local initiatives in educational innovation. He is a fellow of the International Society for Learning Sciences. He currently is the Associate Vice-President for Education at EPFL.

Patrick Jermann

After studies in Geneva (TECFA) and Pittsburgh (LRDC) I joined EPFL in 2003 to coordinate eLearning projects and conduct research in the field of Computer Supported Collaborative Learning (CSCL). Starting 2013 I am responsible for MOOCs production at the Center for Digital Education (CEDE).Former Associate Editor for the IEEE Transactions on Learning Technologies and former Member of the Editorial Board for the International Journal of Computer Supported Collaborative Learning (iJCSCL). Specialties: Interaction analysis, research methods, statistical methods, prototyping, software development, pedagogical design.

Official source

Related concepts (32)

Statistical hypothesis testing

A statistical hypothesis test is a method of statistical inference used to decide whether the data at hand sufficiently support a particular hypothesis. Hypothesis testing allows us to make probabilistic statements about population parameters. While hypothesis testing was popularized early in the 20th century, early forms were used in the 1700s. The first use is credited to John Arbuthnot (1710), followed by Pierre-Simon Laplace (1770s), in analyzing the human sex ratio at birth; see .

Selection bias

Selection bias is the bias introduced by the selection of individuals, groups, or data for analysis in such a way that proper randomization is not achieved, thereby failing to ensure that the sample obtained is representative of the population intended to be analyzed. It is sometimes referred to as the selection effect. The phrase "selection bias" most often refers to the distortion of a statistical analysis, resulting from the method of collecting samples. If the selection bias is not taken into account, then some conclusions of the study may be false.

Formative assessment

Formative assessment, formative evaluation, formative feedback, or assessment for learning, including diagnostic testing, is a range of formal and informal assessment procedures conducted by teachers during the learning process in order to modify teaching and learning activities to improve student attainment. The goal of a formative assessment is to monitor student learning to provide ongoing feedback that can help students identify their strengths and weaknesses and target areas that need work.

Summative assessment

Summative assessment, summative evaluation, or assessment of learning is the assessment of participants in an educational program. Summative assessments are designed to both assess the effectiveness of the program and the learning of the participants. This contrasts with formative assessment, which summarizes the participants' development at a particular time in order to inform instructors of student learning progress. The goal of summative assessment is to evaluate student learning at the end of an instructional unit by comparing it against a standard or benchmark.

Electronic assessment

Electronic assessment, also known as digital assessment, e-assessment, online assessment or computer-based assessment, is the use of information technology in assessment such as educational assessment, health assessment, psychiatric assessment, and psychological assessment. This covers a wide range of activities ranging from the use of a word processor for assignments to on-screen testing. Specific types of e-assessment include multiple choice, online/electronic submission, computerized adaptive testing such as the Frankfurt Adaptive Concentration Test, and computerized classification testing.

Related lectures (35)

Understanding Statistics & Experimental Design

Explores unequal variances, replication, power, effect size, biases, and their impact on research outcomes.

Red Influence: Attractiveness, Desirability, and Status

Explores the effects of red on attractiveness, desirability, and status, emphasizing statistical analysis and the challenges of replication and publication bias.

Statistics & Experimental Design

Explores conditional probability, Framingham studies, effect size, t-test, and sampling error in statistics.

Student Evaluation of Teaching: Effective Strategies

Discusses the importance of student evaluations in improving teaching effectiveness and understanding student perspectives.

Hypothesis Testing and Confidence Intervals: An Overview