Research Design

Learning Objectives Formulate a testable hypothesis for a given computer science problem. Differentiate between independent, dependent, and confounding variables in a computational experiment. Design a controlled experiment to evaluate the performance of an algorithm or system. Compare and contrast experimental, quasi-experimental, and observational research designs. Identify potential threats to internal and external validity in a research study. Select appropriate metrics for measuring system performance, user experience, or algorithm efficiency. How does a company like Netflix *know* that changing a button's color from red to blue will increase user engagement? 🤔 They don't guess; they use research design! This lesson will introduce you to the structured proce...

TermDefinitionExample HypothesisA precise, testable statement about the expected outcome of a study. It proposes a relationship between variables.Null Hypothesis (H0): 'There is no significant difference in average execution time between QuickSort and MergeSort on arrays of 10,000 integers.' Alternative Hypothesis (H1): 'MergeSort will have a significantly lower average execution time than QuickSort on arrays of 10,000 integers.' Independent Variable (IV)The variable that the researcher manipulates or changes to observe its effect on another variable.In a study comparing two search algorithms, the Independent Variable is the 'type of search algorithm' used (e.g., Binary Search vs. Linear Search). Dependent Variable (DV)The variable that is measured or tested...

The Controlled Experiment Pattern 1. Formulate Hypothesis -> 2. Randomly Assign to Control/Experimental Groups -> 3. Manipulate Independent Variable for Experimental Group -> 4. Measure Dependent Variable for all groups -> 5. Analyze Results Use this pattern when you can directly control and manipulate the independent variable to determine a cause-and-effect relationship. It is the gold standard for testing hypotheses, such as whether a new caching strategy improves server response time. The A/B Testing Design Pattern 1. Define Goal Metric (e.g., click-through rate) -> 2. Create Control (A) and Variant (B) -> 3. Randomly serve A or B to users -> 4. Collect data until statistical significance is reached -> 5. Implement the winning version. A specific t...