Purpose of theory of signal detectability (TSD)

• Provides a rational basis from statistical decision theory for conceptualizing how subjects make decisions under conditions of stimulus uncertainty
• Allows separation of sensitivity to stimulus differences or ability to discriminate from various kinds of response bias due to prior knowledge, stimulus probabilities, payoffs, or other motivational factors
• Applicable to a wide variety of detection, discrimination, and recognition problems in many different areas of psychology

• Originally developed form statistical decision theory in the early 1940's by communication engineers to analyze the transmission of information through noisy communication channels
• Introduced in psychology by psychophysicists (particularly in psychoacoustics) in the 1950's to separate stimulus sensitivity from other aspects of the decision making process in sensory psychology

• For at least 150 years psychophysicists have attempted to measure the smallest stimulus values subjects can detect (absolute threshold) and the smallest differences between stimulus values that subjects perceive as different (difference threshold or just noticeable difference)
• Such measurements are important because they show the limits of resolution of our sensory systems and provide a starting point for theorizing about how the physical world is mapped into our sensory experience of it
• For many years these measurements were based on a fairly literal interpretation of a threshold as a fixed sensory value or difference in sensory values which was the dividing line between perceptible and imperceptible
• Two major problems with this approach gradually became evident--one having to do with the reality of sensory thresholds and the other with the effect of nonsensory variables (other aspects of the subjects decision making processes) on the measurements

• If the notion of sensory thresholds is taken seriously, we would expect any stimulus value or difference in stimulus values below the threshold to go undetected so that response probability would be zero or some constant Aguessing probability@
• If the stimulus value or difference in stimulus values is above threshold, it should be detected so that response probability would be 1.0
• Resulting psychometric functions would be step functions with a sudden transition from a response probability of zero or the guessing probability to 1.0 at the value of the sensory threshold.
• Real psychometric functions always show a gradual increase in response probability as stimulus value or difference in stimulus values is increased
• This led to thresholds being interpreted as statistical concepts (e.g., the value of the stimulus which a subject would detect 50% of the time)
• Another possibility is that discrete thresholds simply do not exist

• Many factors other than the detectability or discriminability of the stimuli can bias subjects' response
• Such factors include payoffs, a priori stimulus probabilities, instructions, individual differences
• All of these effects can be thought of as part of the decision process, rather than part of the detection or discrimination process
• In classical psychophysics it was thought that subjects could be trained to be unbiased
• This led to the use of trained, professional subjects and catch trials
• It gradually became clear that response bias was probably unavoidable and that it might be better to recognize this fact and attempt to derive independent measures of sensitivity to stimulus differences and response bias
• This provides the context for the entry of TSD on the scene since that is exactly what it attempts to do

• On a random half of a number of discrete trials a subject is presented with a white noise (noise alone or NA trials)
• On the other half of the trials a faint pure tone is presented along with the white noise (signal plus noise or SN trials)
• The subject's task is to simply respond >yes= or >no= at the end of each trial indicating whether or not he/she thinks the signal was present

• On each kind of trial (NA and SN) the subject can give either of two responses (>yes= or >no=)
• The results can thus be summarized in a 2X2 stimulus-response contingency table
• Each cell in the table simply records the number of trials of that type with that response outcome
• The marginal totals show the total numbers of each trial and response type

• >Yes= on a SN trial is called a hit (H)
• >Yes= on a NA trial is called a false alarm (F)
• >No= on a SN trial is called a miss (M)
• >No= on a NA trial is called a correct rejection (C)

• The count in each cell of the table can be divided by the total number of trials of that type (row total) to obtain the conditional probability of that stimulus-response event
• For example, the number of hits divided by the total number of SN trials is called the >hit rate= and is the conditional probability of saying yes given a SN trial
• Note that given a hit rate and a false alarm rate, the miss rate and correct rejection rates are determined
• We can thus reasonably summarize the results of our little detection experiment in terms of a hit rate and a false alarm rate

• The greater the difference between the hit and false alarm rates, the better our subject was at discriminating between the presence vs absence of the signal (i.e., the more sensitive he/she was to the signal.
• A good measure of sensitivity would thus be some function of the difference between these rates (H-F), but what function
• Response bias (to say >yes= vs >no=) in this situation is reflected in the sum of these rates (H+F), but again the question arises as to exactly how to scale it

• TSD assumes that decisions are based on the value of some internal variable x (e.g. Asensory magnitude@) whose average value is monotonically related to stimulus magnitude
• It is further assumed that the value of x varies from trial to trial, even though the same signal is presented
• The form of this trial to trial variability is usually assumed to be normal
• We can thus think of the situation as being characterized by two overlapping normal distributions, the higher one representing the distribution of sensory magnitudes for SN trials and the lower one the distribution for NA trials
• In the simplest case the variance of these two distributions would be the same

• On any trial the subject randomly samples a value of x from the distribution for that trial type and must use that to decide whether the signal was presented or not
• The most reasonable and consistent way for the subject to do this is to set a criterion value on x (x_c) and to respond >yes= if the value of x is above that criterion and >no= if it is below
• Independent measures of sensitivity and response bias are then provided by the separation between the means of the sensory distributions and the placement of the criterion, respectively

• On this interpretation, hit rate is the area under the sensory distribution for SN trials above the criterion
• Similarly, false alarm rate is the area under the sensory distribution for NA trials above the criterion
• Using these two values we can compute the distance between the means of the two sensory distributions in units of their standard deviation using z tables or the @NORMSINV in Quattro Pro
• The z-transform of a proportion converts it into standard deviation units of a normal distribution with mean 0 and standard deviation 1. A proportion of 0.5 is converted into a z-score of 0, larger proportions into positive z-scores, and smaller proportions into negative ones. Two proportions equally far from 0.5 lead to the same absolute z-score.
• The resulting measure of sensitivity is called d= and is computed as: d= = z(H) - z(F) where H and F are the hit and false alarm rates, respectively
• d= can be thought of as the number of standard deviation units the NA distribution would have to be moved to the right to make F the same as H

• For many years the preferred measure of response bias was the likelihood ratio >beta=
• This measure is the height of the SN distribution at the criterion, relative to the height of the NA distribution at the criterion
• Beta = 1 if the criterion is at the intersection of the two distributions, less than 1 if the criterion is below this intersection, and greater than 1 if the criterion is above this intersection.
• Beta = 1 represents unbiased responding or is said to be characteristic of the AIdeal Observer@ because it maximizes probability of being correct.
• Derivation of beta is fairly complex and it is no longer the preferred measure of response bias so I shall not derive it for you
• If you need to compute beta it can be done with the formula beta = exp(-0.5(z(H)²-z(F)²)) where H and F are hit and false alarm rates, respectively, and exp is the base of the natural logarithms (e) raised to the power of the function=s argument. In Quattro Pro there is a function for exp called @EXP

• In recent years it has been shown that there are various measurement advantages to specifying response bias in terms of its location on the sensory axis (c), rather than as a likelihood ratio (beta)
• The major advantage of this measure for you is that its meaning is more intuitively obvious
• We begin by placing the zero point of the sensory value axis at the intersection of the NA and SN distributions, midway between the means of these two distributions
• Thus, computing c as the location of the criterion on this axis will give us its distance from unbiased placement in units of standard deviation of the sensory distributions
• The z transform of the hit rate is the distance between the mean of the SN distribution and the criterion c: z(H) = d=/2 - c
• The z transform of the false-alarm rate is the distance between the mean of the NA distribution and the criterion c: z(F) = -d=/2 - c
• We can now add these two equations and solve for c in terms of z(H) and z(F)
• [z(H) = d=/2 - c] + [z(F) = -d=/2 - c
• z(H) + z(F) = -2c
• c = -0.5( z(H) + z(F) ) which is our formula for criterion placement in units of standard deviation from unbiased placement

• Since the outcome of a particular condition in a yes-no signal detection experiment can be represented as an ordered pair of values (the hit and false-alarm rates), it is useful to have a way to graphically present and interpret them.
• This is usually done by plotting hit rate against false-alarm rate
• Such a plot is called a receiver operating characteristic or ROC

• The positive diagonal is chance performance (d==0)
• Points above the positive diagonal are above chance (d=>0) and points below it are below chance (d=<0)
• The negative diagonal is unbiased performance (c=0)
• Points below the negative diagonal are positive bias (c>0) and points above it are negative bias (c<0)

• An isosensitivity curve is the locus of points in ROC space representing a given level of d= and different values of c such as might be generated by keeping the stimuli to be discriminated constant and varying response bias factors such as payoffs or instructions
• In ROC space, an isosensitivity curve goes from <0,0> to <1,1> as a curve symmetrical about the negative diagonal
• The distance of this curve from the positive diagonal is monotonically related to d=
• If the isosensitivity curve is not symmetrical about the negative diagonal, the underlying sensory distributions did not have equal variance--a condition that can be handled, but we shall not deal with it

• An isobias curve is the locus of points in ROC space having the same response bias but different values of d=, such as might be produced by varying signal to noise ratio while holding instructions, payoffs, etc. constant
• The form of isobias curves is different for different measures of bias, but for criterion placement c they go from <0,1> to the positive diagonal
• Thus, isosensitivity and isobias curves show that in ROC space, sensitivity or d= increases with movement up and to the left along the negative diagonal, while bias c increases with movement along the positive diagonal away from the negative diagonal
• Note once again that these two dimensions in ROC space are orthogonal to each other so as to provide independent estimates of sensitivity and response bias
• Although I am not going to do it, if the axis of ROC space are rescaled from linear to z coordinate, both isosensitivity curves and isobias curves become straight lines which can be used to graphically estimate d= and c

• Isosensitivity curves could be generated by repeatedly running a yes-no signal detection experiment under different response bias conditions (instructions, payoffs, etc) and a constant signal to noise ratio
• Each condition would lead to a separate estimate of d= with a different value of c and plotting the <H,F> pairs for each condition should yield an isosensitivity curve in ROC space
• A much more efficient way to generate an entire isosensitivity curve is to run a single condition (signal to noise ratio and response bias factors), but ask the subject to make confidence ratings instead of simple yes-no responses
• This can be done by asking the subject to rate each trial on an n-point rating scale varying from certainty that the signal was presented, through relative indifference between the choices, to certainty that the signal was not presented
• Alternatively, we could ask the subject to make a yes-no judgement and then to rate his/her confidence in that judgement on an n-point scale

• Any value in the rating scale can be thought of as dividing the response space into yes and no regions with any rating of that value or lower being a >yes= and any rating with a higher value being a >no=
• Each of these binary partitions of the response space should have the same d=, but c becomes less strict (more negative or less positive) as the value at which the rating scale is partitioned becomes higher (closer to certainty that the signal was not presented)
• For an n-point rating scale, n-1 such partitions of the response space are possible
• For each of these partitions, a hit rate and a false alarm rate can be computed in the same manner as for a simple yes-no signal detection experiment and plotted in ROC space to obtain an entire isosensitivity curve
• For each <H,F> pair a value of d= and of c can also be computed just as for a simple yes-no signal detection experiment

• Begin with a frequency table showing the number of occurrences of each rating value under each of the two stimulus conditions (NA and SN)
• Just as in the simple yes-no situation, convert each of these to a conditional probability by dividing each frequency by the total number of trials with the same stimulus (i.e., the row sum)
• Note that these conditional probabilities must sum to one (with rounding error) for each stimulus
• For each response find the proportion of trials leading to that response or any response to the left of it (lower rating) by summing the conditional probability table from left to right
• Each column of this cumulative probability table now contains a hit and a false-alarm rate with different columns representing increasingly less strict criterion values moving from left to right in the table
• These <H,F> pairs can be plotted in ROC space to obtain an isosensitivity curve or used to compute n-1 values of d= and c in which d= is approximately constant and c is decreasing (H and F must increase)

• Use of the analyses you have learned in this section is not limited to the yes-no signal detection experiment
• These concepts and analyses are useful in any situation in which subjects are required to discriminate between two stimuli that are presented one at a time using either a binary response or some measure that can be conceived of a rating scale (e.g., response rates in an animal experiment)
• The stimuli might be the presence vs absence of something (detection), two different values of the stimuli (discrimination), or stimuli which are to be classified into two categories (recognition such)
• Nor is the usefulness of these concepts and methods limited to sensory experiments, but also applies to many other area of psychology such as memory and concept formation
• These ideas have also been generalized to many other types of experimental designs, but measures comparable to d= and c have to be derived for each of these