Why similarity and distractor count interact
A common intuition is that the more distractors there are, the longer it takes to find the target, and that the harder it is to tell the target from distractors, the steeper that cost becomes. In one set of experiments, participants looked for an oddball color target among distractors while the researchers systematically varied both the color similarity between target and distractors and the number of distractors in the display [1]. Search times increased with the number of distractors, but the size of that increase—the search slope—was not a simple mirror of similarity [1].
The relationship was more nuanced than a straight line. When target and distractor colors were extremely similar—within about 20 degrees on a color wheel—search was highly efficient, with near-zero slopes, meaning adding more distractors barely slowed people down [1]. That contradicts the general assumption that high similarity must produce slow, serial search [1].
What efficient versus inefficient search looks like
In an efficient search, response time stays roughly flat as you add distractors; in an inefficient search, each extra distractor adds a noticeable delay. The color experiments found logarithmic search slopes that were inversely proportional to the number of items in the array, and the slope depended on similarity in a non-monotonic way [1]. So the practical answer is not simply 'more distractors always cost more'—the cost depends on where the target sits relative to distractors in feature space.
Other work on face search found that reaction times increased with the number of distractors, but the increase could be minimal under some conditions [2]. That is consistent with the idea that distractor count matters, but its impact is shaped by the stimulus set and task.
Similarity effects unfold over time and in eye movements
Similarity does not act at a single moment. One study examined the time course of similarity effects and found that both target-distractor similarity and distractor-distractor similarity influence perceptual and decision processes, with effects changing within a trial [4]. That suggests the joint impact of similarity and distractor count is not fixed at the start of search but evolves as the search proceeds.
Eye-movement work in a pseudo-realistic search task manipulated target-distractor similarity and the number of less-salient distractors separately [3]. It found that it takes time for saccades to be correctly directed at the target, and that a high-salience singleton captured very few saccades during search even though it was selected early in free viewing [3]. This fits with the idea that early attention can be pulled by salience while goal-directed search takes time to engage, which may help explain why similarity and distractor count interact rather than simply add up.
What this means for predicting search time
If you want to predict how long a visual search will take, you need to know both how many distractors there are and how similar they are to the target—but the effect of similarity is not monotonic. Very high similarity can, under some conditions, produce efficient search with near-zero slopes [1]. The scope of that finding is color-oddball search with systematic color variation; whether it generalizes to other features or real-world scenes is not established by these studies.
The safest summary is that distractor count sets the potential cost, while similarity determines how steeply that cost grows—and in the extreme-similarity case, the cost can nearly vanish [1]. Similarity effects also change over the course of a trial [4], so a single slope or average reaction time may hide important dynamics.
Sources used in this answer
Target–distractor similarity predicts visual search efficiency but only for highly similar features
Target-distractor similarity and the number of distractors jointly affect visual search, but search can be highly efficient with near-zero slopes even for extremely similar colors (≤20° in color space), contrary to the assumption that high similarity always produces slow search.
Impact of quality of the image, orientation, and similarity of the stimuli on visual search for faces
In visual search for faces, reaction times increased with the number of distractors, though the increase could be minimal under some conditions.
Evaluating the contributions of top-down and bottom-up processing on eye movements during parallel visual search
In an efficient visual search task, target-distractor similarity and the number of less-salient distractors were manipulated separately, and results showed it takes time for saccades to be correctly directed at the target, with a high-salience singleton capturing few saccades during search.
The time course of similarity effects in visual search.
Similarity effects in visual search change over time within a trial, with both target-distractor and distractor-distractor similarity influencing perceptual and decision processes.