I give an historical overview of how cognitive (and other) models of users of interactive IR (IIR) systems have been elicited, constructed and used. Cognitive models in the domain of interactive information retrieval (IIR) are understood as models that a system (or a person) constructs of a(nother) person's "information need"; these are called "user models". Context models are models that a system (or a person) constructs of the conditions that led a(nother) person to engage in information seeking behavior, various characteristics of that person, and various aspects of that person's environment, broadly construed. Both types of models are used in personalization of IIR. The review begins with discussion of Robert Taylor's 1968 article in which he proposed four levels of "information need" or "query", and five "filters" according to which the librarian and the information seeker identify and clarify (i.e. model) various aspects of the user, the user's goals, the topic of interest and so on. I then review various approaches to understanding of why people engage in information seeking behavior, and of automatically constructing cognitive and contextual models. Significant change points include the "cognitive turn" in the early to mid 1970s, modeling the human intermediary in the 1980s, and the "interactive turn" in the 1990s. The review concludes with a brief description of, and comments on the current state of cognitive and contextual modeling in IIR.