From bad through good to excellent In- terpretative Phenomenological Analysis (IPA) studies – presenting set of criteria to evaluate IPA papers and to provide high-quality future research

Interpretative Phenomenological Analysis (IPA) is a recently developed qualitative approach, not only in psychology but also in pedagogy. It turns out that not all IPA studies meet the indicated criteria of acceptability developed by Jonathan A. Smith (2011). The aim of this article is to present a set of criteria to evaluate IPA papers and to provide high-quality future research. The article is divided into three parts: rst, I am going to focus on indicating limitations of pedagogical research in which IPA was used, later I shall go on to give examples of good practice. In the last section, I have correlated particular stages of research with the criteria of validity to show what makes IPA study excellent. I also formulate two main requisites for making high-quality IPA study.


Introduction
Interpretative Phenomenological Analysis is a framework methodology, which determines theoretical and practical tips on constructing a research project -this approach is more and more frequently used by representatives of the humanities, social sciences and health sciences.The focus of research is the position of the human beings with their dilemmas and troubles, or more broadlycoping with the world and in the world.IPA is a qualitative research approach, which examines how people give meaning to their important life experiences.Those experiences have personal meaning for individuals, actively involving their thoughts and feelings into the interpretation of phenomena, objects considered good; a further 55% were judged acceptable and the remaining 18% were rated unacceptable (Smith, 2011, pp. 23, 25).The least positively evaluated papers were often not consistent with the theoretical principles of IPA, insufciently transparent and interesting for the reader to see what was done and poorly evidenced.The last criterion is essential and determines preponderantly the quality of an article.J. A. Smith has pointed to typical ways how it can occur: "large number of descriptive/super cial themes from a large number of participants; each theme has short summary and one or two extracts without interpretation; insuf cient extracts from participants to support the themes being illustrated; no explanation for how prevalence of the themes was determined; and analysis is crude, lacks nuance" (Smith, 2011, p. 17).When I was reading selected pedagogical IPA papers I noticed similar failures. 5This observation was an incentive to re ect on what makes IPA studies good and what we should do to achieve high-quality research.It is worth mentioning that I decided not to give instances of particularly poor papers deliberately.There are two reasons for this: ethical and practical issues.It will be more useful to present good practices 6 and propose a set of criteria to evaluate IPA studies than to emphasise weaknesses of particular articles.This solution is bene cial to the reader, because the guide is provided and can be used to evaluate the quality of a paper on one's own and it can also be useful for a researcher who can take account of criteria in a future research project to make it better.
I have organised my re ections from the review of articles by putting them into one of the four crucial stages which are necessary to accomplish the aim of the research -from planning through realisation to dissemination of the results.I have selected: (1) conceptualisation of the research project, (2) conducting research, (3) working with data, (4) producing a report and its publication.In the last section I have added Lucy Yardles's (2000) criteria of quality of qualitative research which were used by J. A. Smith to illustrate how IPA can address them.I have divided my article into three parts: rst, I am going to focus on indicating limitations of research in which IPA was used, later I shall go on to give examples of good practice.In the second part I am going to focus more on characterising studies (especially analysis) than comparing studies with a set of criteria.In the last section of my essay I have correlated particular stages of research with criteria of validity to show what makes IPA study excellent.

Limitations of IPA research
Looking at the article, rst of all we can evaluate the concept of research because when we accept speci ed assumptions at the beginning, they determine the results of research.A well-thought, internally coherent, theoretically justi ed project is the primary criterion, however, it is not always ful lled.Choosing a research approach is connected with adopting a different view of what we can explore, what can be our research material, what we can infer based on it and what we can achieve analysing these particular gathered data.Most important in choosing the approach is to make sure that the epistemological assumption hidden in our research question is consistent with those which are formulated in the methodological approach.The researcher is supposed to keep consistency within the subject of research, what can be inferred from it and what the aim of the study is (Smith, Flowers, & Larkin, 2009, pp. 40-41).In this way epistemology determines the process of conceptualisation and has a practical in uence on decisions concerning the kind of research we want to carry out.When I was doing the review of the published articles, I realised that researchers' declaration to use IPA in their study does not always agree with what we see in the research project.I have an impression that some publications mention the main assumptions, for example concerning candidates' selection criteria, stages of IPA analysis, however, they do not serve to explain the research procedure but play the role of a label.It is useful because it exempts the researcher from being responsible for reasoning in support of decisions and setting down the process of research.What is hidden behind the label, i.e. a group of meanings, does not need to be negotiated and explained, because the meanings are shared, intersubjectively understood and/or legitimated by invoking the authorities.Kerry Chamberlain (2011, p. 48) formulates a similar remark in his article which is a commentary to a publication concerning the evaluation of IPA research by J. A. Smith (2011a).The author compares the present-day popularity of employing IPA to a similar situation when grounded theory was well-known and also applied unre ectively several decades ago.
On another occasion, the key elements of a research project can be incompatible with important assumptions referred to.Formulating an appropriate research question is crucial in a research project.When the question was phrased from a completely different approach we can say that it is a mistake.It happens that questions are in direct contradiction to the main assumption of IPA, which is focusing on personal experience.It is hard to evaluate this kind of study as a good one or even acceptable, because it has not met a primary condition to preserve epistemological coherence.
Conceptualisation of research de nes a virtual process of doing researchdata collection or sampling.All undertaken decisions exert an impact on the quality of the gathered data which will be analysed and have in uence on the nal results.There are lots of reasons why plausibility and transparency of research can be lost, for example: lack of theoretical justi cation of participants' selection, arbitrariness in deciding to what extent the groups should be homo-geneous, lack of respondents' description, insuf cient information about conditions in which the interviews were conducted, absence of interview schedule or information about the elds of interests.Insuf cient transparency for the reader to understand what has been done is one of the criteria which decides whether the study can be evaluated as not good enough.
A description of the analytic process is directly linked to the previous comments.Stages of analysis were vaguely rehearsed.These declarations were not re ected in the results of research.The authors declared that their work re ected the "double hermeneutic" of the IPA approach and ndings conveyed the researchers' interpretation of the participants' interpretation of their experience, however, their interpretation was only descriptive and super cial.The enclosed single utterances and small passages were not interpreted, we do not know how the themes were chosen and how the quotes were organised.The presented analyses take the form of paraphrasing what the respondents said without offering any comments about it.Readers are seldom informed why the emergent themes have been considered signi cant and why these and not others have been important and deserve to be analysed.It is a common mistake that authors do not point to the frequency of emergent themes and do not try to explain their prevalence -how this prevalence of a theme was determined.In some reports we could nd the following terms: "the informants," "several of the informants" etc. and it was dif cult to recognise how many participants the authors had in mind.A paper should have strong data; therefore, authors are supposed to provide suf cient evidence for each theme but I found it was one of the most neglected criteria.
There are different ways in which authors used the existing literature in their analysis.It is important to present what the connection between the present research and the previous one is.However, it is not enough to say only that there was convergence or divergence because it does not mean there has been any discussion with literature.It could be useful to make an effort to interpret the outcomes in the light of theories but not only empirical.Most of them were used to con rm or contradict the results of the research and the theoretical context is limited.IPA aspires to theoretical translatability, in this way readers can evaluate the results of research with regard to existing professional and empirical knowledge.
Good qualitative work requires good writing.Extracts should be selected and presented in some context.I found that in some papers the authors did not use any indication of the person whose words were quoted.Readers are not able to track the process of interpretation because they do not know who is speaking, whose perspective has been presented and so on, and as a result they may feel confused.Some leading narration was based on quotes and it was dif cult to match them with participants and the context of their life.When researchers present some extract without a name or a nickname and later describe the next emergent theme and also insert some extracts, the readers do not know whose perspective has been shown.Readers are not able to compare the point of view of participants if they do not know who is who.

Examples of good practice
Rachel L. Shaw (2011, p. 30) wrote that "Smith has generated another highly useful training device: a theoretical commitment to phenomenology, hermeneutics and idiography are framed almost as de ning characteristics of IPA research; issues of sample size and selection are dealt with within the framework of idiography; coherence, plausibility and interest become requirements for work considered acceptable; engaging and enlightening the reader are set goals for those wanting to produce good IPA work; while the issue of keeping focused and yet offering depth, and presenting strong data and interpretation are indicators of high-quality IPA research."I think that based on this quotation we can propose the distinction into acceptable, good and excellent IPA study.
I have presented my perception of some weak points of pedagogical research whose authors declared the use of IPA.Some part of this research has not ful lled the formal requirements of good IPA study.There are a few studies which are acceptable, but not excellent.It turns out that just following guidelines and ticking them off does not guarantee success, that is good and high quality IPA research.An additional advantage of the study can be achieved by scrupulousness, for example in terms of procedures transparency, detailed characteristics of research groups, enclosing interview disposition (cf.Redmont, Larkin, & Harrop, 2010), giving degree of prevalence for emergent themes (cf.Hanková, & Vávrová, 2016), noticing patterns and presenting a scheme of relationships and in uence between themes (cf.Hanková, & Vávrová, 2016), de ning limitations of researchers decision (cf.Redmont, Larkin, & Harrop, 2010), but the most important thing which determines the quality of research is the analytic process.According to R. L. Shaw (2011, p. 30) good research is conducted when the reader is engaged and enlightened.In this context, one article was worth mentioning.It concerned the experiences of young bystanders, people who witnessed bullying.The presented results were interesting, well-written and with sustained narrative.Matthew Hutchinson (2012) dialogued with existing theories when he carried out the process of interpretation.However, theoretical knowledge sometimes obscured the perspective of participants.He used a metaphor of theatre to explain the results of research.The meaning of silence, which appeared in a situation when young people witnessed their schoolmate being bullied, changed during the analysis in the context of the metaphor used.It can be read as a passive approval for the bully (it is emotionally easier to be silent than to defend the victim) although the silence is broken by the victim and then the meaning is changed -from quiet tolerance to barely suppressed disapproval.In relation to a metaphoric theatre, the previously mute character is a new voice and the audience can also become potential actors.The situation of being a bystander is complicated because of moral (experience of moral ambiguity), social (the power of social norm, being surrounded by peers, threat of ostracism) and psychological (experience of confusion, fear, isolation, disappointment, loneliness) in uences and conse-quences.Defending victims is connected with one's future project of identity, for example one participant said: "I'd like to be a bit more myself.I would like to do the things I like to do, and I would like to say the things that nobody else does, but I think you've got to be really brave."However, it can also be a sign of gained resilience and assertiveness: "It has made me happier than when I rst did it [defending] and I feel a lot more con dent now… It really doesn't bother me now if I was bullied, because I would just ignore them."Unfortunately, this thread was not developed enough.It could be worth thinking how to take advantage of these ndings, however the author pointed to implications for the professional practice of school staff.

What makes IPA study excellent
What constitutes the paramount value of good IPA study is intangible in a sense that it is not a collection of simple guidelines which can be followed and applied in one's own research project.There is also a need to ful l criteria of the quality of research, the validity of the studies.I have added L. Yardles Smith, 2004) and my own re ections after reading IPA studies.When I was browsing articles I tried to recognise examples of good practises and tailored them to one of the criteria.The rst two columns concern knowledge and to some extent also experience in conducting research.Researchers usually know that they should answer three major questions: what they want to do, why they want to do it and how they want to do it.The chosen approach determines the whole process of conceptualisation and conducting research, however a researcher should preserve transparency as long as it is possible.In relation to the indicator of high-quality IPA research -the issue of keeping focused and yet offering depth, and presenting strong data and interpretation, columns "working with data" and "producing a report and its publication" are the most extended part in the whole table.In this part of the essay, I have focused only on describing the last two columns.The included table is my proposition to be used in future IPA study evaluation.
The process of analysis is crucial; full immersion in data, not only declarative but also actual.It is not a mystical moment when the emergent themes appear and the tracks of interpretation are known.It is hard, time-consuming, demanding work with data, when suddenly we can be inspired by one extract, one deviant case or pattern which is sought-after but we were led by our presumptions and now we need to deconstruct them to make a move.The process of interpretation is not only a simple passing through subsequent stages, but constantly moving within different stances, which can refer to a part and the whole of the analysed text, being close to raw material but also moving beyond the text to a more interpretative and psychological level (Smith, 2004, p. 44)."The second measure of quality is much more concerned with particulars.It is to do with the utterance that stands out and has added value to the analysis as a whole, the single case that is particularly potent."As a result, a researcher should "have two views in mind at the same time: a) an awareness of realistic endpoints and of what is required for an analysis to be good enough for the particular task in hand; b) an eye to the possibility of pushing the analysis further, recognising that often this remains a possibility rather than a necessity" (p.46).J. A. Smith (2004) has shown three levels of interpretation.I called them by virtue of the level of obviousness: literal meaning; implied meaning of the text; hidden, secret or mystic meaning of a text and by virtue of abstractedness: content of utterance; microanalysis: the way how content and meaning were presented; more conceptual and abstract ndings i.e. about time frame used, unsaid meanings, contradictions, functions.
In this regard the concept of a gem is extremely inspiring because it concerns the most demanding level of interpretation (Smith, 2011b).However, it is easier for a novice to nd and deconstruct metaphors -the second level of interpretation (Shinebourne, & Smith, 2010) than peer, probe and investigate gems.But still, without being sensitive to the context neither metaphors nor gems appear.Researchers should also show in their analysis sensitivity to historical, social, and political context, as it happened in relation to narration ( Commitment is connected, to some extent, with sensitivity to context.That is why analysis should be nuanced -the reader should be able to learn something about both the important generic themes in the analysis, but also about lives of particular participants who have told their stories (Smith et al., 1995in: Smith, 2004).It is also persistently getting closer to the participants' perspective and walking away from it to have a more critical attitude (hermeneutics of empathy vs. hermeneutics of suspicion).Apart from gems, metaphors can also hold the meaning and be powerful tools for communicating and sharing experiences which are dif cult to express directly (Shinebourne, & Smith, 2010).Nevertheless the researcher has to peer, probe and investigate to nd gems and/or metaphors and to reveal their meaning by deeper analysis.What is more, an analysed quotation should be set in context.It is not located in complete isolation from the rest of the analysis and requires providing good evidence based on raw material.Researchers should judiciously select good and appropriate illustrations for the track of interpretation, as happened in the following articles: de  This kind of concentration on data is known in Poland as objective hermeneutics.Danuta Urbaniak-Zaj c conducted pedagogical research using objective hermeneutics developed by Ulrich Oevermann.In practice a researcher identi es latent and objective structures of meaning.The rst one is a body of reconstructed opportunities during interpretation.The second are particular structures which characterised the case of analysis and were chosen from all alternatives at the end of interpretation (Kirsch, 2008 in: Urbaniak-Zaj c, 2010, p. 10).In this way I have understood the process of deconstruction which was presented in the study on the personal impact of chronic back pain (Smith, & Osborn, 2007).One respondent asked whether they were going to be surrounded and taken to a camp somewhere.In this case the latent structures of meaning are all meanings of the word "camp" and their later veri cation in the context of a sentence, an extract, a whole interview or other participants' utterances to choose which options are the most adequate.
A more visible, shiny gem has appeared in a research conducted by John Rhodes & Jonathan A. Smith (2010).It is a great study which presents one case of a participant who suffered from depression.The image of experience is vivid, sometimes frightening because of the extent of negative thoughts and emotions accumulated in the participant's narration.The main utterance which is also the part of the title is reconstructed in relation to the whole interview and presents depression as a sort of wounding, reduction or morti cation of one's self; the person is forced to change narration of life and rede ne their project of identity.The authors provided enough quotations to support their interpretation.We can see the time frame of one's life, trajectory of changes and nally the powerful expression of dying of this part of life to which somebody was accustomed.
Another example of IPA study which matches the criteria of rigour and commitment but also sensitivity to context is research about unprotected sex in relations between gay men (Flowers, Smith, Sheeran, & Beail, 1997).The reader can understand the reason why gay men were engaged in unprotected anal sex.The powerful symbolism of penetration was connected with expression of love, commitment and trust.In this study the shining gem also appeared (Smith, 2011b, p. 11), but the whole article is full of vivid utterances (i.e."Like my life's in his hand and his life's in my hand and it's like that" or "Doing it with your body but not with your mind" or we can add "heart").In my opinion it is closely tied with the last criterion -importance and impact because when the subject is important and personally engaging to participants, it is more likely that gems and/or metaphors appear.In this way the researcher can reveal something which is dif cult to come out directly or sometimes it even strikes the eye.These are examples of excellent work because they tell the reader something important, interesting or useful.First and foremost, it is worth asking whether these ndings would have been obtained if a different approach had been employed.When we ask whether IPA was the best approach to answer the research question and the ndings con rm it, we have examples of great work.It could be dif cult to explore and catch the meaning made by young men about alcohol and their experiences (de Vissero, & Smith, 2007; de Visser, Wheeler, Abraham, & Smith, 2013) or health-related behaviour in context of discourses about masculinity (de Vissero, & Smith, 2006) using a different approach.Moreover, the value of this research justi es why IPA was appropriate to have been employed in these studies.It should also be included in the report as well as it is necessary to indicate the reason why the research was conducted, what in uence it has on the discipline's development, what the bene ts from research are for professionals and also participants (it is more about transparency but it is also connected with the criterion of impact and importance).The ndings should always be related to relevant literature in the discussion by comparing them with other research, notions, models or theories.In each of the mentioned articles the analytic process was pushed further and the results showed the phenomenon in a new light.Research about women's anger and aggression (Eatough, Smith, & Shaw, 2008) was extremely inspiring in terms of understanding emotions as judgments.The relational core of emotions was emphasised and explained in relation to women's behaviour.Strong embodiment of anger and the meaning of crying, especially in adult women, are the issues which can be revealed only by phenomenological insight in participants' experiences.A broad spectrum of theoretical sources was used by Pnina Shinebourne, J. A. Smith (2008) to start the analysis of multiple selves from psychiatric perspective through more social-related in uences to psychological point of view on identity, but also attitude to alcohol and the use of metaphors (the last issue was developed in later articles written by the same authors in 2010).
The criterion of rigour is closely connected with the need for transparency.Firstly, before readers would be able to follow the track of interpretation and analysis, they should be informed about decisions which were undertaken by researchers before conducting research and over time.The decisions should be presented and justi ed, conditions in which the research was conducted should be described in detail (most of the articles provided justi cations of the major decision, it is required in all types of research).Johanna Spiers, J. A. Smith, Philip Simpson & Adam R. Nicholls (2016) and another research team (de Visser, Wheeler, Abraham, & Smith, 2013) have explained why they gave up analysing data as per participants' gender.In this study the authors thought about sociodemographic features of interviewer and their potential in uence on participants.However, the most important aspect of transparency again is tied in with the analytic process.There is a need to describe: how it happened that a researcher paid attention to a particular utterance, how they peered at it, why particular case/emerging themes/quotations were chosen to be presented.The evaluation of validity of the analysis can also be provided, i.e. emerging themes were negotiated with another scientist, one tried to bracket gained knowledge or held presumptions or introduced a special procedure of testing the interpretations made.It is worth including a table of the participants, the schedule, elements of the analytic process because "For the researcher, this table is the outcome of an interactive process in which she/he has moved back and forth between the various analytic stages ensuring that the integrity of what participants said has been preserved as far as possible.If the researcher has been successful then it should be possible for someone else to track the analytic journey from the raw data through to the end table" (Eatough, & Smith, 2006).A well written report -planned, with structure, can be the evidence of ful lling all the presented criteria.

Impact and importance
Researcher should know why they want to engage in investigating this particular topic.
The impact and importance occurred when participants are interested in taking part in research and they are engaged in sharing their experiences.
When the subject is important and personally engaging for participants it is more likely that gems and/or metaphors appear.In this way researcher can reveal something which is difcult to come out directly.
Test of research validity lies in whether it tells the reader something important, interesting or useful.Good justi cation of the research, in uence on discipline development, ll the gap in literature, bene ts from research for professionals and also participantseach of these aspects should be presented in the report.
After reading and analysing a great number of articles I realised that there are two main requisites for making high-quality IPA study.The rst term is crucial at the beginning, but it does not guarantee success if the second term is not ful lled.In my opinion, without an important and personally engaging topic it is unlikely to gain rich and interesting data from participants and as a consequence the analysis is more super cial (assuming that interview was conducted properly).Not only is the subject of research signi cant, but also the chosen approach.Good IPA study is when the researcher found something that could be dif cult to explore or reveal using another approach.However, even the choice of the most in uential topic can be insuf cient without demanding work with data so that the ndings will be illuminating, useful and cutting-edge.The process of analysis is crucial; full immersion in data, not only declarative but also actual.It is not a mystical moment when emergent themes appear and tracks of interpretation are known.It is hard, time-consuming, rigorous work with data, when suddenly we can be inspired by one extract, one deviant case or pattern which is sought-after but we were led by our presumptions and now we need to deconstruct them to make a move.As Shaw (2011, p. 30) said the indicator of high-quality IPA research is focused, in-depth and well-documented interpretation.That is why the process of interpretation is not only a simple passing through subsequent stages, but constantly moving within different stances -being sensitive to context vs. being suspicious; being close to raw material vs. being more creative in analytic process; being focused on person vs. being focused on totality; depending on intuition vs. verifying presumptions and checking validity of interpretation.In addition, if a researcher can write a good, planned and engaging report with adequately chosen and interpreted quotations, in which they proved the importance and usefulness of the research and when it is easy to follow the narrative of participants' stories and watch the process of interpretation, then we can say it was an excellent IPA study.
's (2000) criteria of quality of qualitative research which were used by J. A. Smith to illustrate how IPA can address them.I have correlated distinguished stages of research with the criteria of validity.In Table 1.I have presented more general statements based on publications (Smith, Flowers, & Larkin, 2009; Smith, 2011; Shinebourne, & Smith, 2010; Smith, 2004; Smith et al., 1995 in:

Table 1 .
Comparison of two sets of criteria