Data collection.

Summarise the data collection approach(es) and procedure(s).

How complete were descriptions of data collection procedures? Was there any obvious information missing?

How systematic were data collection procedures? Were protocols for data collections provided, e.g., interview protocols?

How was data recorded? E.g., audio recording; note taking; multiple observers or one observer?

Was data collection and analysis cyclical or ‘one-shot’. Was this appropriate?

Were changes in data collection procedures described and justified eg, changes in the interview protocol as a result of initial analysis?

Bearing in mind the previous discussion, were data collection procedures appropriate, and appropriately implemented, for the research purposes?

In summary, how dependable were data collection methods?

1. Was a target population specified? Can a target population be inferred?
2. Describe the overall sampling strategy. How was the sample chosen? If possible, label the strategy (e.g., random sampling, maximum variation sampling, homogenous sampling, extreme case sampling, revelatory case sampling, convenience sampling etc). Was this appropriate for the study?
3. What was the final sample size? Was a response rate provided? How can it be calculated from the information provided? Show how, or what information is lacking. Is this size sufficient for the research?
4. Why was that sample size used? If claims of saturation were made, were they of data saturation, or of analytical or theoretical saturation? What claims were made or evidence provided that saturation was reached?
5. Was theoretical sampling used to saturate categories emerging from a first phase of the research. If so, in what specific ways was it theoretical or purposeful? How relevant to the purposes of the study was the sampling?

Data Analysis.

Describe the procedures used to analyse the data.

How complete were descriptions of data collection procedures?

Was there any obvious information missing?

How systematic were data analysis procedures?

Were protocols for data analysis provided?

Were themes, concepts and codes/categories derived from the data or derived from existing theory?

Is it clear how these were derived?

How systematically were these applied?

Did data analysis cycle with data collection? Were subsequent changes to themes, concepts or codes described and justified?

In summary, how dependable were data analysis procedures?

Knowledge claim?

1. what does the authors claim to be true as a result of the application of their method

2. Are conclusions clearly   linked back to the evidence within the data?

3. In what ways does the author generalise the conclusions? On what basis are these generalisations justified? (Think about analytic generalisabilty?)

Conditions for causal claims.

Were causal claims made or inferred? How and how well met are the conditions required for making causal claims (that is, internal validity)?

Alternative explanations for the results (internal validity and threats to internal validity)

What procedures were undertaken to ensure/increase the internal validity of the study? Are other plausible explanations possible?