Source
r/statistics•
Fourpointtwo

[Q] How do I check to see if x has any effect on the outcome of an intervention?

Question

Hello,

I'm currently writing my bachelors thesis and it seems like I'm in a little bit above my own and my supervisors head.

We have conducted an experiment through a survey where the respondents first answer some questions that score their level of self efficacy on a scale from 10-40.

The next step measures their intent to buy a totalt of 6 different food groups through likert scales, they then get to see a randomized video (2 videos in total) after which their intent to buy is measured again.

We used Wilcoxon's rank sum test to measure if there is a difference on the intent to buy before and after the video since we have a rather small sample size (115).

However, we would also like to measure if the measured level of self efficacy has an effect on which video yields the stronger change in intent to buy.

The level of self efficacy can be split into low or high, but preferrably we would like to use the actual measured level of self efficacy.

We have been in contact with some people in our institution with greater statistical knowledge then our supervisor, but we have not recieved any satisfying answers (yet).

If this is the wrong place for asking these types of questions I do apologize. Also please apologize any spelling and/or grammatical errors, english is not my native tongue.

Posted Dec 15, 2020, 12:58 PM UTC

19
lightsnooze•

Just so I understand:

Intent to purchase is measured pre- and post-test, is that right? Whats the observed range of scores for this intent measure?

Was self efficacy only measured at pre-test? Whats the observed range?

Which of the following best describes your research question involving self efficacy?

  1. Is self efficacy associated with intent to purchase at pre-test?

  2. Is self efficacy associated with intent to purchase at post-test?

  3. Is self efficacy associated with intent to purchase at post-test, even after controlling for pre-test levels of intent?

2
FourpointtwoOP•

The range of scores for the intent is 1-7 for each food group and is measure pre- and post-intervention.

Self efficacy is only measured pre intervention, the range is 10-40.

The research question involving self efficacy would be better described as "does the level of self efficacy effect which intervention has the best effect regarding change in intent to purchase". In other words the change in intent is what we are interested, the levels pre- and post-intervention not so much.

1
lightsnooze•

Okay, there are two experimental conditions: video A and video B?

Just so I am absolutely clear, are you saying that one video maybe increase intent more than the other video, but this increase depends on the initial level of self efficacy?

1
FourpointtwoOP•

Yes, there is two experimental conditions: A & B.

The videos are using different ways of framing the messages in them. The framing in it self might have an effect on changing the intent. The case might also be that the level of self efficacy plays a role in which way of framing the message yields the larger change (if any).

In other words we want to examine which way of framing the information yields the greater change.

In addition to this we want to see if the level of self efficacy has any effect on which way of framing the messages yields the better change of intent.

1
lightsnooze•

Ok I think I have all the info I need. You might be looking at a moderating effect of self efficacy on experimental condition. Have they taught you moderation in your stats class?

You would use multiple linear regression and the equation would be as follows (where the ~ operator means 'regressed on'):

Post-intervention intent ~ pre-intervention intent + experimental condition + self efficacy + experimental condition * self efficacy

I'm assuming you're using SPSS as it's for a Bachelor's but let me know if youre using sonething else and I can send a diff set of resources.

You can use the PROCESS macro by Hayes to run moderation.

https://www.processmacro.org/index.html

Here is a video guide as well

https://youtu.be/JZ2fl8exNTE

3
Soctman•

The simplest test that you could do is to perform a moderation analysis on the change scores (Pre- minus Post-video scores). You're essentially examining the interaction between self-efficacy and the effect of the video.

I noticed that you are using Jamovi. Go ahead and install the "medmod" module - I haven't personally used it yet, but it seems to be a pretty good tool. Make sure that you examine how the scores change (or don't) at -1 SD, Mean, and +1 SD reported self-efficacy. I'm assuming that your a priori hypothesis is that individuals with greater self-efficacy are more resistant to outside influence and will therefore have lower change scores. You may also need to break it down by food group as self-efficacy may not have any effect on preference changes for certain foods.

And be mindful of ceiling effects: If consumers overwhelmingly intend to buy a certain product in the beginning (e.g. pizza), then you will not detect any changes because they gave high initial ratings. In fact, you may even see a non-significant negative trend as an artifact of this effect.

1
FourpointtwoOP•

Pre- minus Post-video scores

Is there any specific reason you suggest this order? We have been subtracting the pre-score from the post-score when calculating the changes.

I'm assuming that your a priori hypothesis is that individuals with greater self-efficacy are more resistant to outside influence and will therefore have lower change scores.

Not quite, our priori hypothesis is that individuals with greater self efficacy will be more susceptible to a specific type of message framing.

I will have a look at doing a moderation analysis on the change scores as you suggested. After a quick glance it does seem less complicated then a moderated multiple linear regression.

1