paperclipsThe position to date (July 2016); 

166 good datasets have now been received in total, thus forming the datapool.

The dataset generated from each eQNR reply has been received as a .csv file so in the first instance these have been collected together into a ‘master’ Excel spreadsheet.

SPSS will be used later for a more detailed interrogation of the data but for the moment, Excel has been an entirely satisfactory tool to enable me to reflect on the data as a whole and to execute an initial analysis which has provided interesting results, indeed, the results that had been hoped for:

 

Introduction

The research groups referred to in the analysis summary below have been labelled carefully so that I will know what I’ve done at this stage for when I come to analyse the data in more detail later and write it all up more thoroughly. However to make it easier to follow… :

In the eQNR, respondents either declared that they had dyslexia or they didn’t. Hence forming two primary groups:

Two subgroups from research group: ND have been established:

Another subgroup, from research group: DI, has also been established:

So in summary, where a research group has been created as a subgroup of either of the main research groups DI or ND, these are designated by the parent research group they’ve come from with a suffix indicating the Dyslexia Index boundary point that has been used to separate these respondents out. The exception to this is research group: DNI, the group of primary interest.

 

Now as reported in an earlier post, a fourth research group has emerged which represents a kind of overlap between the two primary research groups ND and DI and this is respondents who presented a Dyslexia Index, 400 < Dx < 600 which is interesting. However for the moment, inspecting data in this group has been parked although I will return to this later.

 

In analysing the stats, summarized below, where Student’s T-test has been used, the version applied has been to test independent means between groups assuming equal population variances. More work will be conducted later on additional tests that are part of this (for example, testing for equal population variances) and this will be done through SPSS when the data has been exported to that application.

Additionally, a one-tailed test has been used as I am interested in whether one ‘test’ statistic is greater than the other rather than just different (for which a two-tailed test would be appropriate).

 

Where effect size has been reported below, I am using at present, Cohen’s ‘d’ but of course even a raw difference between means is one measure of effect size, as is a correlation coefficient. All of this will be explored in more detail later and reported.

 

The Summary  Table of results so far is available here (opens in a new window) in which the rows of data in pale blue are the ones of greatest interest, that is, summary data for research groups: DNI and DI-600.

 

So in summary, this is what has emerged so far:

ABC_normals95CIs

Effect size results:

On the basis of these figures, Cohen’s ‘d’ effect sizes for the differences in mean ABC for the three research groups: DI-600, DNI and ND-400 have been calculated using using weighted means (degrees of freedom) pooled sample standard deviations (eg: Cumming, 2012, p156) as the calculating process which generated these results:

 

Student’s t-test results:

Even though I am converted to the measure of effect sizes and confidence intervals as more relevant when reporting research results in education and social sciences research, I have also run a conventional t-test to look for a significant difference between the 2 independent Academic Behavioural Confidence sample means of research groups DI and DNI.

For a quick result, I used one of the many online t-test calculators here: http://www.socscistatistics.com/tests/ which generated the results listed below and in the summary table at the end of this section.  I have been able to confirm these t-test results in my own Excel spreadsheet of the complete datapool using Excel’s built-in t-test function.

So it is also pleasing to record this as an additional indicator that the ABC mean value for research group: DNI is statistically significantly higher than the ABC mean value for research group: DI-600. Together with the effect size calculation this is beginning to indicate that students with an unknown dyslexic profile present a significantly higher Academic Behavioural Confidence than students with a known dyslexic profile. Cool 🙂

 

An alternative perspective on analysing these data:

perspectiveHowever, it occurred to me that although the result above looks pleasing, I am keen to avoid falling into the trap of just reporting the data analysis that appears to be providing the results hoped for. To this end, I felt that in order to justifiably conclude that there is a difference in ABC between research groups: DI and DNI I need to be more confident that in other respects, the students’ data results are broadly similar – at least in relation to the mean Dyslexia Index of each group.

At the moment, I have used a somewhat arbitrary discriminator for the boundary point Dx = 600 where a student whose Dyslexia Index is higher than this I am considering as pretty definitely presenting a dyslexic profile. So research group: DNI is comprised of students from research group: ND who present a Dx > 600 taken from this datapool where the Dyslexia Index ranges from a very undyslexic Dx = 100.16 to an astonishingly high value of Dx = 910.20.

As reported above, the mean Dx for this subgroup (DNI) is Dx = 690.8 so in order to be able to more properly compare these students’ mean ABC to the mean ABC for students in research group: DI-600 – the dyslexic control group – this group (DI-600) needs to present a mean Dx that is not significantly different/higher than the mean Dx for research group: DNI.  However for research group: DI-600, the mean Dx = 723.6 which at face value at least is certainly higher (cf: 690.8) but is it significantly higher as to therefore imply that the two groups are less than equal in terms of their mean Dyslexia Indicies?

To test this, I reverted to the independent means t-test as a quick way to establish whether there is a significant difference between the Dx means of research group: DI-600 and research group: DNI.

This leaves me a little uneasy, so I will explore the impact of adjusting the catchment for the subgroup of research group:DI-600 so that the mean Dx is not only NOT significantly different from the mean Dx for research group: DNI, but which also generates an effect size that is at most, ‘small’ (that is, Cohen’s d < 0.2 (Sullivan & Feinn, 2012)) and this will be established by adjusting the boundary value Dx and then using the t-test to determine if the significant difference between the means has been eradicated:

1st attempt:

For the first attempt, I have adjusted the boundary value to Dx = 500. That is, the new subgroup of research group: DI comprises datasets that present a Dx > 500.  This research subgroup is identified as DI-500.

So the effect size for ABC is still close to ‘medium’ when using this adjusted research subgroup: DI-500 as the ‘control’ group.

However, with a p-value in the t-test of 0.4444, one might say that this is very not significant at the 5% level and that an alternative boundary point might therefore be sought that produces a t-test p-value that is closer to the 0.05 critical point.

2nd attempt:

To try to establish this, the t-test has been applied repeatedly to the mean Dx for research group: DNI and a subgroup of research group: DI to try to find a more appropriate boundary value for Dx that is between Dx = 500 and Dx = 600 in research group: DI. A few trials indicated that Dx = 580 could be the cut-off point.

Will there be much impact on re-setting the boundary Dx value to Dx = 580 on the effect size analysis for Academic Behavioural Confidence?

 

Conclusion

We are left with the conclusion that the differences in effect size between using research subgroup DI-500 or research subgroup DI-580 in comparison to using research group: DI-600 data are very small.

My view is that this exercise in trying to get the best, that is, most statistically substantiated result has been very worthwhile and with results for the effect size working out to be broadly similar in all cases surely this adds weight to the robustness of the analysis process and which suggests that my original boundary Dx = 600 is a pretty good one. So at this stage, I am planning to leave it as this. I will be reporting this exercise in data-tinkering more fully in the final write-up as I am hoping that by doing this, I am demonstrating a good awareness of some of the peculiarities of stats analysis.

 

 

References:

Cumming, G., 2012, Understanding The New Statistics; effect sizes, confidence intervals and meta-analysis, Hove, Routledge.
Sullivan, G.M., Feinn, R., 2012, Using Effect Size – or why the ‘p’ value is not enough, Journal of Graduate Medical Education, 4(3), 279-281.

Leave a Reply