reversecodingThe eQNR that has collected my data contains 8 Likert Scales which each comprise collections of stem statements.

I’ve tried hard to word the stem statements in such a way to minimize an overall sense that the questionnaire takes either a generally negative or generally positive sense.  My questionnaire asked respondents to provide a level of agreement to stem-and-leaf statements grouped to form the Likert scales.

For example:

… against which respondents moved a slider control on the questionnaire form in a range 0-100% agreement to indicate their reply.

The questionnaire contained statements that necessarily had either negative or positive sense to them so that respondents were able to judge their agreement level.

So it has been vital to devise a means to adjust the numerical values that answers generated in order that negative answers didn’t cancel out positive ones in each of the 8 Likert Scales.

I’ve achieved this using the simple process of reverse coding and this StudyBlog post is to record this data adjustment event.

 

Academic Behavioural Confidence Scale

This metric has been used completely ‘as standard’ and in its original, 24-item format originally developed by Sander and Sanders (2006).  The stem-and-leaf statements are neutrally worded (in my view) and therefore none of the data collected has been adjusted. In a later review of the ABC Scale factor analysis was applied to establish a reduced, 17-item scale (Sander and Sanders, 2009) in which the statements were grouped into four factors (grades, verbalising, studying and attendance) and I am glad that I used the original, 24-point scale since by excluding the 7 items dispensed with by Sander and Sanders as not contributing to the four factors they identified, I have data to which I can apply the discriminative power of the factor structure as part of my analysis if I so wish. More about this later.

 

Pscyho-educational (PE) constructs – my 6 Likert Scales

The 6 PE constructs that formed the middle section of my questionnaire each comprise 6 Likert items on each of the 6 Likert Scales.

In each scale, that is, for each construct, I wanted to attribute a polarity for the complete scale. So I tried to ensure that all the statements of the scale had either a positive or a negative sense or there was either a balance of positively- and negatively-worded statements for respondents to agree/disagree with.  Where a scale’s statements were mixed, I attributed the polarity of the complete scale and then reverse coded the data items for which the statement’s polarity was opposite to that for the complete scale.

The full list is presented below (again) in the order in which they appeared in the QNR.

+/- bias  RC? scale [polarity]  / statement
 Learning Related Emotions (LRE) [scale polarity set to -ve,  => reverse code +ve statements]
 +  RC  I am able to settle down to my work anytime, anyplace
 –  I feel too embarrassed to ask for help with my studies
 –  I feel guilty about my learning challenges
 –  I think my student-peers mostly regard my learning challenges as excuses, for laziness for example
 –  I don’t use any of the learning support services because it makes me feel different
 +  RC  I don’t think about my learning challenges much
***
 Anxiety Regulation & Motivation (ARM) [scale polarity set to +ve, => reverse code -ve statements]
 – RC  I find it quite difficult to concentrate on my work most of the time
 +  I don’t think my learning challenges make me any more anxious than anyone else
 +  I use my learning strengths to help me with study strategies
 –  RC  I need to work much harder than my friends to get similar grades
 –  RC  I often feel frustrated when trying to study
 +  I enjoy my studies even more when the work becomes difficult
***
 Academic self-efficacy (ASE) [scale polarity set to + ve]
 +  I believe that my learning strengths really make a difference to my academic progress
 +  I plan and organize my work carefully which I believe helps me to get good grades
 +  I don’t think my learning challenges make any difference to the way I tackle my work
 +  I approach my written work with a high expectation of success
 +  I believe my learning strengths help me to be more creative or innovative
 +  I can manage my studies quite adequately without any help
***
 Self-esteem (SE) [scale polarity set to +ve, => reverse code -ve statements]
 –  RC  I often felt pretty stupid at school
 +  If I try hard, I can achieve just as much as anyone else
 +  I think I’m good at studying, perhaps even academically talented sometimes
 +  I approach my written work with enthusiasm
 –  RC  At times, I think that I’m just hopeless at tackling academic work
 –  RC  My contributions in class are usually rubbish, so generally I don’t bother
***
 Learned Helplessness (LH) [scale polarity set to -ve]
 –  When I start a new course or topic I usually think it will be too difficult for me
 –  I’ve had help for dealing with my learning challenges but it hasn’t made any difference
 –  I’m generally not surprised when I get a low grade
 –  I will always be held back by my learning challenges
 –  I think that my grades are as much to do with luck as with any effort on my part
 –  However hard I try, this rarely makes a difference to my grades
***
 Academic Procrastination (AP) [scale polarity set to -ve, => reverse code +ve statements]
 +  RC  I usually finish my essays or assignments well in time for the deadline
 –  I generally put off getting started on my essays or assignments until I really have to
 –  For one reason or another, I often have to request extra time to complete my work
 +  RC  As soon as I’m given an essay or assignment title, I’m usually eager to get going on it straight away
 –  My essays or assignments would probably be better if I didn’t have to rush to finish them
 –  I often find other things to do rather than working on my studies

 

Dyslexia Index (Dx)

This metric has been devised and developed partly by reviewing dyslexia self-identifying evaluators such as the BDA’s Adult Checklist developed by Smythe and Everatt (2001), the original Adult Dyslexia Checklist proposed by Vinegrad (1994) upon which many subsequent checklists appear to be based, and the much later, York Adult Assessment (Warmington et al, 2012) which has a specific focus as a screening tool for dyslexia in adults. Also consulted and adapted has been work by Burden particularly the ‘Myself as a Learner Scale’ (Burden, 2000) which helped in designing the eQNR deployed in the pilot study to this PhD research project and which constituted the dissertation part of my own MSc (Dykes, 2008).

However I feel uneasy about most of the existing screening tools used in HE contexts being either totally or significantly focused on the literacy-deficit aspects of dyslexia because these appear to be neglecting ways to gauge many of the study skills and academic competencies, strengths and weaknesses of students with dyslexia that co-exist with literacy based deficits, a concern shared by many educators working face-to-face with university students (eg: Chanock et al, 2010).

So I developed two preliminary enquiries that sought to find out more about how practitioners supporting and working with students with dyslexia in UK universities firstly adopted some kind of working definition of dyslexia that they felt comfortable with but more significantly, about the prevalence of attributes and characteristics associated with dyslexia that were encountered  by these practitioners in their direct interactions with dyslexic students at university on a day-to-day basis.

I’ve used the results and analysis of the data that these enquiries generated to combine with everything I’ve learned about existing identifying and screening tools to develop my own identifying evaluator, the Dyslexia Index Scale, this then constituting the 8th metric that my main research questionnaire aimed to measure and as reported in an earlier post and described elsewhere in these project webpages.

This final metric measuring Dyslexia Index (Dx) comprised 20 statements following the same design as others in the QNR such that respondents used a slider to record their level of agreement with each of the statements.  For each statement I was expecting either a high score, which indicates strong agreement, or a low score, indicating strong disagreement, to be a marker of a dyslexic profile.  Since the scale is designed to provide a numerical indicator of dyslexia it seems appropriate to aggregate scores in such a way that a high total score points towards a strong dyslexic profile.  As such, I had been planning to reverse code scores for some statements so that the overall calculation to the final Dyslexia Index would not be upset by high and low scores cancelling each other out where a high score for one statement and a low score for a different statement were both indicating a dyslexic profile.  Below is the complete list of 20 statements showing whether I was expecting a ‘high score=strong agreement (H)’ or a ‘low score=strong disagreement (L)’  to be the dyslexic marker.  Also shown is the weighting (w) that I attributed to each score for aggregating them into the Dyslexia Index.  I established these weightings from the prevalence of attributes of dyslexia that my earlier preliminary enquiry had revealed. The highest weightings are an indication of the highest prevalence of that attribute. This also seemed an appropriate way to adjust the aggregate score for each respondent rather than simply take the sum of all the scores (and reverse-coded scores).

However, for the statement: ‘my spelling is generally good‘ where it is widely acknowledged that individuals with dyslexia tend to be poor spellers, I was hence expecting a low score – indicating strong disagreement to be the marker for dyslexia – the picture was less clear for many of the other statements once the data had been collated and tabulated. To explore this further, I ran a Pearson Product-Moment Correlation to calculate values for r, for each statement with the final aggregated Dyslexia Index.  These correlation coefficients are also presented in the table below and I used this to confirm or not whether to reverse-code a statement or not. The criteria I used was this: if my expectation is to reverse-code a statement’s data and this is supported by a strong negative correlation coefficient, hence indicating that Dyslexia Index is negatively correlated with that statement then I proceeded to the reverse-coding process. If the correlation coefficient indicates anything else – that is ranging from weak negative to strong positive – I would leave the data as it is.

So in summary, the table below indicates each of the stem statements for the metric Dyslexia Index (Dx) together with:

w  statement  H / L  r (RG:DI)  r (RG:ND)  RC ?
 0.80  When I was learning to read at school, I often felt I was slower than others in my class  H  0.51  0.73  –
 0.53  My spelling is generally very good  L  – 0.52 – 0.38  RC
 0.70  I find it very challenging to manage my time efficiently  H  0.13  0.30  –
 0.71  I can explain things to people much more easily verbally than in my writing  H  0.61  0.41  –
 0.43  I think I am a highly organized learner  L  – 0.09  0.09  –
 0.48  In my writing I frequently use the wrong word for my intended meaning  H  0.68  0.73  –
 0.64  I generally remember appointments and arrive on time  L  0.20  – 0.10  –
 0.75  When I’m reading, I sometimes read the same line again or miss out a line altogether  H  0.41  0.71  –
 0.76  I have difficulty putting my writing ideas into a sensible order  H  0.52  0.74  –
 0.80  In my writing at school, I often mixed up similar letters like ‘b’ and ‘d’ or ‘p’ and ‘q’  H  0.61  0.60  –
 0.57  When I’m planning my work I use diagrams or mindmaps rather than lists or bullet points  neutral  0.49  0.55  –
 0.75  I’m hopeless at remembering things like telephone numbers  H  0.40  0.53  –
 0.48  I find following directions to get to places quite straightforward  L  -0.05  0.19  –
 0.57  I prefer looking at the ‘big picture’ rather than focusing on the details  neutral  0.22  0.27  –
 0.63  My friends say I often think in unusual or creative ways to solve problems  H  0.19  0.45  –
 0.52  I find it really challenging to make sense of a list of instructions  H  0.49  0.63  –
 0.52  I get my ‘lefts’ and ‘rights’ easily mixed up  H  0.38  0.44  –
 0.70  My tutors often tell me that my essays or assignments are confusing to read  H  0.36  0.69  –
 0.64  I get in a muddle when I’m searching for learning resources or information  H  0.58  0.71  –
 0.72  I get really anxious if I’m asked to read ‘out loud’  H  0.35  0.60  –

It can be seen from the summary table that the only stem statement that I eventually reverse-coded was ‘my spelling is generally very good’ as this was the only one that presented not only a high(ish) negative correlation with Dx of r = -0.52 in the ‘control’ research group: DI, but also correlated negatively against Dx in research group: ND, albeit not so strongly.

This was quite surprising as I was planning to reverse-code data for all statements where I had expected a low score to be a marker for dyslexia. However what is also very interesting is that for the remaining statements once ‘spelling…‘ is dealt with, the value of r for each statement is higher (that is, more positive) for research group: ND than for research group: DI with the exception of the statement ‘I can explain things to people much more easily verbally than in my writing‘.

What is this telling me? First of all, I might be able to argue that this is adding robustness to my Dyslexia Index as a indicator for dyslexia amongst respondents in research group: ND because respondents in this group presented a much wider range of Dx when compared with research group: DI – as we would expect because we know that respondents in research group: DI have dyslexia.

Secondly, by picking out statements where r is medium to high for research group: DI, r is higher, thus indicating an even stronger correlation with Dx, for respondents in research group: ND.  Let us think a bit about this group of statements:

w  statement  H / L  r (RG:DI)  r (RG:ND)  RC ?
 0.80  When I was learning to read at school, I often felt I was slower than others in my class  H  0.51  0.73  –
 0.53  My spelling is generally very good  L  – 0.52 – 0.38  RC
 0.70  I find it very challenging to manage my time efficiently  H  0.13  0.30  –
 0.71  I can explain things to people much more easily verbally than in my writing  H  0.61  0.41  –
 0.43  I think I am a highly organized learner  L  – 0.09  0.09  –
 0.48  In my writing I frequently use the wrong word for my intended meaning  H  0.68  0.73  –
 0.64  I generally remember appointments and arrive on time  L  0.20  – 0.10  –
 0.75  When I’m reading, I sometimes read the same line again or miss out a line altogether  H  0.41  0.71  –
 0.76  I have difficulty putting my writing ideas into a sensible order  H  0.52  0.74  –
 0.80  In my writing at school, I often mixed up similar letters like ‘b’ and ‘d’ or ‘p’ and ‘q’  H  0.61  0.60  –
 0.57  When I’m planning my work I use diagrams or mindmaps rather than lists or bullet points  neutral  0.49  0.55  –
 0.75  I’m hopeless at remembering things like telephone numbers  H  0.40  0.53  –
 0.48  I find following directions to get to places quite straightforward  L  -0.05  0.19  –
 0.57  I prefer looking at the ‘big picture’ rather than focusing on the details  neutral  0.22  0.27  –
 0.63  My friends say I often think in unusual or creative ways to solve problems  H  0.19  0.45  –
 0.52  I find it really challenging to make sense of a list of instructions  H  0.49  0.63  –
 0.52  I get my ‘lefts’ and ‘rights’ easily mixed up  H  0.38  0.44  –
 0.70  My tutors often tell me that my essays or assignments are confusing to read  H  0.36  0.69  –
 0.64  I get in a muddle when I’m searching for learning resources or information  H  0.58  0.71  –
 0.72  I get really anxious if I’m asked to read ‘out loud’  H  0.35  0.60  –

With the exception of slowness in learning to read at school, all these statements are related to core academic competencies at university, whether these be related to reading and writing or related to information organization skills. It seems more than possible that the reason for the lower correlation coefficients between these statements and Dx for students in research group: DI is that these students are probably receiving study-skills support through their disclosure of dyslexia whilst students in research group: ND may not. We could argue that for dyslexic students, the impact that deficiencies in these academic competencies have on their university progress is reduced due to the compensatory support they receive and hence these deficiencies are to some extent ameliorated.  Hence this may be leading these students to perceive themselves to be ‘fixed’, to some extent at least, whilst non-dyslexic students who are in fact presenting a dyslexic profile are still struggling on their own. A pertinent study by Mortimore and Crozier (2006) widely reports, both from previous studies and from their own primary research, that the difficulties encountered in studying at university by students with dyslexia are diverse and much more extensive than just challenges with literacy – both in reading complex university-level texts and in developing their academic writing maturely enough to enable them to properly express their ideas and communicate their knowledge.  I will write more about this in a future StudyBlog post.

For the time being and in summary, this post documents and diarizes the process that I’ve devised and utilized to make the data I’ve collected ready for statistical analysis. More will be written about this as the analysis process proceeds.

 

Update:  19th September 2016

As part of the preparation for transfer from MPhil to PhD, scheduled for next month, I am ‘writing up’ the process that has led to the development of my Dyslexia Index (Dx).

This has caused me to inspect the data again, rightly so, and tinker with it to try to ensure that I’ve considered everything when it comes down to finally and properly analysing the complete datapool.

Since the Dx values are the differentiators for the research groups, it is very important to make sure that the process I’ve devised for calculating the Dx values make sense and are statistically robust, not the least so that this can stand the scrutiny of defending my PhD later!

To summarize the calculating process to date:

However, it occurred to me that item statements which appeared to be showing very low correlations with Dx are quite likely to be making little contribution to the overall Dx value, indeed may be working to conflate the final value in an unhelpful way that makes Dx a less reliable indicator from which the research groups are determined.

To resolve this, I ran a Student’s t-test for independent population means for all 20 Dx item statements to compare means between the two, base research groups  – that is the complete research group: ND and the complete research group: DI.

The rationale for this is that for statements that are likely to be ‘good’ indicators of dyslexia, I would expect the t-test to reveal a statistically significant difference between the population means and for statements that are not good discriminators, I would expect the difference between the population means to be not significant.

The table below sets out the results. These were calculated using the t-test function in SPSS, and the figures quoted are for equal variances assumed, 164 degrees of freedom (because I had 166 good datasets overall) and present the two-tail result, i.e. reporting just whether there is a difference between the means rather than whether one is statistically significantly higher (or lower) than the other (which would be a one-tail test).

As can be seen, this analysis highlights four item statements where there isn’t a statistically significant difference between the means. This enables me to consider that these four item statements are making little or no contribution to the Dx calculation and therefore, their values might be omitted.

item #  item statement value means (100 max) t-test results two-tail test
RG:ND RG:DI t p stat significance
 3.01  When I was learning to read at school, I often felt I was slower than others in my class 39.96 68.69 -6.762 0.000 ridiculously significant
3.02  My spelling is generally very good 64.20 32.81 -6.681 0.000 ridiculously significant
3.03  I find it very challenging to manage my time efficiently 60.02 65.43 -1.1592 0.113 not significant
3.04  I can explain things to people much more easily verbally than in my writing 54.49 73.38 -5.268 0.000 ridiculously significant
3.05  I think I am a highly organized learner 46.59 44.13 -0.363 0.717 not significant
3.06  In my writing I frequently use the wrong word for my intended meaning 40.93 66.40 -8.234 0.000 ridiculously significant
3.07  I generally remember appointments and arrive on time 68.27 64.59 0.816 0.416 not significant
3.08  When I’m reading, I sometimes read the same line again or miss out a line altogether 53.65 83.72 -8.793 0.000 ridiculously significant
3.09  I have difficulty putting my writing ideas into a sensible order 50.85 80.00 -8.704 0.000 ridiculously significant
3.10  In my writing at school, I often mixed up similar letters like ‘b’ and ‘d’ or ‘p’ and ‘q’ 18.81 53.54 -8.803 0.000 ridiculously significant
3.11  When I’m planning my work I use diagrams or mindmaps rather than lists or bullet points 37.48 51.49 -4.516 0.000 ridiculously significant
3.12  I’m hopeless at remembering things like telephone numbers 43.99 63.09 -4.704 0.000 ridiculously significant
3.13  I find following directions to get to places quite straightforward 56.16 46.13 0.488 0.626 not significant
3.14  I prefer looking at the ‘big picture’ rather than focusing on the details 53.80 64.40 -3.545 0.001 highly significant
3.15  My friends say I often think in unusual or creative ways to solve problems 53.89 72.46 -6.164 0.000 ridiculously significant
3.16  I find it really challenging to make sense of a list of instructions 37.52 51.76 -4.886 0.000 ridiculously significant
3.17  I get my ‘lefts’ and ‘rights’ easily mixed up 35.59 64.99 -6.129 0.000 ridiculously significant
3.18  My tutors often tell me that my essays or assignments are confusing to read 33.47 57.10 -7.968 0.000 ridiculously significant
3.19  I get in a muddle when I’m searching for learning resources or information 44.07 65.81 -6.971 0.000 ridiculously significant
3.20  I get really anxious if I’m asked to read ‘out loud’ 44.83 77.40 -7.196 0.000 ridiculously significant

It is of note that the same, four item statements show a negligible correlation with Dx for both base research groups: ND and DI, as highlighted in a reprint of that table, below.

item #  statement  H / L  r (RG:DI)  r (RG:ND)  RC ?
 3.01  When I was learning to read at school, I often felt I was slower than others in my class  H  0.51  0.73  –
3.02  My spelling is generally very good  L  – 0.52 – 0.38  RC
 3.03  I find it very challenging to manage my time efficiently  H  0.13  0.30  –
 3.04  I can explain things to people much more easily verbally than in my writing  H  0.61  0.41  –
 3.05  I think I am a highly organized learner  L  – 0.09  0.09  –
 3.06  In my writing I frequently use the wrong word for my intended meaning  H  0.68  0.73  –
 3.07  I generally remember appointments and arrive on time  L  0.20  – 0.10  –
3.08  When I’m reading, I sometimes read the same line again or miss out a line altogether  H  0.41  0.71  –
 3.09  I have difficulty putting my writing ideas into a sensible order  H  0.52  0.74  –
 3.10  In my writing at school, I often mixed up similar letters like ‘b’ and ‘d’ or ‘p’ and ‘q’  H  0.61  0.60  –
 3.11  When I’m planning my work I use diagrams or mindmaps rather than lists or bullet points  neutral  0.49  0.55  –
 3.12  I’m hopeless at remembering things like telephone numbers  H  0.40  0.53  –
 3.13  I find following directions to get to places quite straightforward  L  -0.05  0.19  –
 3.14  I prefer looking at the ‘big picture’ rather than focusing on the details  neutral  0.22  0.27  –
 3.15  My friends say I often think in unusual or creative ways to solve problems  H  0.19  0.45  –
 3.16  I find it really challenging to make sense of a list of instructions  H  0.49  0.63  –
 3.17  I get my ‘lefts’ and ‘rights’ easily mixed up  H  0.38  0.44  –
 3.18  My tutors often tell me that my essays or assignments are confusing to read  H  0.36  0.69  –
 3.19  I get in a muddle when I’m searching for learning resources or information  H  0.58  0.71  –
 3.20  I get really anxious if I’m asked to read ‘out loud’  H  0.35  0.60  –

So I am left with the conclusion that statement items #3.03, #3.05, #3.07 and #3.13 make little or no contribution to the weighted mean calculation for Dyslexia Index (Dx). With this in mind, I have adjusted my master Excel spreadsheet to generate a fresh column for Dyslexia Index calculated using the weighted mean average of the remaining 16 item statements.  Although this resulted in some minor jiggling of values and hence boundary points for establishing the two most interesting research groups: DNI and DI-600, the relatively small impact that this has had is reflected in the most up-to-date summary table below.  It can be seen that there remains a moderate but significant effect size difference in ABC between research groups:DNI and DI-600.

 

References
Burden, R., 2000, Myself as a Learner Scale, Windosr, NfER-Nelson.
Dykes, A., 2008, A small-scale study of feelings about dyslexia in relation to the uptake of specific learning support amongst students with an identified dyslexic learning difference in an HE institution. unpublished MSc dissertation, University of Southampton.
Chanock, K, Farchione, D., Paulusz, W., Freeman, S., Lo Giudice, L., 2010, In search of a simple assessment instrument for identifying dyslexia in university students, Australian Journal of Learning Difficulties, 15(1) 35-49.
Hatcher, J., Snowling, M.J., Griffiths, Y.M., 2002, Cognitive assessment of dyslexic students in higher education. British Journal of Educational Psychology, 72(1) 119-133.
Mortimore, T., Crozier, W.R., 2006, Dyslexia and difficulties with study skills in higher education. Studies in Higher Education, 31(2), 235-251.
Sander, P., Sanders, L., 2006, Understanding academic confidence, Psychology Teaching Review, 12(1), 29-42.
Sander, P., Sanders, L., 2009, Measuring academic behavioural confidence: the ABC scale revisited. Studies in Higher Education, 34(1), 19-35.
Vinegrad, M., 1994, A revised adult dyslexia checklist, Educare, 48.
Warmington, M., Stothard, S.E., Snowling, M.J., 2013, Assessing dyslexia in higher education: the York adult assessment battery revised, Journal of Research in Special Educational Needs, 13(1) 48-56.

Leave a Reply