Recent Developments in the Cluttering Severity Instrument (CSI)

About the presenters: Klaas Bakker, associated with Missouri State University since 1990; specializes in fluency disorders; research focus on fluency disorders (assessment and diagnostic evaluation of cluttering and stuttering); develops new technologies for the assessment and measurement of clinical aspects of speech (dys)fluency; Associate Editor for the Journal of Fluency Disorders; Chair of the Taskforce on Technology of Special Interest Division 4 (Fluency and Fluency Disorders).

Florence L. Myers has presented a number of papers on fluency disorders. She has (co)authored books, book chapters and journal articles on cluttering. She is a former Associate Editor of the Journal of Fluency Disorders. She is Professor at Adelphi University in Garden City, New York.

Recent Developments in the Cluttering Severity Instrument (CSI)

by Klaas Bakker (Missouri, USA) and Florence Myers (New York, USA)

INTRODUCTION

Most specializations in communication disorders have established severity measures. This is not the case for cluttering. The need for a cluttering severity measure has been repeatedly referred to in recent years (Bakker, 2007; Bakker, St. Louis, Myers & Raphael, 2005 a, b). Not being able to measure cluttering is very limiting. Assessment of cluttering severity should be an integral part of therapy and is essential for most basic and clinical research on cluttering. In this online presentation we will describe how cluttering severity may be measured in these contexts, and discuss some current issues regarding the development of the Cluttering Severity Instrument.

To our knowledge there are no cluttering severity measures that are available for widespread clinical use. It is not known how most clinicians establish (1) if a client needs treatment, (2) if treatment is effective, and (3) the determination of when treatment is not needed anymore. Our suspicion is that these determinations occur through subjective means that may vary significantly among individual clinicians.

Severity measures are also lacking in much basic and clinical research regarding cluttering. For example, any attempt to find out more about how a variable of interest is related to cluttering, or how effective a cluttering intervention is, requires the ability to measure cluttering.

Without the ability to measure cluttering we are confined to opinion-based descriptions. As a result, conclusions may not meet typically expected levels of scientific rigor. Of course, a careful clinical description does contribute to scientific progress, as this has been the predominant methodology that brought cluttering to our attention to begin with. In follow-up research, however, we need to represent cluttering in a measurable form.

Purpose

The purpose of this presentation is to demonstrate how cluttering severity may be quantified and show the status of development of the components of our system. The case was made that this development is supportive of clinical practice as well as most forms of basic and clinical research. Because of the quantitative aspects of our system it has potential to be evaluated with regard to validity and reliability.

The Cluttering Severity Instrument (CSI) contains the essential cluttering features such as described in one popular definition of cluttering (St. Louis, Myers, Bakker & Raphael, 2007); moreover, the CSI model expands this range of clinical features by including others that are commonly considered descriptive for many individuals who clutter but are not by themselves considered obligatory in the aforementioned definition (Daly & Burnett, 1999; Ward, in press). Finally, the assessment framework proposed by us is meant to be a "work in progress." Much about cluttering is unknown at this time. Our system, in order to stay current will need to have the ability to grow with future developments and insights.

Online presentation of this material with an associated threaded discussion forum presents the unique option for all those interested in cluttering worldwide to participate in exchanging ideas and further refinement of the assumptions basic to cluttering severity assessment. It is our intention to take note of such contributions and allow the ultimate system to be shaped after these global scientific roots.

Rationale of the Cluttering Severity Instrument

Cluttering severity should be based on perceptually driven strategies rather than objective quantifiable approaches that treat component behaviors such as disfluencies as discrete and isolated phenomena that can be dissociated with other parameters of speech and language (such as rate, speech or language organization). There are no such things as "clutterings." Cluttering, rather, is characterized by someone's perception that a speech system goes out of control in a unique way that deserves this diagnostic classification.

A unique aspect of cluttering is its multifaceted nature. The diagnosis of cluttering rests on observed compromises in speech fluency, precision of articulation, speech rate and so forth. This is a first step toward achieving face and content validity in a severity measurement instrument. That is, we need to identify the facets of communication that are affected during cluttering and need to quantify the degree to which these facets are impaired.

In fact, the study of speech fluency itself presents similar dilemmas. Finn (2007) articulated some of these dilemmas, as related to the definition of fluency, the criteria for what would be considered "normal fluency," and how to apply these criteria for our study of stuttering and its treatment. We feel an urgency to develop a cluttering assessment procedure at this time, fully acknowledging that we need to continue to conduct research in the area, for two major reasons. First, changes in how cluttering is conceived are slow, and yet we need to support this process. Secondly, the need for treating cluttering--and measuring treatment efficacy--is very strong today. The fact is that clinicians are now treating cluttering with minimal means to measure it. Van Zaalen, Wijnen and DeJonckere (2009), for example, found only fifty percent agreement between two SLPs who specialized in fluency disorders in the identification of disfluency cases based on subjective clinical judgments without the use of perceptual scales or more defining objective data such as "ratio of disfluencies and accuracy and speech flow error scores." Our proposed system represents a "snapshot" or "prospectus" for an ultimate assessment protocol that is user-friendly and captures severity of the various dimensions of communication that are compromised during cluttered segments.

Development of the Cluttering Severity Instrument

Our proposed cluttering severity instrument has a precursor: "Cluttering Assessment Program", or CLASP 2.0, which is explained in another paper on this conference and is available as a download from Bakker et al (2005a); and in slightly improved form from the website of the International Cluttering Association (see Note 1). This computer program contains tools to (1) determine % talking time cluttered, and (2) provide qualitative ratings thought to be descriptive for the severity of a cluttering problem. The CSI is perhaps the first-ever attempt to systematically capture the various dimensions of communication (including aspects of discourse management) often implicated in cluttering.

Although it is technically possible to take all CSI measures completely by hand (using a stopwatch and a paper and pencil rating instrument), it is considerably more efficient to use a computerized version, with very likely added benefits in terms of accuracy and reliability. Nevertheless, the option to conduct the measures by hand may continue to be needed in some situations.

Research evidence for the % talking time component

While there is ample evidence that clinicians can reliably mark "talking time" by hand (e. g., Bakker, 2007) the difficulty to assess "talking time" and "cluttering intervals" simultaneously led to a new approach in which "talking time" does not have to be scored by hand. Software determines talking time automatically from digital recordings allowing a clinician to concentrate on the cluttering identification process.

The fact that cluttering identification by itself is difficult was demonstrated in preliminary research by Palmer (2009). While the reliability levels of overall cluttering time estimates proved reasonable, when scrutinizing the actual individual cluttering interval identification process across participants it became evident that substantial inter-individual differences existed. It became evident that for clinicians to make these selections accurately and consistently there needed to be the option for editing cluttering interval selections. As a result, a cluttering interval editor is now being developed.

There may be other reasons why cluttering interval identification is difficult for clinicians. After all, the presence of cluttering in our system is based on perception, which in turn relies on the individual's conception and understanding of what cluttering is. It is reasonable to assume that improved (and commonly agreed) definitions should be instrumental in improving the cluttering interval selection process. Moreover, it may be desirable to develop a standardized training system for clinicians to calibrate their cluttering interval selection skills.

Research evidence for the behavioral ratings

Research toward the validity of the chosen rating scales in the present assessment tool will go hand-in-hand with continued research toward the validity of current definitions as well as approaches to cluttering assessment. Myers and St. Louis (2006) found perceptual judgments and disfluency counts for a participant with severe cluttering to be moderately correlated (r = .66); however, the correlation between perceptual judgment of rate and syllables per minute was only r = .20. A later study (Darby, 2010) determined that perceptual ratings of speech rate and several methods for measuring speech rate in a physical sense had lower than expected correlations when listening to normally fluent speakers (for example, SPM and perceptual ratings correlated at r=.69, df=10). Perhaps "rate," as a supra-segmental, is a phenomenon less easily captured by discrete measures such as syllables per minute, and perhaps this is even more problematic in cluttered speech. In researching the validity of the Predictive Cluttering Inventory (Daly, 1996) using factor and cluster analyses, Van Zaalen, Wijnen and DeJonckere (2009) concluded that there are two major clusters of behaviors to be considered: speech planning and a language component. These findings may contrast with the most current definition offered by St.Louis and colleagues (St. Louis and Schulte, in press). They do support consideration in the development of a cluttering assessment tool. Clearly the field has just begun its investigations on the behaviors and behavioral ratings of cluttering.

Also needed is research into the potential effects of observation training for improving the clinical observational skills for clinical signs of cluttering (to improve inter- and intra-rater reliability) and improving the technology for collecting observer ratings (e. g., the construction of the rating scales, and our added option for marking "ranges" rather than "point estimates").

The qualitative ratings component:

To allow for qualitative communication behavior ratings in the cluttering severity instrument, the Visual Analog Scale (VAS) procedure was chosen. This technique is now widely used in our discipline and has as an advantage that it produces results with more detailed distinction than is possible, for example, with Likert type rating scales. As the qualitative judgments are made based on entire speech samples, and most of the characteristics are likely to be variably present, a special type of Visual Analog Scoring scale was designed. That is, our VAS scales include the option to express the range of how a clinical characteristic is observed to vary throughout the speech sample. It will then automatically determine the midpoint of this range (as the center value for this characteristic). A link is provided below that allows the reader to download (Note 2) a prototype program for this type of scoring.

Conclusions

Use of % talking time cluttered measure needs a cluttering interval editor to refine selections. Probably there should be a training process with available samples for practicing and calibrating one's observation skills (for accurate and reliable scoring). Work will be needed also to refine the descriptions of what should be considered cluttering even though this is expected to remain a subjective determination for the time being.
The upgrade of the qualitative rating scales needs to be empirically tested. The ability to mark both a center value and range is new and has not been tested. Traditional Visual Analog Scales, while popular, have not been tested either and certainly their use for describing the clinical features of cluttering is in need of supportive clinical scientific evidence. The availability of our prototype of the scoring instrument should make it possible to test its use; nonetheless its clinical use should be considered guarded until supportive evidence becomes available. The availability of our prototype to quantify cluttering severity is a step in the right direction; however, this is a work in progress.

DISCUSSION POINTS (Use threaded discussion to participate):

What are arguments for or against emphasizing perceptual/rating procedures for cluttering severity assessments?
Is it acceptable to include characteristics that according to definitions may not be present in all those considered PWC (e.g., language and pragmatic functions)?
Is it acceptable to develop a measure of cluttering severity while the specialization hasn't completely settled on a specifically defined diagnostic classification?

NOTES

NOTE 1. The latest version of the CLASP 2.02 software may be downloaded from the website of the International Cluttering Association (http://associations.missouristate.edu/ICA.)

NOTE 2. A prototype for the perceptual qualitative ratings component of the Cluttering Severity Instrument will be available by the time this paper appears online from the website of the International Cluttering Association (http://associations.missouristate.edu/ICA.)

References

Bakker, K., (2007). Objectifying cluttering severity. Presentation at the First World Conference on Cluttering, Katarino, Bulgaria.

Bakker, K., St Louis, K. O., Myers, F., and Raphael, L., (2005a). Computer-Aided Assessment of Cluttering. The Eighth International Stuttering Awareness Day Conference, Online Conference: link to paper: http://www.mnsu.edu/comdis/isad8/papers/bakker8/bakker8.html

Bakker, K., St Louis, K. O., Myers, F., and Raphael, L., (2005b). A freeware software tool for determining aspects of cluttering severity. Annual National Convention of the American Speech Language and Hearing Association, San Diego, CA, USA.

Daly, D. A., & Burnett, M. (1999). Cluttering: Traditional views and new perspectives. In R. F. Curlee (Ed.). Stuttering and related disorders of fluency, 2nd ed. (pp. 222-254). New York: Thieme Medical Publishers.

Darby, M., (2010). A Comparative Study of the Validity of Different Types of Speech Rate Measures. Unpublished Master's Project, Missouri State University, Springfield, MO, USA.

Finn, P. (2007). Defining and measuring normal fluency. Perspectives on Fluency and Fluency Disorders. Vol. 17(2). ASHA Division 4. pp. 14-17.

Myers, F. L., & St. Louis, K. O. (2006). Disfluency and speaking rate in cluttering: Perceptual judgments versus counts. Bulgarian Journal of Communication Disorders, 1, 28-35.

Palmer, T., (2009). Reliability for durational measurement of cluttered speech intervals. Unpublished Master's Thesis, Missouri State University, Springfield, MO.

St. Louis, K. O., Myers, F. M., Bakker, K., & Raphael, L. J. (2007). Understanding and treating cluttering. In R. F. Curlee & E. G. Conture (Eds.) Stuttering and related disorders of fluency, 3rd ed. NY, USA: Thieme.

St. Louis, K. O. & Schulte, K. (in press). Defining cluttering: The lowest common denominator. In D. Ward & K. S. Scott (Eds.). Cluttering: Research, intervention, education, East Sussex, UK: Psychology Press.

Van Zaalen, Y., Wijnen, F., & De Jonckere, P. H. (2009). Differential diagnositic characteristics between cluttering and stuttering. Journal of Fluency Disorders, 34(3). 137-154.

Ward, D. (2007). Stuttering and normal nonfluency: Cluttering spectrum behaviour as a functional descriptor of abnormal nonfluency. Presentation at the First World Conference on Cluttering, Katarino, Bulgaria.

March 5, 2010