A CAT with Caveats: Is the Consensual Assessment Technique a Reliable Measure of Graphic Design Creativity?
Year: 2015
Editor: Amaresh Chakrabarti, Toshiharu Taura and Yukari Nagai
Author: Jeffries, K. K.
Institution: School of Art, Design & Performance, University of Central Lancashire, Preston, UK.
Section: Design Creativity Assessment and Capturing
Page(s): 003-010
ISBN: 978-1-904670-60-5
Abstract
The CAT is considered one of the gold standards for creativity assessment, and graphic design, arguably, the most ubiquitous domain within the "creative industries". For the first time, this study tests two tasks to measure graphic design creativity, and by extension, the reliability of the CAT as a measure of graphic design creativity. Initial research suggested low inter-rater reliability may be unduly influenced by a judge’s preference for technical execution. Sixteen professional graphic designers were randomly assigned instructions to discount technical execution from creativity ratings, or instruction that gave no stipulation, for 60 artworks. Inter-rater reliability were acceptable for each task and experimental condition, but were higher for judges that received instructions to discount technical execution. These and other results are discussed, and the argument presented that, for future CAT studies in this domain, specific instructions to discount technical execution offers a more reliable measure of graphic design creativity.
Keywords: Consensual Assessment Technique, Graphic Design, Design Creativity Assessment