Join Us Sign In

Knowledge Base Repository

In addition to research papers, the Design Society is developing several valuable resources for those interested in the study of design. These include a repository of PhD theses, a library of case studies and transcripts of design activities, and an archive of our newsletters. Please note that these resources are accessible exclusively to Design Society members.

How good is ChatGPT? An exploratory study on ChatGPT’s performance in engineering design tasks and subjective decision-making

Wanyu Xu, Maulik Chhabilkumar Kotecha, Daniel A. McAdams

Type:
Papers
Proceedings of the DESIGN2024 18th International Design Conference
Year:
2024
Editor:
Mario Štorga, Stanko Škec, Tomislav Martinec, Dorian Marjanović, Neven Pavković, Marija Majda Škec
Author:
Wanyu Xu, Maulik Chhabilkumar Kotecha, Daniel A. McAdams
Series:
DESIGN
Institution:
Texas A&M University, United States of America
Section:
Artificial Intelligence and Data-Driven Design
Page(s):
2307-2316
DOI number:
10.1017/pds.2024.233
ISSN:
2732-527X (Online)
Abstract:
This study explores how large language models like ChatGPT comprehend language and assess information. Through two experiments, we compare ChatGPT's performance with humans', addressing two key questions: 1) How does ChatGPT compare with human raters in evaluating judgment-based tasks like speculative technology realization? 2) How well does ChatGPT extract technical knowledge from non-technical content, such as mining speculative technologies from text, compared to humans? Results suggest ChatGPT's promise in knowledge extraction but also reveal a disparity with humans in decision-making.
Keywords:
artificial intelligence (AI), design knowledge, design research, design theory
18th International Design Conference (DESIGN 2024)
Conferences

This site uses cookies and other tracking technologies to assist with navigation and your ability to provide feedback, analyse your use of our products and services, assist with our promotional and marketing efforts, and provide content from third parties. Privacy Policy.

Cookie Settings Accept Cookies