Using web crawler technology to support design-related web information collection in idea generation

DS 75-6: Proceedings of the 19th International Conference on Engineering Design (ICED13), Design for Harmonies, Vol.6: Design Information and Knowledge, Seoul, Korea, 19-22.08.2013

Year: 2013
Editor: Udo Lindemann, Srinivasan V, Yong Se Kim, Sang Won Lee, John Clarkson, Gaetano Cascini
Author: Wang, Zhihua; Childs, Peter R N; Jiang, Pingfei
Series: ICED
Institution: Imperial College London, United Kingdom
Page(s): 229-238
ISBN: 978-1-904670-49-0
ISSN: 2220-4334


Effective information gathering in problem and task related fields with which designers or design teams may not be familiar is a key part of the design process. Designers usually consult with subject experts to access expert information. An Effects Database system that includes design-related effects to provide ready access to expertise at any stage within the design process can be used to prompt areas to consider and explore. To maintain the efficiency of the system, its data must be regularly updated and new effects populated from the open source knowledge base. Web crawler technology has integrated into an information gathering and analysis system to rapidly mine design-related information from published data sources in order to update an effects database for use in design. This paper describes the effectiveness and efficiency of the system for updating the database. Comparing with manual information collection, the test results demonstrate that this system can dramatically increase the efficiency on selecting design-related information from un-restricted internet sources.

Keywords: Design methods, information management, database, web crawler


Please sign in to your account

This site uses cookies and other tracking technologies to assist with navigation and your ability to provide feedback, analyse your use of our products and services, assist with our promotional and marketing efforts, and provide content from third parties. Privacy Policy.