AI business writing assistant tools in practice: Text mining and statistical analysis of user experience

Abstract

This study employs a multi-method quantitative research design to review various popular AI writing assistant tools using 11562 actual users’ satisfied and dissatisfied reviews. The research design applies Kruskal-Wallis test to differentiate between AI writing assistants based on user rating, functionality, ease of use (EOU), value for money, and customer support attributes; uses the quantitative text analysis to identify satisfaction and dissatisfaction attributes; and employs topic modeling to uncover the specific related attributes that influence user satisfaction. The statistical analysis results indicated that there is a significant difference among AI writing assistant tools according to user rating, functionality, EOU, and customer support attributes. The text analysis revealed the attributes leading to user satisfaction and dissatisfaction. Topic modeling further identified both common and tool-specific attributes. This study provides a relatively comprehensive set of attributes and sub-attributes for evaluating AI writing assistant tools, which can serve as a foundation for future research.