Quality of Telephone-Based Spoken Dialogue Systems phần 9

Tham số này COMP khoảng tương ứng với một phiên bản nhị phân của bản án trên B1 câu hỏi. Đối với việc phân tích các dữ liệu thử nghiệm 6,3, các tùy chọn sau đây để mô tả thành công nhiệm vụ đã được lựa chọn: công tác thành công tính trên | Table . Task-related parameters 1 . For a legend see Table Name Definition Unit Syst. transp. Int. partn. Meas. method Int. level TS Task success Also Transaction success Percentage of dialogues in which the user has reached his her goal . the system provided the information or action desired provided that this information is available or the action possible. The following categories are differentiated - TS S Succeeded tasks for which answers exist - TS SCs Succeeded with constraint relaxation by the system - TS SCu . Succeeded with constraint relaxation by the user tasks involving unknown objects - TS SCuCs Combination of both types of TS .SC - TS SN Succeeded in spotting that no answer exists tasks involving unknown objects tasks with no answer - TS Fs Failed because of the system s behavior due to system inadequacies - TS Fw. Failed because of the user s behavior due to noncooperative user behavior Fraser 1997 Danieli and Gerbino 1995 Simpson and Fraser 1993 l bb s u exp. dial. Definition of Interaction Parameters 371 372 Table A. 9. Task-related parameters 2 . For a legend see Table A. 1 Name Definition Unit Syst. transp. Int. partn. Meas method Int. level Task solution Existence of a task solution reported by the user in percent of all tasks. Goodine et al. 1992 Polifroni et al. 1992 1 bb s u exp. dial. Solution correctness Correctness of a task solution as reported by the user in percent of all tasks. Goodine et al. 1992 Polifroni et al. 1992 1 bb s u exp. dial. Solution quality For scalable tasks Judgment on the quality of the reported solution. . the quality of a shortest possible routing task can be measured in terms of the resulting route length. Sikorski and Allen 1997 bb s u instr. dial. k Kappa coefficient Percentage of task completion according to the kappa statistics. Determined on the basis of the correctness of the result AVM of a dialogue with respect to the scenario key AVM. A confusion matrix M i j can be set up for the attributes .