Performance
Under the Performance tab of a single Task page, you can view performance statistics in Task&Worker dimensions, including Data Performance, Object Performance, and Accuracy, etc. It also supports filtering and displaying performance statistics based on Data Completed/Submitted, Data Validity, Data Stages, and Submitted/Completed Time.
Personal Performance & My or Team Tasks Performance
For a worker, clicking the
View Performancetab on the top right corner of theTo-dopage will jump to thePersonal Performancepage, which displays the performance statistics from the perspective of a worker him/herself.For Task Admin and Team Admin, clicking the
View Performancetab on the top right corner of theMy/Team Tasks Managementpage will jump to theMy/Team Tasks Performancepage, which displays the performance statistics from the perspective of a Task Admin or Team Admin.These performance pages' display and logic are similar to what is concluded and explained below.
1. Task Performance
Under the Task Performance tab, data-wise and object-wise performance information is available. Data Performance statistics include submitted data item counts, valid work durations, pass rate as quality, and redo counts. The Object Performance chart indicates the specific counts of actions related to different tool types and their corresponding data types.
1.1 Data Performance
| Indicator | Description |
|---|---|
| Submitted Data | Number of data items that were submitted during the filtered time period. |
| Valid Work Duration | Total: The cumulative work duration Average = Total Work Duration / Number of Data Items. This represents the average work duration for an item. |
| Quality | Pass Rate = 1 - Total Number of Redo Times in the Stage / Total Number of Submission Times. This is averaged across tasks. |
| Redo | Rejected Items: Number of Items that were rejected or reassigned. Reset Items: Number of Items that were rejected or reassigned and had their results cleared to an earlier stage. |
1.2 Object Performance
The object performance for each stage can be broken down into the following dimensions:
| Stage | Dimension | Description |
|---|---|---|
| Annotation | Create | Number of new objects created by an annotator |
| View Imported | Number of pre-annotated objects | |
| Review | Review | Number of objects reviewed and passed by a reviewer |
| Create | Number of new objects created by a reviewer in the modification mode | |
| Edit | Number of objects edited by a reviewer in the modification mode | |
| Delete | Number of objects deleted by a reviewer | |
| Acceptance | Accept | Number of objects accepted by an inspector |
| Create | Number of new objects created by an inspector in the modification mode | |
| Edit | Number of objects edited by an inspector in the modification mode | |
| Delete | Number of objects deleted by an inspector |
1.3 Accuracy
Accuracy shows Object Accuracy of Instance or Segmentation. Object Accuracy - Create = 1 - Number of Wrong Objects / Total Number of Created Objects. This is averaged across tasks.
2. Worker Performance
The Worker Performance section displays all the performance statistics in regard to each individual worker, including submitted item counts, total&average work durations, pass rate, rejected item counts, reset item counts, object performance, and object accuracy. The meanings of these concepts are the same as those in Task Performance explained above.
Updated 14 days ago