Skip to main content

How to Create a New Quality Evaluation Project in KantanLQR™

Professional Services
  • Updated

How-to Guide for Project Managers

Creating a new Quality Evaluation Project

KantanLQR is an online quality review tool dedicated to making human evaluation of Machine Translation output faster, seamless and more efficient. It allows Project Managers and Reviewers to collaborate, automate and streamline the process.

  1. Log on to KantanMT.
  2. On the left navigation pane, click Dashboard under KantanLQR. The KantanLQR Dashboard page appears.
  3. Click New. The Create New Project dialogue box opens.
  4. Under Name, in the Project name box, enter a project name.
  5. In the Project Type box, select Quality Evaluation and click Next.
  6. Under Languages, in the Source Language box, select the source language.
  7. In the Target Language box, select the target language for translation, and click Next.
  8. Under Settings, in the Start Date and End Date boxes, select the project duration. Then, choose whether the test data should be shared among reviewers or not, and click Next.
  9. Under Finish, click Create. The KantanLQR Dashboard page appears and the new project is created.

 

Selecting Key Performance Indicators (KPIs)

You can select different industry-standard quality metrics for your active project to test your KantanMT engine. Project Managers can either select the pre-defined KPIs or create their own project-specific, customised KPIs.

    Pre-defined KPIs

  1. In the KantanLQR Dashboard page, click KPIs. The KPIs for [Project Name] page appears.
  2. Click Add to select the pre-defined KPIs.
  3. Under Compulsory KPIs, in the Select Compulsory KPIs box, select the KPIs from the four groups: General, Custom, MQM, Simplified. Click Next.
  4. Under Optional KPIs, in the Select Optional KPIs box, select KPIs if desired, and click Next.
  5. Under Summary, review and click Add. The KPIs are added to the project.
Note                    At least one compulsory KPI needs to be selected.
   
 
    My KPIs
  1. Click My KPIs. The My KPIs page with customisable KPIs appears. 
  2. Read the information at the bottom of the page about the types of KPIs.
  3. Click Create. The Create KPI dialogue box appears.
  4. Under KPI Name, in the Name box, enter a KPI name.
  5. In the KPI Type box, select the relevant option. In the KPI Group box, select Custom, and click Next.
  6. Under Values, in the Values box, enter up to 6 values, and click Next.  
  7. Under Default, in the Description box, enter a description of the KPI. Under Default Value, select a value, if required, and click Next.
  8. Under Summary, review and click Create.
 
Note 1                            Available KPI types are Radiobox, Checkbox, Multiple, Select, Stars, Input and Text area.
Note 2 Values are a range of standards assigned to each KPI. For instance, Terminology and Added Text are values of the KPI Style.
 

Managing Reviewers

Project Managers can assign KantanLQR projects to one or more reviewers and observe progress in real time. 

  1. In the KantanLQR Dashboard page, click Reviewers. The Reviewers for [Project Name] page appears.
  2. Click Add. The Add Reviewers dialogue box appears.
  3. Under Reviewers, in the Reviewers box, type the reviewer’s name and click Next.
  4. Under Summary, review and click Add
  5. The reviewers will receive an email inviting them to take part in the Quality Evaluation project.
 
Note 1                   

You can add more than one reviewer to each job if required. In addition, if you have used KantanLQR previously, the reviewer's email address will appear in the Reviewers box, as email addresses are stored after the first time they are added. To choose an existing reviewer, click their email address.
Note 2 Project Managers cannot be added as Reviewers in KantanLQR. If you are a project manager and you want to act as reviewer, you need to add a different email address.

 

Uploading Test Data for a Project

This is the target data subject to the evaluation, together with the corresponding source data.

  1. In the KantanLQR Dashboard page, click Test Data. The Test Data for [Project Name] page appears.
  2. Click Add. The Add Test Data dialogue box appears.
  3. Under Source Test Data, in the Source Test Data field, paste the source segments of the text to be reviewed, and click Next.
  4. Under Target Test Data, in the Target Test Data field, paste the target segments to be reviewed, and click Next.
  5. Under Summary, review and click Add. The Add Test Data confirmation box appears.
  6. Click OK. The test data is added.
Note                        In the Summary box, the number of lines for source and target need to be the same.

 

Monitoring KantanLQR Projects

KantanLQR projects can be monitored in real time.

  1. The Quality Evaluation project is set up and an automatic email notification is sent to the reviewers to start the project.
  2. In the KantanLQR Dashboard page, click Projects. The project progress bar appears in the Progress area.
  3. In the KantanLQR Dashboard page, click  to edit the project. In the Edit properties dialogue box, you can change Name, Start Date, End Date and Status. You can change Status from Active to Inactive. Click Save to implement the changes.
  4. In the KantanLQR Dashboard page, click  to view the Project Summary.
  5. There are several tabs in the Project Summary page. The number of tabs depends on the number of KPIs selected, as there is one tab per KPI. You can view three other tabs: Overview, Reviewers and Test Data.
  6. In the KantanLQR Dashboard page, click  to preview the project as a reviewer.
  7. In the KantanLQR Dashboard page, click  to pause the project.
 
The Project Summary Page
  1. In the Project Summary page, under Summary, click Download Summary Report to download the project summary file. 
  2. In the Project Progress area, check Overall Progress, Project Progress and Reviewed Test Data.
  3. Scroll down to see visual representations of evaluation results for each KPI.
  4. In the Project Summary page, click Adequacy, Fluency or any other KPI tab for detailed summary. To see details on the post-editing effort, click .
  5. In the Project Summary page, select the Test Data tab, which displays the reviewed segments with all the corresponding scores and comments.
  6. In the Project Summary page, select the Reviewers tab. The Reviewers tab displays information about the progress of each reviewer and whether they have accepted the project or not.
 
Note 1                    

The Project Summary file contains several columns: Row no, Source, Target, Translation, Comment, Reviewer, Email, Processed time. The Project Summary file also contains columns that display information related to the selected KPIs.
Note 2 The Project Summary page contains one tab per KPI with a visual summary of the segments scored.
 

 If you encounter any problem or need help, please contact us at support@kantanmt.com.

Related articles