How to Create a New A/B Test Project in KantanLQR™

How-to Guide for Project Managers

Creating a new A/B Test Project

KantanLQR is an online quality review tool dedicated to making human evaluation of Machine Translation output faster, seamless and more efficient. It allows Project Managers and Reviewers to collaborate, automate and streamline the process.

1. Log on to KantanMT.
2. On the left navigation pane, click Dashboard under KantanLQR. The KantanLQR Dashboard page appears.
3. Click New. The Create New Project dialogue box appears.
4. Under Name, in the Project name box, enter a project name.
5. In the Project Type box, select A/B Test and click Next.
6. Under Languages, in the Source Language box, select the source language.
7. In the Target Language box, select the target language, and click Next.
8. Under Duration, in the Start Date and End Date boxes, select the project duration. Then, choose whether the test data should be shared among reviewers or not, and if they project should include two KPIs. Click Next.
9. Under Finish, click Create. The KantanLQR Dashboard page appears and the new project is created.

Managing Reviewers

Project Managers can assign KantanLQR projects to one or more reviewers and observe progress in real time. 

1. In the KantanLQR Dashboard page, click Reviewers. The Reviewers for [Project Name] page appears.
2. Click Add. The Add Reviewers dialogue box appears.
3. Under Reviewers, in the Reviewers box, type the reviewer’s email and click Next.
4. Under Summary, review and click Add.
5. The reviewers will receive an email inviting them to take part in the AB Test project.

Note 1	You can add more than one reviewer to each job if required. In addition, if you have used KantanLQR previously, the reviewer's email address will appear in the Reviewers box, as email addresses are stored after the first time they are added. To choose an existing reviewer, click their email address.
Note 2	Project Managers cannot be added as Reviewers in KantanLQR. If you are a Project Manager and you want to act as a Reviewer, you need to add a different email address.

Uploading Test Data for a Project

This is the target data subject to the AB test, together with the corresponding source data.

1. In the KantanLQR Dashboard page, click Test Data. The Test Data for [Project Name] page appears.
2. Click Add. The Add Test Data dialogue box appears.
3. Under Source, in the Source Test Data field, paste the source segments of the text to be reviewed, and click Next.
4. Under AB Target, in the Target A Test Data and Target B Test Data fields, paste the target segments to be reviewed, and click Next.
5. Under CD Target, in the Target C Test Data and Target D Test Data fields, you have the option to upload additional target segments for review.
6. Click Next.
7. Under Summary, review and click Add. The Add Test Data confirmation box appears.
8. Click OK. The test data is added.

Note	In the Summary box, the number of lines for the source and the different targets need to be the same.

Monitoring KantanLQR Projects

KantanLQR projects can be monitored in real time.

1. The A/B Test project is set up and an automatic email notification is sent to the reviewers to start the project.
2. In the KantanLQR Dashboard page, click Projects. The project progress bar appears in the Progress area.
3. In the KantanLQR Dashboard page, click  to edit the project. In the Edit properties dialogue box, you can change Name, Start Date, End Date and Status. You can change Status from Active to Inactive. Click Save to implement the changes.
4. In the KantanLQR Dashboard page, click  to view the Project Summary.
5. There are three tabs on the Project Summary page: Summary, Test Data, and Reviewers. In the Summary tab, you can view results in real time.
6. In the KantanLQR Dashboard page, click  to preview the project as a reviewer.
7. In the KantanLQR Dashboard page, click  to pause the project.

The Project Summary Page

1. In the Project Summary page, under Summary, click Download Summary Report to download the project summary file. 
2. In the Project Progress area, check Overall Progress, Project Progress and Reviewed Test Data.
3. Scroll down to see a visual representation of the A/B Test. 
4. In the Project Summary page, select the Test Data tab, which displays the reviewed segments with the corresponding scores.
5. In the Project Summary page, select the Reviewers Tab. The Reviewers tab displays information about the progress of each reviewer and whether they have accepted the project.

Note	The Project Summary file contains several columns: Row no, Source, Target A, Target B, Reviewer, Email, Score A, Score B.

 

 If you encounter any problem or need help, please contact us at [e-mail].