A few days ago a draft of the questionnaire was released and some users provided some feedback and also some critics. We considered and discussed the points that were made in the comments but it was obvious that some users misunderstood how the scoring system worked. In our opinion when releasing the questionnaire it should have been specified that because of devs requirements, the questionnaires started from a value of 100 and we had to substract points instead of adding, that is the reason why there were no positive questions. Probably that point caused the most outrage and it was just a perception issue.
Others mentioned that there were too many questions regarding the contribution post and that it was a bad idea to start from these questions. Some users asked to be more punitive on average to mediocre translations that contain several errors but reward more work well done and error free.
We came out with two possible questionnaires, the first option (Q1), is something implementable in the current utopian framework. The option (Q2) may not be currently implementable that it's what we feel should be done.

First example questionaire (Q1):
Q1 Google Sheet example link

Second example questionaire (Q2):
Q2 Google Sheet example link

The two examples are in the same file, but on different tabs

Q1

The first option is compatible with the current framework in Utopian: it starts from a maximum score of 100 and it subtracts or adds values based on the bonus / malus obtained from the answers to the questions. Now we know that the final score can go beyond the value of 100 so we added also positive questions.

Limits:

In order to reward quality, we must assign greater weight (score) to the questions regarding the quality of translation. Thus, it is not possible to differentiate in a balanced way the economic remuneration for translations of different volumes. For example, a translation of 2000 words should not reward twice as much as a 1000 words translation. On the other hand, if we assign more points to the volume of the translation, the quality loses weight and a long but average translation could receive more than an excellent but short translation.

Meaning of the questions

1. How do you rate the overall accuracy of the translated text?
2. How many major mistakes were found in the translated text? (that can be change the meaning of the text)
3. How many minor mistakes were found in the translated text? (that do not change the meaning of the text)
4. Has the same style of translation been maintained for this project? If more translators are working on this project, are they respecting the original style?
5. How would you rate the formatting, grammar, writing style and overall presentation of the contribution post?
6. Did the translator provide in the contribution post all the information needed to fully evaluate the translation? (For example, did he specify if he needed to research the definition of unfamiliar terms and which tools he used in the translation?)
7. What was the total volume (words) of the translated text in this contribution (excluding duplicate phrases and non-translatable content)?

Question 1: It represents an overall judgment of the translation also taking into account factors not specified in the following questions.

Questions 2 e 3: This is a more specific question, it requires a minimum count to identify the proportion between the errors committed and the length of the processed text, distinguishing the weights in serious and minor errors.

Question 4: It is important that the style is consistent between the different parts translated, such as when you decide not to translate technical or medical words. The consistency of methods and style is also important when different translators work on the same project or on one already started by others. Please note that if you find issues in the previously translated strings you should translate them again avoiding the same mistakes.

Question 5: This question refers to the post presentation. It has less weight than the questions relative to the quality of the translation but it gives more granularity to the final score. Nonetheless, a clumsy post, with several grammatical mistakes, would seriously undermine the professionalism of the translator.

Question 6: Second and last question about the contribution post. Here we are asking if the translator has provided all the necessary information for the correct evaluation of his work, such as: number of translated words, project name, github link, crowdin link, name of translated files (if applicable) and other if necessary.

Question 7: The length of the translation is another important parameter, a standard post is minimum 1000 words. We have introduced bonuses for longer posts and panalities for shorter ones.

Note: The scoring system is a short blanket, it is difficult to give due weight to both quality and length, we tried to mediate.

Q2

Our second proposed questionnaire may not be implementable (it would require some changes in the utopian voting bot). But, we believe it would provide a score that reflects better the quality of the translations and it multiplies the result by a coefficient proportional to the volume of translated words. We would like to highlight two values: the score, which indicates the quality of the translation regardless of its volume and the actual vote that it will receive, which considers also to the amount of translated text. This allows us to discriminate between quality represented by the score and to tune the reward taking into consideration the volume of text translated.
Why is this important? To avoid that short but excellent translations are skipped by utopian voting bot only because there were longer translations in the cue.

Please:

When providing feedback/criticism it would be helpful if you could also provide constructive proposals instead of just saying "I don't like it". We will have a chance to discuss this on Monday in the utopian discord server.

---

If you would like to delegate to Davinci.Witness you can do so by clicking on the following links:
50 SP, 100SP, 250SP, 500SP, 1000SP, 5000SP.
Be sure to leave at least 50SP undelegated on your account.

CC0 Image - Creative Commons, thanks to @mrazura for ITASTEM logo. CLICK HERE AND VOTE FOR DAVINCI.WITNESS DAVINCI WITNESS TRAIL
If you are the owner of an Open Source project on Crowind platform and you want to translate it with our help and the support of @utopian-io please join our discord server and chat with us.
DAVINCI WITNESS DISCORD SERVER