!!!! WIP !!!! The last editing was from Leda on 27.05.2023.
A team coordinator is a person, who is willing to coordinate the work of others and manage all the boring bureaucracy of the Test team. It requires a lot of additional time, patience and responsibility. Testers will be lazy, developers will lie, marketing will make promises and other teams will have no time for your problems. However, the work must be done somehow. If you think you can help the team in this way, this article will give you some closer looks on principles how your work should look.
It is expected that you understand the process how tasks are being managed on [Team] Test board and you understand why. Your membership in other Ryzom teams and projects can be very helpful, especially membership in the Bugs project that allows you to also manage related issues on Bugs board. You should also be familiar with ways how to make an evidence for testing purposes.
Contents
Theory
In case of the Test team, all the team the work is closely related to team board tasks. The team is requested to perform a testing on a target server, does the job and cares about all discovered issues. There are three main jobs that belong to a coordination and connect all the testing process together, however it is not possible to give you exact points to do. Every task is different so you should be creative within process principles.
New task creation
This is the begining of all tasks. One of team coordinators must create a new task and prepare some initial documentation for a testing. It means that you create new task in Waiting column of the target server, describe all request details like what to test, under which circumstances, background lore information and similar. All the information that is necessary for the future test leader to fully understand the matter of the change. This job is very often done by developers to request a testing and you can read How to request new testing article that describes this process from opposite side.
You should make sure that all aspects of the change (testing) are clarified and that there are no doubts about any of general questions, for example how high an event reward should be or how long a mission cool-down should take. Remember that the future test leader needs to know as much as possible to prepare valid testing for testers.
Testing leadership
To lead a testing of a task is the most common job that a coordinator can do. You choose from new (yellow) tasks waiting on the board and find the one you want to care about. Review the task changes on the target server to make sure it is running, create a testing PAD with all necessary instructions and do at least quick test yourself. Make sure you provide all necessary instructions for testers, remember that testers might be just volunteers who want to help but have no previous experience.
Once the test is completed or when it is not possible to continue in testing, it is your work to evaluate the testing, track all known issues and notify other teams about results. You decide if a task can progress or if it will return.
Requesting other teams
It is almost rare that a testing will end with no issues and even when it happens, you still need guide the task to finish.
Task with issues
In case there are any issues, there are three options how to resolve them:
- Issue is confirmed but not actually an issue - clarify all details with related teams and developers and close it if works as intended
- Issue is confirmed and can be fixed along the testing - pass the issue to the appropriate team to solve
- Issue is confirmed and can't be fixed along the testing - add the issue to the Bugs board at the end of the task lifetime
Tasks without issues
Those tasks that have no known issues can progress or be closed. Notify the task developer, team manager, marketing team, translators and/or encyclopatysts at right moment so they can do their work - move the change to next server, complete translation, write documentation for wiki, announce the change, write patch notes... whatever their work is.
Principles
The work of a coordinator is not about to certain amount of tasks right or write number of task comments. Make your work to fit current task needs but please follow these principles. Every single one has a reason to exist and will help you to do better job after all.
It is up to you
To act as a coordinator is your decision and your responsibility. Mostly everyone can mark an issue as resolved, sometimes by accident, and you will also hear lovely stories about happy future. It works, we did it, it is solved. But it is you who (is going to) lead a test so you should make sure yourself that a subject of testing is actually present and running on a target server and that it is possible to even start the testing.
It is you who must confirm that a fix was really delivered to the game server. By own testing or by opening new testing for testers. It is you who re-test all issues reported by testers and confirm them for developers. Of course that you do not have to do everything yourself, ask other (different) testers to help. But it is your responsibility what instructions will be saying to testers and if they will be able to test.
It is up to you not to accept issues that actually are not issues and to explain all issue aspects to a developer, translators, lorists. It is your job to properly close all issues and make sure all was resolved or moved to Bugs board. It is your work to notify other related teams and projects about existing issues from their domains or about a task progress.
All issues are resolved
Make sure that you track all issues that were discovered during a task lifetime and that all of them are properly resolved before that task is closed.
- Issues that can be clarify must be clarify before end of testing, because it might lead to additional other issues that will be consequent of such issue.
- Issues that can be solved withing the testing task must be fixed, re-tested and the fix confirmed before the task can progress. The only exception are minor issues in case of progress from one server to another, related to features that are not fully completed yet. Such issue can remain open but still must be properly resolved at the end of the task lifetime.
- Issues that are not possible to fix withing the task, usually bugs related to a system functions on which the change is based on, must go to the Bugs board and then be closed within the Testing task.
To properly close an issue on the Test team side you need to mark the issue as done on the board as well as in the latest PAD and also mark the PAD record with date, time and details about how the issue was resolved. Fixed and confirmed, transformed, splitted or moved to other issues or other tasks, resolved as not being an issue, simply your summary over the specific issue.
Make suggestions not decisions
All the testers feedback and suggestions are most welcome, but always remember that testers are not those who make decisions. This belongs to developers, managers, game designers and lorists, depending on a matter of the problem. Your work is to warn other teams about possible troubles, inconsistency or regression or suggest better solution but not to decide how will a fix be implemented (in case it really fix an issue), how will rules be set or if an information is (lore) valid.
When you manage task issues, always expect that a change can be used by not experienced players with different needs and ideas. Try to consider all as neutral and try to find common game principles in a feature and make sure it is balanced and fair to all players. Think about impact on the current game because developers or manages focused on the task might not see it in relation. Testers are those who experience and use a feature as first.
It is also not up to testers nor coordinators to decide if something is or is not a bug and if it will or will not be fixed. On the other hand you must carefully consider every issue that any of the task testers rise and decide if you will pass the problem to someone who can make a decision or if it is attended or fixed already.
Relate importance to players not to other issues
We are basically using three type of issues, red for really serious issues, orange for minor issues and blue for all other like clarification. It is up to a test leader if he decide to mark an issue with red, orange or blue, but the importance should be determined from how the issue affects the game-play and not to other issues. Following example will explain it a little better.
Testers have discovered several issues during task #123 testing. It is possible to combine two things to exploit game mechanism, there is one NPC dialog missing, various missing translations, wrong instructions in Journal and no image with event instructions displayed for neutral characters.
From what you read above, it is clear we will have 5 issues for the fictional task #123:
- Combine items to exploit - this might look serious, but consider circumstances that allow to use it, consider variations with other items. Consider you might need to clarify the issue first. If still an exploit, it will be red issue.
- Missing NPC dialog - this one will be a little harder, because we must consider how much it affects the game. If the dialog just says "Thanks, bye", it is a orange minor issue and will be probably not hard to solve. On the other hand, if this specific dialog breaks a chain, makes the mission impossible or hard to complete, then it will be the red issue and the fact that the exploit above might look much more important playes no role. Both have same importance from game-play point of view, because it strongly affects the game experience or balance.
- Various missing translations - missing translations are usually only orange minor issues unless it causes other troubles like when a player will not get important information for example. Then consider to make them red issue
- Wrong instructions in Journal - you must to consider if you need to clarify information first, what the instructions supposed to say and what is the creator's intention. In case instructions are really wrong, consider how serious the problem is, just a little typo for minor issue or something serious that will break game experience and is a red issue?
- No image with event instructions for neutrals - this might be serious red issue in case it is a common problem that signalize something very wrong and the fact it is not that important as the exploit plays no role. On the other hand, when the missing NPC dialog above should tell you that this feature requires non-neutral allegiance, it is just an orange minor issue that will be probably solved when the missing dialog is fixed
Never progress tasks by default
When you go to manage tasks on the Test team board, never progress them by default. The first column where the task goes after a testing is done is back to waiting. The testing must be evaluated first and the task can progress only in case that no known issue prevent that.
To progress a task from Yubo to Gingo, there is no problem with translation issues, not everything was probably done before testing on Yubo and translators will work on strings when it is final.
On the other hand, when a task should progress from Gingo to Atys, there must be no known issues opened except those that are not too serious and can't be solved within the testing. Such issues will go to Bugs board at the end.
Keep eye on your tasks
Once you decided to guide a task and carry about a testing as the test leader, you should not just properly prepare the testing and review its results at the end, but you should also checking the current PAD every few days to answer questions of testers and to see progress. Mark all parts of a tester's report that you have evaluated and give them comments. All your PAD changes and updates should be documented in your own report and marked with date and time, every closed issue should be marked with date time and reason.
In case the PAD is getting too long it is a signal that the tested feature needs work of a developer, clarify details or test restart. Close the test, check the situation yourself. Discuss it with developers, translators or lorists to resolve what is possible. review changes yourself, create new PAD with only current issues and properly close all issues in the old PAD. Start new testing. Where there were any promises done by developers, make sure they really performed all changes and track progress.
Work never ends
Even after a task has landed in the Done column, the work is not over. The Test team should know the most about new game changes and you should try to track all issues players have experienced. Do not hesitate to go and test the issue yourself on Gingo and, with respect to players, on Atys.
While the task is still on the Test team board, do not hesitate to move it back and rise new issues to solve. In case the task is already completed, consider contacting the original developer, opening new task on Bugs board or request new testing so the Test team knows current situation on Atys.
In case any task related issue is reported on the Ryzom forum, give your feedback to the player and provide the task number where the issue was added so the player can ask about specific task in the future and we will be able to find the issue and details about its solution.