This document is a collection of tips that are very important for every testing report. Each topic is important and has a reason. It is not possible to write a document that will teach to to be a good tester or that cover all situations that can occur. These are some highlighted important points, however your personal effort and will is the second and not less important part of your work.
Contents
Report into latest PAD
As a tester, do all your reports into the latest Testing PAD related to a task you are working on. In case the task includes more than one PAD link, always use the latest. There exists a self-explaining PAD example as well as a PAD with some basic help about PAD organization. The actual form of each Testing PAD is fully up to the current Test guide (a coordinator who manages the current test) however all PADs will be similar because of need to track similar information.
In case that the task has no PAD but the testing is open (very rare situation), do your report directly into the task comments.
Provide testing environment details
This is the most important and essential information you provide. Your current character name that was used for testing is not that important and might be useful only in case when a developer needs to review database details related to it. Much more important is to let us know:
- Tester name - the name that you are using in the Test Team. Not all might be clear from your report and a coordinator might need to talk about details with you.
- Platform - your operating system name and version including information if it is 32 or 64 bit. Some issues might be platform specific for various reasons.
- Language - your Ryzom client locales you have used during your testing. Some issues might be language specific for various reasons.
Besides these there might be some specific information needed for a specific testing (for example desktop screen resolution). All these information are usually stored in Testers section of the PAD. In case that you use various combinations remember to list them all and mention if any discovered issue applies to all or just some of those combinations.
Read instructions carefully
Every testing PAD usually has a section with general Description. You will find there some details about the change you are testing and, optionally, details about known issues, fixed issues, possible rewards, general mechanics of the change or debug/tester tools related to your testing.
Another very important section is Instructions where you will find more detailed information about which character you need to prepare for the testing, optionally with tips how to do it, and usually a list of things that we need to test. Your work should however not stop here.
Provide date and time of your testing
All your reports, questions and feedback go to Reports section. Always start new report with date and time when the testing was performed, including time zone. This is important for several reasons:
- Actively developed things might change in time and we might need to know if your report was done before a specific change or after
- Server might have technical difficulties by the time of your report and some discovered issues might not be directly related to the task
Ask if you are lost
No one knows everything. In case you do not understand the matter of a change, feature mechanics is not clear or in case you have discovered a behaviour that is not covered by the PAD instructions, ask. Every task has a developer mentioned in the PAD and also as a tag sticker on the task. Every opened test has also a Test guide, a coordinator who takes care of the current test. Ask them for details. We all work to make Ryzom better so you should not hesitate to also ask directly on the #t-test channel.
Report everything
Yes, you should reflect all topics in Instructions, but it is also important if you can provide all possible related details you can. Your report should describe what and how you did perform the test and all related circumstances. Describe your activity, especially in case you discover an issue. Be ready to take and provide screenshots along with your report or record a video that captures whole situation. Read Making evidence article for help with how to do that.
Your report should also be partially a feedback for the Developer or the Test guide and do not hesitate to suggest better solutions for certain situations. Think the player way:
- Think about what players will experience with the feature
- Think about troubles they might have using the feature alone and in combination with other game mechanics
- Keep in mind that players might have limitations (language barriers, colour blindness, disablement...)
- Keep in mind that players might not be as experienced as you are
Keep eye on your work
Your work do not end with nice shiny report full of images. Come back in few days and review if your report was reviewed and all issues noted in Issues section. Ask coordinators and developers about progress. Repeat your questions in case you did not get any answer, either in the PAD or in the chat.
Do not be lazy to prepare new character and repeat your testing. Within one report (testing session) to examine and fully describe discovered issues, but also after a progress. Come back and re-test everything, provide new report reflecting latest changes. Confirm issues discovered by others, confirm issue fixes. You have the intelligence, experience and creativity. That is why we need you. Otherwise a machine could do your work.
Break testing servers but be discreet on Atys
When testing on Yubo on Gingo, always look for ways how to break or exploit things. Do not hesitate to do. Better break a testing server then let players to do with Atys. And don't be naive to think players will not try. Play stupid, look for ways how to use things the unexpected way. Combine things even when it looks pointless. Damage feature configuration file. Misuse the user interface. Does it ask for number? Input string. Does it expect you to click a link or button? Reload the window few times. Break it.
Absolutely different situation is when testing on Atys. Do your testing but always mind regular players who came to enjoy the game. In case you have suspicions that something might affect others...
- Do a similar test on Gingo first
- Consult it with developers as next
You should always be inconspicuous and you should definitely avoid doing following things:
- Do not spoil details to pubic, let players have fun and discover things themselves
- Do not bother other players, let them enjoy game features and do not, for example, occupy place/game
- Do not use exploits for other reasons then re-testing (or you will get ban)
