Beta Report 0.2.6

On this one I wanted to make sure the added features were working properly and as users might expect them to do. Also I wanted to gather insight into how users perceive the first available plugin. It was important to see whether they would understand how to use the plugin and what it does. So for this test, I tasked the users with the following:

  • Create account [If you don’t already have on]
  • Try to update your profile
  • Create or join a group
  • Create a Post/Event
  • Delete a Post/Event
  • Create an Action. Choose the “Member Goals” plugin. Try to use the action by adding goals to the action after you created it. Also make sure to checkout the report page on that new action.


Test Channels

To carry out the beta test I am using any tests channels available that allow me to distribute the app to beta testers as well as to track their response and evaluations. This does not include simple app directories/publishers. This time around I managed to get feedback from various sources. Also I received some feedback through Collapp, which was also helpful in many ways. I will try and keep the Overall Followers count for this particular channel aligned with the member group size of the Collapp Community within the app. I was also very satisfied with the turnout of Betafamily. This time around I got a lot of reports, without putting as much effort into inviting testers as in the previous tests runs. The social channels groups the feedback through various social media, predominantly discord and reddit.


Test Channel Overall Followers Submited Reports Requested Changes
icon Betafamily 75 25 28
Social ~10 ~10 13
icon Collapp 109 4 12


Tickets by Category

The submitted reports from the test channels are broadly categorized into four major categories. This will help to understand which areas of the app need more focus and are most important for the users. I plan on removing/replacing the Stability & Performance section by adding a Feature Request section.

User Interface

9

User Experience

13

Bugs

8

Major Issues

Evaluating the submitted reports gives insight into which issues are most recognized by beta testers. Submissions pointed out that there were inconsistency when it comes to deleting entries from groups. People also noticed an increase in bugs, compared to previous version. Overall the major group seems to be consistently UX, which means that there is some noise/friction while using the app. Also people, although not a lot of them,voice their issues in fully understanding how the app works and what it is supposed to be. The most common reported issues referred to bugs.

Deleting entries

Deleted posts, events & actions were still visible on home screen

Missing media on post

Users could not add media to posts on creation page

Account verification

Users asked to add account verification functionality to protect email identity

Understanding app

A recurring theme amongst a minority of users is that they do not fully comprehend the purpose of the app or other aspects of it


Conclusion

Overall this was a great test with lots of different opinions from a broad audience. It helped to guide me in what aspects I need to improve and from which angle I might need to think about things like marketing and task priorities. I think I should focus on one particular group of people and build value around their needs first, instead of working in a broad context and understanding of what this app will and should be.