What we're prototyping: project Themis

As most of you have seen, we've been experimenting with AI implications within products at Kybo much more recently and intend to upkeep the benefits of which it can provide. Originally the concept for a more advanced game moderation system, including chat history alongside avatar based detection has been a thought concept for a while but never came to fruition—that's why we're pretty excited to share some advancements we've made with some prototypes of a new product, project Themis.

Behaviour

Before we get into the implications of how Themis can benefit users, group owners and Kybo in player behavior research, we'll share some insight into how each course of action is decided before it's applied in-game. How Themis moderates in your game is up to you, which is why we chose to fine-tune 3 AI models. Themis High, Medium and Low. All models follow the same set of rules and guidelines, but vary in fine-tuning and severity of detection when it comes to moderating any content provided to it. Content that Themis will be able to moderate includes but is not limited to

• Trolling based requests/conversation/questions

• Targeted harassment

• Multilingual character used within bypassing

• Purposely causing issues with hostility

Analytics

There are numerous other cases that Themis can address, and we look forward to exploring these areas further. However, at this early stage of prototyping, our focus is primarily on the foundational aspects of generic moderation, as mentioned above. Themis not only offers advanced chat and player behavior moderation tools for immersive experiences but also empowers group owners to gain deeper insights into their player base.

By leveraging Themis alongside Roblox's sophisticated analytics, we can analyze the chat behavior of users and how they interact with one another within the experience. The statistics we provide can include, but are not limited to:

• Most common topics covered

• Most asked questions

• Most X item requested

• Most common problems

• Most common complaints

• How often an issue is experienced

• Hostility percentage of player-base

• Levels of literacy competence of player-base

Initial prototyping results

The initial versions of the Themis assistants have undergone testing across a variety of scenarios, reflecting a diverse range of user behaviors—some more pronounced than others. Although we can't disclose all the details, we are pleased to report that Themis successfully detected 100% of the clear cases. However, more complex and varied scenarios posed greater challenges, yielding inconsistent results. We are committed to refining the models to enhance their accuracy in determining whether the provided content complies with established guidelines.

Themis improves over time

The landscape of AI is rapidly evolving, and Themis is primed for ongoing enhancement. Whether in its prototype phase or as a fully developed product, we possess the capability to refine and upgrade the core systems that power Themis. This commitment ensures that we will consistently deliver high-quality performance, and likely even better, in the future. While the current prototype is not yet ready for public use, we are dedicated to exploring how this cutting-edge technology can enhance your experiences moving forward.

If you're interested in assisting us with testing Themis, please visit the Careers page and look for the Hyre Tester position. If you have already submitted your application to become a Hyre Tester, rest assured that your response will be automatically taken into consideration.

These projects & prototypes, such as Artis and Themis, help Kybo learn, test and understand what's best suited for our future products, and benefits our development and product performance in the long run. We know it's nothing shocking, but your involvement and hype around these is recognized, and we appreciate your support.