Office Hours: Dec 17, 2025

[cassidyjames]

Our next office hours are happening Wednesday, December 17, 2025 at 1730–1800 UTC!

Join us to share what’s working and what’s not in real time, and connect directly with T&S peers and RMC model partners.

• Discord event
• Google Meet
• #roost-model-community

Agenda

Brief introductions

• Recap of what the ROOST Model Community is, and the purpose of office hours
• Brief round of personal intros

Q&A/show and tell:

• Follow-up from hackathon
• Progress testing/evaluating safety models
• Interesting uses of safety models
• What's missing from current safety models?

Admin:

• Office hours scheduling: bi-weekly cadence? Are different days/times better?

What's Next:

• Introduce yourself in GitHub Discussions
• Start a new discussion! Whether it's an implementation question, research findings, or model specific feedback

[cassidyjames]

Notes from the call:

Attendees

• ROOSTers: @cassidyjames, @julietshen, @andrewmchang
• Partners: Ludo from OpenAI
• Travis: Matrix.org Foundation
• Jon: software engineer at glif.app, heading up trust & safety engineering; previously using omni model from OpenAI; early days, have set up a playground for T&S team to explore
• David Gros
• Naoya

Hackathon

• Hugging Face space can still be used until we run out of credits!
  • Uses the Harmony format automatically; if using yourself, you need to apply/use the Harmony format yourself

Matrix.org Foundation

Real-time chat protocol with public-facing rooms where any of 90-million users can show up and chat. Team currently has near-zero experience with AI models, especially locally-running ones. Looking at everything from "is this on topic" to policy violations, illegal content, etc. Mostly text-based as images are disabled in public rooms.

Most helpful:

• How do I install this/use it locally?
• What shape do requests need to be?
• Hoping to add as additional protection rather than replacing anything they're currently using

Open sourcing their policy server, which is a firehose! Blog post incoming soon!

• Currently uses the Omni moderation model through the OpenAI; what are the differences between that and gpt-oss-safeguard?
  • Omni is not reasoning, safeguard is (big difference when it comes to latency)
  • Omni is only via API, safeguard is available via API and locally
  • Safeguard is customizable for your own policy, which is a huge difference; if what you're looking for is exactly in the realm of the 13 policies offered by OmniMode, then OmniMode is a good fit

Glif.app

• Playground set up with inputs and outputs to get a baseline from existing policy, following community guidelines, getting comfortable with the model
• First use case might be looking at a single vertical, e.g. check appropriateness of usernames
• Next might be to handle cases that are right on the threshold using Omni, kick it to safeguard for reasoning

[andrewmchang]

Adding a few other helpful resources shared during the call!

gpt-oss-safeguard user guide

OpenAI's gpt-oss cookbook (helpful for getting gpt-oss models running locally)

Guidance on the harmony format, which is the optimal way to format system prompts with gpt-oss models

[DNGros]

Thanks for the notes @cassidyjames ! It nice to join. Sorry I was only very partially present on the call; timing worked out unfortunately while I was traveling.

Some notes:

• Good know this is also on OpenRouter. It looks like it still only routes to Groq. I could potentially look if this gives the same output as HF Inference API. If there is bugs that limit replication of results, it would likely be in Groq though.
• I'll just FYI my utils folder that we merged in. I ported over the formatting from the HF space, so this is an option if people want to try to match the space but run the code locally/programatically. Just FYI for anyone.
• There was some discussion of applying GPT-OSS-Safeguard in a few contexts that I'll just loosely throw in a word of caution (such processing all utterances from a high volume firehose like Matrix or ATProto). Some of this I think just for experiments, but I'll note could be very expensive if deployed.
  • Most use cases you would want a cheaper high recall initial check and only pass off the suspicious ones to a larger model. 95+% of your messages will be obviously fine. You'll waste a lot of compute if you run on every message and realtime contexts like Matrix will sacrifice latency.
    • One can use off the shelf models, or train one from generated labels from GPT-OSS-Safeguard or a more frontier model. Then use other model for finer labeling.
  • At some point I want to experiment with GPT-OSS-Safeguard's ability to give a quick single-token (no reasoning) labels in comparison to some other models.
  • The things we talked were not for chatbots, but I will also mention how in chatbot contexts, techniques where you that reuse the same model itself as much as you can is good (since you already doing an encoding of all user tokens anyways).

Thanks for organizing!