Skip to content

feat: implement turn-based conversation modalities (addresses #1103)#1119

Open
jacyanthis wants to merge 14 commits into
PAIR-code:mainfrom
jacyanthis:turn-based
Open

feat: implement turn-based conversation modalities (addresses #1103)#1119
jacyanthis wants to merge 14 commits into
PAIR-code:mainfrom
jacyanthis:turn-based

Conversation

@jacyanthis
Copy link
Copy Markdown

This adds feature #1103

This is a turn-based mode where each agent sends one message in a cycle. The moderator speaks first, then a random order of participants, then the moderator and a new random order... Participants see their text entry disabled and a "Waiting" banner then a "It's your turn" banner.

There is a checkbox in Conversation Settings for the Group Chat stage.

This PR was heavily assisted with an LLM agent, and it's my first time contributing, so thank you for the review! (maybe @rasmi for API regression? @jimbojw ?) I hope to add a few other related features.

@google-cla
Copy link
Copy Markdown

google-cla Bot commented May 14, 2026

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@jimbojw
Copy link
Copy Markdown
Collaborator

jimbojw commented May 18, 2026

Hi @jacyanthis, I'm working on running this locally to check it out. In the mean time, it looks like the schemas need to be updated (there's a failing check to this effect).

@jimbojw
Copy link
Copy Markdown
Collaborator

jimbojw commented May 18, 2026

@jacyanthis What are the steps to test this out manually? Do you start as an Experimenter and make an experiment from scratch? Thanks in advance for step-by-step instructions.

@jacyanthis
Copy link
Copy Markdown
Author

@jimbojw thanks! Schemas are updated. Yes, this is a checkbox in Conversation Settings for the Group Chat stage when editing an experiment.

  1. Run ./run_locally.sh.
  2. Visit localhost:4201.
  3. Click Build experiment from scratch (or use one of your existing experiments).
  4. Create a Group Chat stage or use one.
  5. In the settings menu that appears, click the new Turn-based checkbox (see screenshot).
  6. Launch the experiment as usual. The Group Chat stage should operate in a turn-based manner.
turn-based_checkbox

@jimbojw
Copy link
Copy Markdown
Collaborator

jimbojw commented May 19, 2026

@jacyanthis OK thanks, I was able to test locally using simulated human participants.

There seems to be a bug at the end of the first turn of conversation.

To reproduce:

  1. Create a new experiment with a Group Chat stage enforcing the turn-based conversation (feature under test).
  2. Create a new cohort with at least two "human" participants.
  3. For each simulated participant, select the participant and click the "Click to begin experiment button".
  4. With the experiment started, iterate through the participants as prompted, entering a chat message.

Expected: After the last participant has entered a message, the first participant should be prompted to continue.

Actual: The last participant is prompted a second time. After they enter a second comment, then the conversation rotates back to the first participant.

In the following screenshot, you can see that the third active participant, camel-red-7804, was prompted for both the third conversation turn (C, correct) but also the fourth (D).

Screenshot demonstrating the last-participant-double-prompted bug

@jimbojw
Copy link
Copy Markdown
Collaborator

jimbojw commented May 19, 2026

Note: In my test example, I included only simulated "human" participants, with no mediators. That may be relevant to the bug.

@jacyanthis
Copy link
Copy Markdown
Author

Thanks! I've modified it to prevent back-to-back turns of the same participant when in a no-mediator Group Chat. Specifically, it will use a single turn order for each 'cycle' of the chat, so the person who goes first in a cycle never went last in the previous cycle. This can still occur if there is a mediator, but then there is an intervening mediator turn between cycles.

I checked for other DL functionalities that might not mesh intuitively with turn-based chat. For new participants or mediators being added mid-conversation, I modified the PR so it appends new mediators to the initial mediator 'phase' of the cycle and appends new participants to the secondary participant phase. When a new mediator is added to a no-mediator Group Chat, the turn order will remain fixed.

These seem like the most intuitive options to me, though I don't expect these corner cases to come up with turn-based conversations, or at least not with our intended usage. Alternatively, some other functionalities could be disabled when turn-based is enabled.

@cjqian
Copy link
Copy Markdown
Member

cjqian commented May 25, 2026

LGTM! I haven't pulled this locally, but one minor P2 improvement
image
may be having a little indicator in the participants panel to show whose current turn it is. Something to consider!

@jacyanthis
Copy link
Copy Markdown
Author

Thanks! I added several commits to address various possible issues for turn management (e.g., handling repeated API failures, turn skips for ineligible participants), and I added that turn indicator to the participants panel.

@jimbojw
Copy link
Copy Markdown
Collaborator

jimbojw commented May 26, 2026

@jacyanthis I'll try the latest. You'll want to rebase against main. We had to bump a dependency version to restore GitHub Actions functionality. See #1132

@jacyanthis
Copy link
Copy Markdown
Author

@jimbojw sounds great! Rebase done.

Copy link
Copy Markdown
Collaborator

@jimbojw jimbojw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code looks good. I ran it and can confirm that things appear to be working correctly.

I did notice a bug, but I don't know if it's a new one, or it was there previously. In the following screenshot, bear just finished their turn. For conversational simplicity, I was piloting each participant to just enter letters of the alphabet sequentially ('A', 'B', and so on).

Screenshot showing multiple message bug

On bear's turn, I entered 'F', then rapidly hammered the Enter button. This caused a sequence of 'F' messages from bear even though, presumably, only one message should be allowed.

The multiple messages only go through during the intermediate period in which the send button has been replaced with a spinner. Once the first such message goes through, later taps of Enter do not spawn additional messages.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants