#332 – AI-Generated Summaries
Friday Ship #332 | January 27th, 2023
This week, we’re beginning to roll out our new AI-generated meeting summary feature. With the help of GPT-3, we hope that the feature will save users time by helping them quickly grok what happened in a meeting.
How does it work?
In a retro meeting summary, we currently show you the reflections that were added as well as information about the tasks and comments that were added.
These summaries are helpful for team members to understand what happened in a meeting when they weren’t able to attend.
However, when there are dozens of reflections in each group and hundreds of comments, it can take a while to digest what happened.
We’re hoping that our AI-generated summaries change that.
At the end of the meeting, we now send the text from all of the comments, tasks, and reflections to GPT-3 and ask it to summarise what happened. We then share the response in the meeting summary (which is emailed to the members of the team), as you can see in the screenshot above.
We add an explainer sentence, and we include a 🤖 emoji to indicate that the feature was generated by an AI and could therefore be inaccurate.
We also include a Topic Summary for each reflection group and a Discussion Summary which summarises all of the comments and tasks in the discussion thread.
Finally, we add a Topic Summary to the discussion thread to help everyone quickly understand what the reflection group is about so they can begin the conversation.
What’s it like working with GPT-3?
This idea originated during a recent company Hack Week where one of our developers, Bruce, demoed how the feature could work.
Those of us working on making our ‘Team’ tier more tantalising felt that it would be a good premium feature. This is because larger companies, who typically have larger budgets, are more likely to run into the problem of having many reflections in a meeting and have managers that want to understand what many teams are discussing.
We got the feature ready for production and released it internally to users with the parabol.co email domain. Strangely, we soon noticed that there were formatting issues in some of the summaries.
The summaries would sometimes begin with a full stop or a random word.
After reading through this OpenAI blog on prompt engineering, we changed our prompt from:
prompt: `Summarize this for a second-grade student in one or two sentences: ${text}`,
to:
prompt: `Below is a comma-separated list of text. Summarize the text for a second-grade student in one or two sentences.
Text: """
${text}
"""`,
This solved the issue – the summaries are now correctly formatted and more accurate.
It’s been fun learning about the new world of prompt engineering and seeing what GPT-3 can do! This is the first feature we’ve built using OpenAI’s model, and we’ve got many more ideas in the pipeline.
We’re excited to hear what you think about this feature! To gauge how users feel about this feature, we’re tracking the emojis that are added to the AI comment, we’ll add a chatbot to the summary to get qualitative feedback, and we’ll be using the feature internally.
If you have any thoughts on the feature or ideas on how it could be improved, please drop us an email at love@parabol.co.
Metrics
A small dip in weekly meetings but a strong bump in web metrics and monthly active users.
This week we…
… had three days off in celebration of the Lunar New Year following our inclusive holiday policy
… completed our designs for a new checkout flow
… began rolling out our AI-generated summaries feature
Next week we’ll…
…wrap up a Batting Practice for a Senior Full-Stack Engineer candidate
