Measure your Retrospectives Effectiveness with the Agile ROTI
At the end of an Agile retrospective, workshop or meeting, you may have already participated in a ROTI.
This technique aims to measure the level of effectiveness of the meeting by focusing on the return on time invested by each participant.
Intuitive and fun, ROTI is a great tool that will help you improve the facilitation of your meetings by exploring your team’s feelings and ideas about the team’s retrospectives.
In this article, we will give you our best advice for building your ROTIs and effectively analyzing the feedback from your colleagues.
ROTI? But why?
This technique takes its name from the acronym Return On Time Invested.
To fully understand the raison d'être of ROTI, let's look at this famous saying:
“Remember that time is money.”
- Benjamin Franklin
Everyone wants to optimize their time invested as much as possible, and as Ben Franklin so eloquently puts it in the above quote, time is money. Team meetings are no exception and are arguably one of the most important places to optimize the team’s time.
Evaluating the ROTI of a meeting means measuring the effectiveness and value of the meeting from the perspective of team members. By allowing everyone to express themselves, you truly make the group’s voice the top priority for improving all aspects of the team’s dynamic.
Here are three direct benefits of using ROTI:
Measure the effectiveness of the retrospective through the prism of the time invested
Reinforce the level of psychological security by allowing everyone to express themselves authentically
Create a safe space to suggest ideas to improve the quality of the next retrospective
The problem with the ‘regular’ ROTI
Traditionally, the ROTI uses a show of hands from the team in order to evaluate the return on the time invested on a scale from 1 to 5. The facilitator then invites all the participants to comment on the shared score, with particular emphasis on the lowest score and the highest score.
But, to be honest... this is the ROTI format that we really don't like at Neatro.
Why? Two factors lead us to think this way.
First, the grading scale is often not clearly defined. For example, a 3/5 rating can have one meaning for one colleague, but then a completely different meaning for the other. Therefore, we cannot really trust the results of this exercise because of its lack of clarity and accuracy.
Secondly, voting by a show of hands and sharing comments without the ability to be anonymous is certainly not the best way to create a space where everyone feels free to share their genuine feelings.
Suppose you have a complicated relationship with your manager. Let's say your boss just facilitated the team meeting, and this session was a complete waste of time for you. Are you really going to be gutsy enough to rate the meeting 1/5 in front of everyone, including your manager? Are you going to feel comfortable justifying your score if the risk of possible consequences outweighs the benefit of expressing your opinion?
These types of emotions and relationships surrounding team meetings are not rare and are still very common with many companies these days, as more and more of them focus solely on creating fast-paced and productivity-fueled environments. All of these factors, in addition to not feeling comfortable sharing your opinion with your boss or colleagues, can make for a dangerous work environment.
Aino Vonge Corry describes this phenomenon as ‘The Political Vote’ in her book named Retrospectives Antipatterns. When the team is operating in a dangerous work environment, team members might tend to withhold their opinions and be influenced by the rest of the group.
An efficient and safe ROTI structure
We have seen that two components structure a ROTI: providing a score first and sharing comments second:
1/ Assign a score
First, make sure the vote remains anonymous - you will reinforce the level of psychological safety of the group that way. We will see later that having Neatro as a tool offers you this guarantee.
Here is an example of ROTI that will help your colleagues evaluate the meeting with an unambiguous rating scale. Feel free to take inspiration from this effective retrospective meeting template and use it for any kind of team meeting outside of your retrospectives.
1/5 - Useless meeting
I feel like I’ve completely wasted my time.
The retrospective hasn’t been productive at all.
2/5 - Inefficient meeting
I could have used my time in a better way.
We can definitely do better as a team.
3/5 - Decent retrospective
I think my time has been quite well spent.
The retrospective was fairly productive.
4/5 - Good retrospective!
Overall I’m happy with the time I’ve invested in this meeting.
The retrospective was productive.
5/5 - Great retrospective!
Every minute I spent was valuable.
I feel like this retrospective has helped us grow.
2/ Share comments
We recommend that you make your comments anonymous, but you can take this one step further!
If you have the chance, it is even better to link the overall score to a comment while preserving the author's anonymity. Tools like Neatro make this possible for you.
If you're hosting a face-to-face meeting, things can get a little trickier here. In that case, we suggest that you use a tool that enables anonymity during the entire ROTI activity.
Reveal the ROTI results
Once team members have completed their ROTI, it's time to reveal the results. Below, we share some tips on how to approach the facilitation of this exercise properly.
As a meeting facilitator, you can start by announcing the average ROTI score and describing the distribution of scores (how many rated 1/5, how many rated 2/5, etc.).
Depending on the number of participants and the nature of the meeting, you can ask participants to react to the average score. Make it optional, though. You can also compare it to the previous meeting's ROTI (it is valuable only if the meeting is a recurring event, like a Sprint Retrospective).
Next comes the reveal of the anonymous comments. An intuitive way to navigate through all of the comments is to separate constructive comments from positive comments. Simply put, begin with checking all of the improvement suggestions and then finish with the positive comments.
Sharing comments helps to bring more transparency and collaboration within the group, no matter what their content is. In addition, each participant can learn from these comments and put their learnings into practice for future meetings.
Ultimately, these comments may help your next meeting's facilitator better address current challenges.
How to interpret ROTI results
Remember that ROTI is a way for the team to express themselves and share their feelings about the value brought by the meeting. Therefore, the concept of continuous improvement will always be connected and directly tied to your ROTI.
Make sure you adopt an open-minded perspective and a willingness to listen to others. After all, you are the meeting facilitator, and you want to find out what went well and what you could improve for the next session. Being open to what your team has to say is what will allow you to better and more accurately evaluate your retrospectives.
Of course, two potential scenarios emerge. Through the ROTI, your meeting delivers either a positive return on investment… or a negative one.
The ROTI shows a negative result. What should I do?
By a 'negative return on investment,' we mean an average ROTI score under 3/5. Remember that the lowest score possible is 1/5.
The result that emerges here suggests that the participants did not perceive enough value in the meeting and that their time was not well spent.
Do not be too harsh on yourself. It happens to everyone to facilitate or participate in meetings that seem unproductive. You must know how to separate the evaluation of the meeting from the assessment of your skills as a facilitator.
After studying the score and the comments, it is very likely that you will detect what did not work well.
Were the expectations of the meeting well established?
Have the individuals fully understood their role in the project - including the meeting that has just ended?
Has everyone been able to express themselves on the plan and the next steps?
Was it necessary to mobilize the team for 1 or 2 hours to review the project progress?
Discovering, analyzing, and understanding team feedback is a crucial part of the exercise.
In order to gain a positive learning experience and improve the quality of the next meeting, we recommend that you craft at least one or two action items that you can commit to by the next meeting.
👉 Pro tip: Consider using the SMART methodology to build your goals.
The ROTI shows a positive result. What should I do?
By positive result, we mean any average ROTI greater than 3/5 (or equal to 3/5). For the sake of brevity, we will not distinguish a positive result from a very positive result - simply because the conclusion remains the same.
You may think at first glance that if there is a positive result, then there would be no need to think about ways to improve future meetings.
This is an easy mistake to make! In your ROTI you will always commit to continually identifying areas for improvement 🙃
Positive feedback means something is working and you could apply what is working to the other areas in need of improvement. You can learn from this positive retrospective by extracting the reasons and root causes that led to its success. Gather all of these positive observations and shape your own guide to best practices for conducting a team retrospective.
These anonymous comments may have also helped your colleagues express suggestions for improvement in an authentic way. Continue to take this opportunity to build an action plan made of one or two action item(s) to boost the quality of your next performance as a facilitator.
Continuous improvement is a process that fits all situations and is not exclusively related to negative feedback. There is always room for improvement, positive or negative; you just have to look close enough and have an open mind.
Get the most out of your retrospectives with Neatro’s ROTI
At Neatro, we are driven by a powerful mission: we want to help modern teams improve continuously. We offer tools that provide an open, transparent and safe space for conversation to unlock the potential of every team.
In our opinion, the Agile retrospective is a healthy exercise that supports the team in its quest for performance and well-being. If you're not a retrospective connoisseur yet, don't miss our introductory guide to the Agile Retrospective.
As previously discussed, a successful practice of ROTI requires allowing everyone to express themselves without being influenced by the rest of the group as well as an uncompromising respect for anonymity.
Thanks to Neatro, you benefit from a retrospective experience that respects these principles to offer your team a high level of psychological safety.
So what does a ROTI actually look like in the Neatro experience?
Each participant in the retrospective is first invited to rate the meeting.
To do this, we have prepared an accurate scale that makes it easy for everyone to understand how to evaluate the meeting's return on time invested.
Of course, your individual vote remains anonymous at all times, even once the results are revealed.
Immediately after giving a rating, each participant has the opportunity to share comments, suggestions for improvement, or general remarks about the meeting while still remaining anonymous.
Note that the person who facilitated the meeting can abstain from voting.
For the sake of transparency, the ROTI results (scores and comments) are shared with the entire team. Through a simple and fun interface, you will observe at a glance the trends emerging from this ROTI.
The more retrospectives you complete with your team, the more you will be able to measure the progress being made in these meetings.
If you are committed to helping your team improve and become more efficient, don't hesitate to launch a Neatro retrospective. It's free, no credit card is required.
Happy retrospectives!