Retrospectives

Retrospectives is an Azure DevOps extension to perform smart and efficient retrospectives from within the Azure DevOps pipeline.

Retrospectives are an important practice in becoming an effective team, allowing the team to gather feedback and continuously improve based on the feedback.

Research from the 2018 State of DevOps report indicates that Elite teams are 1.5 times more likely to consistently hold retrospectives and use them to improve their work. Furthermore, a 2013 meta-analysis on teams indicates that teams that effectively debrief/conduct retrospectives are 20-25% more effective.

Teams often use external retrospective tools, white boards with Post-its, OneNote, etc. to conduct retrospectives. The data then needs to be collated, and any actionable items need to be created in your Azure DevOps pipeline.

The Retrospectives extension allows you to do all this and more from within Azure DevOps.

Table of Contents

• Retrospectives
  • Retrospective Features
  • Install
  • Use
  • Best Practices
  • Contribute
  • License

Retrospective Features

• Retrospectives in Azure DevOps
• Real time support for distributed retrospectives
• Ability to create Azure DevOps work items associated to feedback
• Create an emailable summary of a retrospective

Install

The extension can be installed from Azure DevOps Marketplace.

Use

Please note: the screenshot examples use light mode and dark mode interchangeably because both are supported!

1. Open Retrospectives Extension

Navigate to the Azure 'Boards' tab in your account on the left hand navigation. Select the 'Retrospectives' tab under 'Boards'.

2. Pick The Team

You are now on the Retrospectives page. Select your Azure DevOps team from the selector in the header.

3. Create New Retrospective

Create a new retrospective using the 'Create Board' button. If this is your first retrospective for your selected team, then press the "Create Board" button in the center of the screen.

If you have created a retrospective before for your selected team, then you can select "Create a new retrospective" from the navigation drop down.

4. Choose Your Retrospective Settings

When you select New Board or Create new retrospective as above, you will see the following dialog:

Please enter the appropriate information:

• Retrospective Title: Title for the Retrospective.
• Max Votes per User: The maximum number of votes a participant has to use in the "Vote" Phase.
• Include Team Assessment: Include a Team Assessment link at the top of the board.
• Obscure the feedback of others until after Collect phase: When selected, users cannot see other users input until they have moved to another phase. Other users' feedback will be blurred.
• Display 'Retrospective Prime Directive': Include a link to the 'Prime Directive' at the top of the board.
• Do not display names in feedback: When checked, anonymize who creates individual feedback items.
• Columns Settings: You can either Apply from a pre-populated template or individually select and configure columns yourself.

Note: Retrospective Title is the minimum 'required' information (other fields can stay at default as needed). Once the title is provided, the Save button is enabled. Save the retrospective using the Save button.

... Or Select Another Retrospective Board

Once you have created the retrospective board and you want to select a retrospective board different from the currently displayed board, click on the retrospective board name and select the desired one. You can use the search box to find the appropriate retrospective boards if you have a large number of boards.

5. Share the Link to the Retrospective

Once you create a new retrospective, you can share a link to it to all participants. Users can then access that link even from their mobile browsers to participate in the retrospective. The extension offers real time synchronization, so all users will see the most up-to-date information without having to refresh the page.

6. Phases of the Standard Retrospective

The number of columns in each retrospective is determined by the selection made when making the new board. Regardless of the template, they still follow the same sequence of phases.

The Prime Directive

If you had selected the 'Display 'Retrospective Prime Directive', you will see the option to view that directive which sets the stage for the retrospective. It is recommended that you click on the Prime Directive and read it out loud for everyone to hear. Remind everyone that any issues discovered will be assumed to be process problems, not people problems.

Collect

In this phase feedback is collected from all participants. Users can add feedback under any of the columns using the 'Add new card' button or by double clicking the empty space of the column's background.

If when creating a retrospective, the checkbox for "Do not display names on feedback" is checked, then the cards will show no names, appearing anonymous.

If when creating a retrospective, the checkbox for 'Obscure the feedback of others during the Collect Phase' was checked, then while typing, the feedback of others will not be shown, even if anonymous. You can only edit items that you have created in this mode.

Once feedback from all users is collected, move on to the next phase of Group.

Group

In this phase, any similar feedback can be grouped together beneath a "header" or champion item. If you feel 2 feedback items are similar, drag one onto another to group them together. Dragging any item onto a group, will add items to that group.

If you would rather not drag you can use the feedback item's action menu (the ellipsis '...') to "Group Feedback". This will bring up a search box and you can type in the feedback item you would like to group your selected item under.

Both methods of grouping will result in the item being grouped beneath the other.

Once all similar items are grouped together, move on to the next phase of Vote.

Vote

In this phase, participants will individually go through all the feedback items and vote on the ones they feel are important, by clicking on the 'Upvote' icon. Users can reduce their number of votes on a specific item by clicking on the 'Downvote' icon.

Note the Max Votes per User set by the board creator are displayed as the denominator in a ratio of "Votes Used". Votes used by the user are updated in real time as the user clicks on 'Upvote' and 'Downvote' icons.

Once everyone is done voting, move on to the next phase of Act.

Act

In this phase, the team will go through each feedback item. You will notice that the items with the most votes will appear at the top of the column. It is recommended that each time you do this exercise, you only select one or two of the feedback items to take action against. Click on the 'Add action item' button on a feedback card, and select the type of work item that needs to be created in Azure DevOps.

This will open up the standard Azure DevOps work item creation form. Enter the work item details and save. This will create the work item in your Azure DevOps account and also associate it to the feedback item.

7. Optional: The Team Assessment

You can optionally include a "Team Assessment" that anonymously collects feedback from participants. To get the feedback form, you can click on the "Team Assessment" link at the top of the board.

The assessment tracks 6 categories:

• Clarity
• Energy
• Psychological Safety
• Work-life Balance
• Confidence
• Efficiency

The users will rank on a scale of 1 to 10; scores of 1-6 are categorized as "Unfavorable", 7 and 8 are "Neutral," and 9 and 10 are "Favorable."

Once the team assessment has been completed, the summary of answers can be viewed in the Retrospective Summary Dialog (see below).

Improving Based on Team Assessment Scores

Teams may choose to address the issues found in the team assessment in multiple ways:

• Through open conversations during the standard retrospective process to identify opportunities for improvement.
  • While this approach works, it may not be effective if the team isn't psychologically safe, and our research indicates that only 10% of the teams that do team assessments are psychologically safe.
• Through a Team Assessment Retrospective
  • A Team Assessment Retrospective is a new approach to retrospectives allowing teams to identify the largest opportunities for improvement based on each team members' responses to the team assessment.
  • Section 7.1 below outlines the Six Steps in a Team Assessment Retrospective

7.1 The Six Steps To a Team Assessment Retrospective

Setup

Create a new retrospective using the steps above with the following exceptions:

• 'Include Team Assessment' is checked
• 'Display 'Retrospective Prime Directive'' is checked
• 'Do not display names in feedback' is checked
• Note: You do not need to select a template. The template will be modified after the assessment is complete
• Share the link to the retrospective with the team

The Prime Directive

In this phase the facilitator sets the stage for the retrospective. It is recommended that you click on the Prime Directive and read it out loud for everyone to hear. Remind everyone that any issues that discovered will be assumed to be process problems, not people problems.

Team Assessment

In this phase each team member clicks on the Team Assessment and anonymously answers all questions and hits "submit". It is critical that you remind everyone that all feedback is anonymous and confidential.

Throughout this phase you will periodically check the retrospective summary.

If the majority of the team hasn't responded to the assessment, close the summary dialog and encourage everyone to participate, reminding them that if their voice isn't expressed in the assessment it can't be acted upon.

Re-open the retrospective summary and when the majority of the team has responded, discuss which dimension the team would like to improve upon. Note: If more than 20% of the responses are unfavorable for a dimension, we recommend focusing on that area first. Otherwise, it is beneficial to focus on psychological safety first. This is because without psychological safety the more difficult conversations that need to happen to make improvements in the other dimensions will not happen.

Once the team has decided which dimension to focus on, the facilitator will click the button "Discuss and Act" which aligns to the dimension the team wants to focus on. This will create retrospective with a template that aligns to the dimension.

Collect

In this phase team members will provide feedback to help improve the dimension they've chosen to focus on. Ask team members to input feedback cards on the left 2 columns.

After a few moments, the facilitator reads the cards outloud. When it's apparent that no additional feedback cards are being added, the facilitator asks the following question: "Reflecting on the feedback we've collected, if you could take only one action in the next sprint that would yield the biggest improvement on [dimension team is focused on improving from Team Assessment] what would it be? Capture this in a feedback card in the third column (One action in the next sprint)"

Group

In this phase the team will group similar items together. To expedite the process we recommend only grouping the action items together in the last column. After the majority of the team has added feedback cards to the third column, the facilitator switches to the "group" tab and collectively groups like ideas together with the team (see above).

Vote

In this phase the facilitator asks participants to vote only on the third column and the top items for improvement are identified. The facilitator then waits for votes to compile and after the majority of the team has voted (the facilitator can view the retrospective summary to quickly determine how many votes have been cast), the facilitator clicks on the "Act" tab which automatically sorts the cards from the most to the least voted upon.

Act

Follow the steps above to assign actions as work items in Azure DevOps. Note: We recommend only selecting 1-2 actions per sprint to ensure completion prior to the next sprint.

8. Retrospective Summary

The Retrospective Summary dialog is accessible from the "Board Actions" menu in the top navigation.

Once there, there is a summary of the Retrospective. This includes:

• Who created the board
• Number of participants
  • If not anonymous, it will show who participated
• Number of feedback items created
• Number of votes cast
• Number of action items created

If there is a team assessment, then a charted summary will show the vote breakdown by favorability, along with an average score.

10. Board History

There may be times that a team may want to reflect on past retrospectives they have run. There is a History tab in the top navigation of the tool, next to "Board" header.

From here, one can look at past retrospectives and see a few high level stats and the action items created after each by clicking the individual retrospective.

Best Practices

Plan for a Team Assessment Retrospective

Planning in advance will help you make the best use of the team's time in completing a Team Assessment Retrospective.

In Advance of the Event

When	Action
Week in advance (more or less depending on your team’s rhythm of business)	Create and send a calendar invite for 50 minutes for the Team Assessment. Make sure that all team members can be present.
3-5 days prior	(1) Create the Retrospective in ADO; (2) Create the schedule and timebox each section; (3) Update the meeting invite with the schedule; (4) Review action items from previous retro to familiarize yourself and be prepared to share

At the Event

Activity	Time	Guidance
Welcome & Kickoff	10 min	(1) Welcome the team; (2) Review Agenda; (3) Review actions from last retro and make any updates or clarify next steps; (4) Kick-off with the retro
Conduct Team Assessment	5 min	Share link and give team members 2 minutes to complete team assessment
Identify area for deep dive	10 min	Discuss results and agree on area for deep dive
Conduct the deep dive retro	15 min	Collect, Review & Group Columns 1 & 2
Determine action plan(s)	10 min	Collect, Review, Group and Vote on Column 3
Close the meeting	2 min	Thank attendees for participation and summarize

Use Retrospectives to give Kudos

Giving kudos, or recognizing others for their achievements, is one of the highest drivers of employee satisfaction and retention. When people feel appreciated and supported, they will be more engaged and motivated. Who doesn’t want that? You have an opportunity to provide a space for kudos in every team retrospective. Make sure to add a column for “Kudos” in your retrospective board. It is that simple and it just takes a moment for people to add a few pieces of feedback to acknowledge the great work of others. When you finish your retrospective, make sure to read through each kudos feedback item and take a moment to celebrate!

Use Retrospectives to create a Team Working Agreement

Team Working Agreements are shared commitments to ways of working together. They establish team members' work preferences. A shared agreement will balance individual and team needs, and set clear expectations for how the team will work together.

A good team working agreement must include everyone's background, needs and voices so a retrospective is a great way for a team to collect that information and then bring it together. The agreement should be relatively short, 5-7 items.

Bring your team together and create a retrospective board with 1 column:

• Good Working Preferences

Follow the phases through from Collect, Group and Vote. Make sure every team member contributes feedback items to the Board before you Group and Vote. Read every card outloud before Grouping like items together. After voting, you will see those working preferences that got the most votes come to the top.

Select the top 5-7 items and talk through them together as a team to ensure that they are clear and that everyone agrees. Put those items in a document that you can keep in front of your team (at the top of a Microsoft Team Channel, or in your Azure DevOps Dashboard). Review the Working Agreement at the beginning of Team ceremonies or events. Whenever there is a change in the team composition, the Working Agreement should be reviewed again to ensure that all team members agree to each item.

Team Assessment Best Practices

Below are a number of best practices for driving team effectiveness and using the “Team Assessment” measurement within the Retrospective Board.

• The ‘Team Assessment’ measurement questions were selected based on significant research indicating improvements in these areas will result in significant improvements in overall team effectiveness. It is recommended that teams conduct a ‘Team Assessment’ measurement as a part of their standard retrospective process once a month at a minimum. Deep dive discussions to debrief on the area of the biggest opportunity with actions identified should be a part of that process.

• Best practices for improving clarity include creation of and alignment to OKRs. Regular review of OKRs with stakeholders combined with the implementation of agile development methodologies to continually prioritize the work that will have the biggest impact on achieving the OKRs. A well groomed backlog that is visible to the entire team and reviewed regularly will ensure clarity. The backlog should be stack ranked in order of priority with minimum work in process will ensure team clarity on the work that matters the most. Drive continuous improvement in clarity by using the “Clarity” Retrospective Board template, following the standard retrospective process.
• Best practices for improving energy include ensuring that all team members are spending at least 20% of their time on the team doing work that they love. Each team member should embark on a journey of self-discovery to identify what they love by keeping a record of the work that they loved doing on a daily and weekly basis until their strengths are clearly identified. Managers should endeavor to understand the work that their employees love by conducting a simple weekly check-in and asking 4 simple questions: "What did you do last week that energized you (you loved)?”, “What did you do last week that drained your energy?”, "What are your priorities this week?", and "How can I (your manager) help?". Drive continuous improvement in energy by using the “Energy” Retrospective Board template, following the standard retrospective process.
• Best practices for improving psychological safety include conducting a baseline retrospective using the "Psychological Safety" template in Azure DevOps Retrospectives (anonymously). Identify top actions that the team will implement to improve psychological safety. Repeat the psychological safety retrospective every 4-6 weeks until there are no scores below a 7. We've learned that every team is different and, in most cases, simple changes can make a profound impact, but it requires asking anonymously and taking action to improve. Drive continuous improvement in psychological safety by using the “Psychological Safety” Retrospective Board template, following the standard retrospective process.
• Best practices for improving work-life balance include the implementation of fundamental agile practices including backlog grooming and prioritization, limiting Work In Process by establishing WIP limits and adhering to them, and only committing to deliver at the same velocity as the data indicates the team has predictably delivered in the past. Finally, simply asking the question in the Team Effectiveness survey and conducting deep dive discussions to identify drivers combined with follow-through on the associated actions should result in increased sustainability for the team. Drive continuous improvement in work-life balance by using the “Work-Life Balance” Retrospective Board template, following the standard retrospective process.
• Best practices for improving Confidence include sharing Objectives and Key Results (OKRs) from different levels across your organization. Bringing transparency into the OKRs can help teams and individuals better understand how their work aligns to the organizations' priorities and can build confidence in doing the right work to deliver against those OKRs.
• Best practices for improving Efficiency can also include the implementation of agile practices including backlog grooming and prioritization. Limiting Work In Process by establishing WIP limits and adhering to them enables teams to be more efficient so they can "stop starting and start finishing"!

Contribute

See Contributing Guideline

License

Copyright (c) Microsoft Corporation. All rights reserved.