Case study
Measuring mentoring impact
Justifying continued funding and support for professional migrant & refugee women
UX
UI
Web
Case study
Measuring mentoring impact
Justifying continued funding and support for professional migrant & refugee women
UX
UI
Web
Case study
Measuring mentoring impact
Justifying continued funding and support for professional migrant & refugee women
UX
UI
Web
Summary
Professional Migrant Women (PMW) is a community focused on increasing the visibility and representation of migrant women in professional Australia. It offers mentoring programs to help migrant and refugee women overcome unemployment or under-employment. PMW relies on external funding but struggles to gather long-term impact data to justify support.
As part of a broader program of research by Dr. Anh Nguyen Austen, I designed a responsive web-based solution to collect and present long-term impact data to attract support.
User research indicated the need for a solution catering to multiple roles (mentee, mentor, mentor admin) and addressing the unique challenges migrant and refugee women face in reestablishing their careers. A key focus was helping them understand themselves and their strengths in a new context.
I prototyped and tested high-fidelity screens for mobile and desktop for each role. The designs were well-received for ease of use and value but still have room for improvement.
Methodology
User interviews
Competitor analysis
Personas
Finding opportunities (POVs/HMWs)
Defining the solution & features
User flows
Low fidelity wireframes & initial testing
Prototyping & usability testing
Revisions
Summary
Professional Migrant Women (PMW) is a community focused on increasing the visibility and representation of migrant women in professional Australia. It offers mentoring programs to help migrant and refugee women overcome unemployment or under-employment. PMW relies on external funding but struggles to gather long-term impact data to justify support.
As part of a broader program of research by Dr. Anh Nguyen Austen, I designed a responsive web-based solution to collect and present long-term impact data to attract support.
User research indicated the need for a solution catering to multiple roles (mentee, mentor, mentor admin) and addressing the unique challenges migrant and refugee women face in reestablishing their careers. A key focus was helping them understand themselves and their strengths in a new context.
I prototyped and tested high-fidelity screens for mobile and desktop for each role. The designs were well-received for ease of use and value but still have room for improvement.
Methodology
User interviews
Competitor analysis
Personas
Finding opportunities (POVs/HMWs)
Defining the solution & features
User flows
Low fidelity wireframes & initial testing
Prototyping & usability testing
Revisions
User research
In conjunction with the researcher (who asked additional related questions), I chose to interview 5 people remotely for around 60 minutes each:
Each was from a different cultural background
2 have been mentees in various mentoring programs
3 have either mentored and administered mentoring
The line of questioning was focused on understanding:
How information is collected
What information is or can be collected
How that can be interpreted to make decisions, so we can streamline data collection and output data in a meaningful way so it can be used to make decisions.
As whatever solution is designed would inherently have multiple roles (whether mentee or mentor), and the interviewees themselves varied in their experience of those roles, I created separate sets of questions for each role.
User research
In conjunction with the researcher (who asked additional related questions), I chose to interview 5 people remotely for around 60 minutes each:
Each was from a different cultural background
2 have been mentees in various mentoring programs
3 have either mentored and administered mentoring
The line of questioning was focused on understanding:
How information is collected
What information is or can be collected
How that can be interpreted to make decisions, so we can streamline data collection and output data in a meaningful way so it can be used to make decisions.
As whatever solution is designed would inherently have multiple roles (whether mentee or mentor), and the interviewees themselves varied in their experience of those roles, I created separate sets of questions for each role.
User research
In conjunction with the researcher (who asked additional related questions), I chose to interview 5 people remotely for around 60 minutes each:
Each was from a different cultural background
2 have been mentees in various mentoring programs
3 have either mentored and administered mentoring
The line of questioning was focused on understanding:
How information is collected
What information is or can be collected
How that can be interpreted to make decisions, so we can streamline data collection and output data in a meaningful way so it can be used to make decisions.
As whatever solution is designed would inherently have multiple roles (whether mentee or mentor), and the interviewees themselves varied in their experience of those roles, I created separate sets of questions for each role.
Findings
Roles
The interviews made clear that there are three roles to consider in any solution, each with their own goals: mentees, mentors and mentor administrators, who manage the mentoring programs. However, sometimes people may fulfil multiple roles.
Data
Currently PMW do not collect much data on mentees, let alone detailed long-term impact data. Limited data is collected in voluntary surveys after activities. Understanding economic return on investment would be beneficial, as would be the ability to tailor mentoring programs.
Connections
A primary purpose and benefit of the mentoring programs is for refugees and migrants to foster their own local networks, both professional and social. When professional refugees and migrants arrive in Australia, they generally have little to no social network that can help them continue their careers here. In addition there can be cultural and language barriers that make it extra difficult for them.
Understanding strengths
Moving to a foreign country can cause migrants self-confidence to plummet. A key part of the mentoring experience is mentees rebuilding that confidence, as well as learning to understand the value they can bring to workplaces in Australia, and how they can communicate that value.
Perspective
Moving to a new country with a different culture can cause people to feel lost, depressed and like they have lost their identity. Over time they can adjust, appreciate the new freedoms they now have (if they come from a more conservative culture), foster a new sense of identity, and understand the new possibilities available to them in terms of their live and career.
Findings
Roles
The interviews made clear that there are three roles to consider in any solution, each with their own goals: mentees, mentors and mentor administrators, who manage the mentoring programs. However, sometimes people may fulfil multiple roles.
Data
Currently PMW do not collect much data on mentees, let alone detailed long-term impact data. Limited data is collected in voluntary surveys after activities. Understanding economic return on investment would be beneficial, as would be the ability to tailor mentoring programs.
Connections
A primary purpose and benefit of the mentoring programs is for refugees and migrants to foster their own local networks, both professional and social. When professional refugees and migrants arrive in Australia, they generally have little to no social network that can help them continue their careers here. In addition there can be cultural and language barriers that make it extra difficult for them.
Understanding strengths
Moving to a foreign country can cause migrants self-confidence to plummet. A key part of the mentoring experience is mentees rebuilding that confidence, as well as learning to understand the value they can bring to workplaces in Australia, and how they can communicate that value.
Perspective
Moving to a new country with a different culture can cause people to feel lost, depressed and like they have lost their identity. Over time they can adjust, appreciate the new freedoms they now have (if they come from a more conservative culture), foster a new sense of identity, and understand the new possibilities available to them in terms of their live and career.
Defining the target audience
Based on my research, I defined three archetypes each with their own challenges, frustrations and goals. These archetypes represent the three roles of users within the solution: mentees, mentors and mentor admins.
These are their goals:
Fatima, the under-employed mentee
Wants to develop skills she can use to find meaningful employment in Australia
Wants to know her own value to herself and to others in Australia
Wants to be more connected and part of a community in Australia
Wants to grow as a person and develop her potential
Mei, the mentor
Wants to more easily understand her contribution as a volunteer mentor
Wants to know how what their mentoring is affecting mentee’s lives
Wants to be able to quantify their contribution as a volunteer
Wants to understand how she can improve the mentoring she gives, so that it meets mentees needs better
Camila, the mentoring administrator
Wants to more easily quantify the long-term impact mentoring programs are having on mentees
Wants to know how and why mentees enter the programs
Wants to know how what their mentoring is affecting mentee’s lives
Wants to be able to quantify the long-term impact of mentoring in terms of key indicators - self-advocacy, representation, empowerment, identity, visibility, leadership, connection and skills
Wants to quantify the economic return of the mentoring program
Defining the target audience
Based on my research, I defined three archetypes each with their own challenges, frustrations and goals. These archetypes represent the three roles of users within the solution: mentees, mentors and mentor admins.
These are their goals:
Fatima, the under-employed mentee
Wants to develop skills she can use to find meaningful employment in Australia
Wants to know her own value to herself and to others in Australia
Wants to be more connected and part of a community in Australia
Wants to grow as a person and develop her potential
Mei, the mentor
Wants to more easily understand her contribution as a volunteer mentor
Wants to know how what their mentoring is affecting mentee’s lives
Wants to be able to quantify their contribution as a volunteer
Wants to understand how she can improve the mentoring she gives, so that it meets mentees needs better
Camila, the mentoring administrator
Wants to more easily quantify the long-term impact mentoring programs are having on mentees
Wants to know how and why mentees enter the programs
Wants to know how what their mentoring is affecting mentee’s lives
Wants to be able to quantify the long-term impact of mentoring in terms of key indicators - self-advocacy, representation, empowerment, identity, visibility, leadership, connection and skills
Wants to quantify the economic return of the mentoring program
Finding opportunities
From POVs/HMWs I identified some key problems to solve for (one for each persona/role):
How might we help mentees to see their improvement over time of their skills developed through mentoring so they can feel positive about their future in Australia?
How might we help mentors to track their volunteer contributions so they can receive compensation for their in-kind contributions?
How might we help people administering the mentoring programs to better track and understand the long-term impacts of mentoring on mentees, so they can attract and justify support and investment?
Finding opportunities
From POVs/HMWs I identified some key problems to solve for (one for each persona/role):
How might we help mentees to see their improvement over time of their skills developed through mentoring so they can feel positive about their future in Australia?
How might we help mentors to track their volunteer contributions so they can receive compensation for their in-kind contributions?
How might we help people administering the mentoring programs to better track and understand the long-term impacts of mentoring on mentees, so they can attract and justify support and investment?
Defining feature set
Based on the key problems, I decided what features should be included in any solution and prioritised them.
Required for MVP
Mentee self-reflection
Surveys to collect mentee impact data
Visualisation of impact data
Scheduled interventions - automatically gather more data from mentees at specific intervals over time
Next in priority
Mentor time tracking
Mentoring network tracking - to see network effect/impact of mentoring
Mentor visualisation of data - to see their impact on their mentees
Mentoring program info and schedules
Defining feature set
Based on the key problems, I decided what features should be included in any solution and prioritised them.
Required for MVP
Mentee self-reflection
Surveys to collect mentee impact data
Visualisation of impact data
Scheduled interventions - automatically gather more data from mentees at specific intervals over time
Next in priority
Mentor time tracking
Mentoring network tracking - to see network effect/impact of mentoring
Mentor visualisation of data - to see their impact on their mentees
Mentoring program info and schedules
Key flows
I diagrammed preliminary key user flows for:
Mentees to submit surveys
Mentor admins to view mentee impact data
Diagramming the flows helped in particular with considering how to organise data in a way that makes sense to the users.
Key flows
I diagrammed preliminary key user flows for:
Mentees to submit surveys
Mentor admins to view mentee impact data
Diagramming the flows helped in particular with considering how to organise data in a way that makes sense to the users.
Low fidelity wireframing & testing
Through discussions with the researcher, I initially developed some simple low fidelity wireframes to illustrate key screens for the different personas/roles:
Survey questions and a progress metrics dashboard for mentees
Progress metrics dashboard for mentors to view their mentees’ progress
Hours dashboard for mentors to view/add their hours spent mentoring
Aggregated mentee goals, mentor hours and mentee employment dashboards for mentor admins
As the solution was intended to be responsive, wireframes were created for both mobile and desktop.
I also tested these low fidelity wireframes with a user from PMW, where I stepped them through flows and asked questions. This highlighted some minor issues such as:
Expectations of what data needed to be included and the structure of it
Difficulties in understanding how the data presented is derived
Language and contextual understanding
In conjunction with the researcher I also drafted a complete set of questions to be included in mentee impact surveys.
Low fidelity wireframing & testing
Through discussions with the researcher, I initially developed some simple low fidelity wireframes to illustrate key screens for the different personas/roles:
Survey questions and a progress metrics dashboard for mentees
Progress metrics dashboard for mentors to view their mentees’ progress
Hours dashboard for mentors to view/add their hours spent mentoring
Aggregated mentee goals, mentor hours and mentee employment dashboards for mentor admins
As the solution was intended to be responsive, wireframes were created for both mobile and desktop.
I also tested these low fidelity wireframes with a user from PMW, where I stepped them through flows and asked questions. This highlighted some minor issues such as:
Expectations of what data needed to be included and the structure of it
Difficulties in understanding how the data presented is derived
Language and contextual understanding
In conjunction with the researcher I also drafted a complete set of questions to be included in mentee impact surveys.
High fidelity prototyping & testing
With the feedback from my low fidelity wireframes I created high fidelity prototypes for the following flows for each of the personas/roles using PMW’s existing branding:
Mentees
Submitting a baseline survey and using the dashboard
Submitting a follow-up survey and using the updated dashboard
Mentors
Understanding their mentee’s goals and submitting hours
Mentor admins
Understanding and filtering each of:
Mentee goals
Hours
Employment
These flows were then tested in moderated tests over video chat with 5 different users, of varying roles. The particular roles each participant had experience with determined which flows I tested with them.
Every participant rated the flows between 5 and 7 out of 7 for ease of use. The mentor admin screens were found more difficult to understand.
Overall, it was felt the design would be beneficial to all (mentees/mentors/mentor admins) even in its current state.
High fidelity prototyping & testing
With the feedback from my low fidelity wireframes I created high fidelity prototypes for the following flows for each of the personas/roles using PMW’s existing branding:
Mentees
Submitting a baseline survey and using the dashboard
Submitting a follow-up survey and using the updated dashboard
Mentors
Understanding their mentee’s goals and submitting hours
Mentor admins
Understanding and filtering each of:
Mentee goals
Hours
Employment
These flows were then tested in moderated tests over video chat with 5 different users, of varying roles. The particular roles each participant had experience with determined which flows I tested with them.
Every participant rated the flows between 5 and 7 out of 7 for ease of use. The mentor admin screens were found more difficult to understand.
Overall, it was felt the design would be beneficial to all (mentees/mentors/mentor admins) even in its current state.
“I didn’t know I needed this until I saw it”
- a test user with mentee, mentor and mentor admin experience
Priority revisions
It was indicated that the economic cost and return of mentoring should be derived and displayed (through multiplying an hourly rate by the number of hours shown), as this is very valuable to attract funding & support
→ I added a display of hours in terms of money to both the mentor’s hours dashboard and the mentor admin’s dashboards
→ As the actual time is still useful in different scenarios, I included a toggle on the mentor admin’s dashboard to toggle between representing time in terms of hours and representing time in terms of money
Mentor admins also indicated that being able to drill down and see specific mentors workloads would be very helpful in helping them to better manage the mentoring
→ Added a responsive tabular breakdown of individual mentors and their workloads
All mentors and mentor admins were a little confused about categories of time records
→ Added a legend to clarify as this was simplest for the layout
Priority revisions
It was indicated that the economic cost and return of mentoring should be derived and displayed (through multiplying an hourly rate by the number of hours shown), as this is very valuable to attract funding & support
→ I added a display of hours in terms of money to both the mentor’s hours dashboard and the mentor admin’s dashboards
→ As the actual time is still useful in different scenarios, I included a toggle on the mentor admin’s dashboard to toggle between representing time in terms of hours and representing time in terms of money
Mentor admins also indicated that being able to drill down and see specific mentors workloads would be very helpful in helping them to better manage the mentoring
→ Added a responsive tabular breakdown of individual mentors and their workloads
All mentors and mentor admins were a little confused about categories of time records
→ Added a legend to clarify as this was simplest for the layout
The future
Overall, the prototype is in a good state - it fulfils most of the goals for the established roles and is relatively easy to use.
However, testing also illustrated other areas for improvement:
It is clear that different participants were prone to interpreting the questions, multiple choice answers and data presented differently. Further work should be done to iterate on the language and presentation of data used so this is as clear as possible to as many users as possible.
The presentation of employment data was seen as most confusing and not clearly communicating the meaning of the data collected - this could be iterated on
A few users were slightly confused by the presentation of goal progress in the dashboards - this could be improved
Visually speaking, the prototype could also be improved in other ways such as:
Iconography for goals
Use of animations to “celebrate” mentees meeting their goals
The future
Overall, the prototype is in a good state - it fulfils most of the goals for the established roles and is relatively easy to use.
However, testing also illustrated other areas for improvement:
It is clear that different participants were prone to interpreting the questions, multiple choice answers and data presented differently. Further work should be done to iterate on the language and presentation of data used so this is as clear as possible to as many users as possible.
The presentation of employment data was seen as most confusing and not clearly communicating the meaning of the data collected - this could be iterated on
A few users were slightly confused by the presentation of goal progress in the dashboards - this could be improved
Visually speaking, the prototype could also be improved in other ways such as:
Iconography for goals
Use of animations to “celebrate” mentees meeting their goals
