Being a Mentor in open source projects

Abstract

Mentoring is a well-known way to help newcomers to Open Source Software (OSS) projects overcome initial contribution barriers. Through mentoring, newcomers learn to acquire essential technical, social, and organizational skills. Despite the importance of OSS mentors, they are understudied in the literature. Understanding who OSS project mentors are, the challenges they face, and the strategies they use can help OSS projects better support mentors’ work. In this paper, we employ a two-stage study to comprehensively investigate mentors in OSS. First, we identify the characteristics of mentors in the Apache Software Foundation, a large OSS community, using an online survey. We found that less experienced volunteer contributors are less likely to take on the mentorship role. Second, through interviews with OSS mentors (n=18), we identify the challenges that mentors face and how they mitigate them. In total, we identified 25 general mentorship challenges and 7 sub-categories of challenges regarding task recommendation. We also identified 13 strategies to overcome the challenges related to task recommendation. Our results provide insights for OSS communities, formal mentorship programs, and tool builders who design automated support for task assignment and internship.

1 Introduction

Mentorship is frequently adopted in Open Source Software (OSS) projects as a strategy to help newcomers overcome the barriers they face in their first steps toward contribution [1–5]. In offline communities, assigning mentors to new members has proven effective at helping them overcome challenges [6]. OSS communities also offer mentoring initiatives for newcomers [2, 4, 7], including well-known and established programs like Google Summer of Code [8] and Outreachy. Through mentoring, newcomers are trained to acquire the technical, social, and organizational skills they need [2, 3, 9, 10]. Mentors are usually peers who succeeded in overcoming the project challenges themselves and are willing to help others onboard [11]. In this context, a mentor is a peer who was assigned to or who volunteered to support newcomers onboarding to a project.

Past work has focused on developing strategies to recommend mentors automatically [10, 12–14] and to assess the impact of mentorship [2, 9, 15]. However, it is not easy to be a mentor, especially if experienced contributors are resource constrained [7]. While, past work has identified the challenges that contributors, especially new contributors, face when attempting to contribute to OSS [4, 5], the challenges in contributing yourself is different than guiding newcomers to become contributors. Currently, an understanding of who serves as mentors, what challenges they face, and the strategies they use are under-investigated, which might, in turn, contribute to the lack of specific support and training for mentoring in OSS. In our previous work [16], we found that typical OSS mentors are neither formally trained nor paid for their mentoring. Without formal training, mentors—even if they are technically resourceful—may not be aware or prepared for the challenges they may encounter in mentoring.

Therefore, to close this gap, we ask the following research questions:

• RQ1. What are the characteristics of OSS mentors?

  First, we take a comprehensive view of the characteristics of mentors in the Apache Software Foundation (ASF), a large, mature OSS ecosystem that employs both paid and volunteer contributors. Having a baseline understanding of mentor characteristics is essential for tailoring specific approaches to attracting and retaining these important contributors and providing context to understand their challenges and strategies. We answer this question by conducting a large-scale survey (n=624) with contributors in the ASF and comparing mentors’ demographics with those of non-mentors.

• RQ2. What are the mentorship challenges that mentors face?

  Next, we investigate the challenges that mentors face. Understanding these challenges for different aspects of mentoring can help OSS communities understand the state of affairs and where mentors are struggling to devise solutions. To answer this research question, we interviewed 18 mentors in a wider OSS landspace, and not just in one ecosystem (ASF). We found 25 challenges grouped into process, technical, personal, and interpersonal categories.

• RQ3. What are the specific challenges and strategies for recommending tasks to newcomers?

  Past work has shown that identifying appropriate tasks for newcomers is one of the key challenges that mentors face [17], therefore, in this research question, we conduct a deeper investigation of (a) the specific challenges related to task recommendation and (b) the strategies mentors employ to overcome them from the interviews. A list of strategies that current mentors employ to help in task recommendation can not only help other mentors use these strategies but also help researchers devise automated task recommendation approaches.

In this paper, we extend and integrate our previous works [16, 18] by investigating mentors’ characteristics and by providing a comprehensive view of the challenges they face and the strategies they employ. Scientifically identifying challenges and strategies that mentors adopt to support newcomers can help OSS communities, mentors, and researchers create better tools and processes to support mentorship.

In summary, our contribution is a comprehensive view of: 1) the characteristics of mentors in a large OSS community and how they differ from non-mentors; 2) the challenges mentors face to support newcomers onboarding to OSS projects; 3) the challenges of recommending tasks for newcomers; and 4) the strategies mentors employ to mitigate these challenges. Our contributions can guide future investigations and programs to better support mentors, mitigating their challenges, and better supporting their strategies.

2 Background

Mentoring consists of an interpersonal relationship wherein an experienced individual (the mentor) provides practical advice and personal guidance to an inexperienced individual (the protégé or mentee) [19]. Mentoring takes place between participants of all ages and with different levels of engagement and formality [20]. Mentoring relationships build a strong interpersonal bond between the parties and ease the transference of various types of knowledge. In addition to factual knowledge, mentors’ technical guidance helps mentees develop job-related skills [21–23].

Mentoring relationships are commonly employed in communities of practice, in which mentors provide mentees with support in developing the skills and competencies necessary for full and productive participation in the community. For example, in fan fiction communities, informal peer mentors help authors improve their writing skills by providing feedback and answering questions [24]. In software development projects, having a mentor is particularly important for deliberate practice [25, 26], and is a central aspect of increasing expertise [27]. In OSS projects, the role of an OSS mentor is to support software developers in their technical activities —- e.g., explaining the architecture of the project’s software, recommending tasks, and assisting in software development activities. Besides helping in technical activities, mentors can encourage developers to improve their communication and collaboration skills [28].

In OSS, in addition to the traditional dyadic form of mentoring, mentors also take part in a network-enabled form of collective feedback [29]. Early mentoring of newcomers in an open collaboration system such as OSS improves their engagement and retention, increasing their chance of staying in the project and becoming long-term contributors [21, 30].

3 Method

To answer our research questions, we designed a study that comprised two stages. Stage 1 provides an understanding of the characteristics of mentors in a large organization (RQ1). Stage 2 investigates mentorship challenges and strategies from the point of view of the mentors (RQ2 & RQ3). Figure 1 depicts the overview of the two stages in our research method, which we discuss next.

Fig. 1

Research Method Overview

For Stage 1, we employed a survey in collaboration with the Apache Software Foundation (ASF). Focusing on a single organization to answer RQ1 helped us in multiple ways. First, it allowed us to better define our population in the context of a single organization and compare and contrast mentor characteristics with non-mentors. Second, our partnership with the Apache Software Foundation allowed us to reach many respondents (624) across multiple projects. Such a large survey allowed us to obtain data on about 175 mentors. Before this large-scale survey, we tried to reach a large number of mentors, but it was difficult since their role is often not formally defined. Third, the ASF serves as a relevant case study as it is one of the world’s largest OSS foundation with more than 460,000 people involved, with more than 350 projects and initiatives and partners with mentorship programs such as Outreachy and Google Summer of Code. Finally, Apache projects are often studied in scientific research, which allows our work to be placed in the context of existing research [31–35].

For Stage 2, we conducted in-depth interviews with mentors. For this stage, we interviewed mentors from the ASF and included mentors from several other OSS communities to provide a broad view of the mentors’ challenges and strategies.

3.1 Stage 1 - study planning

We engaged with the ASF Diversity & Inclusion committee of ASF, which seeks to understand the state of diversity at the ASF and the challenges faced by its community, and closely collaborated with us in this stage of the study design. The ASF Diversity & Inclusion committee comprises 18 experienced contributors who have different roles within the foundation, including committers, Project Management Committee (PMC), and board members. We drafted the initial set of questions by leveraging questions from past surveys in OSS such as the 2016 ASF Committer Diversity Survey [36], Open Demographics Survey [37], OpenStack Gender Diversity Report [38], Stackoverflow Members Survey [39], and best practices recommended by the Linux Foundation’s Community Health Analytics Open Source Software (CHAOSS) Diversity & Inclusion Working Group [40]. We then collected feedback from the ASF by collaborating on a Google document in a two-month window. Anyone with an ‘apache.org’ valid email could access this document and comment on the survey questions. We used different channels to reach out to the community for feedback (e.g, JIRA ticket, developers mailing list, ASF slack - Diversity & Inclusion channel). After multiple iterations of resolving the comments provided by the community and editing the survey, we scheduled a two-hour listening session with the Apache community to collect additional feedback on the survey questions. We edited the consent form to include the statement that data would be handled as per Apache Privacy Policy and updated the survey questions based on the feedback we received.

We defined the survey’s target population as any contributor in the community to be able to compare mentors with non-mentors. The survey comprised 25 questions, and we used for the present paper 10 of those questions, including how often the respondents contribute as mentors to the projects; if the contributor had a mentor when they started; gender; age; education level; experience as contributor; compensation (paid/mix/unpaid); and English proficiency for technical and social interactions. All the questions were optional to increase the response rate [41]. We present the 10 questions used in the present study in Table 1.

Table 1 Survey Questionnaire

The recruitment was based on sending 7010 direct e-mails to the community’s committers, Diversity & Inclusion (D&I) lists, and posting in social networks. The survey was available between January 1 and February 24, 2020. We received 624 survey responses resulting in a response rate of 8.5%, considering the total community size of 7,500 committers in the ASF.

3.2 Stage 1 - data analysis

To characterize mentors, we used the survey question about how often the contributor performed mentorship activities. This question had four possible answers: Never, Rarely (less than once a month), Sometimes (more than once a month), and Often (once a week or more). From the 624 respondents, 65 did not answer this question. We considered a respondent as a mentor if the options “Sometimes” or “Often” were marked, and not a mentor if the options “Never” or “Rarely” were marked. Using this filter, we found 175 mentors and 383 non-mentors.

We used regression modeling techniques to identify demographics that positively or negatively influence the likelihood of a contributor being a mentor. According to previous studies in literature, contributors who collaborate on online communities can present different motivations according to gender, age, experience level [42], compensation [43] and education level [44]. We selected those demographics to analyze if those independent variables also impact the likelihood of a contributor becoming a mentor. Table 2 presents the independent variables we use.

Table 2 The categorical independent variables, respective levels and possible values they could assume

To check for collinearity among the explanatory variables in the regression models, we first checked for Variable Inflation Factors (VIF). A recommended practice is to remove any variables in the final model that have VIF scores >5 [45]. Based on the VIF scores, we selected the following variables to be part of the model: compensation, had-mentor, experience, and gender. The VIF scores of these variables were <2, signifying low collinearity among the variables.

In this paper, we report all analysis results at α<0.05. We used the R statistical software to perform our analysis.

3.3 Stage 2 - study planning

Our goal in Stage 2 was to find challenges that OSS mentors face and the strategies they employ to mitigate them. We recruited mentors from the Stage 1 survey who said they would be available for a follow-up interview. Additionally, to get a broader understanding of mentors’ challenges, we also recruited mentors from several other OSS communities using convenience sampling and snowballing techniques. Specifically, we sent out recruitment emails to two OSS contributors who we knew to be mentors. Additionally, we asked participants to recommend other mentors we could contact at the end of each interview.

In total, we interviewed 18 experienced OSS mentors, 10 from several OSS projects (P1-P10), and eight from the ASF community (P11-P18). Out of these eighteen participants, 11 self-identified as men, six as women, and one preferred not to disclose. All of our interviewees had a minimum of 1-2 years of experience in the OSS project they are currently in. We kept interviewing until we agreed that no new challenges were identified for at least two interviews. According to [46], sampling can be discontinued once the collected data is considered sufficiently dense and data collection no longer generates new information. Table 3 present the demographic information of the interviewees.

Table 3 Stages 2 and 3 - Demographics for the Interviewed Mentors

3.4 Stage 2 - data collection

We used semi-structured interviews, which consisted of a mixture of open-ended and specific questions designed to elicit foreseen and unexpected information types [47]. In this kind of interview, the questions are planned, and we seek to answer them, but they are not necessarily asked in the same way or order as they are listed [48]. We designed our interview script according to the literature recommendations [47, 48]. The interview script is in Table 4. Interviews included a central question about the challenges that mentor faces while mentoring other contributors, and questions about how the mentor recommends tasks for newcomers. We used follow-up questions to deepen the interviewee’s answer, looking for concrete examples and more details. Before interviewing the participants, we conducted four pilot interviews with Ph.D. students who had experience working in industry or OSS environments to validate the script and confirm whether the interview would fit in a 1-hour time slot. The pilot participants answered all the interview questions and provided us feedback about the flow of the script. We discarded the results of these pilot interviews. We also analyzed the questions and answers to ensure that they provided data to answer our research questions.

Table 4 Central Questions of Interview Script

Two researchers ran the semi-structured interviews, which lasted around 40-minutes: one researcher led the interview while the other observed and took notes. All interviews were remotely undertaken using Skype, Google Hangouts, or phone. With participant consent, we audio-recorded all interviews and transcribed them later for the analysis. Before each interview, we shared a consent form with the participants asking for their agreement. After the interview, we thanked our participants and compensated them with a gift card or an equivalent value in a donation to the OSS project/organization of their choice.

3.5 Stage 2 - data analysis

Two authors of this paper manually transcribed each interview, and then three authors employed a card sorting approach [49] to analyze the data. They started by unitizing each interview into individual cards and applied open coding to classify strategies and challenges. After reading the transcripts, we identified challenges and strategies, and assigned them a code, i.e., a 2–3 word statement that summarizes the challenges or strategy. During five weeks, the partial results were discussed and validated weekly with three more experienced authors. We conducted the whole process using continuous comparison [46] during the coding sessions and negotiated agreement [50] (as a group). In the negotiated agreement process, the researchers discussed the rationale they used to apply particular codes and reach a consensus [50, 51].

We found a total of 25 challenges as presented in Table 6. Mentors from the diverse set of OSS communities (P1-P10) reported 19 of these challenges. Mentors from ASF (P11-P18) mentioned 7 challenges that were common to these challenges and 6 new types of challenges.

4 Results

Our study reveals characteristics of mentors (RQ1), challenges they face in mentorship activities (RQ2), and specific challenges and strategies for recommending tasks for newcomers (RQ3). In this section, we report our findings per research question.

4.1 Characteristics of OSS mentors (RQ1)

In our survey, 559 participants answered the mentorship question, out of which 175 indicated they play a mentorship role at least once a month. Figure 2 presents the demographics of the mentors and Fig. 3 the non-mentors.

Fig. 2

Demographics of Survey Participants who are mentors

Fig. 3

Demographics of Survey Participants who are not mentors

When considering mentors, a large majority were men (92.7%), experienced in the ASF (only 12.2% have less than 1 year experience), had some component of their work paid (62.9% when combining those fully and partially paid), and reported good English proficiency for both technical and social interactions. Circa 7% of mentors said that they face some difficulties for interacting in English during social interactions, and no mentor from our sample reported any problems during technical interactions. There are only slight differences in the demographics of the non-mentors. Non-mentors included 95.5% of contributors who were men and had a similar distribution of work experience, compensation structure, and English proficiency for social interactions.

We used a linear regression model to answer RQ1 as discussed in Section 3.2. Table 5 presents the result of the regression model. The dependent variables “Compensation” where its value is unpaid and “Experience”level <1 year had negative estimates and were significant (p<.05). The negative coefficients indicate that an increase in these factors is linked with a decrease in the likelihood of the contributor with those characteristics to be a mentor. Based on our data, we conclude that contributors are less likely to be a mentor when they volunteer (not compensated) and have less than 1 year of experience.

Table 5 The linear regression model with standardized coefficients showing how different factors influence an OSS contributor to be a mentor

In our dataset, proportionally, both women and men had a similar distribution of being mentors—31% (153 out of 495 men are mentors) as compared to 32% of all women who serve as mentors (12/28). Indeed, we did not find statistical significance for the impact of gender on being a mentor. However, we highlight that we only have a few women in our sample—5% (28 out of 559), which might impact our analysis. (We do not include those individuals who preferred not to state their gender in this analysis). This proportion of women in our survey respondents concurs with the overall proportion of women (5.2%) in ASF found earlier in [52].

4.2 Mentorship challenges (RQ2)

In this paper, we extend our previous work [16], in which we qualitatively analyzed data collected from interviews with mentors. We found 19 categories of challenges that affect mentors classified as:

• Process challenges – imposed by the organization or by internal procedures or practices;

• Technical challenges – directly related to or caused by technology, including frameworks, programming languages, and tools used in the project.

• Personal challenges – related to personal characteristics of mentors;

• Interpersonal challenges – related to the relationship between community and mentees;

In the present study, we extended [16] by interviewing mentors from the ASF. We found 13 categories of challenges in this new dataset, seven already previously identified and six new categories. We classified the new challenges using the above schema. Table 6 presents the aggregated results, and we explain the challenges below. P1-P10 are participants from [16] and P11-P18 are the ASF participants from the present study. We use uppercase FOR THE NAME OF THE CHALLENGE.

Table 6 General Challenges Faced by Mentors

4.2.1 Process challenges

Table 6 presents the five process-related challenges that mentors face; with three that come solely from the ASF interviewees—identified with a ∗ in the table.

The challenge DIFFICULTY IN IDENTIFYING APPROPRIATE TASKS FOR NEWCOMERS was highly pointed as a challenge for mentors. According to P3, “to keep them [the newcomers] engaged you need […] to pick a task that is appropriate for them…, which can be a challenge for mentors.” When a newcomer’s background and goals are unclear, it can be difficult for the mentor to point them to a specific task. Choosing a task was also previously identified as challenging for newcomers as well [17]. Due to the identified relevance of this challenge, we went deeper into it in RQ3.

In addition, if processes are unclear, the mentor must figure out how to get their mentees the information they need. NOT HAVING A FORMAL PROCEDURE FOR INTRODUCING THE COMMUNITY was reported as a challenge by P9, who stated that

“[…] the challenges I have faced are related to how to decide which part of the community to introduce first to the students. It is not totally clear in [project-name] since we have many processes and don’t have a formal procedure for the introduction.”

Mentors also reported they miss a “definitive guide to participate” (P11) or a “contribution path that helps people progress through the project” (P17).

ASF participants also mentioned a LACK OF ESTABLISHED GOVERNANCE WITH PROCESSES AND RULES leads to misinformation and behavior issues. When having “unstated rules” (P14) or when “things are not documented, there are no rules” (P11). Associated to the rules, “there need to be consequences for action when people derail in the mailing lists” (P12). Besides rules and consequences, mentors complain about the lack of a standard and documented guideline with the main regular processes that contributors need to follow. When not having a documented process to follow, mentors need to remember “how we did last time” (P15) or each one can decide “what is the best way to do it” (P15). Moreover, there is a LACK OF PROCESS TO SHIFT AND SUBSTITUTE MENTORS. Finally, although they “try to influence decisions” (P14), MENTORS DO NOT PARTICIPATE IN DECISIONS TO PROMOTE PEOPLE.

4.2.2 Technical challenges

Only one technical challenge was mentioned by mentors.

DIFFERENCES IN THE DEVICES THAT MENTORS AND MENTEES USE is a challenge that happens when mentors and mentees are not using compatible devices or operating systems; it is hard for a mentor to help resolve a newcomer issue. P2 stated “the operating system and distribution my computer is running is very different to what the newcomer is running. If a newcomer has an issue, I try to reproduce it, and I may not have this issue which makes it harder to help.”

4.2.3 Personal challenges

We identified five personal challenges that impact mentors, with one specifically about the ASF (marked by ∗ in Table 6). The challenges relate to their ability or lack of ability to manage the responsibilities that come with mentorship.

HANDLING A LARGE NUMBER OF MENTEES can be overwhelming, as stated by P6: “I really wish it was easier to deal with a lot of people.” This is a personal challenge related to scheduling, which also creates DIFFICULTY IN SWITCHING CONTEXT between helping mentees and doing their work. P10 explained that “if you are not actively focusing your attention on [your mentee] continuously, context switching can be difficult between doing my work and helping them with theirs.” As a part of the mentorship, mentors are expected to complete their own work and be available to help their mentees.

Three participants from previous work and two from ASF mentioned that DIFFICULTY IN TIME-MANAGEMENT could be challenging, since mentors must choose how to allocate their time to the project, sometimes weighting different activities, such as working on code, mentoring, and reviewing. Mentors feel “overcommitted” (P12), as mentorship is “time-consuming” (P17) and finding “time is the hardest thing” (P12). Mentors can also encounter problems in aligning their schedule with newcomers, as mentioned by P4: “being able to contact them [the newcomers] and give feedback was sometimes difficult.”

Finally, the interviewees mentioned DIFFICULTY IN MANAGING DIFFERENT ACCOUNTS as a challenge, since “it’s really annoying to have a lot of accounts to keep track of.”

In the present study, we identified one new personal challenge: NOT GETTING PAID TO BE MENTOR. This problem can affect the mentor’s motivation and availability, as mentioned by one of them: “[...] as long as you are not working on the project full time, or you are not getting paid, it would be really difficult to mentor […] unless we get something in return” (P18).

4.2.4 Interpersonal challenges

Interpersonal challenges is the category with the highest number of challenges reported by the mentors; twelve already identified in [16] and two new ones from the ASF context. Indeed, social interactions play a key role in mentoring.

First, since people who work in an OSS project come from diverse cultures, CULTURAL DIFFERENCES is challenging for mentors. P8 mentioned “in some cultures, people get more upset when people criticize their code…which can be tough.”

Moreover, when newcomers and mentors are geographically distant, they do not have the opportunity for face-to-face interaction, which can, for example, inhibit informal communication and reduce trust. Therefore, COMMUNICATION ISSUES RELATED TO TIME ZONE AND PLACE affect the communication process during mentorship, as people usually “can’t talk in person” (P14) and “most of the communication happens async on Slack” (P18). Mentors can feel they “haven’t communicated them [mentees] well enough” and be disappointed when “[mentees] write an email and have to wait. Sometimes one day, two days” (P16).

Also related to communication in global settings, LACK OF ENGLISH LANGUAGE SKILLS was mentioned by P9 as hindering the mentorship process: “My English is so-so …when both parties have difficulties communicating, it is challenging to overcome and we don’t have good tools for that.” Having a “English [that is] not good makes [mentors] afraid to participate in tech discussions” (P13). The language can also impact reading documentation, as “if that documentation is not equally legible by everybody who claims to be part of [the community], there is a problem” (P13). We also observed that a mentor’s inability to interact with newcomers (LACK OF MENTOR’S INTERPERSONAL SKILLS) can significantly impact a newcomer’s decision to continue contributing to the project. “People might not have the skills to be a mentor, or knowing how to teach somebody” (P17) and “have patience [with mentees]” (P15, P17). Mentors frequently highlight the importance of social aspects, as evidenced by P3: “…the biggest pitfalls of the mentor are: not being responsive and not engaging in other ways than just coding. These projects are about community effort and more than just the code.”

Mentors also face challenges adapting to how different types of people learn and take in the information presented to them. Two mentors reported that ADJUSTING INTERACTION STYLE TO DIFFERENT MENTEE PERSONALITIES is a challenge since mentors are likely to collaborate with diverse people who have unique personalities and working styles, as stated by P9: “[…] you always have to adapt based on each individual newcomer […] one solution doesn’t always work for everyone.” Mentors need to understand their mentees and tailor aspects of the coaching to fit them. For a mentor, determining how to be an effective teacher for a mentee can be difficult.

Four mentors mentioned DIFFICULTY GUIDING MENTEES WHO ARE RESISTANT TO COACHING. Sometimes mentors are required to face the challenge of teaching newcomers who lack a desire to learn. In this sense, P5 mentioned “But I still don’t know how to help people who don’t want to learn.

Also related to coaching, mentors reported that PROVIDING CONSTRUCTIVE FEEDBACK BASED ON THE MENTEE’S BACKGROUND is challenging. Mentors must tailor their comments and criticism to the way a newcomer learns while considering their prior experience and level of self-efficacy. P4 reported that “being able to understand the student’s background and the way they see this stuff and give proper feedback is kinda hard.” Some mentees value feedback. In contrast, others may not readily perceive it constructively.

Moderation is sometimes required for mentors when dealing with newcomers. For example, newcomers who are eager to contribute something relevant to the project tend to start with a task that may be too large or complex for their skillset. CONVINCING PEOPLE TO START SMALL RATHER THAN BIG was reported as a difficulty, as explained by P6: “the other challenge is convincing people to start small rather than big because lots of people want to make big changes, but I can’t help them with those.”. This challenge relates to the process challenge called “difficulty in identifying appropriate tasks for newcomers.”

P3 reported that ENSURING MENTEES FINISH THEIR WORK is a challenge: “the biggest challenge is making sure they are working and making sure they will finish the project. Otherwise, it is a fail for the mentor if the mentee doesn’t finish.”

Mentors also mentioned a DIFFICULTY TO KEEP THE MENTEES ENGAGED, because “people come, they fix their bug, and they go away.” (P16) Indeed, [53] showed that a great number of newcomers place a single contribution and do not return to the project. Additionally, [54] showed that many newcomers submit one or more contributions, which are not accepted by the community, and they do not come back.

As the community grows, the diversity of contributors grows in parallel. Inclusion is important for attracting newcomers, as well as retaining them and increasing their productivity [55]. The participants of our study seemed to be aware of this and placed particular emphasis on this challenge. Mentors mentioned the DIFFICULTY IN CREATING AN INCLUSIVE COMMUNITY as a challenge. Mentors try to ensure that newcomers feel comfortable and are not discriminated. P3 explained, “it is about the community. There has been a lot of discussion about gender pronouns, and this is very important to take into account to make sure the community is inclusive of all, especially for newcomers.” The community can miss diversity also in terms of tenure, when not bringing novice leaders, as stated by P17: “no one with less than 10 years of experience is well regarded to lead anything at the foundation, which then imposes this chicken egg analogy into diversity and inclusion.”

A frequently mentioned challenge was HARSH PROJECT ATMOSPHERE (cited by 8 out of 18 mentors). This challenge affects mentors since they face difficulty supporting newcomers who fear disagreements among committers in the community: “I may find a patch to be fine and ready to commit, but some other committer may look at it and not agree that it is fine” (P1). This is particularly challenging for mentors since it is largely out of their control.

DIFFICULTY TO TRACK THE MENTEES’ PROGRESS is a new category from the present study. As mentioned by one of the interviewed mentors: “They [the mentees] do not come out easily. And that is when I had to go in to [help]: Did you do this step? What was the output for this? [….] I’m not so extensive on doing this, but [do] from time to time […]” (P15). This challenge can be related to a lack of management skills of the mentor on monitoring the progress of remote teams or to a lack of attitude of the mentee on reporting pitfalls and asking for help. Mentors and mentees should have clear responsibilities to know the boundaries of each one’s work to mitigate this challenge. Communication hurdles can also be a cause for this challenge, as mentioned “[Having] the visibility of the work they were doing, because everything was so async” (P18).

DIFFICULTY TO MANAGE A FINANCIALLY MIXED TEAM (PAID AND VOLUNTEERS) OSS projects in foundations, such as the Apache Software Foundation (ASF), include a mix of paid and volunteer contributors working either in standalone OSS projects or open source arms of commercial companies [56, 57], resulting in a variety of motivations and pathways for contributing [42, 58–60], and heterogeneity in contributor goals and project ideas. Mentors from the present study mentioned facing “difficulty of having paid and unpaid volunteers [...] the difference in availability of time, makes a huge difference in how people can contribute”. While the paid contributors dedicate more time to the contribution, “by the time [volunteers] come back around to contribute, again, it’s done [by paid contributors.]” This situation is “demotivating and disenfranchising” (P2).

4.3 Challenges in recommending tasks for newcomers (RQ3)

Recommending tasks for newcomers is one of the most recurrent barriers for newcomers onboarding to a project [17], which was confirmed by our interviews. To further investigate this barrier, we go deeper in the challenge DIFFICULTY IN IDENTIFYING APPROPRIATE TASKS FOR NEWCOMERS (Pro1 from Table 6). The analysis of the interviews resulted in seven sub-categories for this challenge. We also cataloged thirteen strategies employed to overcome those challenges, which we grouped into five categories. The challenges are shown in Table 7 and discussed as follows.

Table 7 Challenges faced by mentors during the process of task recommendation for newcomers in OSS projects

(Pro1.1) Challenging tasks can create social fears in the newcomers. Due to the socio-technical nature of OSS projects, newcomers may feel fearful of exposing a weakness or failing. P1 noted that “Usually, because of social fear, they just back off. They think they are not good enough, or they don’t know enough.” P4 also mentioned that “the biggest barrier is being afraid of being judged.” P10 similarly stated: “sometimes newcomers will be shy to ask for help or not actively engaged when not knowing where to start.” Mentors need to deal with these issues when directing newcomers to appropriate tasks.

(Pro1.2) Mentors have to deal with newcomers’ lack of holistic understanding about the project and its culture. Projects are a complex socio-technical landscape, and newcomers have contact with isolated parts of the code or tasks. As mentioned by P1, “when a newcomer comes in, one thing is they don’t understand all the moving pieces of the codebase. Even if they get the code, they don’t get the impact on the bigger scheme of things.” To make things worse, projects have specific conventions and protocols, and, as mentioned by P2, “sometimes the people are so used to these conventions, they have a hard time communicating them because they don’t realize they exist.”

(Pro1.3) Lack of information about how newcomer-friendly a task is. Often, there is a large number of tasks in the issue tracker—including potentially easy ones—but “no direct way to spot tasks suitable for newcomers”. P1 mentioned that “we don’t know if things are suitable for newcomers.” They complain that people who add issues do not add relevant information and tags that would indicate that the issue is simple to fix. Moreover, the tags that indicate easy tasks only appear for tiny and isolated tasks. P10 identified a concern about such tasks for newcomers: “I dislike having things labeled as ’bite-size’ because it may cause someone to skip talking to anyone and just grab something. You miss out on a lot of important social interaction.” According to this mentor, isolation and lack of interaction for newcomers can increase their social fear.

(Pro1.4) Difficulty in identifying the complexity of a task. Some mentors also reported difficulties in estimating a task’s complexity. P8 said “We don’t have a good way [to evaluate the difficulty of a task]. […] There is a lot out there to analyze the complexity of the code, but I’ve found none of it to be helpful in the < project-name>.” Therefore, developers most often rely on their experience to evaluate a task. However, experts suffer from the “curse of knowledge” — the difficulty of seeing something from an outsider’s point of view [61]. This was observed by P7: “A task may be more involved and technical than we thought initially. Then we may realize a task isn’t suitable for someone because it’s actually an expert level task.”

(Pro1.5) Difficulty in estimating the time necessary to finish a task. Complementary to the complexity of the tasks, mentors are also concerned about their duration. Usually, mentors do not want to give newcomers long-term assignments and, sometimes, even low complexity tasks can take longer. However, determining the time to complete a task is not trivial, even for easy/non-complex tasks. P8 mentioned this difficulty: “It’s hard to estimate the time needed for a task. I’m not good at gauging what is appropriate for a newcomer.” This can be even worse when newcomers’ tasks need to be reviewed by other stakeholders for compliance or other reasons, as mentioned by P1:

“Newcomers have to wait a while sometimes for a review, and it may even take months to get a review because people are busy.”

(Pro1.6) Lack of newcomer-friendly tasks available. Sometimes, there are no easy-enough tasks available at the moment. Some interviewees reported that:

“if you’re a newcomer and come at a time when there aren’t many tasks open for newcomers, …then the difficulty lies in finding something in a certain application to turn into a newcomer task. [P2]”

(Pro1.7) Lack of available information about newcomer’s skills, interests, and expertise. Since mentors may lack information to help them assess newcomers’ characteristics, they often misjudge newcomers’ abilities, as P9 mentioned: “sometimes you think they can take on more than they, in reality, are capable of.” Mentors often do not have access to a portfolio or a set of previous work to evaluate the newcomers’ abilities, as P4 said “some [newcomers] are very good but they don’t have some work portfolio to show their previous skills.” As P8 explained, “we don’t have a good way to match a task with a developer.”

4.3.1 Strategies for recommending tasks to newcomers

We identified 13 strategies employed by mentors to alleviate the challenge of recommending tasks for newcomers, as presented in Fig. 4. We classified the strategies into five main categories, as described below.

Fig. 4

Overall view of the strategies

Strategies to identify task complexity Identifying task complexity is challenging even for mentors (Pro1.4). We explicitly asked our participants if they use a tool to help determine a task’s difficulty. Eight out of ten participants mentioned that they do not use any tool but reported several strategies, as follows.

REPRODUCING THE BUG For bug-related tasks, mentors reproduce the error to further understand it. As P2 explained, “First of all, we reproduce the bug and try to see what is causing it and find out what is necessary to fix it. Then we assess whether the skill level is similar to what a newcomer would have to fix it.”

COMPARING THE COMPLEXITY OF NEW BUGS WITH EXISTING BUGS IN THE BUG TRACKER To further understand task complexity, mentors use previously tagged issues as a scale, comparing the complexity of new tasks with existing ones to find newcomer-friendly tasks, as P2 stated: “we use, for example, the bug tracker to figure out if a bug is at the same level as other newcomer bugs we have.”

TAGGING THE TASK BASED ON DIFFICULTY Six mentors reported that, in their projects, tasks are tagged based on difficulty. Tags make newcomer-friendly tasks more visible and identifiable by mentors and newcomers. As P5 also pointed out, “Mentors are strongly encouraged to tag tasks for newcomers based on complexity (how many concepts does a newcomer need to know, how deep should one’s knowledge be).” In Fig. 5, we can see examples of issues labeled as “good first issues.” As P4 described, “we have this tool called Bugzilla, which we are tracking all the bugs. When we file a bug, we find it to be easy; instead of going directly to fix it, we tag it with a newcomer tag so that we have in our website a list of bugs that are suitable for newcomers.” P6 added that “we basically tag things that are good starter issues. If someone asks, we show them this starter tasks list. It’s not perfect, but it works.” On the other hand, P7 complemented that newcomers do not want to only work on beginner tasks and that labeling a task “beginner” is not very rewarding to those working on the tasks. This mentor also reported having “medium” and “major” tags, which people can progress through as they go. However, P10 remembered that evaluating task difficulty is complicated: “You’re describing the relationship between the person solving it and the task itself. For one person, it may be easy, and for someone else, it may not; it depends on the background. Some tasks are more difficult for more people, so you can say it’s difficult overall.”

Fig. 5

JabRef’s issue tracker with labels (“good first issues”) to help newcomers identifying appropriate tasks

However, as previously mentioned, the lack of information about how newcomer-friendly a task is (Pro2.3) was frequently cited as a challenge. P10 affirmed that tags might not be enough: “it can be tough if a project doesn’t do a good job grooming tasks. When you label something as a good task for newcomers, it’s necessary to check out some things before making that assessment that it’s an easy task.”

ADDING DOCUMENTATION Sometimes, project members also add information in the issue tracker to help newcomers, as explained by P4: “we put in the description how we want the contribution to be and which file they should look at. So we are pretty much tagging bugs and creating tutorials on how to fix this specific bug so they can learn the whole process of how to make their contribution.” P4 further explained that the required skills should also be clear: “You look at the description of the task, you should be able to see if it requires familiarity with C, JavaScript, regular expressions, etc.” P2 added: “usually when mentors assign the newcomer tag, we add clarifying info for the newcomer to read... we also provide specific resources for specific projects.”

DISCUSSING TAGGING IN CONFERENCES Due to the relevance of tagging tasks in OSS project environments, members of OSS projects hold discussions to improve tagging, as P1 explained: “Some committers get together at conferences and meetups to discuss ideas to help newcomers when getting involved... We discuss tagging at those conferences.”

Strategies used to identify skills required for finishing a task Besides determining task complexity, mentors and newcomers need to understand the skills required to complete a task. We found that mentors usually employ two strategies to identify the necessary skills for working on a task, as listed in Fig. 4 and described below.

BREAKING THE CODE INTO SECTIONS AND IDENTIFYING THE INVOLVED CONCEPTS/ALGORITHMS may help to identify required skills, as explained by P5: “To measure the skills, I go through the code line by line and break it into sections to tell what concepts are needed. I determine what algorithms they would need to know, what hardware they need to know, and what coding skills they need to work the bug.” However, recognizing the required skill set for finishing a task is not an easy endeavor for mentors, as P10 stated: “[identifying what skill is required for finishing a task] is a little tough because there may be some unexpected elements.”

FORMING A MENTAL MODEL TO HELP ASSESS A TASK We noticed that our participants reported forming a model of the project structure in their minds, which allowed them recommending the appropriate tasks for newcomers. For instance, P6 mentioned “When I see a task, I can make a mental model of where it may fit.” Mental models are internal representations of the world that help humans understand, describe, and anticipate events and situations [62, 63]. P10 reported that she thinks through a workflow model to consider the required skills and possible locations for a bug.

Strategies to identify newcomer’s characteristics Another important aspect of supporting task selection is determining newcomers’ characteristics, including but not limited to their interests, expertise, and areas of improvement. Mentors report identifying newcomers’ characteristics as challenging (Pro1.7). Eight of our participants mentioned they evaluate newcomers’ expertise, and five mentioned they look for what newcomers are interested in before recommending tasks to them. Our interviewees highlighted that they avoid recommending tasks that newcomers do not like, as P3 stated: “I like to ask them if they would be interested in a task before assigning.” Therefore, it is important that mentors identify newcomers’ interests and areas of expertise. From the analysis of the interviews, we found two strategies mentors apply to help identify newcomers’ characteristics, as presented in Fig. 4 and described below.

ASKING NEWCOMERS DIRECTLY ABOUT THEIR INTERESTS AND EXPERTISE Our participants learn about newcomers’ interests and characteristics by directly asking them what they like and what their past experiences involved. P10 mentioned, “I try to talk to them and help them before sending them off to look at the issue tracker. I gauge their interests and their mindset about the process to find a task for them.” P6 complemented this view: “I will ask someone what they are excited about and what they think their skills are, and then I’ll tell them where to look, and they’ll come back with some issues.”

P10 explained that her first question is usually: “what is your background?” She also explicitly asks about previous experience with the platform: “If a person says they haven’t worked on GitHub, that lets me know a big part of their first contribution will be getting to know how to use these basic tools. So maybe they will do something like fix a typo, very straightforward.”

Mentors also consider the initial confidence of newcomers toward a specific task before assigning it to them. When people are initially confident, they associate success with their ability and failure with bad luck [64]. In this sense, P1 stated that “If they’re confident with their technical skills, we point them towards stuff involving what they know.”

Besides the current skills, mentors also ask newcomers what they want to improve, as P10 explains: “The two main things are: their [newcomers’] experience and how much they are looking to learn within a specific task. Do you want something easy for you? Or do you want to jump into something above your skill level?” Mentors report that having had some experience with the task creates a positive impact on their performance and motivation, as P1 stated: “If someone is good with ZooKeeper interactions, we would point them towards areas that focus on that.”

EVALUATING PREVIOUS CONTRIBUTIONS Previous contributions provide evidence about newcomers’ expertise, as explained by P4: “If we have some way to see past contributions of the newcomer, you can easily see it is something related to that they have done before.” However, the mentor added that “Some students are very good, but they don’t have some work portfolio.”

Scaffolding newcomer’s skill acquisition Mentors frequently mentioned how they recommend a sequence of tasks to onboard newcomers, as presented in Fig. 4 and described below.

ASSIGNING A SMALL TASK FIRST AND THEN CHALLENGING THE NEWCOMERS WITH BIGGER TASKS According to our interviewees, offering newcomers small starter tasks provides mentors the opportunity to evaluate newcomers’ skills and interests and support them through the learning curve. Regarding this, P7 stated: “Sometimes …you have to start on the basic tasks and go from there. We see how they are doing and then move forward.” Furthermore, mentors state that assigning small tasks that newcomers can complete keeps them motivated. P3 explained the task “has to be technically very simple[...] For example, one of the tasks we have for newcomers is modifying strings on the UI, so they get excited about having made that first contribution and see that everyone is using it. Since they face so many other challenges, the first task should not be technical. They have to figure out the tooling and Bugzilla, so there is a lot they must overcome.” Mentors adjust the trajectory of tasks based on how they see the newcomers’ performance: “if the task should take about a week and the person finishes in a couple of days, you think ‘hmm this may be too easy for them’.”

RECOMMENDING REPETITIVE TASKS Mentors also use a strategy of assigning newcomers repetitive tasks to help them master specific skills and gain confidence before they move to more complex tasks. For example, P3 mentioned that“I usually choose tasks that are repetitive but not very technical, so maybe modifying many lines of code in the same way. It is not very technical, but then they learn a bit more about the programming language or the API.” P6 complemented this view: “Usually, it is better if they don’t have to learn a new skill right away.”

LETTING NEWCOMERS CHOOSE THEIR TASKS This was not classified as a strategy, but we want to highlight that some mentors prefer to let newcomers find tasks that suit their expertise and interest, as P5 explained: “I don’t assign tasks. One step of being a contributor is choosing your own task. We use Bugzilla, which lists all the bugs and tasks, and newcomers go there and pick something to work on. People come to me when they don’t know what to do. Then, I guide them about what is easier and more complex and give them pointers.” This view is shared with P1: “they usually just pick up something they know they would like.” Seven of our interviewees mentioned that they usually identify a subset of tasks and allow newcomers to choose the tasks they are more willing to accomplish.

Restructure project’s task landscape There are times during the project life cycle in which no newcomer-friendly task exists [16]; this was mentioned as a challenge by our mentors (Pro1.6). In these cases, mentors apply strategies to restructure the project’s task landscape to explore or define newcomer-friendly tasks. We found two strategies that mentors employ to achieve this goal, as can be seen in Fig. 4 and described below.

DIVIDING TASKS INTO SMALLER PIECES Whenever possible, mentors divide tasks into smaller pieces, as P3 explained: “usually we can divide the tasks based on the project itself and knowledge about specific parts of the project.” This mentor preferred to create tasks related to the user interface: “There are parts of the projects that are low-level and parts using the UI. The UI parts are less technical and less demanding for a contributor while low-level tasks are more demanding.”

ENCOURAGING NEWCOMERS TO ADD FUNCTIONALITIES Another strategy reported by our interviewees involves encouraging newcomers to propose and add new functionalities to the project. Regarding this, P5 stated: “In Gnome To Do, in the beginning, there were 6 tasks which were big and not suited to newcomers. Because I didn’t have any easily fixable tasks, I encouraged newcomers to add new functionality as a way to contribute.”

5 Discussion

Mentorship in OSS projects is beneficial [16, 28]. However, our work reveals that mentoring in OSS comes with its own challenges. Additional work is needed to better support mentors in these communities. In the following, we discuss the implications of this study for research and practice in light of our results and related literature.

5.1 Supporting mentors

Many challenges exist outside of the technical realm. As the answer to RQ2 shows (see Section 4.2), most challenges reported by the mentors are interpersonal (14 out of the 25 challenges). Additionally, as shown in RQ1’s answer (Section 4.1), several mentors report that they do not feel confident in English for non-technical conversations. Indeed, lack of English language skills is a barrier reported by our interviewees. As the literature points out [65, 66], interpersonal and social interactions are core in mentoring processes, mainly to onboard newcomers to a community and to create effective bonds and a sense of belonging [67]. P3 stated this as a problem with mentors: “…the biggest challenges for a mentor are: […] engaging in other ways than just coding. These projects are about community effort and more than just the code.” The mentoring literature shows that a mentor can help shield a mentee from flaming wars with more senior members and intervene to help them appropriately resolve such conflicts [19]. To mitigate these challenges, OSS communities may design training programs focused on improving social communication skills (including coaching and other psychosocial support [68]), which can be decisive for the newcomers’ onboarding success. However, as our participants revealed, often there is no available training for mentors—none of the participants in our study had received any mentoring-related training.

Lack of payment is a challenge for some mentors. Out results show that being a mentor in OSS is challenging. [28] argue that some benefit should be offered to mentors to compensate them for their efforts. Indeed, mentors are more likely to be paid than other contributors (RQ1 - see Section 4.1): around 63% of the surveyed mentors receive some form of payment. However, a still large number of mentors are volunteers and report NOT GETTING PAID TO BE A MENTOR (Per5) as a challenge (RQ2 - see Section 4.2). OSS communities that cannot provide financial compensation for mentorship could design mechanisms to acknowledge and encourage mentorship through other motivators. The literature points out that mentoring remains “invisible” work [69], which does not receive the same level of recognition as technical contributions. For example, leadership boards (e.g., GitHub contributions insights) only account for commits. Recognition is a key driver for participation in OSS [42, 58] and future work can investigate how this and other motivators play a role for mentors in OSS. Future work can also further investigate payment for mentoring activities. We posit that these (compensated) mentors might take on this role to help their colleagues from the same company.

Men and women are equally likely to perform mentorship in OSS. Although the literature has shown that women frequently perform community-centric roles [60], our results from RQ1 (Section 4.1) show that mentor and non-mentor gender distributions are similar. Nevertheless, the literature also underscores the importance of mentorship to onboard women to OSS [70], and that the duration of women’s engagement in OSS is negatively affected by the differences in men and women mentors’ perspectives about gender personalities; underestimation of women’s capabilities by both the open source community at large and women newcomers themselves; and men mentors’ lack of awareness about the community’s unequal and often harsh treatment of its women members [16]. As part of the soft skills training, OSS communities can discuss the use of gender-inclusive language, adopting a code of conduct, gender-specific strategies, and other strategies to better support women newcomers. Creating an inclusive community was reported as a challenge by our interviewees (RQ2 - see Section 4.2).

Personal differences should also be addressed. Our results show that some challenges (RQ3 - Section 4.3) are related to how diverse newcomers face challenging tasks. Some newcomers prefer to start big rather than small (RQ2 - see Section 4.2), women are particularly risk-averse [71], and cultural differences were reported as a challenge by our interviewees. Therefore, mentor-specific training may also address how to adjust the guidance to different personal and cultural traits.

Recommending tasks is particularly challenging. Our results from RQ2 (see Section 4.2) indicated that recommending tasks is particularly challenging for mentors, and we further explored these challenges in RQ3 (Section 4.3). We found that this process involves multiple challenges related to understanding newcomers’ backgrounds and having comprehensive knowledge about the tasks in the project. We also observed that the mentors use different strategies to overcome the challenges, rarely relying on automated processes. In the following, we discuss some suggestions to better support task recommendation in OSS projects.

5.2 Supporting task recommendation

Provide easily accessible information about tasks. When mentors assign tasks to newcomers, they need to filter them based on complexity and required skills. Without proper information, they must sift through the available tasks and read descriptions to triage tasks that are newcomer-friendly or “low-hanging fruits” [72]. Depending on the project size and complexity and the number of available tasks, this can be complicated and tedious. For example, the project kubernetes¹ had more than 2,000 open tasks when this paper was written.

Labeling/tagging is a widespread practice for providing support in task selection. During this process, project members add labels to tasks, as described by our interviewees (Section 4.3). Our survey respondents acknowledged that this strategy aids in overcoming most of the challenges (see Table 6). Adding detail to a task can help both mentors and newcomers develop a more holistic picture of its complexity and solution, facilitating assessing whether a task matches a newcomer’s profile. We suggest that tags inform the tasks’ appropriateness for newcomers in terms of required skills, priority, estimated effort, and difficulty. Although tagging is an approach recommended by GitHub and OSS project guidelines, sometimes projects do not have the capacity to triage and label tasks. This may lead to another problem: the lack of (explicitly) available tasks (Prod2.3). When a project adopts the labeling strategy, it is important to maintain and update the labeled issues to prevent newcomers from taking on already fixed or outdated issues [73]. Providing automated ways of tagging and better supporting human annotators are still open research opportunities that could provide great help to mentors and newcomers [74].

Explore different ways to understand newcomers’ skills. Our results indicate that expertise and skill identification (Prod2.7) is a key challenge for mentors who recommend tasks to newcomers. The evidence collected here extends the understanding that awareness of developers’ skills is the foundation for building productive teams [75, 76]. [27] presented a theory mapping the main traits of software developers’ expertise. Mentoring is explicitly presented as a central part of the theory since it helps “building knowledge and thus contributes to the development of expertise.” This theory also highlights that mentorship is a feedback mechanism that may help developers gain task-specific and general knowledge. Although the theory shines light on the importance of mentorship in knowledge acquisition, the authors could not find appropriate ways to assess expertise objectively. Our results echo this evidence since we noticed that the mentors did not provide any objective way to assess newcomers’ expertise.

Mentors usually interact with newcomers to collect additional information to match their current skill level with the skills required for a specific project or task. This unstructured communication creates a solid mentor-mentee bond and facilitates ties that may influence future engagement. This strategy is in line with the landscape feature of proactive assistance and mentoring culture stated by [77], which was considered as the most influential in how pleasant and efficient the integration experience was for the newcomer. Seven participants also mentioned it as an effective way to reduce social fear (Prod.1). However, sometimes the gathered information can be subjective and influenced by low or high levels of self-confidence. Their self-assessment may not be accurate [27], because “if a person’s pattern of past performance was highly variable in relation to relatively constant evaluation criteria [they] would not be able to form a stable assessment of his ability …highly variable tasks may not be characterized by a person as skill task at all but as involving factors beyond his control” [64].

Communication issues may also impact the strategy of directly asking newcomers to identify their skills. The literature points out that professional developers experience gaps in communication (both written and oral communication) [78] and negotiation skills [79]. Additionally, the completely remote mentoring (e-mentoring) approach usually employed in OSS, as opposed to face-to-face interactions, may reduce trust between the parties [80] and likewise decrease communication effectiveness [81]. Thus, in some cases, it may not reduce or eradicate newcomers’ social fears [82], especially in the initial (or introductory) phase of the joining process [83]. Miscommunication on text-based channels is common since context and expression are often lost [80]. However, as the literature indicates, e-mentoring—a computer-mediated alternative to classic face-to-face mentoring—can scale up mentoring, providing more information to mentees and enabling them to connect with more people than classic face-to-face mentoring would allow [84]. E-mentoring has particular applicability for OSS communities, as the work is conducted in a distributed way [85].

As several of our interviewees reported, another way to identify existing skills is by relying on a portfolio built on data from previous interactions with repositories and technical communities. Indeed, software development leaves traces of development activities in the repository, which can then be used for inferring the expertise of developers [86]. Existing tools such as “My GitHub Resume”² and “Visual Resume” [87] may help to assess developers’ skills based on real interactions. However, these tools are not widely used by OSS developers.

Moreover, sometimes no historical data of a newcomer’s contributions are available to evaluate expertise. Therefore, mentors who cannot promptly identify newcomers’ skills can instead evaluate them by giving small tasks to the newcomers. This strategy allows them to explore the project while they have the chance to assess the newcomer’s performance. This manner of assessing skills may be effective for contextualizing the newcomer’s background within the project boundaries. However, it may create further issues if the tasks used for this evaluation are too easy or too complex, including demotivating newcomers and reducing retention. Therefore, mentors should explore multiple ways to understand newcomers’ skills.

Create a pathway for newcomer learning and retention. According to our survey respondents, starting with smaller tasks and following them with more complex tasks is an effective strategy. Creating an adequate path and providing appropriate encouragement and support may also help in engaging and retaining newcomers. Creating these pathways depends not only on the skills of the newcomers but on aspects such as developers’ goals, motivations, and availability [74]. One option is to provide tasks that require higher competence of a particular skill and then move on to more complex and broad tasks. Another option is to keep newcomers working on tasks that fit their current skill set. The choice for the most appropriate alternative should align with the newcomers’ goals, which requires constant follow-up from the mentor. The theory of Legitimate Peripheral Participation (LPP) has been widely adopted and describes how participation, situated learning, and identity construction are interrelated and can evolve as a person joins a community of practice [88]. Instead of processing information in an isolated style, situated learning occurs through social interactions and puts a strong focus on practicing the acquired knowledge [89]. Fang et al. [90] suggested that OSS developers’ learning behavior is situated in their everyday activities and that newcomer retention can occur after having repeated positive situated learning and identity construction social interactions. The strategy mentioned by our participants of creating an appropriate and evolutionary pathway of tasks with appropriate encouragement and support is also aligned with the situated learning of identity construction LPP.

5.3 Implications

Education and Training Personnel: People interested in Education and Training can make use of our findings to better understand the challenges faced by mentors in OSS and provide mentoring-specific training. When asked whether they had been trained to act as a mentor, all participants answered “no.” It is important to offer training on the skills needed to be a mentor, either at the undergraduate level or even in a professional environment. Moreover, given the number of social challenges revealed by the participants, (future) professionals must acquire the proper “soft” skills that will better prepare them for mentoring. The challenges evidenced here also serve as a starting point for making instructors aware of what to expect when incorporating OSS projects in their curricula, which is becoming more common [91–94].

Online communities: Given the number of challenges mentors face (25), communities must provide appropriate support to those who volunteer or act as mentors. We found that providing up-to-date and straightforward documentation of available open issues and precise tagging of available open issues are some techniques that OSS communities can utilize to support newcomers and mentors. We also identified a set of task recommendation strategies that mentors can use to recommend tasks to newcomers. Tagging the tasks (and keeping them up-to-date) was frequently recommended. In fact, this strategy is already in place in several well-established projects, like LibreOffice, Apache Open Office, Mozilla, Gnome, Media Wiki, and Ubuntu. Moreover, we found that “difference in the devices that mentors and mentees use” is also a challenge. Thus, providing ways to make it easier to build the system locally greatly benefits the onboarding process. A potential solution would be a pre-configured environment, utilizing a Virtual Machine with a built environment [72] or a container management tool, such as Docker.³

Mentors: Mentors can benefit from the set of challenges uncovered in this paper by becoming more aware of what they can expect when dealing with newcomers and better prepare themselves for supporting those willing to contribute to or join the community. Mentors can also take advantage of the strategies presented in Section 4.3, employing them to support newcomers.

Researchers. This research described several challenges and strategies that could be further investigated and supported. Research is needed to help the community develop a more precise description of newcomer-friendly tasks. Observational studies may be conducted to understand what information newcomers look for when selecting a task and provide insights about the dimensions that should become labels. Information needs and mentoring strategies for newcomers with different learning styles [95] could also be investigated. In addition, exploring how the newcomers actually find information may inform machine learning approaches to automatically suggesting labels for issues. Moreover, identifying better ways to elicit newcomers’ characteristics, evaluate their effectiveness, and propose novel and more effective methods of evaluating newcomers’ characteristics can be a potential research topic. Furthermore, future work may focus on investigating how each challenge and strategy influence newcomers with different motivations [42], retention in the project, and assertiveness in recommended tasks. Understanding motivation and demotivation factors influencing mentors can also be investigated in future research.

6 Limitations and threats to validity

Sampling bias could have affected our survey. However, the survey was widely deployed, receiving 600+ respondents representing a broad set of demographics and projects. Nevertheless, although it mirrors the distribution of our population, we received very few answers from women, which may have contributed to the lack of statistically significant results when comparing men and women.

Sampling bias can also affect our interviews, including self-selection and social desirability biases. However, we counteracted this effect by seeking different perspectives, inviting people with diverse profiles and backgrounds, and from various projects. We also acknowledge that the size of our sample (18 interviews) can be considered small. However, we continued interviewing until we could not identify any new challenges or strategies in two interviews in a row. The amount of participants is in line with the anthropology literature, which mentions that a minimum of 10 knowledgeable people is enough to uncover and understand the core categories in a study of lived experience [96]. Finding participants for this study was challenging because mentoring in such non-structured environments can take place through private, not publicly visible communication channels [2]. To increase the number of respondents in our study, we deployed multiple tactics to reach mentors, contact survey respondents, as well as reaching out to personal contacts (and snowballing) and previous contacts with Google Summer of Code mentors, social media posts, and OSS-related mailing lists. We relied on self-reported experience in mentoring to select our participants. During the analysis of the interviews, we looked for evidence of their mentoring experience, and we could not find any case for which we identified a mentor with low experience.

Another threat to validity can arise in the qualitative analysis process. However, to avoid misinterpretation in the qualitative coding of the data, we used the constant comparison method. As new codes emerged, we compared them with the existing code set and frequently met with the research team to discuss and clarify the codes. To increase the construct validity, we worked closely with the Diversity & Inclusion (D&I) committee at the ASF to craft the survey and interview questions such that it was accessible to the community.

However, we likely did not discover all possible challenges and strategies or provide a complete explanation of them. We are aware that the OSS universe is vast, and challenges and strategies can differ according to projects.

Finally, this research focused on OSS settings to gain a deeper understanding of this specific community. Challenges and strategies may be different for companies, other online communities, and different types of users. Future research should focus on analyzing the commonalities and differences among challenges and strategies in different domains to build generalized models and theories about onboarding and mentorship in open collaboration communities.

7 Related work

Newcomers joining OSS projects face many challenges [4], and task assignment is a recurrent hurdle [17]. In the following, we provide more details about the existing literature related to mentorship in OSS projects, choosing appropriate tasks for newcomers, and strategies to support task recommendation.

7.1 Mentoring

Mentoring is explored in several domains and activities: in management literature is a way to help new employee socialization [97–99] (e.g., helping newcomers understanding the company’s processes, the internal culture), and in education (teaching) literature is a way to help new teachers acclimate [100–102] and students overcome learning challenges [103–105]. For our purposes, a mentor is someone of advanced rank or experience who guides, teaches, and develops a novice [106]. Some existing literature analyzes the challenges faced during the mentorship. For example, [107] conducts a literature review analyzing the challenges related to gender in the mentor-mentee relationship. In the education domain, [100] explores the problems encountered in mentoring new teachers, while [108] explore the challenges faced by faculty members while mentoring online doctoral students.

Software Engineering has also taken up mentoring as an object of study [109–111]. In fact, the importance of mentorship as part of the knowledge acquisition process for novices is evidenced in the theory of software development expertise developed by [27]. In closed source settings, formal mentorship is a common practice to support newcomers [111]. [77] reported teams that proactively mentor newcomers make integration easier.

Formal mentorship in OSS has also been investigated. Some researchers focused on automatically identifying and recommending mentors [10, 12–14], claiming that mentoring benefits newcomers’ onboarding. [2] conducted a case study to assess the impact of mentoring and found that it significantly improves newcomer onboarding. In addition, [15] studied the impact of mentoring on developers’ training and retention and introduced a measure to assess mentoring capacity to facilitate learning and retention. In contrast, [9], who studied Mozilla, evidenced that onboarding programs may not result in long-term contributors, even though mentored newcomers considered the mentorship program valuable. [8] investigated the strategies OSS projects use to mentor and onboard newcomers in the context of Summer of Code programs. Mentorship is adopted by several prominent OSS projects (e.g., RedHat,⁴ KDE,⁵ Apache,⁶ and OpenStack ⁷).

Although the literature shows some interest in OSS mentorship and approaches to support finding an appropriate task for newcomers, to the best of our knowledge there is no systematic identification of strategies that mentors employ to help newcomers select their tasks and the challenges they face in this process. The present work adds to the OSS onboarding and task recommendation literature and the scarce literature about mentoring in open collaboration environments.

7.2 Task recommendation

The OSS literature widely reports the dilemma of finding an appropriate task, because new developers find it challenging to identify bugs that interest them, match their skill sets, are not duplicates, and are important for the community [112] [17]. For example, [113] reported that newcomers need specific guidance on what to contribute (e.g., open issues, required features, and simple tasks to start with), while [114] reported that communities expect new participants to find tasks to work on, although they sometimes assign tasks. The literature also shows that newcomers struggle to find a task [17] while they often also face an arduous learning curve to handle the technical complexity, given the lack of domain knowledge or project information available for starters.

Some initiatives aim to support newcomers to find appropriate tasks, like Up For Grabs,⁸ First Timers Only,⁹ and Awesome for Beginners,¹⁰ which aim to aggregate easy issues from several OSS projects. GitHub¹¹ encourages projects to tag issues that are easy for newcomers, which is a strategy also used by communities such as LibreOffice,¹² KDE,¹³ and Mozilla. ¹⁴

The literature also includes findings related to recommending and filtering out tasks. For instance, [115] presented Hipikat, a tool that builds a group memory and recommends source code, email messages, and bug reports to newcomers. Similarly, [112] previously developed a Tesseract extension that enables newcomers to identify similar bugs through synonym-based searches and to visually explore a bug’s socio-technical dependencies.

Another way to support task recommendation is through systems that match people to tasks. [116] used a voting heuristic based on analyzing the modification history of each artifact related to the task. [117] similarly use machine learning in the project’s history to identify and suggest the developer most familiar with certain artifacts, identifying this person as an expert. Finally, DebugAdvisor [118] also aims to recommend developers based on expertise on the source code related to the task.

Although the literature explores strategies for task recommendation in OSS projects, they rely on extensive manual work to tag the issue tracker since issue trackers usually either do not consider newcomers’ skills and interests or only work with developers who have previous experience in the project. In this work, we extend the existing literature by uncovering mentors’ strategies to recommend appropriate tasks for newcomers.

8 Conclusion

OSS communities frequently rely on mentors to guide newcomers to become long-term, active contributors. According to our data, contributors are less likely to be a mentor when they volunteer (not compensated to contribute) and have less than 1 year of experience.

We also identified 25 challenges faced by mentors in OSS projects. Further, we had a deep investigation of one of those challenges - Difficulty in identifying appropriate tasks for newcomers (ID Pro2 from Table 6). We identified 7 challenges that mentors face when recommending tasks to newcomers. The identified challenges range from handling newcomers’ low self-efficacy and understanding their background to deal with poor information about the tasks available at hand. We also identified 13 approaches to recommending tasks for newcomers. These strategies relate to identifying newcomers’ characteristics, scaffolding newcomers’ skill acquisition, identifying task complexity, identifying skills required to finish a task, and restructuring the task landscape.

To conclude, mentorship in OSS is still an under-explored research area. Our results provide initial insights about characteristics of mentors and challenges they face, and strategies they employ, which OSS communities and mentors can leverage to improve the current state of practice. Providing better support for mentorship can help to attract more volunteers to this important role and, indirectly, provide smoother onboarding of newcomers to OSS projects.