In May, Automattic’s engineering hiring team launched a user research study to better understand how our approach to tech hiring resonates with women and non-binary folks who may experience similar gender discrimination in the workplace and are experienced developers. We asked participants questions about how they saw their careers and next opportunities, as well as specifically asking about their reactions to our job postings. It’s easy to find advice on this topic, but the volume is overwhelming and the information is often conflicting. We wanted to cut through the vast quantity of Think Pieces On The Internet to make some changes based on actual data. Our goal, unsurprisingly, is to identify ways to increase gender representation on Automattic’s engineering teams.

This was a fascinating process, and we are truly grateful to the 71 people who responded to our request for feedback. Given that we had this level of interest within just five days speaks to the relevance of the topic, as attracting contributors to a research project can often be far more of a challenge. While much of what we learned (discussed below) lines up with research you can find online, listening to people share their individual stories was profoundly impactful on our interviewers. It also revealed further insights that we are eager to apply. Automattic’s hiring team has been able to make some immediate changes, but have much more to think about, and expect this to influence our ongoing efforts.

Methodology

Out of our pool of 71 engineers, we conducted in-depth interviews with 14 participants, and a further 24 answered our follow up survey. The majority of our respondents had over 10 years experience working in software development, and around 60% of them had looked for a new job in the past year. 71% of our respondents were from the US, and all but one participant lived in English speaking countries. All participants self-identified as women or non-binary. The research process was very similar to that used in product user research. We described our initial rationale, invited contributions, used a consistent script, and vetted respondents to ensure the diversity of our interviewing pool.

Results we expected?

• Respondents have strong networks that they use for job hunting and support.
• Respondents are most likely to leave a current role in search of growth – for example, Deutsche Bank discovered that women were leaving to take “bigger jobs” (DB’s term) , than they were offered internally, something that was addressed by adding a sponsorship program.
• Respondents carry out extensive research on companies before applying if they do not already have a personal connection in the organization, and are highly attuned to red flags. This matches what data shows on the experience of women in tech companies – in the Elephant in the Valley survey, 66% of women report being excluded from opportunities because of gender, 60% report being sexually harassed, and 60% of those who reported it were unhappy with the course of action. Womens’ career choices are too frequently based on avoiding a bad experience rather than seeking a place where they can thrive.
• Women in technical roles feel pushed onto the management track, and have to fight to continue to focus on technical work as individual contributors.
• Women receive a significant amount of personal outreach, and many have come to expect it. Generic requests from recruiters are unlikely to be successful.
• It’s important that other women are visible in the hiring process. Many women have experienced being “the only,” and at this point in their careers prefer that not to be the case.
• Hiring processes that are collaborative and responsive – a growth opportunity rather than purely evaluative – were perceived most favorably.
• Respondents care a lot about a respectful process. Responsiveness is a key metric, and was also a theme in interactions.

What was surprising?

• Respondents read job descriptions almost exclusively for red flags; for instance, they search for language encouraging all relevant employees (not just women) to take parental leave.
• Glassdoor is not considered useful, being seen as mainly a place for negative reviews by former employees or unsuccessful candidates.
• Many interviewees mentioned using Stack Overflow to look for jobs. We had previously advertised on that platform and discontinued it based on the demographics of applications it drove; diversity of applicants was even lower than the already too-low industry average (as also seen in their survey). However based on these findings, we are running another test on that platform.
• For some participants, interviewing is seen as a skill to maintain. These people are continually looking at job listings (though at a less-intensive level than a full job search), to ensure that they are aware of their options – before reaching a point where they’re desperate to leave their current position.
• Consistently being able to have an impact (including leadership opportunities) stood out as important, and they will leave their current role if they cannot find this. This is a key point related to research showing women are evaluated more on their past performance and men more on potential.
• Women are looking for more communities focused on connecting to other senior women, and around more technical topics (many communities focus on entry to mid-level folks). Concerns around online harassment can put women off trying to build their network online.
• Respondents look to have as much information as possible up front – about the job itself, salary ranges, the team they would join, and the hiring process, and expectations in terms of responsiveness.

What changes did we make?

• Existing work and life commitments mean that it is important to know the details of the hiring process at the outset: we have published a public page that clearly outlines our hiring process so that people have a concrete understanding of the expectations. We’ve also put together a more detailed document about what to expect during interviews, and will be creating similar materials for the other phases of our process.
• We are revisiting our employment branding strategy to take into account the extent of research by candidates, revamping our developer blog, collecting resources on what it’s like to work as a developer at Automattic.
• We are working on supporting our existing developers to increase their visibility externally, through encouraging them to write (our OSS culture goes beyond our willingness to share code), speak, and engage more in their communities. Many co-located companies use their office space to host community events, and it’s clear that is a smart strategy, because it builds employer brand, awareness, and personal connections between candidates and the host/prospective employer. We are looking to find a more distributed, office-less way to approach this.
• Historically, we’ve resisted asking existing under-indexed employees for second shift work – it’s a too-common reflex- to put the work of “diversity” ahead of and at the expense of those for whom we should be doing the work of “inclusion.” This research made it clear that the highest-leverage activity for our existing under-indexed employees was chatting with potential candidates, and we are making casual chats with other women more available to potentially interested applicants. We see this as more impactful than asking women in particular to interview. While this still amounts to asking for second shift work, we are aiming to identify the highest leverage activities and ask just for participation in those, rather than putting the full work of “diversity” on the diverse, and expecting them to do everything on top of their current job.
• We removed all the little games from our job posting page. We were trying to test people’s attention to the job posting and filter out unmotivated candidates; it turned out we were also putting people off who we want to apply.
• We removed all the language that emphasized that hiring is a competitive process -for instance, removing language about application volume.
• We will be testing adding “(Senior)” into all our technical job postings. Historically, we’ve been reticent to do this because it implied a job ladder that doesn’t exist at Automattic, but the research clarified that its absence sent the message that our positions are all mid-level roles without the path to growth that women candidates tend to look for.
• We rewrote our job postings highlighting the learning and career opportunities for the candidate, rather than focusing on our needs.

What are we considering going forward?

Some of what we learned was less concretely applicable, or requires broader changes across our engineering organization.

We’re considering how to adjust our messaging with respect to D&I, and whether some of this messaging should be tailored to developers and D&I within the engineering organization specifically. We also need to consider how this interacts with the commitment to Open Source – the data shows that women are less likely to contribute to Open Source projects, in part because of the culture of many of these projects.

The best way to retain the women we have is to ensure that they have opportunities for growth outside of affinity networks. This is something we’re mindful of as we look to improve opportunities for growth across the organization. We’re also considering the research in Women Don’t Ask, and thinking about how to offer under-indexed folk opportunity, rather than expecting them to ask for it.

Another key area to explore is the importance of titles. As an organization, we have an unorthodox approach to job titles, promotions, and leveling; employees are free to create their own job titles, and promotion and leveling are holistic processes focused on the specific employee, their skills and development goals, and current team needs, rather than on a preconceived hierarchical chart. We hire only experienced engineers who will make the whole team better – the way they choose to do that is up to them and their team lead. Bringing in titles would be a significant change to the way we operate, potentially reduce our flexibility, and if not done carefully and thoughtfully would be destined to fail. We have to consider whether we can better facilitate growth, and make the opportunities for growth and recognition clearer in our job postings and hiring process.

Some changes we’ve made relating to this recently:

• We took a group of >20 team leads and developers to the Lead Developer event in London, and facilitated a day together. We will repeat this for US-based team leads in Austin later this year.
• We have started offering candidates the opportunity for one-on-one calls with a member of our Developer Experience team during later stages of our hiring process; we’re starting with under-indexed folks, with a view to rolling this out to everyone. These exist outside the regular hiring process, and is premised on the idea that it both provides an opportunity for the applicant to ask any questions they have and for us to better understand their career goals and motivation so that we can assign them to the team that’s the best fit.

It’s early for all of this, but we’re excited about the concrete changes we were already able to make, and the initial markers of progress we are tracking. As our Creed states, “I am in a marathon, not a sprint” – we have much more to do.

This project was a team effort from @sinitivity, @tobynieboer, @rlwilliams0803, @annahollandsmith, led and coordinated by @annezazu. You can find the official announcement and a more complete whitepaper here.