Insights on the large-scale deployment of a curated Web-of-Trust: the Debian project’s cryptographic keyring

In this section, we present background information needed for better understanding of what prompted our work, the process underwent by the Debian project that prompted for a fuller analysis to gain understanding of the keyring itself.

Throughout this section, a set of Research Questions (RQs) are presented, which guide the discussion that follows.

Given we are already using the developers’ keyring to measure social engagement, connectedness and activity of individuals, it makes sense to study the KSPs. We will thus proceed to compare the size, progression and reach of the KSPs held at the Debian yearly developer’s conference, DebConf (DC).

Before starting with this session’s analysis, we must point out this section was written not only based on properties obtained from the data set, but with personal knowledge of one of the authors having been a participant and organizer of DebConf for most of its editions; we explain some observed trends based on insights of the Debian project community that cannot be supported by formal reference material.

Of course, KSPs are not only held at DC; a long-standing tradition is for developers to announce in the debian-private mailing list their travel plans, and it is customary for them to explicitly mention meeting their peers to exchange key signatures. There are also formal KSPs at different free software meetings. But for the purposes of our study, we decided to focus on the largest and more representative events.

We have analyzable data for the past twelve DC editions. Attendance to DC (and participation in its KSPs) varies by several factors, mainly the geographical location and the world (as well as each country’s) economic conditions, but we will present some hypotheses as to some observed trends. The number of participants per each DC edition, as well as the proportion of its participants who were part of the KSP is shown in Fig. 1.

We say this distribution is natural because the most attended DCs (15, 17 and 7) were held close to important developer density population centers, and the lowest attendances (12, 8) were held in countries further away. Given the developer density in the United States, attendance for DC 10 and 14 rate seems low; this might be caused by several developers not willing travel there due to their ideological positions, or unable to do so, due to their national origin. We found it surprising to find only 246 people attended DC 9, held in Western Spain, in a region that between 2002 and 2013 pioneered free software development [23]; its number, however, is close enough to the following most attended, 6 and 16, to require further information to explain their relative sizes - Issues such as economic conditions on the relevant year ([24], p. 26) or even perceived ease to travel to the destination.

Not every attendee to the conference takes part in the KSP. Looking at the proportion of attendees who signed up to participate shows also an interesting picture. Only three times KSP participation has had over half of the attendees (DC 9, with 72.76%, 6, with 59.46%, and 10, with 53.49%). Other than those three occurrences, participation has remained in a fairly narrow 15% band, between 34 and 49%.

Not all participants in DC, nor in the KSP, are formal members of the Debian project (Debian Developers). Figure 1 captures this as well. We find some interesting patterns when looking at both relations; the match is not perfect: we counted each key as belonging to a DD if it had an identity with an @debian.org mail address. Some (few) DDs are known not to add this identity to their key material.

DC 9 was quite outstanding as it is by far the conference with least non-KSP participants, although it keeps a high proportion of non-DD participants. This can relate to the economic recession mentioned in ([24], p. 26): while fewer Debian Developers attended the conference than usual, it was held in a region with high free software involvement; looking at the list of participants, it contains a high number of local people affiliated with different Free Software projects.

DC 7, 8 and 9 have the lowest percentage of DDs in the KSP. This can be caused by the aftereffects of the Transnational Republic experiment carried out in DC 6: a DD, well known and well connected to the WoT, presented a seemingly official ID document generated by a fictional non-government, the Transnational Republic experiment [25, 26]. This incident led to an important discussion within the project and a change in the way KSPs were held after said conference. As this clearly showed, this had the result of socially diluting the responsibility of verifying each other (that is, if everybody has checked somebody’s identity, each participant does only a very cursory look on the documents. Given KSPs were already over 150 participants long, expecting thoroughness was clearly unrealistic.

Now, how significant are KSPs to the connectedness of their participants? Not everybody in a keyring cross-signs during a KSP. In fact, until the aforementioned incident in DC 6, the KSP protocol in Debian used to be for every participant to form two lines, having ID documents ready, and cross-check every other participant’s identity.

From DC7 onwards, keysigning sessions are held in a continuous KSP fashion: after an initial session where a document stating the key for the full document with the keys of every participant is verified to be correct and the same for every KSP participant, attendees are encouraged to meet and have a relaxed talk and introduction with each other. Of course, the amount of cross-signatures every person will get is much smaller than with the two-line model, but they are also of greater significance.

After DC 9, the keyring maintenance team started raising awareness of the need to migrate to stronger keys, as explained in Section 2.2.1 and detailed at length in [9]; we infer that raised the need to reconnect newer, stronger keys into the keyring, as the high DD participation in KSPs starting at DC 10 shows.

Considering this, how effective are KSPs to build trust in keyrings? To this effect, we measured the ratio between total signatures and keys on each KSP’s keyring (that is, the average percentage of the keyring each key is directly connected to), at the date of the KSP session itself, and weekly for the following 15 weeks; this can be seen in Fig. 2.

KSP effects are not immediate; participants are encouraged to print out a copy of the base document at home, from a computer system they ultimately trust; the first thing shown by Fig. 2 is how long does it usually take to KSP participants to sign, send, receive and upload the keys: while there is a big variation in the initial three weeks, changes quickly converge and change past said period is very minor.

As for the total connectedness for each of the KSP keyrings, the inner increment is quite sensible for all cases during the observed period; DC 7 shows the least improvement, increasing the average percentage of keys signed directly from each key from 14.55 to 21.45 (a 47% increase). A hypothesis towards such a small increase could be that DC 7 was held in Scotland. The Debian-UK community, predominantly centered in Cambridge, is very large and tightly knit socially from the very early days of the project. The KSP started off with a very well connected keyring, and although this has been one of the largest conferences in the project history and the total participants in the KSP is the second highest, being this the first KSP after the Transnational Republic experiment mentioned above, it follows that many people would be reluctant to sign keys from people they weren’t already familiar with.

Two years later, the KSP had the biggest increase: DC 9’s keyring went from 3.20 to 11.55, a 261% increase. Do note that, although it was the smallest conference in Europe, Table 1 shows the absolute numbers for the KSP were coincidentally equal to DC 7’s, and the proportion of DDs was almost equal. We think this might be explained because many of the DDs who attended DC 9 were not strongly connected to the project, in contrast with the mentioned situation of Debian-UK.

As mentioned at the beginning of the present Section, most of the explanations of phenomena depending on social interaction are just hypotheses; we have to acknowledge several data are within the range for uncertainty to play a heavy hand.

The work done for the described keyring migration, as well as the migration process itself, presented a great opportunity to understand the key migration as a social phenomenon as well, using the keyring as a way to measure social cohesion and vitality.

We prepared graphic representations of the keyring at its various points in time, in the hope to learn from it patterns about its growth and evolution that can warn about future issues. For the trust-mapping graphs, we use directed graphs, where each key is represented by a node and each signature by an edge from the signer to the signee. For starters, we were interested in asserting whether the characteristics observed on the whole OpenPGP WoT [11] repeated in the subset of it represented by the Debian keyrings. Of course, said work was done as a static analysis on the keyring back in 2011; back then, the whole OpenPGP keyring stood at 2.7 million keys; at the time of this writing there are 4.5 million keys, growing by 100 to 400 keys every day [27].

Figure 3 presents seven snapshots of the main developers keyring, processed by Graphviz using the neato layout program, which implements the spring minimal energy model [28]. Of course, at the scale they are presented, each individual edge or node becomes irrelevant; there is too much density at the center, and the outlying nodes and edges appear as just noise. However, the shape of the strong set⁴ does lend itself to analysis.

Ulrich notes that the WoT graph is similar to a scale-free one and exhibits a hub structure, but is not scale-free in the strict sense.

Something happened, however, in the course of 2010 that led to the WoT acquiring the shape shown in Fig. 3c by the end of the year - Instead of a seemingly uniform blob, there is a distinct protuberance. This horn grew throughout the following years, and by 2014, the keyring consisted of two roughly equivalent blobs somewhat weakly linked together, as Fig. 3d and e show.

We find this protuberance to be the portrait of a social migration: The project is often portrayed as unique among free software projects due to the close personal ties among developers; its yearly developers’ conference, DebConf, has a very high repeating attendance rate. However, given the project has lived for over 20 years, it is understandable many of the original members have grown inactive and moved on to other life endeavors; formal retirement is requested from developers, but many people reduce their engagement gradually, and just never formally retire.

While the geographical dispersion makes it quite hard for some developers to meet others and obtain new certificates, as we already mentioned there is a tradition in Debian to announce travels in a (private, developers-only) mailing list, and active developers often will gladly meet people traveling to their region just for a key signature exchange.

Although the number of developers that by late 2010 had migrated to a stronger key was still quite small, the call for key migration was initially answered by those with most active key activity -hence, probably more conscious about the importance of this migration. Of course, although it was not a targeted removal, it was a socially self-selected one: trust hubs were among the first to migrate to stronger keys. And even though they attempted to re-build their WoT relationships and cross-sign with other developers at gatherings such as DebConf, the group of developers that -as explained in Section 4- had drifted away from project activity didn’t reconnect with them.

While the migration to keys longer than 1024 bits took much longer than originally expected, the initial push was not bad: during 2010, it reached from practically zero to close to 10% of the keys - But many of those keys were hubs, people long involved in the project, with many social bonds, and thus very central keys. When those people migrated to newer keys, the signatures linking their long-known fellow developers to the project were usually not updated, and several old keys could have even become islands, gradually losing connectivity to the strong set.

Given Debian’s longstanding practices, rather than isolated, many such keys started drifting apart as a block, growing separated from the center of mass. This explains why the migration started as a lump to later become two large, still somewhat strongly connected bodies, mostly stable over the years. Of course, as more developers migrated to strong keys, by late 2014 the remaining group started losing cohesion, and by December 2014 (before it was completely removed), it is barely noticeable - All of its real hubs had migrated to the new center of mass, with many previously connected keys becoming isolated, as Fig. 3f shows.

In order to prove this hypothesis, we generated again the same graphs, but factoring in the trust aging: if individual signatures are colored by their age, it is possible to visually identify if a significant portion of the group’s trust is aging - That is, if social bonds as reflected by intra-key signatures are over a given edge. The seven subfigures of Fig. 4 correspond with those of Fig. 3, but with color-coded edges (according to the image caption)⁵.

Surprisingly, even Fig. 4b shows a clear grouping of keys by signature age - But this grouping does not appear a year earlier, in Fig. 4a. This can, again, be indicative that the first people to migrate to stronger keys, even before it altered the overall shape of the WoT, migrated during 2009; by early 2010, they might constitute the tight, new (blue) group still in the periphery, that eventually became the core of the newer blob.

Following from the same data set, we started a further statistical analysis; this section presents the preliminary results we gathered from applying survival analysis techniques.

The general focus of survival analysis is on the modeling the time it takes until a specific event occurs, in social sciences one often speaks of event history [29]. We have found interesting findings from studying how many people keeps participating in the Debian project throughout the time, that is, to model the time until departure from the Debian project. The main motivation comes from the need to understand keyring population along time and from the the implications of survival as reliability of subjects (it is more likely for someone to be trusted if they’ve been long enough in a community), thus arising a rough measure for trust. Our sampled data is defined by the keys that make up the curated WoT from the Debian Developers keyring [9].

The analyzed data is treated as a longitudinal study. We point out that intervals are not of the same length in time: each data point is a tag in the keyring’s Git history,⁶ and the period of analysis spans between July 2008 and July 2016. During said period, 124 tags were recorded, averaging to 23.96 days each, with a standard deviation of 27.95, with a maximum of 69 days and a minimum of one day.

Given the way the keyring is structured, we used the long key ID (the lowest 64 bits of its fingerprint, in hexadecimal representation) as a unique identifier for each key. For each tag and key we counted the number of signatures made to that key by counting the number of non-zero entries in the corresponding key column of an adjacency matrix at a specified tag.

We identified people’s participation in Debian using their key activity record (has the key stopped getting signatures?) and keyring membership (has the key stopped being part of the keyring of interest?) which codes our data as right-censored because no further information about keyring membership is known afterwards; right censoring scheme constitutes data where all that is known is that the individuals are still alive (the keys are still active) at a given time, [30].

For this analysis, in order to make a key-to-person correspondence, we considered key ownership using the following equivalence relation: keys are equivalent (i.e. they refer to the same person) if they have the same metadata. We considered that using the name as metadata was naturally appropiate to make a distinction among people.

We used the R programming language mainly leveraging routines from survival and flexsurv packages; unless otherwise stated, significance level is assumed to be 5%.

Our line of approach begans showing the proportion of remaining people in the keyring along time through the survival function, that is, keyring permanency. Then, using the accumulated hazard function, we get the expected exits per person that remains in the keyring until the end time (in perpetuity). And lastly, using the hazard rate function, we get the departure rate from the keyring.

For the non-parametric or observed curves we used the Kaplan-Meier product limit estimator for the survival function, [31], the Nelson-Aalen moments estimator for the accumulated hazard function [32], and the kernel density estimator for the hazard rate function, [33].

A parametric estimation to see the mortality law fitting our data was made using a Generalised Gamma distribution through maximum-likelihood estimation [34]. The motivation for using the Gen. Gamma model is due to the closeness and confidence band coverage it has to the observed hazard rate function obtained non-parametrically. Proper justification for said model comes from the fact that it minimizes Akaike’s Information Criterion when compared to the other models, making it a better model in terms of information loss, [35], while also rejecting other models using a log-likelihood test of -1422.395 at 3 degrees of freedom, [36]. The estimated parameters found for our model were μ=2.399, σ=0.2722, Q=2.5594, with standard errors of 0.0719, 0.1432, 1.3519 respectively.

Finally to make inference about the effect of received signatures as a predictor for survival, we used a semi-parametric model using the Cox Proportional Hazards model [37] by taking the average number of signatures received by a person as a covariate. The estimated regressor for the avg. num. of signatures received was β=−0.00839, with a standard error of 0.0024 and a p-value of 0.0046. Proper verification for the proportional hazard assumption for the avg. num. of signatures received was done testing the Scaled Schoenfeld Residuals [38] against a Schoenfeld Individual Test which yielded a p-value of 0.1617. This means the average number of signatures received by someone has a statistically significant risk contribution at any given time [39].

In the non-parametric plot of Fig. 5, we observe downward steps when at least one people stops getting signatures. The crosses represent the followup time for censored observations (for which no further information is known and thus the proportion of keys remains). This plot does reflect the fact that many keys were dropped during the 1024-bits key removal (circa January 2015). Observed proportion of keys being above the theoretical model from year 3.5 to 6.5 years suggests that after three years the keys wouldn’t be much likely to leave; at least not until after 6.5 years, where the probability of remaining afterwards is almost as the 50% chance of heads in a coin flip. It is remarkably that keyring permanency doesn’t really go below 50%, showcasing good health in the keyring.

As we mentioned, due to the 1024-bit key migration, there is a clear skew that introduces a sharp drop around 6.5 years.

Figure 6 shows the people exits given one key in perpetual risk, that is, if it is to remain in the keyring for all its time span. The increasing steps from the non-parametric exits is natural being the accumulated sum per tag of the exits over remaining people ratios. The similarity from previous plot is expected since cumulated hazard is a logarithmic transformation from survival function. We see again that the observed plot lies below the theoretical model starting from year 3.5 through year 6.5 (about 3 years), quickly increasing afterwards more than expected. It is not until near year 3.5 that a someone is expected to be half-way to go, which is better when compared to the expected life of a subject being 5 years.

Figure 7 shows the departure rate is analogous to a mortality rate. The observed behaviour suggest that coming of age there’s a sudden increase on the risk i. e. keys "wear out" to their age around year 5.5, certainly close to the expected life. Yet the parametric departure rate being under the non-parametric rate at the final years shows the dramatic effect from the 1024 removal. Another remarkably finding was that departure rate in general gives empirical evidence to say that 7 out of 100 keys will leave “any time now” (from the fact that hazard rate is the instantaneous probability of failure at a specified time; failure means, the probability of a key will completely cease activity after a given time of life) in a 8 year lapse.

From Fig. 8 we see that the average number of signatures received trend has a negative risk contribution. It is noteworthy that coming of age, the risk contribution starts to be positive by a bit, effectively showing the effect of the hazard rate at later ages. The resulting effect was that for each signature received by someone it reduces the baseline risk by 0.83% on average.

So how many signatures does someone needs to be almost failure-proof? We found a minimal average number of 358 signatures are required to make up for a failure risk statistically close enough to 0 (provided that the subject holds consistent and not by-chance interaction with the people from said 358 signatures).

As a comparison, the max recorded average of signatures received by a person was of 168.45 yielding an absolute failure risk reduction of about 75.67% from the baseline, and as expected, said person is indeed active; the mean average of signatures received was of 11.82 yielding an absolute failure risk reduction of about 9.45% from the baseline.

In summary receiving 12 signatures (which is roughly the expected average number of signatures received) reduces the odds of exiting at any given time by 10% from the baseline risk. In general people in the project will constantly remain active for about 4.5 years, as long as they went through 1.5 years. It is just after six years where people effectively have an uncertain end unless they interacted meaningfully with other people: holding consistent relationship with at least 30% of people in the keyring can almost guarantee a lifetime membership.

The Debian keyring is a very peculiar subset of the whole OpenPGP Web of Trust analyzed in [11]. The work we present here provides data empirically supporting the theoretical observations, particularly regarding the robustness of what he defines as the LSCC (Largest Strongly Connected Component). The migration away from 1024-bit keys provided an opportunity to follow the progression of the connectivity in our WoT after several of its hubs were removed.

The preliminary results for this work have been shared with a group of Debian developers. Historically, the usual practice for key signing has been to generate non-expiring signatures; people that have already cross-signed their keys don’t have an incentive to refresh their trust. There is an ongoing discussion as to whether this practice should change towards time-limited signatures, better modeling ongoing social relationships, or to stick to current practice.

The resulting survival analysis can be used to generate an objective measure for trust; this study was done only on the Debian Developers keyring, it would be interesting to compare with the more loosely connected Debian Maintainers keyring. We also want to further explain the keyrings by stratification. The survival analysis showcases good health of the Debian Developers keyring (which makes up the mass of Debian’s WoT).

Finally, the methodology followed for this study could be applied to other free software projects, aiming to correlate with events and trends spanning a wider population than Debian’s; the applicability of our work to other projects, however, depends on having a proper data set to base the work off. As mentioned in Section 2.3, we are aware of only one large project interested in formally structuring a Curated Web-of-Trust keyring, but other data sets could be taken as inputs - several authors have performed studies based on the pattern of discussions in mailing lists [40]; most of the analysis we presented here is based on the observation of seven years of history, so it will take a long time before this can be applied over other data sets.