Skip to main content
  • IETF 116 Yokohama registration now open

    Registration is now open for IETF 116 Yokohama

    • Jay DaleyIETF Executive Director
    24 Nov 2022
  • IETF 115 post-meeting survey

    IETF 115 London was held 5-11 November 2022

    • Jay DaleyIETF Executive Director
    22 Nov 2022
  • Catching up on IETF 115

    Recordings are now available for sessions held during the IETF 115 meeting and the IETF Hackathon, where more than 1500 participants gathered in London and online 5-11 November 2022.

      13 Nov 2022
    • Opportunities for university researchers and students during IETF 115

      The upcoming IETF 115 meeting in London on 5-11 November 2022 is a unique opportunity for networking researchers to learn how RFCs are written, to engage with the Internet standards community to begin to develop research impact, and to meet more than 1,000 leading technologists from around the world currently working in industry, academia, and other organizations.

        1 Nov 2022
      • Suggested IETF 115 Sessions for Getting Familiar with New Topics

        These IETF 115 meeting sessions are likely to include discussions and new proposals that are accessible to a broad range of Internet technologists whether they are new to the IETF or long-time participants.

          24 Oct 2022

        Filter by topic and date

        Filter by topic and date

        IETF 111 post-meeting survey

        • Jay DaleyIETF Executive Director

        23 Aug 2021

        The results from our IETF 111 post-meeting survey are now available.

        The survey results for the IETF 111 post-meeting survey are now available on a web-based interactive dashboard.  We are always very grateful for the detailed feedback that we have received and will continue to process over the next few months.  The commentary below highlights a couple of areas where changes we have made based on feedback have been a success, and areas we still need to work on.  

        Analysis

        In total 166 responses were received.  Of those 164 participated in IETF 111 from a population of 1329 giving a margin of error of +/- 7.17%. The total numbers of meeting participants has grown (1196 for IETF 110, 1282 for IETF 109) while the survey response rate is significantly down on previous meetings (299 for IETF 110, 258 for IETF 109).

        The results for satisfaction questions include a mean and standard deviation using a five point scale scoring system of Very satisfied = 5, Satisfied = 4, Neither satisfied nor dissatisfied = 3, Dissatisfied = 2, Very dissatisfied = 1.  While there’s no hard and fast rule, a mean of above 4.50 is sometimes considered excellent, 4.00 to 4.49 is good, 3.50 to 3.99 is acceptable and below 3.50 is either poor or very poor if below 3.00. The satisfaction score tables also include a top box, the total of satisfied and very satisfied, and a bottom box, the total of dissatisfied and very dissatisfied, both in percentages.

        In this commentary a comparison is made with the IETF 110 Meeting Survey results using a comparison of means that assumes the two samples are independent even though they’re not but neither are they fully dependent.  A dependent means calculation may give a different result. A few comparisons are also made using a comparison of proportions.   

        Overall satisfaction

        The mean satisfaction score for IETF 111 (Q10) was 4.13 with 87% either ‘Satisfied’ or ‘Very satisfied’.  This is a statistically insignificant decline from the 4.20 mean satisfaction score for IETF 110.

        Preparedness

        Overall, preparation remains good with 83% reporting that they are either sufficiently prepared or well prepared. 4.35% were definitely under-prepared (Q8), well up on the 1.45% for IETF 110, which itself was down from 5% for IETF 109 but still a very small percentage of the total. The comments on this are mixed, with some pointing to the resources for practising with the tools, some bemoaning the lack of in-person meetings and some asking for WG chairs to provide the agenda earlier.

        Satisfaction with the agenda

        Overall satisfaction with the IETF 111 agenda was at 3.91 (Q12), a statistically insignificant decline from 4.04 for IETF 110 but now rated as 'acceptable' down from 'good'. Looking at the individual parts of the agenda (Q11), most of the satisfaction scores had no statistically significant change except for satisfaction with side meetings, which had a statistically significant improvement, and satisfaction with existing working groups, which had a statistically significant decline. Satisfaction with the opportunities for social interaction remains at 'very poor' and for side meetings, plenary and RGs it rates at 'acceptable'. For all other parts it rates 'good'.

        Satisfaction with the meeting structure

        Overall satisfaction with the structure of the IETF 111 meeting was 4.08 (Q14), a statistically insignificant drop from 4.20 for IETF 110. However, looking at the individual parts (Q13) there were statistically significant drops in satisfaction for the policy of scheduling online meetings in the timezone of the in-person meeting that they replace, for the overall length of the day and for starting at 12pm "local time", though this latter question was asked slightly differently this time. The first two are both rated as 'acceptable' and the latter is rated as 'poor' and only just above the boundary with 'very poor'. Satisfaction with 9 parallel tracks remains only 'acceptable' at 3.60 but for IETF 109 which has 8 parallel tracks, the score was 3.64 suggesting that a reduction to 8 would not improve satisfaction in any measurable way.

        Impact of being fully online

        In response to a hypothesis that moving fully online was allowing more people to participate in the IETF than would otherwise have been able to, a new question was added "If this meeting had been a normal (no COVID) in-person meeting in San Francisco with 8+ hour days and free remote participation then which of the follow best applies to you?" (Q15). Only 2 of the 148 who answered this question said that they would not have participated and 5 would have participated in fewer sessions, offset by 4 who would have participated in more sessions, suggesting that the hypothesis does not hold.

        Sessions

        41% experience no session conflicts (Q18) in IETF 111, up from both IETF 110 and IETF 109 at 35%, though a statistically insignificant increase. Satisfaction at conflict avoidance was 3.76 (Q20), a statistically insignificant increase from IETF 110 at 3.73. This again rates at 'acceptable' and suggests that more work is needed to get it up to 'good'.

        For this survey we omitted any questions on sessions running out of time to keep the survey short.

        Participation mechanisms

        Satisfaction with Meetecho remains 'good' at 4.29 (Q21) and only 'acceptable' for Gather at 3.77 and jabber at 3.68. After experimenting with Gather for four meetings we may need to experiment with alternative mechanisms for social interaction. Satisfaction with the audio streams and youtube videos both experienced statistically significant declines down to 3.84 and 4.09. Further investigation is required to understand this.

        Problem reporting

        Satisfaction with our response to problem reporting declined from 4.31 for IETF 110 to 4.00 (Q25) (the number of people responding to this question are so small that this is a statistically insignificant decline). From the comments it seems that much of this is down to confusion about what sessions require separate registration and the time it takes to fix that when someone try to join a session that is getting underway. A good improvement suggested in the text (Q26) is to put information on how to get support onto the agenda page.

        Final feedback

        Clearly, the majority just want to get back to in-person meetings with some using this survey (Q27) to complain about the decision to switch IETF 112 Madrid to an online meeting. Some are concerned that the lack of in-person meetings is making the IETF less relevant and less well supported by community volunteers.


        Share this page