Skip to main content
  • Suggested IETF 119 Sessions for Getting Familiar with New Topics

    These IETF 119 meeting sessions included discussions and proposals that are accessible to a broad range of Internet technologists whether they are new to the IETF or long-time participants.

      26 Feb 2024
    • Google and consortium of local organizations to host first Australian IETF meeting in over 20 years

      Google, auDA, and Internet Association Australia (IAA) provide key support for Brisbane meeting to be held 16-22 March 2024

        23 Feb 2024
      • JSONPath: from blog post to RFC in 17 years

        Today the JSONPath RFC (RFC 9535) proposed standard was published, precisely 17 years after Stefan Gössner wrote his influential blog post JSONPath – XPath for JSON that resulted in some 50 implementations in various languages.

        • Glyn NormingtonRFC 9535 Editor
        21 Feb 2024
      • Stepping towards a Sustainable Internet

        The IAB’s new Environmental Impacts of Internet Technology (E-Impact) program will hold its first virtual interim meeting over two slots on 15 and 16 February 2024. These interim meetings are open to participation, and we invite all interested community members to join, participate, and contribute.

        • Jari ArkkoE-Impact Program Lead
        • Suresh KrishnanE-Impact Program Lead
        7 Feb 2024
      • What’s the deal with Media Over QUIC?

        In 2022, the IETF formed a working group for Media Over QUIC (MoQ)—a media delivery solution that has the potential to transform how we send and receive media during live streaming, real-time collaboration, gaming, and more.

        • Brett BralleyThought Leadership Content Writer, Cisco
        25 Jan 2024

      Filter by topic and date

      Filter by topic and date

      IETF 111 post-meeting survey

      • Jay DaleyIETF Executive Director

      23 Aug 2021

      The results from our IETF 111 post-meeting survey are now available.

      The survey results for the IETF 111 post-meeting survey are now available on a web-based interactive dashboard.  We are always very grateful for the detailed feedback that we have received and will continue to process over the next few months.  The commentary below highlights a couple of areas where changes we have made based on feedback have been a success, and areas we still need to work on.  

      Analysis

      In total 166 responses were received.  Of those 164 participated in IETF 111 from a population of 1329 giving a margin of error of +/- 7.17%. The total numbers of meeting participants has grown (1196 for IETF 110, 1282 for IETF 109) while the survey response rate is significantly down on previous meetings (299 for IETF 110, 258 for IETF 109).

      The results for satisfaction questions include a mean and standard deviation using a five point scale scoring system of Very satisfied = 5, Satisfied = 4, Neither satisfied nor dissatisfied = 3, Dissatisfied = 2, Very dissatisfied = 1.  While there’s no hard and fast rule, a mean of above 4.50 is sometimes considered excellent, 4.00 to 4.49 is good, 3.50 to 3.99 is acceptable and below 3.50 is either poor or very poor if below 3.00. The satisfaction score tables also include a top box, the total of satisfied and very satisfied, and a bottom box, the total of dissatisfied and very dissatisfied, both in percentages.

      In this commentary a comparison is made with the IETF 110 Meeting Survey results using a comparison of means that assumes the two samples are independent even though they’re not but neither are they fully dependent.  A dependent means calculation may give a different result. A few comparisons are also made using a comparison of proportions.   

      Overall satisfaction

      The mean satisfaction score for IETF 111 (Q10) was 4.13 with 87% either ‘Satisfied’ or ‘Very satisfied’.  This is a statistically insignificant decline from the 4.20 mean satisfaction score for IETF 110.

      Preparedness

      Overall, preparation remains good with 83% reporting that they are either sufficiently prepared or well prepared. 4.35% were definitely under-prepared (Q8), well up on the 1.45% for IETF 110, which itself was down from 5% for IETF 109 but still a very small percentage of the total. The comments on this are mixed, with some pointing to the resources for practising with the tools, some bemoaning the lack of in-person meetings and some asking for WG chairs to provide the agenda earlier.

      Satisfaction with the agenda

      Overall satisfaction with the IETF 111 agenda was at 3.91 (Q12), a statistically insignificant decline from 4.04 for IETF 110 but now rated as 'acceptable' down from 'good'. Looking at the individual parts of the agenda (Q11), most of the satisfaction scores had no statistically significant change except for satisfaction with side meetings, which had a statistically significant improvement, and satisfaction with existing working groups, which had a statistically significant decline. Satisfaction with the opportunities for social interaction remains at 'very poor' and for side meetings, plenary and RGs it rates at 'acceptable'. For all other parts it rates 'good'.

      Satisfaction with the meeting structure

      Overall satisfaction with the structure of the IETF 111 meeting was 4.08 (Q14), a statistically insignificant drop from 4.20 for IETF 110. However, looking at the individual parts (Q13) there were statistically significant drops in satisfaction for the policy of scheduling online meetings in the timezone of the in-person meeting that they replace, for the overall length of the day and for starting at 12pm "local time", though this latter question was asked slightly differently this time. The first two are both rated as 'acceptable' and the latter is rated as 'poor' and only just above the boundary with 'very poor'. Satisfaction with 9 parallel tracks remains only 'acceptable' at 3.60 but for IETF 109 which has 8 parallel tracks, the score was 3.64 suggesting that a reduction to 8 would not improve satisfaction in any measurable way.

      Impact of being fully online

      In response to a hypothesis that moving fully online was allowing more people to participate in the IETF than would otherwise have been able to, a new question was added "If this meeting had been a normal (no COVID) in-person meeting in San Francisco with 8+ hour days and free remote participation then which of the follow best applies to you?" (Q15). Only 2 of the 148 who answered this question said that they would not have participated and 5 would have participated in fewer sessions, offset by 4 who would have participated in more sessions, suggesting that the hypothesis does not hold.

      Sessions

      41% experience no session conflicts (Q18) in IETF 111, up from both IETF 110 and IETF 109 at 35%, though a statistically insignificant increase. Satisfaction at conflict avoidance was 3.76 (Q20), a statistically insignificant increase from IETF 110 at 3.73. This again rates at 'acceptable' and suggests that more work is needed to get it up to 'good'.

      For this survey we omitted any questions on sessions running out of time to keep the survey short.

      Participation mechanisms

      Satisfaction with Meetecho remains 'good' at 4.29 (Q21) and only 'acceptable' for Gather at 3.77 and jabber at 3.68. After experimenting with Gather for four meetings we may need to experiment with alternative mechanisms for social interaction. Satisfaction with the audio streams and youtube videos both experienced statistically significant declines down to 3.84 and 4.09. Further investigation is required to understand this.

      Problem reporting

      Satisfaction with our response to problem reporting declined from 4.31 for IETF 110 to 4.00 (Q25) (the number of people responding to this question are so small that this is a statistically insignificant decline). From the comments it seems that much of this is down to confusion about what sessions require separate registration and the time it takes to fix that when someone try to join a session that is getting underway. A good improvement suggested in the text (Q26) is to put information on how to get support onto the agenda page.

      Final feedback

      Clearly, the majority just want to get back to in-person meetings with some using this survey (Q27) to complain about the decision to switch IETF 112 Madrid to an online meeting. Some are concerned that the lack of in-person meetings is making the IETF less relevant and less well supported by community volunteers.


      Share this page