Skip to main content
  • Suggested IETF 119 Sessions for Getting Familiar with New Topics

    These IETF 119 meeting sessions included discussions and proposals that are accessible to a broad range of Internet technologists whether they are new to the IETF or long-time participants.

      26 Feb 2024
    • Google and consortium of local organizations to host first Australian IETF meeting in over 20 years

      Google, auDA, and Internet Association Australia (IAA) provide key support for Brisbane meeting to be held 16-22 March 2024

        23 Feb 2024
      • JSONPath: from blog post to RFC in 17 years

        Today the JSONPath RFC (RFC 9535) proposed standard was published, precisely 17 years after Stefan Gössner wrote his influential blog post JSONPath – XPath for JSON that resulted in some 50 implementations in various languages.

        • Glyn NormingtonRFC 9535 Editor
        21 Feb 2024
      • Stepping towards a Sustainable Internet

        The IAB’s new Environmental Impacts of Internet Technology (E-Impact) program will hold its first virtual interim meeting over two slots on 15 and 16 February 2024. These interim meetings are open to participation, and we invite all interested community members to join, participate, and contribute.

        • Jari ArkkoE-Impact Program Lead
        • Suresh KrishnanE-Impact Program Lead
        7 Feb 2024
      • What’s the deal with Media Over QUIC?

        In 2022, the IETF formed a working group for Media Over QUIC (MoQ)—a media delivery solution that has the potential to transform how we send and receive media during live streaming, real-time collaboration, gaming, and more.

        • Brett BralleyThought Leadership Content Writer, Cisco
        25 Jan 2024

      Filter by topic and date

      Filter by topic and date

      IETF 118 post-meeting survey

      • Jay DaleyIETF Executive Director

      30 Nov 2023

      IETF 118 Prague was held 4-10 November 2023

      The results of the IETF 118 Prague post-meeting survey are now available on a web-based interactive dashboard. Thank you to all of you who responded to this survey as we use your views to continually adjust the meeting experience.

      Analysis

      We received 336 responses, all of whom participated in IETF 118, 291 onsite and 45 remote. As no the respondents did not participate in IETF 118 the specific questions for them are not shown in the dashboard. With 1806 registered participants, this gives the survey a maximum margin of error of +/- 4.82% (4.9% for onsite and 14.17% for remote).

      The results for satisfaction questions include a mean and standard deviation using a five point scale scoring system of Very satisfied = 5, Satisfied = 4, Neither satisfied nor dissatisfied = 3, Dissatisfied = 2, Very dissatisfied = 1. While there’s no hard and fast rule, a mean of above 4.50 is sometimes considered excellent, 4.00 to 4.49 is good, 3.50 to 3.99 is acceptable and below 3.50 is either poor or very poor if below 3.00. The satisfaction score tables also include a top box, the total of satisfied and very satisfied, and a bottom box, the total of dissatisfied and very dissatisfied, both in percentages. Please note that a small number of questions are on a four point scale.

      Question changes since the last survey

      For this survey we added some new questions about side meetings as this remains an ongoing area of concern. Some questions were removed (e.g. satisfaction with 'new' WGs) to keep the survey of a reasonable length. Not asking about the venue overall was an inadvertent omission.

      Actions taken following the last survey

      For this meeting, we made the following changes, prompted by survey feedback:

      • Extended the meeting to a full day on Friday to provide for more sessions, partly to ease pressure on sessions and partly to give more options to avoid conflicts.
      • Added a new Farewell Reception after the meeting closed on Friday to provide an opportunity for people to unwind and say goodbye.
      • Added an alcohol-free event in the form of a movie night.

      Unfortunately, this meeting did not have a meeting host and so there were no free t-shirts and no social event.

      Satisfaction

      Overall satisfaction is 4.42 which is a very good result. With some key exceptions, the satisfaction scores remain high, reflecting the various improvements made since we returned to onsite meetings.

      The table below shows the satisfaction scores for the last six meetings, along with colour coded indicators for the five point scale above: excellent (🔵), good (🟢), acceptable (🟡), poor (🔴), very poor (⚫️)

      Satisfaction scores for the last six meetings
      IETF 118 Prague IETF 117 San Francisco IETF 116 Yokohama IETF 115 London IETF 114 Phila. IETF 113 Vienna
      Overall satisfaction 4.42 🟢 4.30 🟢 4.30 🟢 4.28 🟢 4.19 🟢 4.36 🟢
      AGENDA
      Overall agenda 4.20 🟢 4.16 🟢 4.18 🟢 4.22 🟢 4.06 🟢 4.16 🟢
      Sessions for new WGs - 4.19 🟢 4.17 🟢 4.12 🟢 4.15 🟢 4.18 🟢
      Sessions for existing WGs - 4.22 🟢 4.22 🟢 4.22 🟢 4.10 🟢 4.24 🟢
      Sessions for WGs 4.32 🟢 - - - - -
      BOFs 4.12 🟢 3.95 🟡 4.11 🟢 4.10 🟢 4.09 🟢 4.04 🟢
      Sessions for existing RGs - 4.12 🟢 4.14 🟢 4.10 🟢 3.95 🟡 4.13 🟢
      Sessions for RGs 4.09 🟢 - - - - -
      Plenary 3.94 🟡 3.99 🟡 3.98 🟡 3.98 🟡 3.98 🟡 3.94 🟡
      Side meetings 3.79 🟡 3.75 🟡 3.73 🟡 3.81 🟡 3.73 🟡 3.52 🟡
      Hackathon 4.27 🟢 4.25 🟢 4.34 🟢 4.35 🟢 4.30 🟢 4.09 🟢
      HotRFC 3.90 🟡 3.89 🟡 3.84 🟡 4.21 🟢 3.94 🟡 4.17 🟢
      Pecha Kucha 4.00 🟢 4.15 🟢 - - - -
      Office hours 4.07 🟢 3.98 🟡 4.23 🟢 4.00 🟢 4.09 🟢 3.96 🟡
      Opportunities for social interaction 4.06 🟢 4.11 🟢 3.72 🟡 3.98 🟡 3.89 🟡 3.51 🟡
      STRUCTURE
      Overall meeting structure 4.34 🟢 4.28 🟢 4.28 🟢 4.28 🟢 4.19 🟢 4.26 🟢
      Start time 4.49 🟢 (9:30am) 4.28 🟢 (9:30am) 4.16 🟢 (9:30am) 4.28 🟢 (9:30am) 4.20 🟢 (10:00am) 4.12 🟢 (10:00am)
      Finish at 5pm Friday 3.65 🟡 - - - - -
      Length of day 4.20 🟢 4.30 🟢 4.30 🟢 4.32 🟢 4.10 🟢 4.20 🟢
      Number of days 4.18 🟢 (5+2) 4.27 🟢 (5+2) 4.30 🟢 (5+2) 4.32 🟢 (5+2) 4.30 🟢 (5+2) 4.23 🟢 (5+2)
      Session lengths 4.38 🟢 (60 / 90 / 120) 4.41 🟢 (60 / 90 / 120) 4.36 🟢 (60 / 90 / 120) 4.32 🟢 (60 / 90 / 120) 4.25 🟢 (60/120) 4.31 🟢 (60/120)
      Break lengths 4.38 🟢 (30/90) 4.32 🟢 (30/90) 4.38 🟢 (30/90) 4.36 🟢 (30/90) 4.25 🟢 (30/90) 4.16 🟢 (30/60)
      Number of parallel tracks 3.94 🟡 (8) 4.08 🟢 (8) 4.01 🟢 (8) 3.90 🟡 (8) 3.86 🟡 (8) 3.92 🟡 (8)
      PARTICIPATION MECHANISMS
      Meetecho 4.47 🟢 4.35 🟢 4.45 🟢 4.45 🟢 4.23 🟢 4.36 🟢
      Gather - 3.52 🟡 3.46 🔴 3.37 🔴 3.06 🔴 3.04 🔴
      Zulip 3.94 🟡 3.66 🟡 3.77 🟡 3.73 🟡 3.56 🟡 2.91 ⚫️
      Jabber - - - - - 3.80 🟡
      Audio streams 4.27 🟢 4.02 🟢 4.21 🟢 4.04 🟢 4.05 🟢 4.14 🟢
      YouTube streams 4.36 🟢 4.32 🟢 4.36 🟢 4.25 🟢 4.22 🟢 4.25 🟢
      CONFLICTS
      Conflict avoidance 4.00 🟢 3.90 🟡 3.94 🟡 3.91 🟡 3.78 🟡 3.89 🟡
      VENUE & ACCOMM
      Overall accommodation 4.34 🟢 4.07 🟢 4.09 🟢 - - -
      Overall venue - 3.90 🟡 - - - -
      Location 4.52 🔵 3.60 🟡 - - - -
      Venue facilities 4.43 🟢 4.07 🟢 - - - -
      Cost of rooms 3.54 🟡 2.87 ⚫️ - - - -
      Availability of rooms 4.32 🟢 4.07 🟢 - - - -
      ONSITE
      Overall 4.50 🔵 4.29 🟢 - - - -
      Badge collection 4.70 🔵 4.69 🔵 - - - -
      WiFi 4.17 🟢 3.98 🟡 4.06 🟢 4.10 🟢 3.82 🟡 -
      QR Codes 4.18 🟢 4.11 🟢 - - - -
      Break F&B 4.05 🟢 4.44 🟢 - - - -
      Breakout seating 3.80 🟡 4.08 🟢 - - - -
      Signage 4.15 🟢 4.22 🟢 - - - -
      Coffee carts 4.64 🔵 4.56 🔵 - - - -
      Childcare 4.12 🟢 4.06 🟢 - - - -
      Welcome reception 4.08 🟢 - - - - -
      Farewell reception 4.34 🟢 - - - - -

      Areas for improvement

      Cost of rooms

      We have only recently started to track this and so this stage we can only note that satisfaction is on the low side, particularly as we book meetings some time in advance.

      Side meetings

      While there is a slight uptick in satisfaction, though not significant, this remains an issue. For this survey we asked some satisfaction questions to help understand the problem better:

      Satisfaction scores about aspects of side meetings
      Subject Satisfaction score
      The number of side meetings 3.71
      The scheduling of side meetings 3.51
      The length of side meetings 3.93
      The remote participation technology in side meetings 3.41
      The content of side meetings 3.95

      These results suggest that the problem is around the organisation and support of side meetings rather than the content. There are lots of comments about side meetings that seem to indicate that the current piecemeal approach to improvement may not be enough and more comprehensive review may be needed.

      Conflict avoidance

      We again follow the new process to reduce conflicts as described in our blog post, and extended the meeting on Friday afternoons. This did not lead to a statistically significant increase in satisfaction with conflict avoidance. This may be something that simply cannot be improved without adversely affecting some other aspect of the meeting, such as length of the day.

      Zulip

      We have seen a statistically significant uptick in satisfaction for Zulip, as we hoped we would see as people became more familiar with it. However the satisfaction score is still not high enough. The comments on Zulip vary from those with views on the integration of chat into Meetecho, and those who use Zulip directly and would prefer a different product. It is really not clear if it possible to choose another product that would get a higher satisfaction score given that there are so many different products out there that people use and prefer.

      Breakout seating

      We thought we had this one sorted but apparently not. Back to the drawing board to understand exactly what people want, which appears to be seats within ~10 meters of the meetings rooms.

      Extending to Friday afternoon

      This meeting experimented with a full day on Friday and the satisfaction score for that is only acceptable not good. Given that these sorts of changes take some time to bed in, we will monitor that for the next couple of meetings. There are a number of comments that this was not advertised enough, but it is mentioned clearly in all of our emails about the meeting organisation.

      Individual comments

      The individual comments cover a number of key themes, some of which have already been covered above:

      • The location of the venue. Almost all of the comments here we positive, both about the city and the ease of getting there. Unfortunately, we will not be returning for several years at least as the venue now charges too much for our budget.
      • Cost of registration, accommodation and travel. There are multiple comments that the cost of participating onsite is too high, particularly from those in the academic community. The various recommendations include: a more general bursary program, lower registration fees, and lower cost/quality venues.
      • Food and beverages provided. While there is a notable amount of positive feedback, there is also a clear theme that the provision for vegans and vegetarians was insufficient and that more care needs to be taken. There is a specific request for batch brew coffee that will be considered and a request that the baristas provide tea, which we almost fell for.
      • Crowding. There are multiple concerns expressed about the density of some events, in particular the welcome reception. The background here is that this was planned to be in a larger room but this was swapped with the Hackathon room because of the very large number of people who signed up for the Hackathon. The actual number who turned up for the Hackathon was significantly lower than the number that signed up, but even then that room was crowded. The underlying issue here is how to estimate in advance the number of people coming to the large events, which is something we have to do many years in advance in order to book the space. Adding new space nearer to the time is almost impossible because it has normally been sold by the venue to other clients.

      Why people participated remotely

      For the second time, we asked people who participated remotely (Q5), why they did and if they would have preferred to participate onsite (Q5a). Once again, the major factor, cited by 27 of the 45, was the lack funding to travel, all of whom would have participated onsite if they could have. This means that there are several hundred who would have participated onsite if funding was available, which should be taken into consideration when considering any possibly bursary program.

      And finally

      Thank you everyone who responded to this survey, your feedback is much appreciated.


      Share this page