Skip to main content
  • JSONPath: from blog post to RFC in 17 years

    Today the JSONPath RFC (RFC 9535) proposed standard was published, precisely 17 years after Stefan Gössner wrote his influential blog post JSONPath – XPath for JSON that resulted in some 50 implementations in various languages.

    • Glyn NormingtonRFC 9535 Editor
    21 Feb 2024
  • Stepping towards a Sustainable Internet

    The IAB’s new Environmental Impacts of Internet Technology (E-Impact) program will hold its first virtual interim meeting over two slots on 15 and 16 February 2024. These interim meetings are open to participation, and we invite all interested community members to join, participate, and contribute.

    • Jari ArkkoE-Impact Program Lead
    • Suresh KrishnanE-Impact Program Lead
    7 Feb 2024
  • What’s the deal with Media Over QUIC?

    In 2022, the IETF formed a working group for Media Over QUIC (MoQ)—a media delivery solution that has the potential to transform how we send and receive media during live streaming, real-time collaboration, gaming, and more.

    • Brett BralleyThought Leadership Content Writer, Cisco
    25 Jan 2024
  • IETF Administration LLC 2024 Budget

    The IETF Administration LLC has finalised its budget for 2024.

    • Jay DaleyIETF Executive Director
    18 Jan 2024
  • Update on the IT Infrastructure Transition Project

    Begun in the last quarter of 2023, work is underway to define and deploy a new, cloud-based infrastructure approach for services that support the work of the IETF, and to move those services onto the new infrastructure.

    • Robert SparksIETF Tools Project Manager
    12 Jan 2024

Filter by topic and date

Filter by topic and date

IETF 112 Plenary Experiment Evaluation

  • Internet Engineering Steering Group

4 Feb 2022

For the IETF 112 meeting, the Internet Engineering Steering Group (IESG) conducted an experiment by holding the plenary on the Wednesday before other IETF sessions occurred, in order to reduce session conflicts during the week.

When we announced the experiment, we announced five evaluation criteria. All of those criteria were imperfect, and there is no target number of improved metrics to declare the experiment a success. Nevertheless, the results are below.

1. An improvement in survey responses reporting session conflicts compared to previous IETF online meetings.

The survey data don’t see any clear change in conflicts.

IETF 112 Conflicts

It is possible that the large 2-5 and 6-10 bins are masking some improvement, but the IESG concludes that there is no change in the survey responses as a result of this experiment.

2. Positive response to a new survey question about subjective satisfaction with the format change.

The survey indicated a mean score of 3.7 on satisfaction with "Holding the plenary the week before the IETF meeting", with a roughly 3:1 satisfied:dissatisfied ratio.

IETF 112 survey format change satisfaction

The IESG concludes that there is strong community sentiment in favor of the change, although it is possible that the positive responses are concentrated among those with no intention of attending the plenary, and the negatives among the minority who usually do attend.

3. Elimination of a ninth track, and a reduction in formal conflicts in the final agenda compared to previous online meetings.

There was no ninth track. The scheduling tool, which has not remained completely stable over time, tracks several different types of conflicts based on information provided in session requests.

IETF Meeting Chair Conflicts Area Director Conflicts Key Participant Conflicts Technology Overlaps
108 0 1 1 0
109 0 3 2 0
110 0 6 2 3
111 2 2 2 0
112 2 1 0 0

Other efforts to reduce conflicts over this time, including a variable number of session requests, further complicate any conclusion. Nevertheless, the IESG concludes that the plenary experiment did coincide with fewer conflicts reported by the software.

4. Little or no reduction in plenary attendance (< 15%) compared to other online plenaries in European time zones (i.e. IETF 108 and 110) 

There were 299 attendees at the IETF 112 plenary, a drop in absolute terms of 23% from IETF 110 and 32% from 108. In relative terms to the number of attendees:

Proportion of Meeting Attendees at Plenary

The IESG concludes that the experiment did have a significant negative impact on plenary attendance, although future attempts might reduce the extent that participants are surprised by the change and unintentionally miss the plenary.

5. The subjective experience of the IESG and Secretariat in attempting to minimize conflicts during IETF 112.

Several members of the IESG expressed that the IETF 112 scheduling process was considerably easier and required fewer tradeoffs. Some Area Directors think that this is mostly due not to the experiment, but instead because of soft limitations on session requests instituted before IETF 112.

Summary

The IESG’s consensus is that the plenary experiment did coincide with a reduction in scheduling conflicts, and most IESG members believe the experiment was an important cause. However, we also believe that the attendance drop at the IETF 112 plenary was unacceptable, although it might not be a permanent feature of this innovation.

We do not believe that the experiment results justify changing standard procedure for fully remote meetings, though we may conduct further experiments to reduce the disparity in attendance.

Regardless of this experiment outcome, the IESG fully intends for hybrid meetings, including IETF 113, to hold plenaries during the meeting week.

Acknowledgements

Thanks to Liz Flynn, Amy Vezza, and Jay Daley for helping to collect the data.


Share this page