Embracing Key Values and Key Values Indicators: How can SNS projects drive sustainable change?
The interactive workshop ‘Embracing Key Values and Key Values Indicators: How can SNS projects drive sustainable change?’ jointly organised by 6G4Society and FIDAL, held online on 25th October, brought together projects working on Key Value Indicators (KVIs) for an engaging session focused on interactive dialogue and active contributions from participants.
The main goal of this workshop was designed around two sets of activities. These explored: 1) how SNS projects are working with Key Value Indicators (KVIs) to address the positive impacts of 6G innovation on society, including their hopes and fears in this process; and 2) challenges and best practices in defining and implementing KVIs in ways that show impact. The aim was to better identify what are the expected outcomes from using KVIs. After short presentations to set the scene, participants were engaged via interactive dialogues and encouraged to actively contribute to guided activities on a MIRO Board. Below are the initial high-level results. Not all participants voiced all statements below, but these are the trends that emerged.
Hopes
There are great hopes that KVIs provide a new route to align innovation with societal expectations in ways that can support: building public acceptance of 6G; creating more responsible and ethical innovation; and putting people and the environment as a priority. By doing this, it is hoped KVIs can support technology developers to focus on the relevant aspects beyond technological performance in their work. Some participants wished for KVIs to create a strategic evaluation framework aimed at more effectively identifying high-potential projects. Overall, there is hope that a common framework can be developed for KVIs, with concrete, practical, step-by-step guidance on how to select and apply KVIs that already exist, and, if needed, practical steps to develop project-relevant indicators. This is tied with a hope that KVIs can take into account different TRLs, project types, structures, and goals.
Fears
A shared fear revolves around measuring KVIs, particularly within a project’s lifetime. Projects currently see measurement or assessment as unclear, hard to define, apply, quantify, or qualify. It is even unclear to the projects if KVIs are to be measured during a project or afterwards, if they are looking at impact in the world, or if KVIs can be measured in lower TRL projects. Similarly, many projects are finding it difficult to map KVIs onto KPIs, fearing this mapping might not be meaningful or readily able to be validated. One reason for the fears expressed during the workshop is that KVIs are not sufficiently concrete for technology developers and discussions tend to remain theoretical rather than providing actionable steps for them. Another concern that some participants raised is that the bridge between what we talk about doing with KVIs and what we can do is too big, with the worry that KVIs will, because of these gaps, become a form of green-washing marketing tool. Another is the lack of engagement with key experts and researchers with the required knowledge and skills.
Assumptions
Quite a few assumptions about KVIs emerged that need to be confirmed or refuted moving forward. Not all fit together neatly into a single picture. These assumptions include that KVIs:
- need to be quantified, though this is not easy
- should be mapped onto KPIs and expressed as such
- represent factors that might not even be present during a project and thus not measurable by the project
- do not always require empirical validation
- can build on information that is available and not costly to obtain
- should be able to work with or without use cases.
Questions
Key Questions projects still have revolved around three primary themes:
- How do you make the values more tangible to developers? The questions within this theme point to the need to address different understandings of the societal values themselves, different ideas about when a KVI can be assessed, and engaging new types of data that is not technical in nature.
- What is the link between KVIs and KPIs? How similar in form (or not) are KVIs to KPIs? This is tied to trying to understand how to have an objective evaluation of a KVI or how to work with subjective measures in ways that are meaningful for technology designers. There were even questions about if KVIs have thresholds like KPIs.
- How can the scope and ambition of KVIs be clearly defined? These questions revolved around how to clearly define what projects need to manage and what they can really have a say about when defining a KVI. Questions considered how to appropriately bound the problem in ways that support filtering priorities and values in order to identify concrete and relevant KVIs, all without losing track of the impact, out there, in the future.
Showing Impact via a KVI
Defining the impact towards a societal value (e.g. how do you know you are making change that supports inclusivity?) for a project seemed quite a challenge for many of the attendees, particularly around articulating the scale and scope of the impact, which remain abstract concepts. In part, this was because of a need to better drill down to what about a value is being engaged in a project (e.g. what about trust? what about well-being? who/what does this matter for?). Consensus is needed on the definitions and objectives engaged, and this requires having clarity about who should be involved in such consensus processes, from project partners to external stakeholders (e.g. publics, businesses, policymakers, users), and how to declare the interpretations of the societal values as relevant and meaningful.
In addition, when projects were asked how they would show an external stakeholder they are creating a foundation for impact, projects realised they did not necessarily have the expertise to identify what that information, data, or features were, beyond technical KPIs. This kind of showing, it became evident, involves a different type of knowledge regarding verticals, contexts, and the non-technical goals of stakeholders in that domain in order to identify what would tangibly and convincingly indicate to someone outside a project that action towards a value was taken. Showing impact also requires interactions with users, with stakeholders more broadly, or with experts in the field to support interpretations and assumptions being made by the projects.”
Next steps
6GSociety and FIDAL will provide a detailed report of the workshop outcomes, which 6G4Society will translate into an action plan for building on these results in collaboration with the projects. This will highlight steps that projects can take or activities that 6G4Society can support across projects in order to address the fears, answer the questions, work towards the hopes, and produce a set of good practices in showing the impact KVIs seek to achieve.
